adp.ingest.transform

Functions that transform dataframes to silver (merge) and bronze (append)

Functions

append(snapshot, table_name[, ...])	Append the snapshot to the table on the table location
clean_column(col_name[, replace_dots, lowercase])	clean_column to make it 100% compatible with PySpark
column_aware_difference(snapshot, df_base)	column_aware_difference returns the rows in snapshot that are not in df_base
merge(snapshot, table_name, ...[, silver_type])	Merge the new snapshot on the silver_table using the keys specified
with_datatypes_escalated(transform_func)	Decorator to make sure that the datatypes of the new data are proper and compatible