adp.ingest.datatypes.cast_dataframes

adp.ingest.datatypes.cast_dataframes(df_delta: DataFrame, snapshot: DataFrame, type_escalation_mode: TypeEscalationModeEnum)

cast_dataframes df_delta and snapshot to the datatypes from get_escalation_target

FOR INTERNAL USE ONLY (None is returned when the df_delta is not casted)

Parameters:

  • df_delta (DataFrame) – Pyspark DataFrame from bronze/silver layer

  • snapshot (DataFrame) – New DataFrame to append/merge to df_delta, this is also a Pyspark DataFrame

  • type_escalation_mode (TypeEscalationModeEnum) – Which escalation mode to use. (strict/loose), see TypeEscalationModeEnum for more information.

Returns:

the converted delta Dataframe if converted else None, the converted snapshot

Return type:

(DataFrame | None, DataFrame)