Rollback

The delivery framework contains rollback functionality to rollback the state of a table to a previous point. Also, the framework can be used to rollback a group of the tables to a previous point in time. This group is defined by all the child %delivery_load statements in for a single a parent.py run.

The framework checks the \layer\name\metadata\run.delta table to determine whether the latest run has been executed successfully. If this is not the case, the user can either manually rollback all the tables, or use the framework to rollback all tables to their previous state.

You can enable checking whether the previous job run has finished successfully by setting the check_previous_run_successful to True in the @delivery_load yaml. When this is set to True, the framework will check whether the previous run has been executed successfully. If this is not the case, the framework will raise an exception by default. Also, with check_previous_run_successful set to True, you can also set restore_automatically to True in @delivery_load. This will automatically rollback all tables to their previous state when the previous run has not been executed successfully. This is useful when you want to make sure that the previous run has been executed successfully before starting a new run.

The rollback functionality can be configured via the @delivery_load yaml, However, the Python functions can also be called manually: The function rollback_table() can be used to rollback a table to a previous point in time. The function rollback_job() can be used to rollback a group of tables to a previous point in time.