Data loading steps
Follow these steps to perform a single load operation of a data source, ensuring data consistency and freshness in your data warehouse.
To complete a single load operation of a data source, Zuora recommends the following steps:
Insert into the query table all the rows whose is_deleted is false in the staging table
Zuora recommends that you script all the steps for all data sources so that you can perform them in a single transaction. In this way, your data warehouse clients always query against a consistent snapshot of the complete data set.
You can keep track of how fresh the data in the data warehouse is by maintaining a data freshness table of data sources and their last update dates.