Hello and welcome everybody to our TNWiki Article Spotlight on Tuesday.
Lately I had to deal a lot with Extract-Load-Transform (ETL) processes. One problem I met several times is to determine which data has already been transferred to a database and which data needs to be processed next. A typical approach for solving this problem is to add extra fields to a database table or adding a processing flag. When working with a SQL Server there is another way ... Change Data Capture (CDC). Ricardo Lacerda has put together an article which walks you through CDC in a typical scenario: A SQL Server on-premise and one hosted in Microsoft Azure. He explains exactly the need and why typical solutions can be difficult to implement. After a short introduction he sets up the scenario and shows step-by-step how to move your data from one database to another with the help of a SQL Server Integration Services (SSIS) package using CDC.
If you have similar problems in your hybrid infrastructure and would like to see an elegant solution, check out this article!