The source of my Copy activity
is the result of calling a REST API
and the Sink
is a Azure SQL Table
that I insert those records in it.
Now I want to know what records we just got inserted in that Sink
so I can do some update statement
on those records. So my question is how can I know what went into Sink so I can now update those records.
Two methods come to mind
If the table has a timestamp of when the records were inserted then you could use that value to know which were just inserted.
Instead of putting the records directly in the final table put them in a staging table. Then you can either update as part of the insert to move them to the final table do the updates OR update them in staging table and then copy over to the final table. Just remember to truncate the staging table every run so it only has the new records.