google-bigquerygoogle-cloud-data-fusionscd2

Implement SCD2 logic in Data Fusion and BigQuery

I am trying to implement an SCD2 table loading with Data Fusion but can't seem to find the necessary building block to do it. (something that was presented here).

I could join the new records (stage table) with the target table, filter the unchanged records, but I could not do merge/update on the SCD2 type table, as there is no option to do this.

Is there any option to do merge/update on a BigQuery table using Data Fusion?

Solution

  • CDF currently does not support update on BQ, but may in the future.