google-bigquerychange-data-capture

CDC from BigQuery


Could you please tell if it is possible to capture CDC on a GCP BigQuery database in order to receive notifications for every INSERT/UPDATE/DELETE performed in BigQuery tables? If so, could you provide the documentation that describes how to do this? I know how to perform CDC to BigQuery, but I don't know if it is possible to perform CDC from BigQuery.


Solution

  • Backin April at Google's NEXT conference, a feature called "BigQuery continuous queries" was announced. The video for it can be found here. This feature continuously monitors one or more tables and when something "changes" in the table, an action is performed. One of the first uses is for outbound CDC. When something changes in the table, the corresponding changed row/details can be published to PubSub.

    As of the time of this post (2024-06), I do not believe that the feature is generally available ... however, if you reach out to your Google Cloud sales rep, there is a private preview programme on-going where you can get early access to it. I am not aware of when the formal general availability is scheduled.