google-cloud-platformgoogle-cloud-functionsgoogle-cloud-dataflowdataflow

How to execute a Cloud Function after Dataflow job has succeeded?


I want to trigger Cloud Function only if Dataflow job execution completed successfully.

Cloud Function should not be triggered if Dataflow job is failed.

I am running a Dataflow job using a Dataflow template (jdbc to BigQuery) from the the Dataflow UI.

There is no option to trigger any Cloud Function or something after job execution. Also, I can't make changes in template code. What's the way to trigger Cloud Function?


Solution

  • There isn't, yet, built it feature for this, but I can propose a workaround.

    resource.type="dataflow_step"
    textPayload="Worker pool stopped."
    

    Like this, every end of dataflow job, a new message will be posted to PubSub and your Cloud Function will be triggered.