google-cloud-dataflowgoogle-cloud-vertex-aigoogle-cloud-automl

Call AutoML prediction model from Dataflow SQL


Is there a way to call a AutoML prediction model from within Dataflow SQL?


Solution

  • No, from Dataflow SQL alone that's not possible. You could write a Beam pipeline, and have some SQL transforms in that pipeline, as well as a DoFn to call the AutoML prediction endpoint.

    Dataflow SQL uses the Zeta SQL variant of Beam SQL (if you want to reuse the same SQL code you are currently using with Dataflow SQL).

    To run the Beam SQL pipeline, you would run it like any other pipeline (a "regular" Dataflow job).