azureazure-synapseazure-machine-learning-serviceazure-synapse-pipelineazure-ml-pipelines

How to run an Azure Machine Learning pipeline from Synapse?

I have a working v2 pipeline in Azure Machine Learning that I want to run from Synapse. However, I cannot select my pipelines from the dropdown menu.

I have a working linked service and a non-self-hosted integration runtime. When I click 'Open in Azure Portal', it brings me straight to the Pipelines page in my Azure ML environment.

Synapse pipeline trying to execute an Azure Machine Learning pipeline

The pipeline is also made available as a batch endpoint in Azure ML.

Deploy (v2) vs Publish (v1) of the pipeline to an endpoint Azure Machine Learning pipeline as a batch endpoint

How do I get Synapse to start a pipeline job in Machine Learning?

Solution

  • I got similar results when I tried the above scenario.

    enter image description here

    To resolve the above issue, make sure you publish the ML pipeline. Go to ML workspace -> Pipelines -> Pipeline endpoints and verify your pipeline endpoint should be here.

    enter image description here

    If not publish the pipeline by creating new pipeline endpoint.

    enter image description here

    Now, the pipeline will show up in the dropdown of the Synapse pipeline activity as shown below.

    enter image description here