azure-synapse

Invoke Synapse (secured behind private endpoint) pipeline using http(s)

We are using a tool called Tidal which has ADF built-Integration. And, ADF is invoking the Synapse Pipeline using Web activity - it makes call over https.

Both the ADF and Synapse Workspace are in the same subnet (public access for both services is disabled).

Now, ADF calls Synapse pipeline using https (ex: https://.dev.azuresynapse.net)

when we invoke the Synapse Pipeline using the above method, we get the below error

enter image description here

The private end points are configured properly and I guess the error is because we are invoking the public end point which travels over internet. How do we invoke the endpoint privately?

Thanks, Praveen

Solution

  • enter image description here

    To resolve the error above, follow the procedure below:

    1. Create an integration runtime with virtual network configuration as shown below:

    enter image description here

    1. Go to the Azure Data Factory manage tab and create a new private endpoint by clicking the "New" option. Select the Azure Synapse analytics option, choose your server name from the list, and select the dev option as shown below:

    enter image description here

    1. Click on "Create." This action will require approval from the Synapse workspace. In the Synapse workspace, go to the networking tab and select "Private endpoint connections." You will see a hyperlink that leads you to approve the private endpoint. Click on the hyperlink and approve it, as shown below:

    enter image description here

    1. After approving successfully, select the created integration runtime in web activity and select the authentication method as shown below:

    enter image description here

    It will debug successfully without any error.