azurewebhooks

How do I create a Webhook in Azure so I can load data into a Datalake?


I apologize if my question is worded poorly, but I've been searching online for a few hours now and I keep running into dead ends and working on something until realizing it's not what I need. I was wondering if anyone had any good resources for creating a Webhook in Azure for loading data? What tool in Azure should I be using to make the Webhook? I have a vendor who will be sending POST calls to the Webhook once I build it to send data that I need to drop into a datalake or into a Synapse Data Warehouse. Any and all help is appreciated.


Solution

  • You can try Azure function apps as a workaround here in this case.

    Go to Azure portal -> Create Azure function app. You can select the Runtime stack as per your desired language. Here, I have chosen python function app. Give the same region as your storage account and set this as the function app storage account.

    In the function app, go to functions overview and create a new function of HTTP trigger type.

    enter image description here

    You can select the Authorization level as Function or Anonymous as per your security requirements.

    enter image description here

    After creating the Azure function, use the python code from this documentation to upload the file to your Data Lake. You can also refer this SO answer by @Rithwik Bojja.

    Now, if your function app authorization is Function level, generate the function app key and you can give this key to your clients.

    enter image description here

    For sample, I have used the URL, and you can see the return response.

    enter image description here