azureazure-data-factoryazure-log-analyticsazure-monitoring

Setting up Alert for ADF Pipeline hang state


Sometimes my adf pipeline shows in progress status but in actual nothing happens. I find it's hung or nothing happening. I am looking for a solution which can atleast raise an alert on same to restart the pipeline. Can someone please suggest an alert rules or ways to tackle this scenarios

I tried few queries for alert rules but I hardly find a solution for this.


Solution

  • To prevent a Data Factory activity from keep running when its actually stuck for some reason, you can use timeout:

    For example, in the copy data activity below, the timeout is set to 30 minutes (the default is 12 hours). After which the activity will fail.

    You can use the retry and retry interval settings to rerun the activity if it fails. In the example below, the activity will try again 3 times, with 60 seconds between retries.

    copy data settings

    If you wish to send an email notification about the failure, rather than rerunning it, you can use logic apps, see documentation here - https://learn.microsoft.com/en-us/azure/data-factory/how-to-send-email?source=recommendations