Retrying failed ADF pipelines which were triggered originaly by blob storage events

I have an ADF pipeline which is associated with two triggers.

1. Storage Event Trigger.
2. Tumbling window.

"Storage Event Trigger" gives me the blob which was landed to the associated container. Assume, there is failure in the pipeline which was triggered by the "Storage Event Trigger". Now, since I have configured "Tumbling Window Trigger" for Retry mechanism, I want to get the data/blob name of failed pipeline which was triggered by "Storage Event Trigger" such a way that my retrying pipeline will get data with which it was failed to get it retriggered.

We have configured 2 trigger. 1. Storage Event Trigger 2. Tumbling Window Trigger.

To the Tumbling window trigger we configured the retry policy with retry attempt 3 & interval for it as 30 Sec. Also, have recurrence as 5 min.

To the storage Even trigger we are passing value as a parameter from triggerBody().fileName. Now, to the Storage Event Trigger we are getting value which got landed to the container associated with the trigger & pipeline is getting triggered.

Within the pipeline we configured 2 activity. 1. Set Variable & 2. Fail activity to intentionally fail the pipeline such way that it will be picked while doing retry by the tumbling window. From Storage Trigger we are trying to the Triggered File Name & referring it into the pipeline via pipeline Parameter provisioning.

AFAIK, The above two triggers will create its own independent pipeline runs and tumbling window trigger cannot re-run the failed storage event trigger pipeline run. It can retry its own pipeline runs.

To achieve your requirement, you can try the below pipeline design as a workaround.

First create variables with default as below in your pipeline.

Now, take an Until activity and give the below condition.

@or(equals(variables('count'),variables('retry')),equals(variables('status'), 'Succeeded'))

Here, this design first initiates the above variables and with pre-defined retry value. You need to give all of your required activities inside the Until activity. When any activity got failed, the until will check whether counter is equal to the given retry count or not and whether the status is Succeeded or not. it uses the Generic error handling pattern.

You need to below activities before and after your activities. Use two Set variable activities like below to increment the counter variable.

@add(variables('count'),1)

@variables('temp')

Now, after your activities, add another set variable activity on the skip and failure of the last activity and give value Failure.

On the success of last activity, add another set variable activity and give the value Succeeded.

On running this pipeline, if any of your activities fails, this will retry all activities for maximum of given retry count. It will stop whenever all activities give success.