azureazure-pipelinesazure-data-factory

How to save error details in Set Variable activity if any sequential activity fails


The goal is to carry out error details if any of activities fails and email those details as notification.

If no activities fail - will be success email if any one or all activities fail - will be error details email.

Set Variable will execute if at least one of the activities fails. Which is fine. But how to make sure I capture Error Message from Expl1 and Expl2 activities? Set Variable has an expression:

@concat(
   activity('ExPl1').Error.Message,
    activity('ExPl2').Error.Message

)

However, if Expl1 fails, then Set Variable is also fails with an error:

The expression for Expl1  cannot be evaluated because property 'Error' cannot be selected.

enter image description here


Solution

  • In your case, if any of the activity gets succeeds, if won't give the error details and that's why it is giving the cannot be evaluated because property 'Error' cannot be selected error.

    In this case, you can use coalesce(exp1,exp2) function which will give the first non-null value. First check whether the value activity('pipeline1').Error is null or not, if not null then get the error details as shown below. For the second pipeline, check whether activity('pipeline2') is null or not, if not null then check for activity('pipeline1').Error is null or not. If it is not null, then get the error details. Here, checking activity('pipeline2') is null or not is required as there are chances that the second pipeline may get skipped.

    @concat(if(not(equals(coalesce(activity('Execute Pipeline1').Error),null)),
       activity('Execute Pipeline1').Error.Message,''),'',if(and(not(equals(coalesce(activity('Execute Pipeline2')),null)),not(equals(coalesce(activity('Execute Pipeline2').Error),null))),
       activity('Execute Pipeline2').Error.Message,''))
    

    Pipeline design:

    enter image description here

    If the pipeline1 fails, it will skip the next pipeline and it will store the error details in the variable as shown below.

    enter image description here

    If pipeline1 succeeds and pipeline2 gets fail, then it will store the second pipeline error details.

    enter image description here

    Result when both pipelines are success: If both pipelines are succeeds, it will execute the wait activity.

    enter image description here

    You can send this variable to your mail and use as per your requirement.