azure-devopsazure-data-factory

Parameterize Azure Data Factory trigger start date

I am trying to deploy ADF to multiple stage at different time. My Pipeline has Tumbling Window Trigger and I would like to parameterize the trigger startTime so that I can set the start time before deploying to avoid backfilling trigger runs.

This is the ARM template I have for the trigger portion. How do I edit this to parameterize the trigger start time? Thank you.

"Microsoft.DataFactory/factories/triggers": {
    "properties": {
        "pipelines": [
            {
                "parameters": {
                    "*": "="
                }
            },
            "pipelineReference.referenceName"
        ],
        "pipeline": {
            "parameters": {
                "*": "="
            }
        },
        "typeProperties": {
            "scope": "="
        }
    }
}
Solution

  • Referring this document Use custom parameters with the Resource Manager template and this Answer, I can complete it with following ARM template.

    {
    "Microsoft.DataFactory/factories/triggers": {
        "properties": {
            "typeProperties": {
                "startTime": "=:-startTime:string"
            }
        }
    }
}

    Steps:

    enter image description here enter image description here

    It is also in the arm-template-parameters-definition.json file:

    enter image description here Result:

    enter image description here

    Update:

    ##[error]TumblingWindowTriggerStartTimeUpdateNotAllowed: Start time cannot be updated for Tumbling Window Trigger.

    To avoid the error, we can add an Azure PowerShell task to stop and delete the trigger before the ARM template deployment task. After deployment, we can also add an Azure PowerShell task to start the trigger.

    Sample:

    enter image description here

    Azure PowerShell task to stop and delete the trigger:

    $triggersADF = Get-AzDataFactoryV2Trigger -DataFactoryName $DataFactoryName -ResourceGroupName $ResourceGroupName
$triggersADF | ForEach-Object { Stop-AzDataFactoryV2Trigger -ResourceGroupName $ResourceGroupName -DataFactoryName $DataFactoryName -Name $_.name -Force }
$triggersADF | ForEach-Object { Remove-AzDataFactoryV2Trigger -ResourceGroupName $ResourceGroupName -DataFactoryName $DataFactoryName -Name $_.name -Force }

    Azure PowerShell task to start the trigger:

    $triggersADF = Get-AzDataFactoryV2Trigger -DataFactoryName $DataFactoryName -ResourceGroupName $ResourceGroupName
$triggersADF | ForEach-Object { Start-AzDataFactoryV2Trigger -ResourceGroupName $ResourceGroupName -DataFactoryName $DataFactoryName -Name $_.name -Force }

    Related Doc: Updating active triggers