The company I'm at, has many Azure Devops YAML pipelines, spread across multiple Git repositories. These multi-stage pipelines are used to build .NET applications and to deploy them to four different stages (dev/test/accp/prod).
To reduce the amount of files being copied from one pipeline to the next, I've created a separate Git repo that contains reusable templates that the various pipelines can reference.
One of the templates is performing some cross-cutting concerns when deploying applications, and I want to extend this template to inject a preDeploy step for each jobs.deployment. To illustrate the problem, I've constructed a very contrived example consisting of 3 files. The first is a template that contains a deployment job; based on the parameter includePredeployStep
a hard-coded preDeploy step will be included:
# File: deployJob.yaml
parameters:
- name: deploymentName
type: string
- name: environment
type: string
- name: includePredeployStep
type: boolean
jobs:
- deployment: ${{parameters.deploymentName}}
environment:
name: ${{parameters.environment}}
resourceType: virtualMachine
strategy:
runOnce:
deploy:
steps:
- powershell: |
Write-Host "Inside job deploy step"
${{if eq(parameters.includePredeployStep, 'true')}}:
preDeploy:
steps:
- powershell: |
Write-Host "Inside job preDeploy step"
To inject a preDeploy step, I've created another template that is heavily inspired on Microsoft's example of how to use each, this is the file that I cannot get to work:
# File: execute-deployment-jobs.yaml
parameters:
- name: deployJobs
type: jobList
jobs:
- ${{ each job in parameters.deployJobs }}:
- ${{ each deployJobProperty in job }}:
${{ if and(ne(deployJobProperty.key, 'dependsOn'), ne(deployJobProperty.key,'strategy')) }}:
${{ deployJobProperty.key }}: ${{ deployJobProperty.value }}
${{ if eq(deployJobProperty.key, 'strategy')}}:
strategy:
${{if deployJobProperty.value.runOnce}}:
runOnce:
${{each runOnceProperty in deployJobProperty.value.runOnce}}:
${{if ne(runOnceProperty.key, 'preDeploy')}}:
${{runOnceProperty.key}}: ${{runOnceProperty.value}}
preDeploy:
steps:
- powershell: |
Write-Host "Inside injected preDeploy step"
displayName: Injected preDeploy step
- ${{each preDeployStep in runOnceProperty.predeploy.steps}}:
- ${{preDeployStep}}
Finally, I use file 'execute-deployment-jobs' in my pipeline:
# File: pipeline.yaml
pool:
name: Development
stages:
- stage: Test
jobs:
- template: execute-deployment-jobs.yaml
parameters:
deployJobs:
- template: deployJob.yaml
parameters:
deploymentName: MyApplication
environment: Test
includePredeployStep: false # <--- Value 'false' does not cause a problem
- template: deployJob.yaml
parameters:
deploymentName: MyOtherApplication
environment: Test
includePredeployStep: true # <--- Value 'true' causes the problem
When I validate the template, I get an error saying "'preDeploy' is already defined".
The problem only arises when includePredeployStep
on the very last line in the pipeline is set to true
; if I change the value to false
, the validation of the template succeeds and the compiled template shows the injected preDeploy step.
Any suggestions on how I can modify file 'execute-deployment-jobs.yaml' such that it will work for jobs with or without build-in preDeploy steps?
Btw, much of the code in 'execute-deployment-jobs' must be duplicated for strategy canary
and rolling
. Don't know if there is a quick fix to solve that.
---------- Update 1 ----------
As a clarification; what I want of 'execute-deployment-jobs' is for it to always inject a preDeploy step into any deployment jobs, regardless of whether the deployment job already has a preDeploy section. In the example pipeline, deployment of application 'MyOtherApplication' causes a compile error because it has a job-specific preDeploy section (that prints out "Inside job preDeploy step"). If I remove this job-specific preDeploy section, the template can compile and both deployment jobs have an injected preDeploy section (printing out "Inside injected preDeploy step"). What causes the compile error?
It seens that the attempt to nest preDeploy
steps within another preDeploy
section leads to conflicts.
I tested the issue and modified the execute-deployment-jobs.yaml
. It will check if preDeploy
already exists and merge the steps if it does, or create a new preDeploy
section if it doesn't.
parameters:
- name: deployJobs
type: jobList
jobs:
- ${{ each job in parameters.deployJobs }}:
- ${{ each deployJobProperty in job }}:
${{ if and(ne(deployJobProperty.key, 'dependsOn'), ne(deployJobProperty.key, 'strategy')) }}:
${{ deployJobProperty.key }}: ${{ deployJobProperty.value }}
${{ if eq(deployJobProperty.key, 'strategy') }}:
strategy:
runOnce:
${{ each runOnceProperty in deployJobProperty.value.runOnce }}:
${{ if eq(runOnceProperty.key, 'preDeploy') }}:
preDeploy:
steps:
- powershell: |
Write-Host "Inside injected preDeploy step"
displayName: Injected preDeploy step
- ${{ each preDeployStep in runOnceProperty.value.steps }}:
${{ preDeployStep }}
${{ if ne(runOnceProperty.key, 'preDeploy') }}:
${{ runOnceProperty.key }}: ${{ runOnceProperty.value }}
${{ if not(deployJobProperty.value.runOnce.preDeploy) }}:
preDeploy:
steps:
- powershell: |
Write-Host "Inside injected preDeploy step"
displayName: Injected preDeploy step
Test result: