azure-devopsazure-pipelinesazure-pipelines-yaml

Azure pipelines filter "each" loop results with an "if"


We have a setup where we deploy libraries if they selected on pipeline run, like below:

parameters:
  - name: librariesToBePublished
    default:
      - name: my-awesome-package-1
        dependsOn: []
      - name: my-awesome-package-2
        dependsOn:
          - my-awesome-package-1
      - name: my-awesome-package-3
        dependsOn:
          - my-awesome-package-1
          - my-awesome-package-2
    type: object

  - name: my-awesome-package-1
    default: false
    type: boolean

  - name: my-awesome-package-2
    default: false
    type: boolean

  - name: my-awesome-package-3
    default: false
    type: boolean

Then we have a job with dependency setup with an each loop, like below:

    jobs:
      - ${{ each package in parameters. librariesToBePublished }}:
        - job: Publish_package_${{ replace(package.name, '-', '_') }}
          displayName: "Publish ${{ package.name }}"
          dependsOn:
            - ${{ each dep in package.dependsOn }}:
              - Publish_package_${{ replace(dep, '-', '_') }}
          condition: |
            and(
              succeeded(),
              eq('${{parameters[package.name]}}', true)
            )

If you check all three boxes, everything works as expected:

enter image description here

The problem is, if you would like to only deploy my-awesome-package-2 and not select its dependency my-awesome-package-1, it is not running since it has a dependsOn with Publish_package_my_awesome_package_1 combined with succeeded(). The first package is not selected so the dependency is skipped, not succeeding:

enter image description here

I would like to add something like this, if possible but no luck so far:

Option 1: Conditionally add dependencies

jobs:
      - ${{ each package in parameters. librariesToBePublished }}:
        - job: Publish_package_${{ replace(package.name, '-', '_') }}
          displayName: "Publish ${{ package.name }}"
          dependsOn:
            - ${{ each dep in package.dependsOn }}:
              - ${{ if eq('${{parameters[dep]}}', true) }}:
                - Publish_package_${{ replace(dep, '-', '_') }}
          condition: |
            and(
              succeeded(),
              eq('${{parameters[package.name]}}', true)
            )

Option 2: extend succeeded

jobs:
      - ${{ each package in parameters. librariesToBePublished }}:
        - job: Publish_package_${{ replace(package.name, '-', '_') }}
          displayName: "Publish ${{ package.name }}"
          dependsOn:
            - ${{ each dep in package.dependsOn }}:
              - Publish_package_${{ replace(dep, '-', '_') }}
          condition: |
            and(
              or(succeeded(), skipped()),
              eq('${{parameters[package.name]}}', true)
            )

Since there is no skipped() check, our only option to put or(succeeded(), eq('${{parameters[dep]}}', false)) there but the condition: part is out of the each loop.


Solution

  • You can simplify your code and get rid of the boolean parameters by using stages instead of jobs.

    Example:

    parameters:
      - name: libraries
        displayName: 'Libraries to be published'
        type: object
        default:
          - name: my-awesome-package-1
            stageName: p1
            exitCode: 1  # 0=success, other values=failure
            dependsOn: []
          - name: my-awesome-package-2
            stageName: p2
            exitCode: 1  # 0=success, other values=failure
            dependsOn:
              - p1
          - name: my-awesome-package-3
            stageName: p3
            exitCode: 0  # 0=success, other values=failure
            dependsOn:
              - p1
              - p2
    
    stages:
      - ${{ each package in parameters.libraries }}:
        - stage: ${{ package.stageName }}
          displayName: 'Publish ${{ package.name }}'
          dependsOn: ${{ package.dependsOn }}
          jobs:
            - job: publish
              displayName: 'Publish ${{ package.name }}'
              steps:
                - checkout: none
                - script: |
                    echo "Publishing ${{ package.name }}..."
                    exit ${{ package.exitCode }}
                  displayName: 'Publish'
    

    Queuing a new build:

    New build

    Choosing the stages:

    Choosing the stages

    Finally, when running the pipeline Azure DevOps will take care of the dependencies between stages for you:

    Pipeline stages