azure-devops azure-pipelines azure-pipelines-yaml

How do I identify the triggering pipeline in the download step when downloading pipeline artifacts in Azure DevOps on-prem?

Clarification:

There are many questions/answers on stackoverflow that talk about using the

DownloadPipelineArtifact@2

and

DownloadBuildArtifacts@1

tasks but they all assume that you already have access to the project name and/or project id of the project that you're downloading artifacts from. My question is specifically how do I get one of those tasks to recognize the project name when I'm using the name of an external project that is triggering my pipeline? For example, I tried this but the project and pipeline parameters don't pick up the values of the variables. I'm sure it's user error in the syntax somewhere, but I'm just not getting it.

- task: DownloadBuildArtifactsV2@0
  inputs:
    buildType: 'specific'
    project: '$(resources.pipeline.$(Resources.TriggeringAlias).projectID'
    pipeline: '$(resources.pipeline.$(Resources.TriggeringAlias).pipelineName)'
    buildVersionToDownload: 'latest'
    downloadType: 'single'
    downloadPath: '$(System.ArtifactsDirectory)'

Original Question:

I've been working with Azure Devops pipelines for a couple weeks now and I've learned a lot, but still feel like I have a long way to go so any help would be greatly appreciated. I'm setting up a pipeline whose only job is to run some tests on a .jar file, generate an output, and drop that output on a file share for other teams to use. There are many Azure DevOps projects in my organization that need this functionality, so I'm setting up my pipeline so that it can be triggered by any of those external projects via pipeline resource triggers like this:

pipelines:
 - pipeline: external-pipeline
   project: external
   source: external-repo
   trigger: 
    enabled: true
    branches:
     include:
       - main
     exclude:
       - test

I can get everything working if I use the "external-pipeline" alias directly to download the .jar file as a pipeline artifact from the triggering pipeline like this:

steps:
#Get the artifacts from the pipeline that triggered this run
- download: external-pipeline
  displayName: Get Artifacts
  enabled: true

However, I need the triggering pipeline alias to be dynamically determined based on which of the many external projects triggers my pipeline. This seems like it would be straightforward since there's a system variable that holds the value of the triggering pipeline alias called Resources.triggeringAlias, but it doesn't seem to work in the download step.

I tried using different variable syntax like this:

steps:
#Get the artifacts from the pipeline that triggered this run
- download: $[ Resources.triggeringAlias ]
  displayName: Get Artifacts
  enabled: true

and this

steps:
#Get the artifacts from the pipeline that triggered this run
- download: $(Resources.triggeringAlias)
  displayName: Get Artifacts
  enabled: true

I also tried assigning Resources.triggeringAlias to another variable first:

variables:
  triggeringAlias: $(Resources.TriggeringAlias)

steps:
#Get the artifacts from the pipeline that triggered this run
- download: $(triggeringAlias)
  displayName: Get Artifacts
  enabled: true

In all cases the pipeline refuses to even start and throws this error:

Download Pipeline Artifacts shortcut: cannot resolve source name

It seems like there should be a simple way to reference the triggering pipeline in the download step so my pipeline can use the published artifacts, but I can't figure out which of the many identifiers and variable scoping methods I need to use. I've found answers to do this within a single Azure DevOps project where you know the specific project name in advance, but has anyone done this dynamically between multiple projects? Thanks!

Working YAML

trigger: none

pool:
  name: default
 
resources:

  pipelines:
  - pipeline: Pipeline1
    project: Test-Sandbox
    source: repo1
    trigger: 
     enabled: true
     branches:
      include:
        - main
      exclude:
        - test
  - pipeline: Pipeline2
    project: Test-Sandbox
    source: repo2
    trigger:
     branches:
       include:
         - main
       exclude:
         - test
  - pipeline: Pipeline3
    project: Test-Sandbox
    source: repo3
    trigger: 
     branches:
      include:
        - main

jobs:
- deployment: Deploy
  environment: 'myEnvironment'
  displayName: The deployment job
  variables:
    system.debug: true
    triggeringAlias: $[upper(variables['resources.triggeringAlias'])]
  workspace:
    clean: all
  strategy:
    runOnce:
      deploy:
        steps:
        - ${{ if eq(variables['Build.Reason'], 'ResourceTrigger') }}:
          - download: none
          - pwsh: |
              echo "Triggering alias: $(triggeringAlias)"
              echo "Project name: $env:RESOURCES_PIPELINE_$(triggeringAlias)_PROJECTNAME"
              echo "Triggering pipeline definition id: $env:RESOURCES_PIPELINE_$(triggeringAlias)_PIPELINEID"
              echo "##vso[task.setvariable variable=project;]$env:RESOURCES_PIPELINE_$(triggeringAlias)_PROJECTNAME"
              echo "##vso[task.setvariable variable=definition;]$env:RESOURCES_PIPELINE_$(triggeringAlias)_PIPELINEID"
          - task: DownloadBuildArtifactsV2@0
            inputs:
              buildType: 'specific'
              project: '$(project)'
              pipeline: '$(definition)'
              buildVersionToDownload: 'latest'
              downloadType: 'specific'
              downloadPath: '$(System.ArtifactsDirectory)'

Solution

Upon checking the requirements to dynamically download the artifacts published by the triggering pipeline resource only, I managed to use a script to get the expected projectName and pipelineID of the triggering pipeline resource and output them as variables for downstream DownloadPipelineArtifact@2 task. I leveraged the environment variables as suggested in this document for troubleshooting.

trigger: none

pool:
  vmImage: ubuntu-latest

resources:
  pipelines:
  - pipeline: Pipeline1
    source: ProjectA-CI1
    project: ProjectA
    trigger: true
  - pipeline: Pipeline2
    source: ProjectA-CI2
    project: ProjectA
    trigger: true

jobs:
- deployment: Deploy
  environment: E-Test
  variables:
    system.debug: true
    triggeringAlias: $[upper(variables['resources.triggeringAlias'])]

  strategy:
    runOnce:
      deploy:
        steps:
        - ${{ if eq(variables['Build.Reason'], 'ResourceTrigger') }}:
          - download: none
          - pwsh: |
              env
              echo "Triggering alias: $(triggeringAlias)"
              echo "Project name: $env:RESOURCES_PIPELINE_$(triggeringAlias)_PROJECTNAME"
              echo "Triggering pipeline definition id: $env:RESOURCES_PIPELINE_$(triggeringAlias)_PIPELINEID"
              echo "##vso[task.setvariable variable=project;]$env:RESOURCES_PIPELINE_$(triggeringAlias)_PROJECTNAME"
              echo "##vso[task.setvariable variable=definition;]$env:RESOURCES_PIPELINE_$(triggeringAlias)_PIPELINEID"
          - task: DownloadPipelineArtifact@2
            inputs:
              buildType: 'specific'
              project: '$(project)'
              definition: '$(definition)'
              buildVersionToDownload: 'latest'
              targetPath: '$(Pipeline.Workspace)'

You can use a bash script instead to generate those variables instead, if you prefer.

          - bash: |
              env
              echo "Triggering alias: $(triggeringAlias)"
              echo "Project name: $RESOURCES_PIPELINE_$(triggeringAlias)_PROJECTNAME"
              echo "Triggering pipeline definition id: $RESOURCES_PIPELINE_$(triggeringAlias)_PIPELINEID"
              echo "##vso[task.setvariable variable=project;]$RESOURCES_PIPELINE_$(triggeringAlias)_PROJECTNAME"
              echo "##vso[task.setvariable variable=definition;]$RESOURCES_PIPELINE_$(triggeringAlias)_PIPELINEID"