azure-devopsazure-pipelinesazure-pipelines-yamlazure-rm-templateazure-pipelines-tasks

Azure yaml pipeline group variables not seen by task in a template file


I have a pipeline stage that is using a template as follows:

# Deploy to AKS
- stage: DeployTEST
  displayName: Test env for my-app
  condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
  variables:
  - group: 'my-app-var-group-test'
  - group: 'package-variables'
  - template: templates/shared-template-vars.yml@templates
  jobs:
  - deployment: TestDeployment
    displayName: Deploy to AKS - Test
    pool:
      vmImage: $(vmImageName)
    environment: env-test
    strategy:
      runOnce:
        deploy:
          steps:
          - template: ./aks/deployment-steps.yml

...and the content of the template deployment-steps.yml is:

steps:
- script: |
    echo AzureSubscription: '$(azureSubscription)'
    echo KubernetesServiceConnection: '$(kubernetesServiceConnection)' # this is working

- task: KubernetesManifest@0
  displayName: Create imagePullSecret
  inputs:
    action: createSecret
    secretName: $(imagePullSecret)
    dockerRegistryEndpoint: $(dockerRegistryServiceConnection)
    kubernetesServiceConnection: $(kubernetesServiceConnection) # this is causing an error

I get an error like this:

There was a resource authorization issue: "The pipeline is not valid. Job TestDeployment: Step input kubernetesServiceConnection references service connection $(kubernetesServiceConnection) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz."

and like this when I try to select individual stages prior manual pipeline run:

Encountered error(s) while parsing pipeline YAML: Job TestDeployment: Step input kubernetesServiceConnection references service connection $(kubernetesServiceConnection) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.

The errors above are misleading, because it is not an authorization issue:

NOTE: The variable kubernetesServiceConnection is defined in the my-app-var-group-test variable group & when I comment out the KubernetesManifest task, the value of the $(kubernetesServiceConnection) variable is properly printed to the pipeline console output without any issues and the pipeline runs successfully!?

I know I could use parameters to pass values into the template, but this setup is already used by all other pipelines (variable group vars are used/references in templates) and this issue appeared on a newly created pipeline. I have used file comparison to compare the yaml of a working pipeline and this one and failed to spot anything...

I might be missing something obvious, but I spent hours on this failing to resolve the error...


Solution

  • The answer is here: https://blog.richardfennell.net/posts/using-azure-service-connection-names-that-are-stored-in-variables-group-ado-pipeline/. The summary version is that due to quirks in how ADO parses YAML, in order for a service connection variable defined in a variable group at the stage level to pass initial parsing, you need to have that variable defined at the top (global) level and then the stage will override its value as expected. If you have different variable groups for different stages, just import one at the top level of your pipeline and that'll fix it.