azure-devopsazure-pipelinesazure-yaml-pipelines

Azure DevOps multistage pipeline YAML: how to checkout multiple repos?


My Azure DevOps pipeline uses yaml templates from two different repos, which is configured as following.

  1. There is an application repository which has deployable application and one yaml file - the pipeline's "root" template
  2. Templates Repository. Root template calls other templates and stages from the Templates repository. Then templates from this repository call other templates and scripts (from the same repository)

Templates repository is referenced as a resource in the root template. I didn't find a way to checkout the templates repo just once and then use templates and scripts across all pipeline stages. For now I have to clone the templates repo manually in each stage where I need to use additional templates or scripts. At the end of each stage Azure Devops clears the cloned repo. Is there a simple way to checkout templates repo just once, or somehow else reference its resources from sub-stages?


Solution

  • I'm not sure since you didn't show your YAML file, but did you use checkout step:

    resources:
      repositories:
        - repository: devops
          type: github
          name: kmadof/devops-templates
          endpoint: kmadof
    
    steps:
    - checkout: self
    - checkout: devops
    - script: |
        echo $(Build.SourcesDirectory)
        ls $(Build.SourcesDirectory) *
    - template: templates/template.yaml@devops
      parameters:
        repo: devops-templates
    

    Above script checout two repos. In devops-templates I have template which is used in main build yaml file (located in self repo).

    Please take a look also here

    EDIT

    I work a bit with this and tried few things. Let me describe first relation between files:

    And you don't have to actually checkout template repo because at running all templates are fecthed and build plan is created. You only need to checkout template repo if you are going to run some scripts (for instance powershell scripts). Here you have my yaml files:

    build.yaml

    resources:
      repositories:
        - repository: devops
          type: github
          name: kmadof/devops-templates
          endpoint: kmadof
    
    stages:
    - template: templates/start.yaml@devops
      parameters:
        repo: devops-templates
        buildSteps:
          - checkout: self
          - checkout: devops
          - bash: echo Test #Passes
            displayName: succeed
          - bash: echo "Test"
            displayName: succeed
    

    start.yaml

    # File: start.yml
    parameters:
    - name: repo  # defaults for any parameters that aren't specified
      default: ''
    - name: buildSteps # the name of the parameter is buildSteps
      type: stepList # data type is StepList
      default: [] # default value of buildSteps
    stages:
    - stage: secure_buildstage
      pool: Hosted VS2017
      jobs:
      - template: process.yaml
        parameters:
          pool:   # this parameter is called `pool`
            vmImage: ubuntu-latest  # and it's a mapping rather than a string
      - job: secure_buildjob
        steps:
        - script: echo This happens before code 
          displayName: 'Base: Pre-build'
        - script: echo Building
          displayName: 'Base: Build'
    
        - ${{ each step in parameters.buildSteps }}:
          - ${{ each pair in step }}:
              ${{ pair.key }}: ${{ pair.value }}     
    
        - script: echo This happens after code
          displayName: 'Base: Signing'
    

    process.yaml

    parameters:
    - name: 'pool'
      type: object
      default: {}
    
    jobs:
    - job: build
      pool: ${{ parameters.pool }}
      steps:
      - template: another-template.yaml
        parameters:
          repo: devops-templates
    

    another-template.yaml

    parameters:
    - name: repo  # defaults for any parameters that aren't specified
      default: ''
    
    steps:
      - pwsh: Write-Host 'Hello form another template'
    

    Please take a look here: enter image description here

    Build job uses template from devops-template repo but I don't checkout repo in this job.

    You may wonder why we can't have one checkout per build. And this is because each job can run a different agent.

    enter image description here

    Here you have few links:

    Last remark, you really need to checkout repo with templates when you template calling a file from that repo. For instance:

    steps:
      - task: PowerShell@2
        inputs:
          filePath: /scripts/myscript.ps1