Get template name from variable and run template?

I have an Azure DevOps repository that I want to fill with a number of (pipeline-)templates.

Each template-file represents an Azure Container App Job.

On merge to production I would like the pipeline to identify changed templates and run these templates. The templates will then update/add Container App Jobs in my Container Environment.

If it can't be done this way what way would be better?

For clarification: I have a container ready that I want to start as multiple Azure Container Apps job with different parameters. I am not asking how to create a container in Container Registry.

Update:

Update 2: First pipeline definition:

variables:
  ubuntuversion: 'ubuntu-22.04'
trigger:
  branches:
    include:
      - main 
stages:
- stage: FindNewTemplates
  jobs:
  - job: FindNewTemplates
    pool:
      vmImage: $(ubuntuversion)
    steps:
    - checkout: self
      fetchDepth: 0

    - powershell: |
          $token = "$(system.accesstoken)" 
          $token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))   
          $prefix_url = "$(System.CollectionUri)"
          $project = "$(System.TeamProject)"
          $buildId = $(Build.BuildId)
          $url = "${prefix_url}${project}/_apis/build/builds/${buildId}/changes?includeSourceChange=true&api-version=7.0"
          $response=Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"}
          $changesCount = $response.count
          Write-Host "The count of commits associated with this build is $changesCount."


          $modifiedFiles= git diff HEAD HEAD~$changesCount --name-only
          echo $modifiedFiles 
          $modifiedFiles1 = @()
          foreach ($file in $modifiedFiles)
          {
            if ($file -like 'acaj-definitions/*tes*.yaml')
            {
              echo $file
              $modifiedFiles1= -join("$modifiedFiles1", ",", "$file")
            }
          }
          $modifiedFiles1 = $modifiedFiles1.TrimStart(",")
          echo $modifiedFiles1

          $url="${prefix_url}${project}/_apis/pipelines/14/runs?api-version=5.1-preview"
          echo $url


          $JSON = "
          {

            `"resources`": {
              `"repositories`": {
                `"self`": {
                  `"ref`": `"refs/heads/main`"
                }
              }
            },
            `"templateParameters`": {
              `"files`":`"[$modifiedFiles1]`"
            },



          }"


          $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"} -Method Post -Body $JSON -ContentType application/json
          echo $response

Second pipeline definition:

parameters:
- name: files
  type: object
variables:
  ubuntuversion: 'ubuntu-22.04'
  acaj_memory: 4
trigger: none

jobs:
- ${{ each file in parameters.files }}:
  #- template: ${{file}}
  #  parameters:
  #    acaj_memory: $(acaj_memory)
  - script: echo ${{file}}

Update 3: ADO output

Description  : Run a PowerShell script on Linux, macOS, or Windows
Version      : 2.239.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
==============================================================================
Generating script.
========================== Starting Command Output ===========================
/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/home/vsts/work/_temp/b5f286c2-1c7d-41b0-b89a-c064e9581e64.ps1'
The count of commits associated with this build is 19.
.azure-pipelines/01_identify_new_templates.yaml
.azure-pipelines/01_identify_new_templatesmine.yaml
.azure-pipelines/02_run_new_templates.yaml
acaj-definitions/intralinks/testtemplate.yaml
acaj-definitions/intralinks/testtemplate.yaml
acaj-definitions/intralinks/testtemplate.yaml
https://dev.azure.com/companyname/projectname/_apis/pipelines/14/runs?api-version=5.1-preview

{

  "resources": {
    "repositories": {
      "self": {
        "ref": "refs/heads/main"
      }
    }
  },
  "templateParameters": {
    "files":"[acaj-definitions/intralinks/testtemplate.yaml]"
  }
}
Invoke-RestMethod: /home/vsts/work/_temp/b5f286c2-1c7d-41b0-b89a-c064e9581e64.ps1:49
Line |
  49 |  $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "B …
     |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     |  {   "$id": "1",   "innerException": null,   "message": "TF400898: An
     | Internal Error Occurred. Activity Id:
     | 647e4983-f3df-41eb-84e0-5d3b0f8bafaf.",   "typeName":
     | "System.NullReferenceException, mscorlib",   "typeKey":
     | "NullReferenceException",   "errorCode": 0,   "eventId": 0 }

##[error]PowerShell exited with code '1'.
Finishing: PowerShell

I am afraid that there is no out-of-box variable can contain the changed files information.

would like the pipeline to identify changed templates and run these templates

Based on your description, you are using the template in Pipeline to run the jobs.

We need to running the script to collect the changed files and send to the pipeline to confirm which templates to run.

Since the YAML templates will be expand at compile time and script run will execute at runtime, we need to create two Pipelines to achieve the requirement.

Here is an example:

Pipeline One: Used Rest API and Git command to collect the changed files and send the result to Pipeline Two.

Pipeline Two: Use Object type parameters and each expressions to loop the changed files to run the template.

parameters:
- name: files
  type: object


jobs:
- ${{ each file in parameters.files }}:
  - template: ${{file}}

Result:

Pipeline One:

Pipeline Two:

Update:

To let the Rest API to trigger the Pipeline successfully, you need to grant the Queue Builds permissions to two service accounts: Project Collection Build Service Account(OrganizationName) and ProjectName Build Service(Organization).

Click the Manage Security Option and set the permissions.

For example:

Then you can use the following updated Pipeline One sample:

steps:
- checkout: self
  fetchDepth: 0

- powershell: |
      $token = "$(system.accesstoken)" 
      $token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))   
      $prefix_url = "$(System.CollectionUri)"
      $project = "$(System.TeamProject)"
      $buildId = $(Build.BuildId)
      $url = "${prefix_url}${project}/_apis/build/builds/${buildId}/changes?includeSourceChange=true&api-version=7.0"
      $response=Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"}
      $changesCount = $response.count
      Write-Host "The count of commits associated with this build is $changesCount."


      $modifiedFiles= git diff HEAD HEAD~$changesCount --name-only
      echo $modifiedFiles 
      $modifiedFiles1 = @()
      foreach ($file in $modifiedFiles)
      {
        if ($file -like 'tes*.yml')
        {
          echo $file
          $modifiedFiles1= -join("$modifiedFiles1", ",", "$file")
         }
      }
      $modifiedFiles1 = $modifiedFiles1.TrimStart(",")
      echo $modifiedFiles1

      $url="${prefix_url}${project}/_apis/pipelines/{Pipeline Two ID}/runs?api-version=5.1-preview"
      echo $url


      $JSON = "
      {

        `"resources`": {
          `"repositories`": {
            `"self`": {
              `"ref`": `"refs/heads/main`"
            }
          }
        },
        `"templateParameters`": {
          `"files`":`"[$modifiedFiles1]`"
        },



      }"


      $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"} -Method Post -Body $JSON -ContentType application/json
       echo $response

Or you can create a full scope PAT and use it in the $token field.

When the Rest Api runs successfully, it will return json object value.

For example:

Update2:

I have checked the YAML code. The cause of the issue is that the Second Pipeline YAML format has issue.

You can use the following format:

parameters:
- name: files
  type: object




jobs:
- ${{ each file in parameters.files }}:
  - job: 
    steps:
    - script: echo ${{file}}