Access ADO Pipeline Output Variables using the Azure DevOps CLI

I have an ADO pipeline which has a single job in it which sets some variables. I need a way to run this pipeline from the cli with a command like this:

run_id=$(az pipelines run --name "mypipeline" --branch "main" --org https://dev.azure.com/MYORG --project "myproject" --query "id" -o tsv)

Then I need a way to get the output variables from the cli with a command like this:

az pipelines runs show --id $run_id --org https://dev.azure.com/MYORG --project "myproject"

However, any attempt I make at getting a variable included in that output does not show up. Does anyone know if there's a way to get a variable outputted in a way that can be seen with an az pipelines runs show command? I can easily pass variables between jobs, display them in the logs, etc. But I can't make them available to be seen from the cli.

I've tried many options of setting the variables like:

      echo "##vso[task.setvariable variable=TEST1]test1"
      echo "##vso[task.setvariable variable=TEST2;isOutput=true]test2"
      echo "Output Variable: test3"

But none of them are ever included.

Solution

The purpose of "output variables" is to create state that can be reused between jobs or stages within the pipeline. They are not a direct by-product of the pipeline that can be queried, but there are other APIs that can expose that state.

The cli is a wrapper around the Azure DevOps REST API. Specifically, az pipeline runs show calls the Build - GET endpoint. This endpoint gives you the details about which pipeline was run and its current status. The internal state of the pipeline, including the number of stages, jobs and tasks are not included in this endpoint.

The Timeline - GET endpoint can retrieve a flat list of all the stages, checks, jobs and tasks for a given build. Again, here, this is providing you with the status of those elements, not their outputs.

Once you have the Timeline, you can query an endpoint to give you specific details for a record.

The process looks like this:

Call the Timeline - GET endpoint to retrieve the timeline for the build.

https://dev.azure.com/<organization>/<project-id>/_apis/build/builds/<build-id>/timeline?api-version=7.1

This returns the current timeline for the build.
```
{
   "id": "<timeline-id>",
   "records": [
      {
       "id": "<record-id>",
       "parentId": "<record-id ref>",
       "type": "...",
       "name": "...",
       "state": "...",
       "result": "..."
      },
      ...
   ]
}
```
The output variables aren't exposed in this endpoint. Make note of the id.
While there is a Records - Update endpoint to manipulate the record, the GET equivalent appears in the client libraries. The undocumented GET endpoint is: https://dev.azure.com/<organization>/<projectId>/_apis/distributedtask/hubs/build/plans/<planId>/timelines/<timelineId>/records/

Where the planId and timelineId is the id of the timeline obtained in the last step.

The JSON result looks similar to the Timeline - GET, except the output variables are included:
```
{
   "value": [
      ...
      {
         "id": "<id>",
         "type": "Task",
         "name": "...",
         "log": { ... },
         "task": { ... },
         "variables": {
           "MYVARIABLE1": {
              "value": "...."
           },
           "MYVARIABLE2": {
              "value": "..."
           }
         }            

      },
      ...
   ]
}
```

It is possible to invoke these calls using the az devops invoke command.

$projectName = "<name>"
$projectId = "<guid-for-project>"
$buildId = 12345
$taskName = "<my_task_with_output_variables>"

# get the timeline for the build
$timeline = az devops invoke `
               --area build `
               --resource timeline `
               --route-parameters `
                  project=$projectName `
                  buildId=$buildId | ConvertFrom-Json

$task = $timeline.records | 
          where-object { $_.type -eq "Task" -and $_.name -eq $taskName }

$timelineId = $timeline.id
$planId = $timeline.id

# get the timeline record
$record = az devops invoke `
             --area distributedTask `
             --resource records `
             --route-parameters `
                 hubName="build" `
                 planId=$planId `
                 timelineId=$timelineId `
                 scopeIdentifier=$projectId | ConvertFrom-Json

# locate the task that has the output variables
$task = $record.value | 
          where-object { $_.type -eq "Task" -and $_.name -eq $taskName }

# party
write-output $task.variables.MYVARIABLE1.value
write-output $task.variables.MYVARIABLE2.value