Is there a way to make agents pickup pipeline jobs instantaneously?
We have a scenario where the ADO web hooks are just falling short in capabilities. I'm using the web hooks to trigger an Azure Logic App but its getting a bit over complicated with the different actions I'd need to compensate for a lot of the information that the Web hook does not have available (custom fields, dynamic project and organization names, etc)
A web hook trigger to run a build pipeline that has a custom script is unfortunately slow since the time it takes for that task to trigger would already be about 10 seconds on a good day, we'd need something to process some work item data and then trigger some API calls in 2-3 seconds from the time of the event (in this case the event is a work item getting created)
I've not included a lot of details as they're not necessary but the end-to-end process looks like this:
- Work Item Created
- Fetch Details
- If a field contains a specific value
- Make a REST API call to an external endpoint
I'd appreciate any suggestions that including only azure and azure devops products. Company policy prevent me from setting some third party service or application to fill in this gap.
Solution
You may consider using Azure DevOps WebHooks to trigger your workflow.
In addition, you may use WebHook resource to trigger pipeline that uses InvokeRESTAPI@1 task in an agent-less job to call API request to external endpoint. Since the pipeline uses an agent-less job, it will skip the waiting time for the pipeline to pick up an agent from the pool. Here is a simple sample for your reference.
- Create an
Incoming WebHookservice connection the Azure DevOps project settings and take notes of the WebHook NameADOWebHookand Service Connection NameADOWebHookSvcCnn;
- Generate YAML pipeline referencing the .yml file from the default branch of the pipeline; refer the WebHook Name
ADOWebHookand Service Connection NameADOWebHookSvcCnndefined in theIncoming WebHookservice connection duringstep 1;
resources:
webhooks:
- webhook: ADOWebHook
connection: ADOWebHookSvcCnn
jobs:
- job: AgentlessJob
pool: server
condition: ${{ eq( parameters.ADOWebHook.resource.fields['System.Reason'], 'New defect reported') }}
steps:
- task: InvokeRESTAPI@1 # Send custom notification to Microsoft Teams (as the external service)
inputs:
connectionType: 'connectedServiceName'
serviceConnection: 'ADOTeamsIncomingWebHookConnector'
method: 'POST'
body: |
{
"@type": "MessageCard",
"@context": "http://schema.org/extensions",
"themeColor": "0076D7",
"summary": "${{ parameters.ADOWebHook.resource.fields['System.CreatedBy'].displayName }} created a new ${{ parameters.ADOWebHook.resource.fields['System.WorkItemType'] }}",
"sections": [{
"activityTitle": "${{ parameters.ADOWebHook.resource.fields['System.CreatedBy'].displayName }} created a new ${{ parameters.ADOWebHook.resource.fields['System.WorkItemType'] }}",
"activitySubtitle": "On Project ${{ parameters.ADOWebHook.resource.fields['System.TeamProject'] }}",
"activityImage": "https://adaptivecards.io/content/cats/3.png",
"facts": [{
"name": "Assigned to",
"value": "${{ parameters.ADOWebHook.resource.fields['System.CreatedBy'].displayName }}"
}, {
"name": "Created date",
"value": "${{ parameters.ADOWebHook.resource.fields['System.CreatedDate'] }}"
}, {
"name": "Status",
"value": "${{ parameters.ADOWebHook.resource.fields['System.State'] }}"
}],
"markdown": true
}],
"potentialAction": [{
"@type": "ActionCard",
"name": "Add a comment",
"inputs": [{
"@type": "TextInput",
"id": "comment",
"isMultiline": false,
"title": "Add a comment here for this task"
}],
"actions": [{
"@type": "HttpPOST",
"name": "Add comment",
"target": "https://learn.microsoft.com/outlook/actionable-messages"
}]
}, {
"@type": "ActionCard",
"name": "Set due date",
"inputs": [{
"@type": "DateInput",
"id": "dueDate",
"title": "Enter a due date for this task"
}],
"actions": [{
"@type": "HttpPOST",
"name": "Save",
"target": "https://learn.microsoft.com/outlook/actionable-messages"
}]
}, {
"@type": "OpenUri",
"name": "Learn More",
"targets": [{
"os": "default",
"uri": "https://learn.microsoft.com/outlook/actionable-messages"
}]
}, {
"@type": "ActionCard",
"name": "Change status",
"inputs": [{
"@type": "MultichoiceInput",
"id": "list",
"title": "Select a status",
"isMultiSelect": "false",
"choices": [{
"display": "In Progress",
"value": "1"
}, {
"display": "Active",
"value": "2"
}, {
"display": "Closed",
"value": "3"
}]
}],
"actions": [{
"@type": "HttpPOST",
"name": "Save",
"target": "https://learn.microsoft.com/outlook/actionable-messages"
}]
}]
}
waitForCompletion: 'false'
# - job: AgentJob
# dependsOn: []
# steps:
# - script: echo "The reason is ${{ parameters.ADOWebHook.resource.fields['System.Reason'] }}"
- Generate the WebHook URL; replace the
<ADO Organization Name>with your Azure DevOps organization name and<WebHook Name>withADOWebHookfrom our example;
https://dev.azure.com/<ADO Organization Name>/_apis/public/distributedtask/webhooks/<WebHook Name>?api-version=6.0-preview
Create WebHook event of
Work item created; add theWebHookURL fromstep 3into this event to trigger this pipeline;
My sample pipeline will be triggered once a new work item is created; it evaluates the value of field
System.Reasonof the newly created work item and then send API request to the external endpoint (Microsoft Teams connector message); you can customize this request sent to external endpoint with the parameter information from the event payload using the format${{ parameters.<WebhookAlias>.<JSONPath>}};
Here is the Test payload of the triggering event to create a new work item;
Method: POST
URI: https://<dev.azure.com/ADO Organization Name>/_apis/public/distributedtask/webhooks/ADOWebHook?api-version=6.0-preview
HTTP Version: 1.1
Headers:
{
Authorization: Basic ********
Content-Type: application/json; charset=utf-8
}
Content:
{
"subscriptionId": "bb3faa0d-6063-42ef-8820-5d5037161241",
"notificationId": 12,
"id": "d2d46fb1-dba5-403c-9373-427583f19e8c",
"eventType": "workitem.created",
"publisherId": "tfs",
"message": {
"text": "Bug #5 (Some great new idea!) created by Jamal Hartnett.\r\n(http://fabrikam-fiber-inc.visualstudio.com/web/wi.aspx?pcguid=74e918bf-3376-436d-bd20-8e8c1287f465&id=5)",
"html": "<a href=\"http://fabrikam-fiber-inc.visualstudio.com/web/wi.aspx?pcguid=74e918bf-3376-436d-bd20-8e8c1287f465&id=5\">Bug #5</a> (Some great new idea!) created by Jamal Hartnett.",
"markdown": "[Bug #5](http://fabrikam-fiber-inc.visualstudio.com/web/wi.aspx?pcguid=74e918bf-3376-436d-bd20-8e8c1287f465&id=5) (Some great new idea!) created by Jamal Hartnett."
},
"detailedMessage": {
"text": "Bug #5 (Some great new idea!) created by Jamal Hartnett.\r\n(http://fabrikam-fiber-inc.visualstudio.com/web/wi.aspx?pcguid=74e918bf-3376-436d-bd20-8e8c1287f465&id=5)\r\n\r\n- Area: FabrikamCloud\r\n- Iteration: FabrikamCloud\\Release 1\\Sprint 1\r\n- State: New\r\n- Assigned to: \r\n- Comment: \r\n- Severity: 3 - Medium\r\n",
"html": "<a href=\"http://fabrikam-fiber-inc.visualstudio.com/web/wi.aspx?pcguid=74e918bf-3376-436d-bd20-8e8c1287f465&id=5\">Bug #5</a> (Some great new idea!) created by Jamal Hartnett.<ul>\r\n<li>Area: FabrikamCloud</li>\r\n<li>Iteration: FabrikamCloud\\Release 1\\Sprint 1</li>\r\n<li>State: New</li>\r\n<li>Assigned to: </li>\r\n<li>Comment: </li>\r\n<li>Severity: 3 - Medium</li></ul>",
"markdown": "[Bug #5](http://fabrikam-fiber-inc.visualstudio.com/web/wi.aspx?pcguid=74e918bf-3376-436d-bd20-8e8c1287f465&id=5) (Some great new idea!) created by Jamal Hartnett.\r\n\r\n* Area: FabrikamCloud\r\n* Iteration: FabrikamCloud\\Release 1\\Sprint 1\r\n* State: New\r\n* Assigned to: \r\n* Comment: \r\n* Severity: 3 - Medium\r\n"
},
"resource": {
"id": 5,
"rev": 1,
"fields": {
"System.AreaPath": "FabrikamCloud",
"System.TeamProject": "FabrikamCloud",
"System.IterationPath": "FabrikamCloud\\Release 1\\Sprint 1",
"System.WorkItemType": "Bug",
"System.State": "New",
"System.Reason": "New defect reported",
"System.CreatedDate": "2014-07-15T17:42:44.663Z",
"System.CreatedBy": {
"displayName": "Jamal Hartnett",
"url": "https://vssps.dev.azure.com/fabrikam/_apis/Identities/e5a5f7f8-6507-4c34-b397-6c4818e002f4",
"_links": {
"avatar": {
"href": "https://dev.azure.com/mseng/_apis/GraphProfile/MemberAvatars/aad.YTkzODFkODYtNTYxYS03ZDdiLWJjM2QtZDUzMjllMjM5OTAz"
}
},
"id": "e5a5f7f8-6507-4c34-b397-6c4818e002f4",
"uniqueName": "Jamal Hartnett",
"imageUrl": "https://dev.azure.com/fabrikam/_api/_common/identityImage?id=e5a5f7f8-6507-4c34-b397-6c4818e002f4",
"descriptor": "ukn.VXkweExUVXRNakV0TWpFME5qYzNNekE0TlMwNU1ETXpOak15T0RVdE56RTVNelEwTnpBM0xURXpPRGswTlRN"
},
"System.ChangedDate": "2014-07-15T17:42:44.663Z",
"System.ChangedBy": {
"displayName": "Jamal Hartnett",
"url": "https://vssps.dev.azure.com/fabrikam/_apis/Identities/e5a5f7f8-6507-4c34-b397-6c4818e002f4",
"_links": {
"avatar": {
"href": "https://dev.azure.com/mseng/_apis/GraphProfile/MemberAvatars/aad.YTkzODFkODYtNTYxYS03ZDdiLWJjM2QtZDUzMjllMjM5OTAz"
}
},
"id": "e5a5f7f8-6507-4c34-b397-6c4818e002f4",
"uniqueName": "Jamal Hartnett",
"imageUrl": "https://dev.azure.com/fabrikam/_api/_common/identityImage?id=e5a5f7f8-6507-4c34-b397-6c4818e002f4",
"descriptor": "ukn.VXkweExUVXRNakV0TWpFME5qYzNNekE0TlMwNU1ETXpOak15T0RVdE56RTVNelEwTnpBM0xURXpPRGswTlRN"
},
"System.Title": "Some great new idea!",
"Microsoft.VSTS.Common.Severity": "3 - Medium",
"WEF_EB329F44FE5F4A94ACB1DA153FDF38BA_Kanban.Column": "New"
},
"_links": {
"self": {
"href": "http://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/workItems/5"
},
"workItemUpdates": {
"href": "http://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/workItems/5/updates"
},
"workItemRevisions": {
"href": "http://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/workItems/5/revisions"
},
"workItemType": {
"href": "http://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/ea830882-2a3c-4095-a53f-972f9a376f6e/workItemTypes/Bug"
},
"fields": {
"href": "http://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/fields"
}
},
"url": "http://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/_apis/wit/workItems/5"
},
"resourceVersion": "1.0",
"resourceContainers": {
"collection": {
"id": "c12d0eb8-e382-443b-9f9c-c52cba5014c2"
},
"account": {
"id": "f844ec47-a9db-4511-8281-8b63f4eaf94e"
},
"project": {
"id": "be9b3917-87e6-42a4-a549-2bc06a7a878f"
}
},
"createdDate": "2024-01-30T03:13:52.0705853Z"
}
Here the output in the external service (a notification card in Microsoft Teams);
- ADO YAML Pipelines | How to get secret variable from a variable group based on the value of another variable
- Azure Devops - Get release definitions by agent pool ID
- Azure Pipelines using YAML for multiple environments (stages) with different variable values but no YAML duplication
- How to specify which version of nuget.exe to use with self-hosted agent in Azure DevOps?
- Login failed for user '<token-identified principal>' while authorizing to SQL Server via Service Principal from AAD group assigned
- Access Azure DevOps Pipeline Artifacts from different organization
- Initiate Azure DevOps Release Pipeline from Logic App and Retrieve Environment Variable Output from the pipeline
- Is it necessary to switch from the Hosted XML to the Inheritance Process Model after a migration
- Why is Azure Pipelines installing the wrong .NET SDK version (according to global.json)
- Azure pipeline does't allow to git push throwing 'GenericContribute' permission is needed
- Azure Devops - How to START one stage before parallel stages?
- Dynamically populate DependsOn in Azure DevOps Yaml job
- Issue with AzurePowerShell@5 task: 'The term 'pwsh.exe' is not recognized as the name of a cmdlet, function, script file, or operable program."
- Azure Devops pipeline, git authentication only works with -c http.extraheader
- How to find the associated pipeline in Azure DevOps from a given yaml file?
- Error Deploying Apps on K8 on-Prem Cluster using Azure Devops CI/CD pipeline
- Git Fetch command not fetching latest changes from Azure DevOps repo
- Post-Job taking a long time in Azure DevOps Pipelines
- How to install private python package from Azure Artifact feed via CLI
- Add environment to Pipeline stuck in "connecting to server"
- Azure DevOps yaml pipeline: Using output variables in conditions does not evaluate correctly
- NuGetCommand@2 in Azure Devops Pipeline is unable to get Azure Artifacts feed
- I am having issues with docker compose task in azure pipeline
- Field Automation Based On Date Field
- Grant Azure Devops ARM Service Connection access to Key Vault
- Azure DevOps Pipelines: Environment Variables Not Available in dotnet test Command
- Terraform source error - Error: Failed to query available provider packages
- Run jobs in `each` expression run in sequence in Azure Devops
- Azure Pipeline not triggered by changes to other than self repository
- Azure cli cant connect due to Invalid client secret provided