I need to retrieve a list of pipelines in Azure Data Factory (ADF) and are associated with a specific branch in a specific repository within Azure DevOps.
Here’s what I’ve tried so far:
Azure DevOps REST API: I used the following API call in powershell to retrieve details from the branch in the repository, as the pipeline details are stored inside ADF/pipelines, but this is not giving me the expected result:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/items?path={path}&scopePath={scopePath}&recursionLevel={recursionLevel}&includeContentMetadata={includeContentMetadata}&latestProcessedChange={latestProcessedChange}&download={download}&$format={$format}&versionDescriptor.versionOptions={versionDescriptor.versionOptions}&versionDescriptor.version={versionDescriptor.version}&versionDescriptor.versionType={versionDescriptor.versionType}&includeContent={includeContent}&resolveLfs={resolveLfs}&api-version=5.0
I got this response from the above request, but my expectation was to retrieve the list of adf pipelines triggered from a specific branch.
count : 1
value : {@{objectId=xxxxxx; gitObjectType=tree;
commitId=xxxxxx; path=/ADF/pipeline; isFolder=True; url=https://xxxx.xxxx.xxx
/xxxx/xxxxxxxx/_apis/git/repositories/xxxxxx/it
ems?path=%2FADF%2Fpipeline&versionType=Branch&version=Main&versionOptions=None}}
PowerShell Module: I used Get-AzDataFactoryV2Pipeline to list pipelines in the Data Factory:
Get-AzDataFactoryV2Pipeline -ResourceGroupName <ResourceGroupName> -DataFactoryName <DataFactoryName>
But, this cmdlet returns pipelines in the live mode in the Data Factory environment, not the Devops pipelines.
As per the json files screenshot, they are ADF pipelines actually. In DevOps git repo, it will have the name in the json file as below:
If you want to get the ADF pipeline name
from DevOps git
for specific branc, eg: Develop, you can checkout the branch, and get the name value from the json.
Yaml sample below:
pool:
vmImage: Windows-latest
steps:
- checkout: self
persistCredentials: true # keep credential for git command latter.
- powershell: |
git fetch --all
git checkout Develop # for Develop branch
# Get all JSON files in the "ADF/pipeline" folder
if (Test-Path -Path "ADF/pipeline") {
$jsonFiles = Get-ChildItem -Path "ADF/pipeline" -Filter *.json -Recurse
foreach ($file in $jsonFiles) {
$jsonContent = Get-Content -Path "ADF/pipeline/$file" -Raw | ConvertFrom-Json
$pipelineName = $jsonContent.name
Write-Output "Branch: Develop, PipelineName: $pipelineName" # output branch name and pipeline name.
}
} else {
Write-Output "Path 'ADF/pipeline' not found, skipping..."
}
It can get the pipeline name: