I have a pipeline which should pull from another repository:
pool:
vmImage: ubuntu-latest
resources:
repositories:
- repository: upstream
type: git
name: foo/bar
ref: refs/heads/master
endpoint: upstream-access
variables:
repourl: $[ resources.repositories.upstream.url ]
steps:
- checkout: self
persistCredentials: true
clean: true
- script: |
git branch -va
git remote add upstream-repo $(repourl) || true
git fetch upstream-repo
# Check if upstream branch exists locally; create it if it doesn’t
if git rev-parse --verify upstream >/dev/null 2>&1; then
git checkout upstream
else
git checkout -b upstream upstream-repo/master
git push origin upstream # Initial push to create the branch on origin
fi
git pull upstream-repo master --rebase
git push origin upstream
displayName: 'Sync upstream branch with $(repourl)'
But this would generate a URL like this:
Job preparation parameters
Variables:
repourl:
Parsing expression: <resources.repositories.upstream.url>
Evaluating: resources['repositories']['upstream']['url']
Result: 'https://dev.azure.com/orgname/840afda9-cc92-42ac-aeb4-fee9f56de5a0/_apis/git/repositories/972c972e-5c39-4420-b779-ec482edb0aa9/'
ContinueOnError: False
TimeoutInMinutes: 60
CancelTimeoutInMinutes: 5
Expand:
MaxConcurrency: 0
...
fatal: repository 'https://dev.azure.com/orgname/840afda9-cc92-42ac-aeb4-fee9f56de5a0/_apis/git/repositories/972c972e-5c39-4420-b779-ec482edb0aa9/' not found
fatal: 'upstream-repo/master' is not a commit and a branch 'upstream' cannot be created from it
error: src refspec upstream does not match any
...
However, if I just remove the endpoint
from resources.repositories.upstream
, the URL would be correctly formed:
Job preparation parameters
Variables:
repourl:
Parsing expression: <resources.repositories.upstream.url>
Evaluating: resources['repositories']['upstream']['url']
Result: 'https://orgname@dev.azure.com/orgname/prjname/_git/reponame'
ContinueOnError: False
TimeoutInMinutes: 60
CancelTimeoutInMinutes: 5
Expand:
MaxConcurrency: 0
fatal: could not read Password for 'https://orgname@dev.azure.com': terminal prompts disabled
fatal: 'upstream-repo/master' is not a commit and a branch 'upstream' cannot be created from it
error: src refspec upstream does not match any
Questions:
Please note that the error fatal: could not read Password for 'https://orgname@dev.azure.com
in your pipeline was NOT a permission error.
Your pipeline only checked out the self
repo without checkout: upstream
. Since persistCredentials
was set as true
, the checkout:self
step was running and added the $(System.AccessToken)
in the .git/config
file for your downstream git commands
to use. But it was only stored there for authenticating access to your self
repo.
In the script, it simply added remote
url to your upstream
repo, but didn't tell the command to use a password
.
With those being said, you may add git config http.$(repourl).extraheader "AUTHORIZATION: bearer $(System.AccessToken)"
before fetch. Here is a sample YAML pipeline.
pool: # Default # Run the agent job on a self-hosted agent to check git config in the pipeline working directory.
vmImage: ubuntu-latest
resources:
repositories:
- repository: upstream
type: git
name: foo/bar
ref: refs/heads/master
# endpoint: upstream-access
variables:
repourl: $[ resources.repositories.upstream.url ]
steps:
- checkout: self
persistCredentials: true
- script: |
echo "repourl - $(repourl)"
git branch -va
git remote add upstream-repo $(repourl) || true
git config http.$(repourl).extraheader "AUTHORIZATION: bearer $(System.AccessToken)"
git fetch upstream-repo
sleep 120 # Pause the agent job to check the git config file on the self-hosted agent
Adding steps for your reference to check the required pipeline service account permissions.
Navigate to your Organization Settings and Project Settings -> Pipelines -> Settings -> Disable Limit job authorization scope to current project for non-release pipelines; this option will have the build pipelines in this project to run as the pipeline service account Project Collection Build Service (<YourOrgName>) in order to access the repos outside this project.
Check the Project Settings of foo -> Repos -> Repositories -> In the Security tab of repo bar
-> Set the permission to Contribute as Allow for the pipeline service account Project Collection Build Service (<YourOrgName>); set other permissions for your pipeline service account based on the requirements of your git commands.
The endpoint that your upstream
repo resource is referring to is likely the name of an Azure Repos service connection, which is typically used to connect to another Azure DevOps organization using basic or token-based authentication.
When creating an Azure Repos service connection, you provide the organization URL along with token-based authentication.
When running a pipeline that defines a repository resource with an endpoint, it authenticates using the token and then searches in the connected organization (via url
) to find the repository (bar
) from the project (foo
).
In your scenario, since the pipeline is in the same Azure DevOps organization as the repository but in a different project, you don't need to authenticate using an endpoint. The pipeline's service account should already have the necessary permissions to access the repository.
For more information, refer to the following documents:
Check out multiple repositories in your pipeline