gitazurepermissionsazure-pipelinesgit-submodules

How do I enable my Azure pipeline to checkout a submodule using Git?

In our Git project, we have the following defined in .gitmodules

[submodule "DBPkg"]
  path = DBPkg
  url = https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg

When building the Azure Pipeline, in my "Get Sources" pipeline step (I'm using "Azure Repos Git"), I have enabled checking out of submodules

enter image description here

In our project settings, I have also made the following configurations ...

enter image description here

However, when building the project, it dies when attempting to checkout the submodule wiht the following ...

2020-10-16T20:29:44.9912145Z HEAD is now at e06abce Modified docker file to remove unnecessary container name.
2020-10-16T20:29:44.9913415Z ##[command]git submodule sync
2020-10-16T20:29:44.9920288Z ##[command]git -c http.https://MainCo@dev.azure.com.extraheader="AUTHORIZATION: bearer ***" submodule update --init --force
2020-10-16T20:29:44.9923206Z Submodule 'DBPkg' (https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg) registered for path 'DBPkg'
2020-10-16T20:29:44.9923842Z Cloning into '/home/vsts/work/1/s/DBPkg'...
2020-10-16T20:29:44.9924246Z remote: TF401019: The Git repository with name or identifier DBPkg does not exist or you do not have permissions for the operation you are attempting.
2020-10-16T20:29:44.9924850Z fatal: repository 'https://dev.azure.com/MainCo/OurProject/_git/DBPkg/' not found
2020-10-16T20:29:44.9925716Z fatal: clone of 'https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg' into submodule path '/home/vsts/work/1/s/DBPkg' failed
2020-10-16T20:29:44.9926288Z Failed to clone 'DBPkg'. Retry scheduled
2020-10-16T20:29:44.9926718Z Cloning into '/home/vsts/work/1/s/DBPkg'...
2020-10-16T20:29:44.9927132Z remote: TF401019: The Git repository with name or identifier DBPkg does not exist or you do not have permissions for the operation you are attempting.
2020-10-16T20:29:44.9928020Z fatal: repository 'https://dev.azure.com/MainCo/OurProject/_git/DBPkg/' not found
2020-10-16T20:29:44.9928801Z fatal: clone of 'https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg' into submodule path '/home/vsts/work/1/s/DBPkg' failed
2020-10-16T20:29:44.9929412Z Failed to clone 'DBPkg' a second time, aborting

What else do I need to do to give permissions to the pipeline step to checkout the submodule? The URL in the .gitmodules is directly cut and pasted from the Repos section where we would normally clone that repository.

Edit: The URLs for the repos in questions as copied from the portals are

https://MainCo@dev.azure.com/MainCo/OurProject/_git/MainAPIs
https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg

So the submodule lives at https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg but not sure how to phrase the above in terms of a Git submodules.

Edit 2:

In response to the answer given, here is the error output reported by the agent ...

...
2020-10-20T13:21:23.9543308Z ##[command]git submodule sync
2020-10-20T13:21:23.9892501Z ##[command]git -c http.https://MainCo@dev.azure.com.extraheader="AUTHORIZATION: bearer ***" submodule update --init --force
2020-10-20T13:21:24.0147532Z Submodule 'DBPkg' (https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg) registered for path 'DBPkg'
2020-10-20T13:21:24.0194148Z Cloning into '/home/vsts/work/1/s/DBPkg'...
2020-10-20T13:21:24.1694357Z remote: TF401019: The Git repository with name or identifier DBPkg does not exist or you do not have permissions for the operation you are attempting.
2020-10-20T13:21:24.1696067Z fatal: repository 'https://dev.azure.com/MainCo/OurProject/_git/DBPkg/' not found
2020-10-20T13:21:24.1726598Z fatal: clone of 'https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg' into submodule path '/home/vsts/work/1/s/DBPkg' failed
2020-10-20T13:21:24.1731117Z Failed to clone 'DBPkg'. Retry scheduled
2020-10-20T13:21:24.1760958Z Cloning into '/home/vsts/work/1/s/DBPkg'...
2020-10-20T13:21:24.3885945Z remote: TF401019: The Git repository with name or identifier DBPkg does not exist or you do not have permissions for the operation you are attempting.
2020-10-20T13:21:24.3892068Z fatal: repository 'https://dev.azure.com/MainCo/OurProject/_git/DBPkg/' not found
2020-10-20T13:21:24.3902720Z fatal: clone of 'https://MainCo@dev.azure.com/MainCo/OurProject/_git/DBPkg' into submodule path '/home/vsts/work/1/s/DBPkg' failed
2020-10-20T13:21:24.3909036Z Failed to clone 'DBPkg' a second time, aborting
2020-10-20T13:21:24.3987155Z ##[error]Git submodule update failed with exit code:

1

Solution

  • Update2

    Pipelines can access any Azure DevOps repositories in authorized projects, as described in the previous Limit job authorization scope to current project section, unless Limit job authorization scope to referenced Azure DevOps repositories is enabled.

    With this option enabled, you can reduce the scope of access for all pipelines to only Azure DevOps repositories explicitly referenced by a checkout step in the pipeline job that uses that repository.

    Try to turn off this option and check again.

    If it's still not work, you need to check repo's security. Find your [project name] build service account and project collection build service account, make sure both of them have enough permission to access your two git repos.

    enter image description here

    Update

    Try to use this in .gitmodules since submodule repo is in the same url as the parent. :

    [submodule "DBPkg"]
  path = DBPkg
  url = ../DBPkg

    You could also check this link: https://stackoverflow.com/a/34618962/5391065

    The build pipeline will check out your Git submodules as long as they are:

    • Unauthenticated: A public, unauthenticated repo with no credentials required to clone or fetch.

    • Authenticated:

      1. Contained in the same project, GitHub organization, or Bitbucket Cloud account as the Git repo specified above.

      2. Added by using a URL relative to the main repository. For example, this one would be checked out: git submodule add /../../submodule.git mymodule This one would not be checked out: git submodule add https://dev.azure.com/fabrikamfiber/_git/ConsoleApp mymodule

    You could also take a look at the Azure DevOps documentation here.

    A similar blog for your reference: Using Git submodules in (private) Azure DevOps repositories