Clone azure devops submodule from another project using pipeline

As the title says, I have a base repo with 2-3 helper apps that speed up my django development.

Lets say I have this nimble_auth git repository as a submodule

https://dev.azure.com/nimble/base_repo/_git/nimble_auth/

And my new project is at django_backend, on a different project, but under the same organization.

https://dev.azure.com/nimble/new_proj/_git/django_backend/

This is my current pipeline and it doesn't work with persist credentials nor does it work with a manual clone script.

To be clear, I want a working solution whichever that is, I don't want to fix my current solution.

trigger:
  - develop

resources:
  - repo: self

variables:
  tag: "$(Build.BuildId)"

stages:
  - stage: Build
    displayName: Build Docker Images
    jobs:
      - job: Build
        displayName: Build Docker Images
        condition: eq(variables['Build.SourceBranch'], 'refs/heads/develop')
        cancelTimeoutInMinutes: 30
        pool:
          name: Nimble
          demands:
            - Agent.Name -equals Pipelinus Maximus
        steps:
          - checkout: self
            persistCredentials: true
            clean: true
          # Clone submodules using HTTPS
          - script: |
              git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" \
                  submodule update --init --recursive
            displayName: Fetch Submodules

          # Build Image from the root directory
          - task: Docker@2
            displayName: Build Docker Image for Django (api-django)
            inputs:
              command: build
              dockerfile: '$(Build.SourcesDirectory)/Dockerfile'
              buildContext: '$(Build.SourcesDirectory)'
              tags: |
                api-django:$(tag)

from that script this is the current error:

Starting: Fetch Submodules
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.250.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
========================== Starting Command Output ===========================
/usr/bin/bash --noprofile --norc /opt/azure/AzureAgent/_work/_temp/2d9074d2-7d50-427d-aaf8-1c74fc1ab69a.sh
Cloning into '/opt/azure/AzureAgent/_work/1/s/nimble_auth'...
remote: TF401019: The Git repository with name or identifier nimble_auth does not exist or you do not have permissions for the operation you are attempting.
fatal: repository 'https://dev.azure.com/nimble/base_repo/_git/nimble_auth/' not found
fatal: clone of 'https://nimble@dev.azure.com/nimble/base_repo/_git/nimble_auth' into submodule path '/opt/azure/AzureAgent/_work/1/s/nimble_auth' failed
Failed to clone 'nimble_auth'. Retry scheduled
Cloning into '/opt/azure/AzureAgent/_work/1/s/nimble_auth'...
remote: TF401019: The Git repository with name or identifier nimble_auth does not exist or you do not have permissions for the operation you are attempting.
fatal: repository 'https://dev.azure.com/nimble/base_repo/_git/nimble_auth/' not found
fatal: clone of 'https://nimble@dev.azure.com/nimble/base_repo/_git/nimble_auth' into submodule path '/opt/azure/AzureAgent/_work/1/s/nimble_auth' failed
Failed to clone 'nimble_auth' a second time, aborting

##[error]Bash exited with code '1'.
Finishing: Fetch Submodules

Hope this is enough context, thanks in advance.

To clone a submodule from another project, you can refer to the steps below.

1. In your new_proj, turn off Limit job authorization scope to current project for non-release pipelines and Protect access to repositories in YAML pipelines option from Project Settings -> Pipelines -> Settings.

   

2. In your pipeline, set submodules to true in your checkout task.

   steps:
   - checkout: self
     submodules: true
   
   - script: |
       tree
     displayName: 'ListAllFiles'
   

3. In your base_repo project, add build service account Project Collection Build Service (yourOrgName) into the Contributors group. (If you still have the same error in the pipeline, add project-level build service account new_proj Build Service (yourOrgName) into the Contributors group as well.)

   

4. Result: