gitnpmhttpspersonal-access-token

npm install from private forgejo via git+https results in "403 forbidden" of "verify"

I recently moved from private GitLab to private Forgejo. Now I see this issue with an older project that uses npm, here's the package.json

{
  "name": "zzz",
  "version": "2.9.0",
  "repository": "https://zzz.de/zzz/zzz.git",
  "license": "BSD-Protection",
  "description": "zzz.",
  "dependencies": {
    "private-dependency": "git+https://my-forge.jo/private-org/private-project.git#0.9.2",
    "bootstrap": "3.4.1"
  }
}

So the current project and the private dependency are on the same Forgejo server. In the workflow, I do run npm ci to install the dependencies.

That doesn't work, I get 403 forbidden.

The known difference between GitLab and Forgejo is, that on GitLab the repo was public while on Forgejo it cannot (org needs to be private to protect private package - different story). Surely this generates the 403 forbidden.

My Question: How to make it work?

Some technical background I found out already: npm dependencies with git+https do call git and git will use curl for the network requests.

So either I can provide proper auth data to git or curl. I did some tests already, all failed. I really must be missing something here. It must work somehow -.-

Approach 1: Checkout with PAT
The most obvious idea was to provide a PAT for the checkout action, which can access both repositories. It failed.

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects clone https://my-forge.jo/private-org/private-project.git /root/.npm/_cacache/tmp/git-clonelDb5Ay --recurse-submodules
npm ERR! Cloning into '/root/.npm/_cacache/tmp/git-clonelDb5Ay'...
npm ERR! remote: Verify
npm ERR! fatal: Authentication failed for 'https://my-forge.jo/private-org/private-project.git/'
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2024-05-31T14_31_19_441Z-debug-0.log

Approach 2: Update git config to use PAT
I leveraged git config to set the PAT via the URL, it failed

  - name: npm login
    shell: bash
    run: |
      git config --remove-section http."https://my-forge.jo/"
      git config url."https://${{ inputs.npm-token }}@my-forge.jo".insteadOf "https://my-forge.jo"

resulted in

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects clone https://my-forge.jo/private-org/private-project.git /root/.npm/_cacache/tmp/git-cloneoRXY7d --recurse-submodules
npm ERR! Cloning into '/root/.npm/_cacache/tmp/git-cloneoRXY7d'...
npm ERR! remote: Verify
npm ERR! fatal: Authentication failed for 'https://my-forge.jo/private-org/private-project.git/'
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2024-05-31T14_05_30_219Z-debug-0.log

Approach 3: "Preinstall" the dependency
Since npm has a local cache folder, I thought it can help to install the dependency with the PAT in the URL before running npm ci. It didn't, BUT at least the npm install worked:

  - name: install dependencies
    working-directory: src/theme
    shell: bash
    run: |
      git config --remove-section http."https://my-forge.jo/"
      npm install -g git+https://${{ inputs.npm-token }}@my-forge.jo/private-org/private-project.git#0.9.2
      npm ci

resulted in

added 1 package, and audited 2 packages in 2s
found 0 vulnerabilities
npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote https://my-forge.jo/private-org/private-project.git
npm ERR! remote: Verify
npm ERR! fatal: Authentication failed for 'https://my-forge.jo/private-org/private-project.git/'
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2024-05-31T14_23_56_288Z-debug-0.log

Approach 4: Provide auth to curl via .netrc
Latest now it get's crazy. I really must be missing something.

Again the idea is, npm calls git which calls curl which loads .netrc from $HOME. So I defined the PAT there, it failed:

  - name: npm login
    shell: bash
    run: |
      echo "machine my-forge.jo" >> ~/.netrc
      echo "  login ${{ inputs.npm-token }}" >> ~/.my-forge.jo

results in

npm ERR! code 128
npm ERR! An unknown git error occurred
npm ERR! command git --no-replace-objects ls-remote https://my-forge.jo/private-org/private-project.git
npm ERR! remote: User permission denied
npm ERR! fatal: unable to access 'https://my-forge.jo/private-org/private-project.git/': The requested URL returned error: 403
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2024-05-31T07_55_07_202Z-debug-0.log
Solution

  • Turns out, it is a combination of Approach 1 and 2.

    Checkout using the PAT

      - name: checkout
    uses: https://github.com/actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
    with:
      token: ${{ secrets.NPM_TOKEN }}

    and then, before calling npm ci, configure git to add the PAT to the URL important: it must be --global

      - name: install dependencies
    working-directory: src/theme
    shell: bash
    run: |
      # Needed to prevent 401 / 403. We're accessing another repository
      git config --global url."https://${{ inputs.npm-token }}@my-forge.jo/".insteadOf "https://my-forge.jo/"
      
      npm ci

    Thanks to a friend of mine, who fed my post into GPT4 ;) Beside many things I ignored, there was one idea to combine everything. Since this did work, I reduced it to what I describe here.