githubazure-devopsnuget

Consuming private NuGet feed hosted on GitHub from an Azure Devops pipeline


I have a repo hosted in GitHub that builds and publishes a NuGet package to a private feed hosted by NuGet. If I create a workflow using GitHub Actions, then I can consume the NuGet package with no issue.

The problem that I have is that I am trying to consume that NuGet package from a pipeline running in an Azure DevOps instance. I've created a NuGet service connection in Azure DevOps named github-nuget, and I reference it in my workflow like this:

trigger: none

resources:
- repo: self

stages:
- stage: Packaging
  jobs:
  - job: buildAndPackage
    steps:
    - task: NuGetAuthenticate@1
      inputs:
        nuGetServiceConnections: 'github-nuget'
    - script: >-
        docker build
        --build-arg NUGET_GITHUBUSER=doesnt-really-matter
        --build-arg NUGET_GITHUBTOKEN=$(VSS_NUGET_ACCESSTOKEN)
        --tag my-api:$(Build.BuildNumber)
        --file api/src/MyProject.Web/Dockerfile .

and then in my Dockerfile, my build stage is as following:

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG NUGET_GITHUBUSER
ARG NUGET_GITHUBTOKEN
ENV NUGET_GITHUBUSER=${NUGET_GITHUBUSER}
ENV NUGET_GITHUBTOKEN=${NUGET_GITHUBTOKEN}
WORKDIR /src
COPY ["api/src/MyProject.Web/MyProject.Web.csproj", "api/src/MyProject.Web/"]
COPY ["nuget.config", "/"]
RUN dotnet nuget update source github --username ${NUGET_GITHUBUSER} --password ${NUGET_GITHUBTOKEN} --store-password-in-clear-text --valid-authentication-types basic
RUN dotnet restore "api/src/MyProject.Web/MyProject.Web.csproj"
COPY . .
WORKDIR "/src/api/src/MyProject.Web"
RUN dotnet build "MyProject.Web.csproj" -c Release -o /app/build --no-restore

The build fails when it attempts to retrieve the NuGet package from the private feed:

#43 [build 34/38] RUN dotnet nuget update source github --username anyone --password *** --store-password-in-clear-text --valid-authentication-types basic
#43 0.492 Package source "github" was successfully updated.
#43 DONE 0.7s

#44 [build 35/38] RUN dotnet restore "api/src/MyProject.Web/MyProject.Web.csproj"
#44 1.332   Determining projects to restore...
#44 4.554 /usr/share/dotnet/sdk/8.0.403/NuGet.targets(174,5): warning : Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured. [/src/api/src/MyProject.Web/MyProject.Web.csproj]
#44 4.571   Retrying 'FindPackagesByIdAsync' for source 'https://nuget.pkg.github.com/my-org/download/my.shared.package/index.json'.
#44 4.571   Response status code does not indicate success: 401 (Unauthorized).

Any ideas? It seems to me like the NuGetAuthenticate step isn't setting the personal access token required by GitHub correctly, but because it's a password, there's no way to verify in the logs what is actually being sent through. The execution of NuGetAuthenticate seems to be successful:

Installing the Azure Artifacts Credential Provider (.NET Core) to '/home/vsts/.nuget/plugins/netcore/CredentialProvider.Microsoft'. This credential provider is compatible with .NET SDK 6 or later.

Setting up the credential provider to use the identity 'MyProject.API Build Service (my-org)' for feeds in your organization/collection starting with:
  https://pkgs.dev.azure.com/my-org/
  https://my-org.pkgs.visualstudio.com/

Setting up the credential provider for these service connections:
  https://nuget.pkg.github.com/my-org/index.json

Finishing: Authenticate private NuGet feed

Solution

  • I can reproduce the similar issue when using the YAML sample.

    When the Nuget Authentication task authenticates external resources, its output variable is not a valid Github PAT/Password. Therefore, it cannot be used directly to update the Nuget source in the Nuget config file.

    To solve this issue, you can refer to the following two methods:

    Method1: if you need to keep using the nuget.config file to set the github source. You need to skip using the NuGetAuthenticate task and run the dotnet nuget update with your github PAT.

    Here is an example: You need to manually set Github PAT as secret variable in Pipeline.

    stages:
    - stage: Packaging
      jobs:
      - job: buildAndPackage
        steps:
        - script: >-
            docker build
            --build-arg NUGET_GITHUBUSER=doesnt-really-matter
            --build-arg NUGET_GITHUBTOKEN=$(GithubPAT)
            --tag my-api:$(Build.BuildNumber)
            --file api/src/MyProject.Web/Dockerfile .
    

    Dockerfile Sample:

    FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
    ARG NUGET_GITHUBUSER
    ARG NUGET_GITHUBTOKEN
    ENV NUGET_GITHUBUSER=${NUGET_GITHUBUSER}
    ENV NUGET_GITHUBTOKEN=${NUGET_GITHUBTOKEN}
    WORKDIR /src
    COPY ["api/src/MyProject.Web/MyProject.Web.csproj", "api/src/MyProject.Web/"]
    COPY ["nuget.config", "/"]
    RUN dotnet nuget update source github --username ${NUGET_GITHUBUSER} --password ${NUGET_GITHUBTOKEN} --store-password-in-clear-text --valid-authentication-types basic --configfile /nuget.config
    RUN dotnet restore "api/src/MyProject.Web/MyProject.Web.csproj" --configfile /nuget.config
    COPY . .
    WORKDIR "/src/api/src/MyProject.Web"
    RUN dotnet build "MyProject.Web.csproj" -c Release -o /app/build --no-restore 
    

    Method2: if you want to keep using the NuGetAuthenticate task, you can use the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable in docker file for dotnet restore process.

    Here are the steps:

    Step1: Remove the github source in Nuget.config file.

    Step2: Remove the following line in docker file:

    RUN dotnet nuget update source github --username ${NUGET_GITHUBUSER} --password ${NUGET_GITHUBTOKEN} --store-password-in-clear-text --valid-authentication-types basic
    

    Step2: Add the following line to the docker file:

    RUN curl -L https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh  | sh
    ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{\"endpointCredentials\": [{\"endpoint\":\"https://nuget.pkg.github.com/my-org/index.json\", \"username\":\"docker\", \"password\":\"${NUGET_GITHUBTOKEN}\"}]}"
    

    Dockerfile:

    FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
    ARG NUGET_GITHUBUSER
    ARG NUGET_GITHUBTOKEN
    ENV NUGET_GITHUBUSER=${NUGET_GITHUBUSER}
    ENV NUGET_GITHUBTOKEN=${NUGET_GITHUBTOKEN}
    RUN curl -L https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh  | sh
    ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS="{\"endpointCredentials\": [{\"endpoint\":\"https://nuget.pkg.github.com/my-org/index.json\", \"username\":\"docker\", \"password\":\"${NUGET_GITHUBTOKEN}\"}]}"
    WORKDIR /src
    COPY ["api/src/MyProject.Web/MyProject.Web.csproj", "api/src/MyProject.Web/"]
    COPY ["nuget.config", "/"]
    RUN dotnet restore "api/src/MyProject.Web/MyProject.Web.csproj"
    COPY . .
    WORKDIR "/src/api/src/MyProject.Web"
    RUN dotnet build "MyProject.Web.csproj" -c Release -o /app/build --no-restore
    

    Pipeline sample:

    trigger: none
    
    resources:
    - repo: self
    
    stages:
    - stage: Packaging
      jobs:
      - job: buildAndPackage
        steps:
        - task: NuGetAuthenticate@1
          inputs:
            nuGetServiceConnections: 'github-nuget'
        - script: >-
            docker build
            --build-arg NUGET_GITHUBUSER=doesnt-really-matter
            --build-arg NUGET_GITHUBTOKEN=$(VSS_NUGET_ACCESSTOKEN)
            --tag my-api:$(Build.BuildNumber)
            --file api/src/MyProject.Web/Dockerfile .
    

    In this case, you can keep the current settings in the Pipeline and it will use the github source to restore package.