EOFError while parsing .netrc in CI job for accessing GitLab PyPI package registry

I have a CI job in repo A that builds an image (Dockerfile via Kaniko), which requires a package from a package repository from repo B.

Since this is a task that will repeat in the future quite a bit, I created a GAT (group access token, rights: api, role: Developer) called CI_GROUP_API with a corresponding group CICD variable with the same name and value, so that my CI job can access all other repos in that group by using it.

I use the GAT to generate a .netrc on-the-fly in my CI job and I'm also adjusting my requirements.txt to add the package (will make this change permanent in the future). The files that are required for the CI job can be seen after the error log below. When I run my job I receive:

Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (24.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://gitlab.company.com/api/v4/projects/58838/packages/pypi/simple
User for gitlab.company.com: ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 44, in _iter_built
    for version, func in infos:
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 297, in iter_index_candidate_infos
    result = self._finder.find_best_candidate(
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 890, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 831, in find_all_candidates
    page_candidates = list(page_candidates_it)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/sources.py", line 194, in page_candidates
    yield from self._candidates_from_page(self._link)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 791, in process_project_url
    index_response = self._link_collector.fetch_response(project_url)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 461, in fetch_response
    return _get_index_content(location, session=self.session)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 364, in _get_index_content
    resp = _get_simple_response(url, session=session)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 135, in _get_simple_response
    resp = session.get(
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/network/session.py", line 520, in request
    return super().request(method, url, *args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/requests/sessions.py", line 710, in send
    r = dispatch_hook("response", hooks, r, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pip/_vendor/requests/hooks.py", line 30, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/network/auth.py", line 500, in handle_401
    username, password, save = self._prompt_for_password(parsed.netloc)
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/network/auth.py", line 455, in _prompt_for_password
    username = ask_input(f"User for {netloc}: ") if self.prompting else None
  File "/usr/local/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 251, in ask_input
    return input(message)
EOFError: EOF when reading a line
error building image: error building stage: failed to execute command: waiting for process to exit: exit status 2

---

My CI job is defined as follows:

build-service:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: ['']
  tags:
    - asprunner
  rules:
    # Build base image only if the Dockerfile or CICD config file have changed and been pushed to the cloud branch
    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "dev"'
      when: on_success
  script:
    - echo "Building image for service X using Kaniko"
    - |
      export BASE_REGISTRY_URL=registry.gitlab.company.com/mygroup/base-service
      export BASE_TOKEN_NAME=CI_GROUP_API
      export BASE_TOKEN_VALUE=$CI_GROUP_API

      cat <<EOT >> .netrc
      machine gitlab.company.com
      login $BASE_TOKEN_NAME
      password $BASE_TOKEN_VALUE
      EOT

      cat <<EOT >> requirements.txt
      --index-url https://gitlab.company.com/api/v4/projects/58838/packages/pypi/simple
      base-service
      
      EOT

      echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_JOB_TOKEN}" | base64 | tr -d '\n')\"},\"${BASE_REGISTRY_URL}\":{\"username\":\"${BASE_TOKEN_NAME}\",\"password\":\"${BASE_TOKEN_VALUE}\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor
       --context .
       --dockerfile Dockerfile
       --insecure
       --skip-tls-verify
       --skip-tls-verify-pull
       --insecure-pull
       --destination "${CI_REGISTRY_IMAGE}:v0.2"

My corresponding Dockerfile can be seen below:

FROM registry.company.com/mygroup/base-service:v1.0
ENV DEBIAN_FRONTEND noninteractive

COPY ./requirements.txt requirements.txt
COPY ./.netrc .netrc
RUN echo "----------- requirements.txt -----------" && cat requirements.txt 
RUN echo "----------- .netrc -----------" && cat .netrc
RUN pip3 install --upgrade pip && pip3 install --no-cache -r requirements.txt

RUN mkdir -p service
COPY main.py service/
COPY gltf_transformer service/

RUN rm requirements.txt .netrc

ENTRYPOINT ["sh", "python3 main.py"]

The resulting pip-related files look like this:

requirements.txt

pygltflib
--index-url https://gitlab.company.com/api/v4/projects/58838/packages/pypi/simple
x-base-service

.netrc

machine gitlab.company.com
login CI_GROUP_API
password [MASKED]

I thought that perhaps the [MASKED], which is how GitLab hides masked variables in the CI job's logs, might have been written to the .netrc as the actual value. However, first the error comes while reading the user and second I added

artifacts: paths: - requirements.txt - .netrc when: on_failure

to my job to get the files and verified that the token's value is there.

I have also downloaded both files locally to my system and ran pip for the requirements.txt using the .netrc from the job's artifacts. It worked just fine.

The error message is very, very inaccurate.

First I need to clarify that the only reason it worked locally was because pip was using my SSH to connect to the package repository and .netrc was actually never used. I found that out when I ran the pip with the requirements.txt file in a clean environment and it asked me for user name and password.

My next thought was that it's the encoding that's the problem. After all the documentation states:

[

However, using iconv for converting the UTF-8 to ASCII didn't do anything and the error was still there.

Then I remembered, while reading the documentation for the Nth time, that the .netrc is related to a specific user. Now, inside a Docker container we don't really have users, however we do have the /root directory as the home directory.

I added

COPY ./.netrc /root/.netrc

to my Dockerfile, followed by fixing all following paths referring to the file and it worked.

The man-pages for .netrc does state:

So NETRC environment variable can also be used if a specific location other than the user's home directory is required.