Keyringrc.cfg Permission Issues when installing packages via a python script using poetry

I have an app being managed by supervisor running as the user app_user. When the script starts, it checks if there are new packages and tries to install them using the code below. It however throws the error [Errno 13] Permission denied: '/root/.config/python_keyring/keyringrc.cfg'

Function to install packages:

import os
import sys
import subprocess
import logging

logger = logging.getLogger("general_log")

def install_app_packages():
    """
    Run poetry install to update application packages
    """
    python_path = sys.executable
    current_dir = os.path.dirname(os.path.realpath(__file__))
    project_directory = os.path.dirname(current_dir)
    poetry_install_command = "poetry install --with prod --no-root"
    activate_path = f"{python_path[:-7]}/activate"
    export_poetry = 'export PATH="/opt/apps/venv/poetry/bin:$PATH"'
    full_command = f"{export_poetry} && source {activate_path} && {poetry_install_command}"
    
    try:

        subprocess.run(
            full_command,
            shell=True,
            cwd=project_directory,
            executable="/bin/ash"
        )

        return True

    except Exception as e:
        logger.error(f"Error while running Poetry Install: {e.__str__()}")
        return False

Detailed error:

[2024-07-29 12:53:01 +0300] [24580] [INFO] Using worker: sync
[2024-07-29 12:53:02 +0300] [24581] [INFO] Booting worker with pid: 24581
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  - Installing semver (3.0.2)

  PermissionError

  [Errno 13] Permission denied: '/root/.config/python_keyring/keyringrc.cfg'

  at /usr/lib/python3.11/pathlib.py:1013 in stat
      1009│         """
      1010│         Return the result of the stat() system call on this path, like
      1011│         os.stat() does.
      1012│         """
    → 1013│         return os.stat(self, follow_symlinks=follow_symlinks)
      1014│ 
      1015│     def owner(self):
      1016│         """
      1017│         Return the login name of the file owner.

Cannot install semver.

[2024-07-29 12:53:03 +0300] [24580] [ERROR] Worker (pid:24581) exited with code 3
[2024-07-29 12:53:03 +0300] [24580] [ERROR] Shutting down: Master
[2024-07-29 12:53:03 +0300] [24580] [ERROR] Reason: Worker failed to boot.

What works though?

1. Installation works when invoked directly using gunicorn

When I run the application directly using gunicorn while running as the same app_user, it successfully installs.

(poetry-venv)project_directory$ gunicorn --bind 0.0.0.0:8030 server.wsgi --error-logfile /tmp/gunicorn_error.log --access-logfile /tmp/gunicorn_access.log --preload

2. Changing Supervisor to run as root

in supervisor conf file for the app, if I specify user=root and group=root instead of the app_user, the installation also succeeds.

So wondering what could by wonky here. App permissions or some poetry settings? Why is the script trying to look for keyringrc.cfg under root yet "root" is not in play here?

I had an earlier question on permissions here but I haven't found the answer. Could it be related?

Frustration, Google and SO to the rescue.

So the reason it was trying to access the /root/* directory is because the process (subprocess in the case of supervisor) was running as root. So I needed to find a way to run this process as the app_user. To do that, one has to define an environment variable in the subprocess file as defined here on SO.

environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8,HOME="/home/app_user",USER="app_user"

This will override any environment variable that's found in the default /etc/supervisor.conf file and force the subprocess to use the values in the subprocess file.