pythondockergithub-actions

An error occurred while using the Python magic library at Docker & Github Action. "E ModuleNotFoundError: No module named 'magic'"


Issue Description: Running a Python project using the python-magic library within a Docker container on Windows 10 OS results in the error "Module magic not found."

Proposed Fix: Initially, changing the dependency from python-magic to python-magic-bin resolved the issue in the local environment. However, when running the project in a Docker container, the same error resurfaced unexpectedly.

Dockerfile Content:

# Dockerfile for API service
FROM python:3.11.0 as base

# Dependencies installations
RUN apt-get update && apt-get install -y \
    poppler-utils \
    tesseract-ocr \
    libgl1-mesa-glx \
    libmagic1 \
    python3-magic

# Non-root user setup
RUN groupadd --gid 1000 user && adduser --disabled-password --gecos '' --uid 1000 --gid 1000 user
WORKDIR /home/user
USER user

# Development stage setup
FROM base as dev
USER root
COPY --chown=user:user ./requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY --chown=user:user ./src /home/user/src
COPY --chown=user:user ./test /home/user/test
COPY --chown=user:user .env /home/user/.env
USER user
CMD ["python", "-m", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "5000", "--reload", "--env-file", "/home/user/.env"]

# Test stage setup
FROM base as test
COPY --chown=user:user ./requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
USER root
COPY --chown=user:user ./src /home/user/src
COPY --chown=user:user ./test /home/user/test
RUN mkdir /home/user/.mypy_cache && chown user:user -R /home/user/.mypy_cache
RUN pip install pytest
CMD ["pytest"]
USER user

# Default target stage
FROM dev

Current Challenge: The error "Module magic not found" persists when running the project in a Docker container despite changing the library dependency. Seeking insights or suggestions to overcome this issue. Your thoughts?


Solution

  • After encountering the "Module magic not found" error in the Docker container, I devised a solution by creating separate requirements.txt files for Windows and Linux versions of the project. The Linux version requires python-magic, while the Windows version necessitates python-magic-bin.

    Dockerfile With Updated Configuration:

    # Dockerfile for API service
    FROM python:3.11.0 as base
    
    # Define build arguments
    ARG PROJECT=src
    ARG TEST=test
    
    # Install dependencies
    RUN apt-get update && apt-get install -y \
        poppler-utils \
        tesseract-ocr \
        libgl1-mesa-glx \
        libmagic1 \
        python3-magic
    
    # Create a non-root user
    RUN groupadd --gid 1000 user && adduser --disabled-password --gecos '' --uid 1000 --gid 1000 user
    
    WORKDIR /home/user
    USER user
    
    # Development stage setup
    FROM base as dev
    
    USER root
    COPY --chown=user:user ./requirements-linux.txt ./
    RUN pip install --no-cache-dir -r requirements-linux.txt
    COPY --chown=user:user ./$PROJECT /home/user/$PROJECT
    COPY --chown=user:user ./$TEST /home/user/$TEST
    COPY --chown=user:user .env /home/user/.env
    USER user
    CMD ["python", "-m", "uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "5000", "--reload", "--env-file", "/home/user/.env"]
    
    # Test stage setup
    FROM base as test
    
    COPY --chown=user:user ./requirements-linux.txt ./
    RUN pip install --no-cache-dir -r requirements-linux.txt
    COPY --chown=user:user ./$PROJECT /home/user/$PROJECT
    COPY --chown=user:user ./$TEST /home/user/$TEST
    RUN mkdir /home/user/.mypy_cache && chown user:user -R /home/user/.mypy_cache
    RUN pip install pytest
    CMD ["pytest"]
    USER user
    
    # Set the default target stage to 'dev'
    FROM dev
    

    By segregating the requirements between Windows and Linux versions and updating the Dockerfile correspondingly, the project now runs smoothly without the "Module magic not found" error in the Docker container.