Python dependency caching in GitHub Actions

I'm trying to optimize CI/CD workflow on GitHub Actions for my project by moving dependency caching to first job. Dependencies are based on two files requirements.txt and requirements_dev.txt. Cache gets created but seems to contain only dependencies from the first file (jobs that use dependencies from second file fail in missing packages). I didn't find any help in actions/setup-python@v5 documentation or any similar Stack Overflow thread (there were some related to caching job, but not to setup-python). What am I doing wrong here?

name: code quality
on: [push, workflow_dispatch]
jobs:

  check-cache:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: python setup
        id: pydeps
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
          cache: 'pip'
          cache-dependency-path: 'requirements*.txt'
      - name: dependencies
        if: steps.pydeps.outputs.cache-hit == 'false'
        run: |
          python -m pip install -r requirements.txt
          python -m pip install -r requirements_dev.txt

  lint:
    needs: check-cache
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: check black and mypy
        run: |
          python -m black --check src
          python -m mypy src

  test-unit:
    needs: check-cache
    name: run unit tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: test code
        id: test
        run: python -m pytest tests/unit_tests
      - name: test artifacts
        if: failure() && steps.test.outcome == 'failure'
        uses: actions/upload-artifact@v4
        with:
          name: log-unit-tests
          path: logs-unit

  test-module:
    needs: check-cache
    name: run module tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: test code
        id: test
        run: python -m pytest tests/module_tests
      - name: test artifacts
        if: failure() && steps.test.outcome == 'failure'
        uses: actions/upload-artifact@v4
        with:
          name: log-module-tests
          path: logs-module

EDIT: I've rerun the workflow with added pip install -r requirements -r requirements_dev.txt in lint and test jobs, and it turns out that requirements.txt were installed again (no message saying "Requirement already satisfied") so it's not just problem with requirements_dev.txt.

• You're caching dependencies using:

  cache-dependency-path: 'requirements*.txt'

  which is correct — this will generate a cache key based on the contents of both requirements.txt and requirements_dev.txt.

• However, in check-cache, you install dependencies:

  python -m pip install -r requirements.txt

  python -m pip install -r requirements_dev.txt

  But in your other jobs, you:

  • do not install Python or dependencies, so the cache isn’t even being used.

  • setup-python@v5 isn't invoked in the lint, test-unit, and test-module jobs, so pip cache logic never kicks in there.

The Fix :

Add setup-python@v5 and dependency install to all jobs

Here's how you should refactor your jobs (e.g., for lint):

  lint:
    needs: check-cache
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: python setup
        uses: actions/setup-python@v5
        with:
          python-version: '3.10'
          cache: 'pip'
          cache-dependency-path: 'requirements*.txt'
      - name: install dependencies
        run: |
          python -m pip install -r requirements.txt
          python -m pip install -r requirements_dev.txt
      - name: check black and mypy
        run: |
          python -m black --check src
          python -m mypy src

Why Your Cache Seems Empty in Other Jobs

Each GitHub Actions job runs on a fresh VM — the pip cache from one job (like check-cache) does not persist unless:

• You use setup-python in each job

• You use cache: pip + same cache-dependency-path

• You re-run pip install in each job to read from the cache