dockergithub-actionsdiskspace

Github runner out of disk space after building docker image

I have two github repos (call them A and B) that both have a .github/workflows/build_and_test.yml file that looks almost identical to this:

name: Build and test

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build-with-docker:
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v3
      - uses: docker/setup-buildx-action@v2
      - uses: docker/build-push-action@v3
        with:
          tags: my_tag:latest
          load: true
          context: .
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - run: docker run my_tag pytest

In repo A

In repo B

The warning is You are running out of disk space. The runner will stop working when the machine runs out of disk space. Free space left: 0 MB.

The error happens in the docker run my_tag pytest step: I get

Unable to find image 'my_tag:latest' locally
docker: Error response from daemon: pull access denied for my_tag, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.

which I suspect is a red herring that would go away if I fixed the disk space issue, although I am not certain. (I suspect Unable to find image is a red herring because I do not get this error in repo A.)

Is there a way to reduce the disk space used by my build-push-action step in repo B without changing its Dockerfile? Should I be using the no-cache-filters input described at https://github.com/docker/build-push-action#inputs?

(Possibly related: How to remove old docker images in continuous integration to save disk space, although I am looking for an answer that better fits my github workflow setup.)

According to https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources github's Linux runners have 14 GB of SSD space.

Update: I tried adding no-cache: true to docker/build-push-action@v3 in repo B, and I see the exact same running out of disk space warning and Unable to find image error as before.

Confusingly, the github UI has a check mark next to Run docker/build-push-action@v3, which indicates that that step ran successfully, so I'm not sure at what exact point in time the runner runs out of disk space:

github workflow screenshot

Solution

  • I got this to work thanks to https://github.com/actions/runner-images/issues/2840#issuecomment-1284059930.

    My new github workflow yml file is

    name: Build Docker image

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build-with-docker:
    runs-on: ubuntu-20.04
    steps:
      - name: Remove unnecessary files
        run: |
          sudo rm -rf /usr/share/dotnet
          sudo rm -rf "$AGENT_TOOLSDIRECTORY"
      - uses: actions/checkout@v3
      - uses: docker/setup-buildx-action@v2
      - uses: docker/build-push-action@v4
        with:
          context: .
          tags: my_tag:latest
          load: true
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: Run pytest
        run: docker run my_tag pytest

    The Remove unnecessary files step fixed the disk space warning and the Unable to find image error went away (so it was indeed a red herring).