Using docker RUN --mount=type=cache

I'm not fully understand docker's RUN --mount=type=cache mechanism, even having read the following articles,

https://docs.docker.com/build/cache/
https://vsupalov.com/buildkit-cache-mount-dockerfile/
Difference between --cache-to/from and --mount type=cache in docker buildx build
mount=type=cache more in-depth explanation https://github.com/moby/buildkit/issues/1673

This is my simple dockerfile:

FROM debian

ENV GOPATH=/go
WORKDIR ${GOPATH}/src/github.com/my/prj

# Copy sources
COPY ./ ./
RUN ls -l
RUN --mount=type=cache,target=${GOPATH} ls -l

What I was trying to do is to cache the go mod download and all go build caches, however, this is what I'm getting:

#5 [internal] load build context
#5 transferring context: 2.04kB done
#5 DONE 0.0s

#6 [builder 2/5] WORKDIR /go/src/github.com/my/prj
#6 CACHED

#7 [builder 3/5] COPY ./ ./
#7 DONE 0.0s

#8 [builder 4/5] RUN ls -l
#8 1.555 total 312
#8 1.555 -rw-r--r-- 1 root root 23796 Dec  2 21:59 History.md
#8 1.555 -rw-r--r-- 1 root root  1097 Dec  2 21:59 LICENSE
#8 1.555 -rw-r--r-- 1 root root   600 Dec  2 21:59 Makefile
#8 1.555 -rw-r--r-- 1 root root    15 Dec  2 21:59 Procfile
#8 1.555 -rw-r--r-- 1 root root 57837 Dec  2 21:59 README.md
#8 1.555 -rwxr-xr-x 1 root root   714 Dec  2 21:59 benchmark.sh
#8 1.555 -rw-r--r-- 1 root root  6093 Dec  2 21:59 controllers.go
#8 1.555 -rw-r--r-- 1 root root  3823 Dec  2 21:59 error.go
#8 1.555 -rw-r--r-- 1 root root   459 Dec  2 21:59 error_test.go
#8 1.555 -rw-r--r-- 1 root root   276 Dec  2 21:59 go.mod
#8 1.555 -rw-r--r-- 1 root root  6439 Dec  2 21:59 go.sum
. . . 
#8 1.555 -rw-r--r-- 1 root root   318 Dec  2 21:59 version.go
#8 DONE 1.6s

#9 [builder 5/5] RUN --mount=type=cache,target=/go ls -l
#9 1.881 total 4
#9 1.881 -rw-r--r-- 1 root root 45 Dec  2 22:58 go.mod
#9 DONE 1.9s

Why all of sudden my files are gone when using --mount=type=cache? How to clear such --mount=type=cache while retaining all other docker caches like step caches or apt caches? Is it still possible to cache the go mod download and all go build caches, when using COPY . .?

Solution

The problem here is that you've set the WORKDIR inside the path that you're later caching. Mounting cache is like any other mount; it replaces the directory tree at the mount point, but the files only exist there while the mount is active.

Your COPY executes without the mount. The first ls also executes without the mount, so you're seeing the files you've copied. The second ls executes with the mount, so you're seeing the files from the mount (and not the files you copied since they were overridden).

Since COPY doesn't support mounts, if you want to merge these files then you will need to COPY elsewhere and then RUN cp or similar with the mount active.

Bear in mind however that files in the mounted cache aren't included in the final image, so you shouldn't be installing application files or dependencies there; it's only for temporary files needed to build your image but not needed to keep (such as downloaded archives).

As far as I can tell, go does not download files in a layout that can reasonably be cached. You may be better served looking into multi-stage builds instead.