ccache has zero cache hits in GitLab CI, even when sources don't change and the cache is persisted.
Furthermore, the cache increases in size every time a build runs, which means it is being rebuilt over and over.
The issue was the default way ccache checks if the compiler is the same - via its timestamp. Since each GitLab CI instance runs a brand new Docker container, this timestamp is always different, hence always building new cache.
To fix this, set CCACHE_COMPILERCHECK to content instead of the default mtime.
From the ccache documentation:
compiler_check (CCACHE_COMPILERCHECK)
By default, ccache includes the modification time (“mtime”) and size of the compiler in the hash to ensure that results retrieved from the cache are accurate. This setting can be used to select another strategy. Possible values are:
content
Hash the content of the compiler binary. This makes ccache very slightly slower compared to the mtime setting, but makes it cope better with compiler upgrades during a build bootstrapping process.
mtime
Hash the compiler’s mtime and size, which is fast. This is the default.