Heroku not using cache for Yarn v3

Trying to get Heroku to use cached modules for Yarn v3 using workspaces. I have the following:

package.json:

"engines": {
    "node": "16.x",
    "yarn": "3.x"
  },
  "cacheDirectories": [
    "node_modules",
    "packages/components/node_modules",
    "packages/lib/node_modules",
    "packages/schema/node_modules",
    "packages/web/node_modules",
    "packages/web/.next/cache"
  ]

heroku-buildpack-features:

cache-native-yarn-cache=true

Heroku output:

-----> Creating runtime environment
       
       NPM_CONFIG_LOGLEVEL=error
       USE_YARN_CACHE=true
       NODE_VERBOSE=false
       NODE_ENV=production
       NODE_MODULES_CACHE=true
       
-----> Installing binaries
        !     You don't need to specify Yarn engine. Heroku will install the latest Yarn 1.x, so that per project version can be used. More information here: https://yarnpkg.com/getting-started/install#global-install
              https://devcenter.heroku.com/articles/nodejs-support
       
       engines.node (package.json):  16.x
       engines.npm (package.json):   unspecified (use default)
       engines.yarn (package.json):  unspecified (use default)
       
       Resolving node version 16.x...
       Downloading and installing node 16.7.0...
       Using default npm version: 7.20.3
       Resolving yarn version 1.22.x...
       Downloading and installing yarn (1.22.11)
       Using yarn 3.0.0

-----> Installing binaries
        !     You don't need to specify Yarn engine. Heroku will install the latest Yarn 1.x, so that per project version can be used. More information here: https://yarnpkg.com/getting-started/install#global-install
              https://devcenter.heroku.com/articles/nodejs-support
       
       engines.node (package.json):  16.x
       engines.npm (package.json):   unspecified (use default)
       engines.yarn (package.json):  unspecified (use default)
       
       Resolving node version 16.x...
       Downloading and installing node 16.7.0...
       Using default npm version: 7.20.3
       Resolving yarn version 1.22.x...
       Downloading and installing yarn (1.22.11)
       Using yarn 3.0.0
       
-----> Restoring cache
       Loading 6 from cacheDirectories (package.json):
       - node_modules
       - packages/components/node_modules (not cached - skipping)
       - packages/lib/node_modules
       - packages/schema/node_modules
       - packages/web/node_modules
       - packages/web/.next/cache

<a bunch of resolve stuff>

       ➤ YN0000: ┌ Fetch step
       ➤ YN0013: │ @apollo/client@npm:3.4.7 can't be found in the cache and will be fetched from the remote registry
       ➤ YN0013: │ @apollo/protobufjs@npm:1.2.2 can't be found in the cache and will be fetched from the remote registry

<and so on>

I've tried adding ".yarn/cache" to cacheDirectories in package.json, but it always says it's empty.

Not really sure where to go from here.

As of 9/2/2021, Heroku's nodeJS buildback doesn't fully support yarn 2. They started adding support in v173, but caching hasn't been fully addressed yet.

The underlying issue is that the buildpack doesn't properly detect that $BUILD_DIR/.yarn/cache exists and sets the environment variable YARN_CACHE_FOLDER to something other than .yarn/cache.(see https://github.com/heroku/heroku-buildpack-nodejs/blob/752f1d5a139a800920cf5bb1bf70aad1a2954525/bin/compile#L140)

When the buildpack gets to the cache step the .yarn/cache directory doesn't actually exist.

The solution:

I was able to work around this by manually setting YARN_CACHE_FOLDER to .yarn/cache in the config vars. You can do this via the CLI by running heroku config:set YARN_CACHE_FOLDER=.yarn/cache.

Ensure you also have .yarn/cache listed in cacheDirectories.

You will know it works when the build logs show:

-----> Caching build
       Saving 1 cacheDirectories (package.json):
       - .yarn/cache

If it did not work there would be a (nothing to cache) note next to .yarn/cache.