node.jsnode-modulespartitionpnpmmender

How can I require node modules across disk partitions?

I have a Node program that I run on Raspberry Pi 4. I've recently started using a OTA deployment system called Mender to push updates to my code on remote RPis. Mender creates a partition system that uses two 3.5GB partitions, one as the main and the other as a rollback in the event of a failed deployment. And it has a 3rd partition /data, that is around 20GB in my case, for things that need to be persisted between updates.

I was unable to get my entire application and all of it's node module dependencies into the 3.5GB partition. So I moved the node_modules directory to the /data partition and created a symlink that points back to my project directory(home/pi/myProject). This works for module installs but when I try to require an installed module from within my project an error is thrown;

internal/modules/cjs/loader.js:883
  throw err;
  ^

Error: Cannot find module '@google-cloud/pubsub'
Require stack:
- /home/pi/myProject/pwrMngmnt.js
- /home/pi/myProject/[eval]
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
    at Function.Module._load (internal/modules/cjs/loader.js:725:27)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/myProject/pwrMngmnt.js:3:20)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Module.require (internal/modules/cjs/loader.js:952:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/pi/myProject/pwrMngmnt.js',
    '/home/pi/myProject/[eval]'
  ]
}

Is there a configuration I need to set to make this work?

Solution

  • Instead of doing a symlink, you could specify the NODE_PATH environment variable. Quoting the NodeJS documentation : "NODE_PATH was originally created to support loading modules from varying paths before the current module resolution algorithm was defined."

    NODE_PATH is still supported and could perfectly fit your use-case IMO. Don't forget to fix file permissions if needed (using chmod and chown).

    For instance:

    export NODE_PATH="/data/node_modules"
node <your script>

    Furthermore, you can ask npm or yarn to install modules in this directory.

    With yarn:

    yarn install --modules-folder /data/node_modules

    With npm:

    mkdir -p /data/node_modules
npm install --prefix /data

    Possible related questions: