reactjsimportnode-modulessubdirectorymediapipe

Is there any way of using node modules packages subfolders?

I am using @mediapipe/tasks-vision npm package in the webapp, I want to use my own wasm folder that's present in 'node_modules/@mediapipe/tasks-vision/wasm'.

I am using next.js as the front end.

Any help or sharing is most appreciated.

Code Snippet:-

import { FaceLandmarker, FilesetResolver, DrawingUtils} from "@mediapipe/tasks-vision"; 
 
Initialize face landmarker

  useEffect(() => {
    const createFaceLandmarker = async () => {
      const filesetResolver = await FilesetResolver.forVisionTasks("https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.14/wasm");
      faceLandmarker = await FaceLandmarker.createFromOptions(filesetResolver, {
        baseOptions: {
          modelAssetPath: "https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task",
          delegate: "GPU"
        },
        outputFaceBlendshapes: true,
        runningMode: "VIDEO",
        numFaces: 1
      });
      setLoaded(true);
    };
    createFaceLandmarker();
  }, []);

I have tried every possible way that are below but didn't get the solution:-

Solution

  • A URL must be passed, and not a path. The way this seems to work is the mediapipe library uses the URL you pass to inject a <script> tag into the page that point towards URLs derived from that one (sub files). The relevant code that ultimately does this (after many nested calls) is here.

    This is really quite unusual, hence your legitimate confusion. The library must have some reason for wanting to download this lib asynchronously from a URL at runtime like this. The likely reason is the JS is dynamically loaded based on some conditions about the browser environment, and also because it wants to then further download the WASM binaries, which can't be bundled direct into a JS app trivially.

    Regardless It means that we can't use ES imports to provide the lib during the build.

    Instead, you would need to make the specific directory inside of node_modules available such that it was served from the NextJS public folder and therefore available on a URL like http://yoursite.com/wasm.

    To do that in an automated way, we could modify NextJS's Webpack configuration such that the relevant files from node_modules are automatically copied into the public dir.

    First install copy-webpack-plugin to your project:

    npm install copy-webpack-plugin --save-dev

    In your next.config.js:

    const CopyPlugin = require("copy-webpack-plugin");

const nextConfig = {
  webpack: (config) => {
    const customPlugins = [
      new CopyPlugin({
        patterns: [
          {
            from: 'node_modules/@mediapipe/tasks-vision/wasm',
            to: '../public/', // Reason for `..` is because next uses the hidden `.next` folder during build
          },
        ],
      }),
    ];

    config.plugins.push(...customPlugins);

    return config;
  },
};

module.exports = nextConfig;

    You will want to add public/wasm to your .gitignore to not accidentally commit these files. We will rely on the build copying them to the right place.

    Now the files should be hosted on a URL on your site e.g. http://yoursite.com/wasm.

    Next, change the reference to this URL in the code:

    const filesetResolver = await FilesetResolver.forVisionTasks("/wasm");

    Restart NextJS to ensure the build config change is applied.