Idiot developer unable to import files dynamically in nodejs with esm modules

I am trying to convert my commonjs repo to ESM and have having issues with the readFileSync command. It seems that I get different behaviour on node 18 and on node 20. I try to load a path:

D:\dev\ts-command-line-args\README.MD

and I get an error:

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file and data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'

I then give it a path with a schema in by:

    const markdownPath = pathToFileURL(resolve(args.markdownPath)).href;
    console.log(`Loading existing file from '${chalk.blue(markdownPath)}'`);
    const markdownFileContent = readFileSync(markdownPath).toString();

and I then get this error on node 18:

Loading existing file from 'file:///D:/dev/ts-command-line-args/README.MD'

node:fs:601
  handleErrorFromBinding(ctx);
  ^

Error: ENOENT: no such file or directory, open 'file:///D:/dev/ts-command-line-args/README.MD'
    at Object.openSync (node:fs:601:3)
    at readFileSync (node:fs:469:35)
    at writeMarkdown (file:///D:/dev/ts-command-line-args/dist/write-markdown.js:15:33)
    at file:///D:/dev/ts-command-line-args/dist/write-markdown.js:46:1
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
  errno: -4058,
  syscall: 'open',
  code: 'ENOENT',
  path: 'file:///D:/dev/ts-command-line-args/README.MD'
}

(the file definitely exists)

and this error on node 20:

Error: ENOENT: no such file or directory, open 'D:\dev\ts-command-line-args\file:\D:\dev\ts-command-line-args\README.MD'

To further complicate this my project has already loaded a file using a pretty much identical method:

        console.log(`Loading args from file ${resolve(parsedArgs[options.loadFromFileArg])}`);

        const configFromFile: Partial<Record<keyof T, any>> = JSON.parse(
            readFileSync(resolve(parsedArgs[options.loadFromFileArg])).toString(),
        );

Loading args from file D:\dev\ts-command-line-args\package.json

this working code is in parse.js. I am running this code with:

node dist/write-markdown

write-markdown.js imports parse.js:

import { parse } from './parse.js';

So it seems (this is an assumption) that loading files in js files that are run directly by node does not work but loading files in js files that imported does work...

I really hope that someone can clear this up for me!

Solution

You were chasing a red herring. It's always important to check the stack of an error and see where it's coming from!

The full stack for your ERR_UNSUPPORTED_ESM_URL_SCHEME error was this:

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at new NodeError (node:internal/errors:372:5)
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/resolve:1078:11)
    at defaultResolve (node:internal/modules/esm/resolve:1158:3)
    at ESMLoader.resolve (node:internal/modules/esm/loader:605:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:318:18)
    at ESMLoader.import (node:internal/modules/esm/loader:404:22)
    at importModuleDynamically (node:internal/modules/esm/translators:106:35)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:35:14)
    at loadArgConfig (file:///C:/Users/david/Temp/ts-command-line-args/dist/helpers/markdown.helper.js:121:23)
    at file:///C:/Users/david/Temp/ts-command-line-args/dist/helpers/markdown.helper.js:114:42 {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}

The important part is:

at loadArgConfig (file:///C:/Users/david/Temp/ts-command-line-args/dist/helpers/markdown.helper.js:121:23)

...which refers to this code:

export async function loadArgConfig(jsFile: string, importName: string): Promise<UsageGuideConfig | undefined> {
    const jsPath = join(process.cwd(), jsFile);
    // eslint-disable-next-line @typescript-eslint/no-var-requires
    const jsExports = await import(jsPath); // <------------------- ERROR HERE!!!

    const argConfig: UsageGuideConfig = jsExports[importName];

    if (argConfig == null) {
        console.warn(`Could not import ArgumentConfig named '${importName}' from jsFile '${jsFile}'`);
        return undefined;
    }

    return argConfig;
}

It has nothing to do with the write-markdown.ts file and its usage of readFileSync! You changed code in that other file though, which changed the error because you introduced a new bug instead, in code that runs before this one, so it now failed earlier.

The solution is to keep the code related to readFileSync as it was because it worked just fine (without pathToFileURL) and instead use pathToFileURL in loadArgConfig (the place the error complained about):

const jsExports = await import(pathToFileURL(jsPath).href);

See my pull request https://github.com/Roaders/ts-command-line-args/pull/45