Typescript, NodeJS and ES6/ESM Modules

I write this as a sequel to a previous article of getting ES6 module imports to work with other NPM modules, from TypeScript.

The CommonJS import framework (require) is how node.js was developed and getting everything smooth and working with modern static import is sometimes not so easy, and often not well documented.

I favor static imports, because the syntax and lack of obvious structure around require is just so full of magic and just works provided you have spent a number of years working inside that universe.

Basic settings

As a starter, whether a Node project defaults to CommonJS modules or ESM is decided by two settings - which must correspond.

tsconfig.json:

{
  "compilerOptions": {
    // ... 
    "module": "esnext", /* Or 'commonjs'. */

package.json:

{
   // ... 
   "type": "module",  /* Or 'commonjs' */ 

The problem here - relating to TypeScript

If I have a local TS file (utils.ts) like this:

// Invert keys and values of 1-level deep object
export function invert(o: Dict<BaseTypes>) {
    // ... implementation here ...
}

and then want to use it from another file:

import {invert} from './utils';

then that import will only work (everything else being default settings in TS and Node), as long as I stay with CommonJS modules.

However, when I change over to ESM modules (see below), the compiled Javascript will no longer work. Because Node.js will be trying to import exactly:

import {invert} from './utils';

And honestly, there is no such file - that is ./utils - without the file extension added to it.

The explanation is that as TypeScript transpiles this, then for CommonJS, it ends up calling require(), and some logic on top of that. And... require() accepts local javascript file names... without file extension.

So if I want my code to work with Node with ESM, I need to change it to:

import {invert} from './utils.js';

That means, I need to have different code bases, if I target CommonJS or ESM. Not very good.

Sort of going forwards...

For a while I accepted the compromise of adding .js to every local import... But then I wanted to add Jest testing on top of this. Which (together with ts-jest) does its own building of the test files (behind the scenes, I think using Babel). And ts-jest (whatever my tsconfig/package.json said) would not accept imports from explicit Javascript files:

import {invert} from './utils.js';  // ts-jest errors here

So I needed to fix it differently. (Understanding how/why ts-jest configures TypeScript/Node differently I did not want to enter).

A couple of solutions

One approach is to use the NPM module esm - however I never went in that direction. And I would like to avoid pulling in dependencies to solve this.

But it turns out there is a Node flag to do solve exactly this problem: --experimental-specifier-resolution=node. (You find it towards the bottom here).

Wow, all settled ?

Well, we also need to launch Node in some different ways (from terminal, from a Node bin script, from the VsCode debugger, and the last two presents small challenges).

Passing Node flags into the VsCode debugger

It turns out there is a launch.json entry for this:

    "configurations": [
        {
            "type": "pwa-node",
            // ...
            "runtimeArgs": ["--experimental-specifier-resolution=node"],
            "program": "${workspaceFolder}/src/cmd.ts",

(Scroll down to the bottom of this page for docs).

And it actually does the job (I found various other suggestions where to put the Node arg, however I think they targeted an old version of the Javascript debugger).

Passing Node flags into an Npm bin command

To run a JS script directly from the terminal (without prefixing it with node) we can use the bin section of package.json:

... 
"bin": {
    "my_cmd": "./path/to/my/script",
  },

However, if the target script is JS/TS (yes, it would be), we need to insert a shebang there, to make it executable:

#!/usr/bin/env node
// Rest of JS/TS here

But the obstacle here is that we cannot pass options to Node in the shebang. Sorry. We stumble into a bash fact of life here, and there's no simple way around it, for Node/TS/JS.

There is a magic hack to this over here. However that fulfills my definition of being so cryptic and non-intuitive (for anyone who did not patiently learn Bash internals) that I cannot recommend it.

The solution instead is to let the bin command point to a shell script, and let that one invoke Node with required options:

For me:

  "bin": {
    "knemm": "./shell/knemm",
  },

and knemm then being:

#!/bin/bash 
# Get directory of calling script
DIR="$( cd "$( dirname "$0" )" &> /dev/null && pwd )"
if [ "$(echo $DIR | grep '.nvm')" ]; then
    DIR="$(dirname "$(readlink -f "$0")")"
fi 
/usr/bin/env node --experimental-specifier-resolution=node $DIR/../lib/cmd-db.js $@

The explanation of the ''DIR'' part is that the current directory (inside my Node project) is lost when the command is invoke as a symlink (see below). I need to point to the JS file in a relative way, so I then need the directory. Here's background how to find the script dir.

To install this, as a global command, I run:

$ npm link

Then a global symlink appears:

$ knemm 
... command outputs, all ESM imports are resolved! 

(At some point I needed to manually remove those symlinks generated by npm link as it wouldn't change those, when I edited package.json.)

Discussion

This has taken some hours (over some weeks) to work out, and I write this to summarize the effort and the learning. Partly so I remember better and then maybe it helps someone.

I hope all rough parts of using modern JS under Node will be gradually smoothened / paved out.

Of course, the last part of my solution is Linux/Bash centric. But nowadays, with WSL/WSL2, also anyone on Windows can access a good Linux environment. So I do not see a downside with that (that's how all of this was developed).

Comments

[Stargator]

I have a TypeScript index.ts file and it compiles down to index.js in a dist directory. But I notice when an external script require() that library, it seems to be trying to run the TypeScript file and not the compiled file. Which causes an issue.

TypeScript and Node is a mess

[Pete Anderson]

You can pass multiple arguments to a shebang. Just pass -S (or the more explicit --split-string) to the env command:

#!/usr/bin/env -S node --experimental-specifier-resolution=node

import {invert} from './utils';
// ...

[Tam CARRE]

YOU SAVED ME!! Thank you.