DEV Community

Heiker
Heiker

Posted on • Updated on

Create a productive environment for your personal nodejs scripts

There are times where I want to automate a boring task using nodejs and I get really excited, because I get to write code for fun. The thing is that half of that excitement goes out the window the moment I have to npm init and then npm install x, those extra steps make me a bit sad. I've never had to that with bash. I want to be able to skip those steps and still have things that just work. How are we going to do that? With the power of bad practices and a few questionable decisions.

The goal

I don't want to get all fancy and mess around too much with node internals, the only thing I want is to have my favorite utility functions and some dependencies preloaded. That's it. I want to just create a something.js and start writing stuff like this.

format_number(74420.5);
// => 74.420,5
Enter fullscreen mode Exit fullscreen mode

Or even better.

sh('node --version');
// => vXX.YY.Z
Enter fullscreen mode Exit fullscreen mode

Without even touching npm. Let us begin.

Step 1: Polute the global scope

Kids, you should never polute the global scope of a node process. Never. However, since this is just for funsies we are going to do just that. I'm going to trust that you, dear reader, are not going to try this in any "production" environment. Only try this at home.

So, the node cli has a handy flag called --require, with it we can tell it to execute any script or module we want before executing the "main" script. It would be something like this.

node --require /path/to/some/script.js ./my-script.js
Enter fullscreen mode Exit fullscreen mode

Let's start with that. Go to that folder where you have all your side projects (I know you have one) and make a new directory (I called jsenv). Next create a main.js or index.js and put this.

function format_number(number) {
  return new Intl.NumberFormat('de-DE').format(number);
}

global['format_number'] = format_number;
Enter fullscreen mode Exit fullscreen mode

Next, create a script in some random location and try to use format_number.

With everything in place try this.

node --require /path/to/jsenv/main.js /tmp/my-script.js
Enter fullscreen mode Exit fullscreen mode

That should have worked. With this simple step we can now "preload" our favorite utilities. But we can take this further.

Step 2: Get your favorite tool(s)

In the jsenv (or whatever you called it) folder run npm init -y and then install something from npm. For this example I will choose arg, this is a library I use to parse command line arguments. If you're going to create a cli tool you'll need one of those, so might as well "preload" that one too.

On jsenv/main.js add this.

global['cli'] = require('arg'); 
Enter fullscreen mode Exit fullscreen mode

On your script add this.

const args = cli({ '--help': String });
console.log(args);
Enter fullscreen mode Exit fullscreen mode

And for the test.

node --require /path/to/jsenv/main.js \
  /tmp/my-script.js --help me
Enter fullscreen mode Exit fullscreen mode

Isn't it cool? Now we can get things from npm ahead of time and not worry about them again. Which leads us to.

Step 3: Get help from the outside

One of the strengths of bash is that we can call just about any tool we have available on our system by just using their name. I know node can do that too, but it's awkward at best. But there is hope, the library execa has a function (execa.command) that can give us a syntax that is more convenient. Before using it in a script I would like to do some adjustments.

const execa = require('execa');

const shell = (options) => (cmd) => execa.command(cmd, options);
const sh = shell({ stdio: 'inherit' });
sh.quiet = shell();
sh.run = (cmd) => sh.quiet(cmd).then(res => res.stdout);
sh.build = shell;
sh.safe = (cmd) =>
  sh(cmd)
    .then((arg) => arg)
    .catch((arg) => arg);

global['sh'] = sh;
Enter fullscreen mode Exit fullscreen mode

I called the variable shell but it's not really a shell. You can't do fancy stuff with it. It's just suppose to work like this.

sh('some-command --an argument --another one');
Enter fullscreen mode Exit fullscreen mode

You can only call commands with its arguments. If you want to get creative you can still call your shell.

sh('bash -c "# fancy stuff goes here"');
Enter fullscreen mode Exit fullscreen mode

I would recommend against this, since is slower and increases that chances of things getting horribly wrong if you pass dynamic values to it.

sh will print the output of the command to stdout. The variant sh.quiet will not do that. sh.safe will not throw an error on fail. And sh.run will keep the result to itself and then it will return the output as a string.

Step 4: Dependencies on demand

As you might have guessed, "preloading" a bunch of libraries can have a negative impact on the startup times of your script. It would be nice if we could "require" a library without npm install everytime for every script. We can do this with the help of the environment variable known as NODE_PATH. With it we can tell node where it can find our dependencies.

We can test this by going to the jsenv folder and installing some tools.

npm install node-fetch form-data cheerio ramda
Enter fullscreen mode Exit fullscreen mode

May I suggest also puppeteer-core, it's the core logic of puppeteer decoupled from the chromium binary. Chances are you already have chrome or chromium in your system, so there is no need to use the puppeteer package.

Now we need some test code.

const fetch = require('node-fetch');
const html = require('cheerio');

(async function () {
  const response = await fetch('http://example.com');
  const $ = html.load(await response.text());

  console.log($('p').text());
})();
Enter fullscreen mode Exit fullscreen mode

Excuse my iife, I was testing this in an old version of node.

We have our tools and our script, now we need to tell node where it can find our packages.

NODE_PATH=/path/to/jsenv/node_modules/ \
node --require /path/to/jsenv/main.js \
/tmp/my-script.js
Enter fullscreen mode Exit fullscreen mode

Now we are getting deep into "unix shell" territory so this might not work on windows if you use cmd.exe or powershell. But it should work on git-bash.

That command should give us this.

This domain is for use in illustrative examples in documents.
You may use this domain in literature without prior
coordination or asking for permission.More information...
Enter fullscreen mode Exit fullscreen mode

We have gain the ability to call libraries located somewhere else. Now we are free from npm init and npm install. We can start hacking on stuff by just creating a single .js file. But we are missing something.

Step 5: Make it convenient

That node command we need to type is not very nice. So, what we would do now is create a script that would call it for us.

#! /usr/bin/env sh

NODE_PATH=/path/to/jsenv/node_modules/ \
  node --require /path/to/jsenv/main.js "$@"
Enter fullscreen mode Exit fullscreen mode

The final step would be putting this somewhere in your PATH, so you can call it like this.

js /tmp/my-script.js
Enter fullscreen mode Exit fullscreen mode

Or make this.

#! /usr/bin/env js

const args = cli({});
const [num] = args._;
console.log(format_number(num));
Enter fullscreen mode Exit fullscreen mode

Assuming you made it executable, it should be possible for you to do this.

/path/to/my-script 12300.4
Enter fullscreen mode Exit fullscreen mode

Extra step: Enable es modules and top-level await

Recent versions of node will allow you to have that but only on .mjs files or if you have a package.json with the property "type": "module". But there is a problem, node ignores the NODE_PATH env variable when using native es modules. Don't be afraid we can still use them, but we need the package esm to enable them.

First step, get the package.

npm install esm
Enter fullscreen mode Exit fullscreen mode

Create an esm.json file and put this.

{
  "cache": false,
  "await": true
}
Enter fullscreen mode Exit fullscreen mode

Modify the node command.

#! /usr/bin/env sh

export ESM_OPTIONS=/path/to/jsenv/esm.json

NODE_PATH=/path/to/jsenv/node_modules/ \
  node --require esm \
       --require /path/to/jsenv/main.js "$@"
Enter fullscreen mode Exit fullscreen mode

Now this should work just fine.

#! /usr/bin/env js

import fetch from 'node-fetch';
import html from 'cheerio';

const response = await fetch('http://example.com');
const $ = html.load(await response.text());

console.log($('p').text());
Enter fullscreen mode Exit fullscreen mode

Show me all the code

I got you fam. It's here, and with some more bells and whistles. But if you're going to use that main.js you might want to delete a few requires, probably won't need all of that.

quick note: all this code was tested against node v10.23.1, and all the specific versions of dependencies are on this package.json.


Thank you for reading. If you find this article useful and want to support my efforts, buy me a coffee ☕.

buy me a coffee

Top comments (0)