Zero delay development & unit testing iterations

If you write js or ts targeting node, and would like to improve your daily life and your productivity, yey 🎉 this article is for you !

TLDR

As devs, our daily life is often a constant back and forth between:

• Writing code
• Running unit tests
• Running the written code "for real"

You may have noticed how joyful it is when this iteration process is fast enough so you can have this instant feedback loop between writing and testing.

This article explains how to leverage Webpack HMR in order to get instant restart/reload/run for your NodeJS server and your Mocha unit tests, whatever your codebase size.

To get a glimpse of what I'm talking about, you can clone this repo and follow "Development" instructions (by the way this is a small OS lib I maintain, I wrote about it here)

What is HMR, and how can it be useful ?

If your project is big enough, you probably experience painful delays... when you save a line of code, It might takes ages (or at least seconds) to:

• reload your unit tests and/or rerun them
• restart your localhost server in order to test this line of code.

That is because in either case, your whole codebase must be reinterpreted by node (and possibly retranspiled if you use Typescript with ts-node), including all your node_module dependencies. That is A LOT of code to reparse and reexecute for a single line of code changed.

That is where HMR (Hot-Module-Reload) come in handy.
I wont go into the details of it (Google is your friend), but in two words, the idea is to keep your server running, and only reload the modules that have changed when you save a code file. No need to restart your localhost sever anymore when writing simple changes.

Neat. How do I setup HMR ?

Webpack is a good way to do this.
It is often seen as a client-side devtool to bundle webapps, but It can also be used to bundle nodejs apps.

In short, you'll have to setup webpack for your nodejs server, enabling HMR. To do this, I can point you at this sample repo. It demonstrates how to setup a simple Typescript+Express server with Webpack HMR + unit testing (read its instructions before cloning).

Here is what happens when you change a line of code when changing your server code:

It just reloaded the changed file (main.ts), but all its dependencies are already in-memory. If you have hundreds of dependencies, and thousands of code files, your new server will be up and running again waaaay faster using this method.

To understand the black magic behind it, check:

• webpack.config.js
• The end of main.ts
• .vscode/launch.json
• The Webpack documentation about HMR

Although it is not that complicated, be aware of the fact that HMR is not pure magic. Try to understand how it works behind the hood, and you'll undersand its constraints (which are mainly due to the fact that the modules that are not reloaded may have static in-memory state which will be persisted between HMR recompilations).

Also make my unit tests fast again, please

If you are serious about developing NodeJS projects (or any kind of project for that matter), you probably write unit tests.

If you use vscode as an IDE, and Mocha as your unit test framework, you may already use mocha test explorer + mocha

In that case, you're lucky 🎉 It so happens that I forked the official mocha test adapter to bring HMR support to your unit tests.

We've been using it for a year in my company, it works quite well, and it is stable - do not hesitate to write an issue here if you have any trouble.
It brung the delay between hitting the ▶ button of a unit test and actually hitting a 🔴 breakpoint from 20 seconds to 0.5 sec 🤯.

Here is how it feels to run a thousand unit tests with it:

Or to debug one:

... And the good news is: It is that fast even if your codebase has millions of lines of code, thousands of dependencies, and you wont experience any delay when changing a line of code ❤

To set it up

If you carefully read the aforementioned sample repository, you might have noticed that it also defines an HMR ready unit testing configuration.

In short, it involves:

• Creating a simple unit test entry point file that will reference all your unit test files (using context.require)
• Creating a webpack bundle which uses this entry file
• Telling mocha-test-adapter to use the bulit bundle as an HMR bundle via .vscode/config.js
• Running the webpack bundler (npm start), then reloading all your tests

Read the instructions of this repo, it should work out of the box !

A tip to yet improve your iteration loop experience

Running a test should now be pretty fast.
However, you still have to click on 🐜 button, then ▶ each time you want to run a test. Which I found to be quite a hassle.

To solve this, in the spirit of my previous article

How to never touch your mouse again 🐭

Olivier Guimbal ・ Aug 18 '20 ・ 3 min read

#productivity #vscode

I recommend using the commands test-explorer.rerun, test-explorer.reload, test-explorer.redebug and test-explorer.run-this-test.

Here is my key bindings for those:

    {
        "key": "ctrl+f5",
        "command": "test-explorer.rerun"
    },
    {
        "key": "ctrl+shift+f5",
        "command": "test-explorer.reload"
    },
    {
        "key": "ctrl+alt+f5",
        "command": "test-explorer.redebug"
    },
    {
        "key": "alt+shift+f5",
        "command": "test-explorer.run-this-test"
    },