Scaling a simple Node.js + Express.js application using node.js modules

Hello DEVs,

This tutorial is going to be about scaling a Node.js + Express.js application.

We will use the very basic express configuration of one-file server logic, and we will scale the application by cloning it, using the native node.js modules 'cluster' and 'process', aswell as we will create a little CLI, so we can interact with our workers(processes/cloned apps).

I hope you are ready, beacause we're just getting started!

So, let's create a new directory, call it testNodeApp or something like that.
We will run

npm init

and then

npm install express

This is the basic app.js file:

const express = require('express');
const app = express();


app.get('/', (request, response, nextHandler) => {
  response.send('Hello node!');
  console.log(`Served by worker with process id (PID) ${process.pid}.`);
});

const server = require('http').createServer(app);

server.on('listening', () => {
  console.log("App listening on port 3000");
})
server.listen(3000);

You can run it with

node ./app.js

, and if you do, you should get an output like:

App listening or port 3000

And when you navigate to http://localhost:3000, or just do

curl localhost:3000/

you should see "Hello node!" as a response. Check you console for the important output - something like:

Served by worker with process id (PID) XXXX.

Where xxxx is the process id.

The next thing we are going to do is to create a cluster.js file in the same directory.

cluster.js - INITIAL

const cluster = require('cluster');
const os = require('os');

if (cluster.isMaster) {
    // Take advantage of multiple CPUs
    const cpus = os.cpus().length;

    console.log(`Taking advantage of ${cpus} CPUs`)
    for (let i = 0; i < cpus; i++) {
        cluster.fork();
    }
    // set console's directory so we can see output from workers
    console.dir(cluster.workers, {depth: 0});

    // initialize our CLI 
    process.stdin.on('data', (data) => {
        initControlCommands(data);
    })

    cluster.on('exit', (worker, code) => {
        // Good exit code is 0 :))
        // exitedAfterDisconnect ensures that it is not killed by master cluster or manually
        // if we kill it via .kill or .disconnect it will be set to true
        // \x1b[XXm represents a color, and [0m represent the end of this 
        //color in the console ( 0m sets it to white again )
        if (code !== 0 && !worker.exitedAfterDisconnect) {
            console.log(`\x1b[34mWorker ${worker.process.pid} crashed.\nStarting a new worker...\n\x1b[0m`);
            const nw = cluster.fork();
            console.log(`\x1b[32mWorker ${nw.process.pid} will replace him \x1b[0m`);
        }
    });

    console.log(`Master PID: ${process.pid}`)
} else {
    // how funny, this is all needed for workers to start
     require('./app.js');
}

So, what we do here is just import the os and the cluster modules, get the number of cpus and start workers with amount equal to the cpus count - we want the maximum.

The next thing, we set up a if-else condition - workers live in the ELSE block, as require('./file') will execute the file if used like this.

In the IF block, we will write down our logic for the master worker.

cluster.fork() starts the child process in the ELSE

To initialize our CLI, we need to listen for user input. This input is the standart input of the process, or stdin. We can access it via:

process.stdin.on("event", handlerFunc); 

Because we are in the master worker.

Something very important to note is that the master worker is not a worker, but a controller - he will not serve requests, but give requests to workers.Requests should be randomly distributed across workers.You can check this by making a benchmark test - if you are under a Linux system, you probably have apache benchmark (ab). Open your terminal and write:

ab -c200 -t10 http://localhost:3000/

This will execute 200 concurrent requests for 10 seconds.
Try it with both 1 worker, and many workers - you will see the difference.

Next, here:

cluster.on('exit', (worker, code) => {
        // Good exit code is 0 :))
        // exitedAfterDisconnect ensures that it is not killed by master cluster or manually
        // if we kill it via .kill or .disconnect it will be set to true
        // \x1b[XXm represents a color, and [0m represent the end of this 
        //color in the console ( 0m sets it to white again )
        if (code !== 0 && !worker.exitedAfterDisconnect) {
            console.log(`\x1b[34mWorker ${worker.process.pid} crashed.\nStarting a new worker...\n\x1b[0m`);
            const nw = cluster.fork();
            console.log(`\x1b[32mWorker ${nw.process.pid} will replace him \x1b[0m`);
        }
    });

We will restart our workers if any worker crashes. You can experiment with this and add those lines in app.js (at the end) :

setTimeout(()=>{
   process.exit(1);
}, Math.random()*10000);

This will kill a process at random time interval.

when you execute

node cluster.js

, you should start recieving input like:

Taking advantage of 8 CPUs
{
  '1': [Worker],
  '2': [Worker],
  '3': [Worker],
  '4': [Worker],
  '5': [Worker],
  '6': [Worker],
  '7': [Worker],
  '8': [Worker]
}
Master PID: 17780
App listening on port 3000
App listening on port 3000
App listening on port 3000
App listening on port 3000
App listening on port 3000
App listening on port 3000
App listening on port 3000
App listening on port 3000
Worker 17788 crashed.
Starting a new worker...
Worker 17846 will replace him
App listening on port 3000
Worker 17794 crashed.
Starting a new worker...

Worker 17856 will replace him 
Worker 17804 crashed.
Starting a new worker...

Worker 17864 will replace him
App listening on port 3000
App listening on port 3000

Note that everything here is async, so you wont get a really ordered output. I stronly advise you to delete the

setTimeout(...)

in app.js from now on.

Now, we are going to continue with the very CLI itself.You should have noticed that we are actually calling an undefined function then we listen to stdin, so we will now create this function.

const initControlCommands = (dataAsBuffer) => {
    let wcounter = 0; // initialize workers counter
    const data = dataAsBuffer.toString().trim(); // cleanse input
    // list workers command
    if (data === 'lsw') { 
        Object.values(cluster.workers).forEach(worker => {
            wcounter++;
            console.log(`\x1b[32mALIVE: Worker with  PID: ${worker.process.pid}\x1b[0m`)
        })
        console.log(`\x1b[32mTotal of ${wcounter} living workers.\x1b[0m`)
    }
    // -help command
    if (data === "-help") {
        console.log('lsw -> list workers\nkill :pid -> kill worker\nrestart :pid -> restart worker\ncw ->create worker')
    }
    /// create worker command
    if (data === "cw") {
        const newWorker = cluster.fork();
        console.log(`Created new worker with PID ${newWorker.process.pid}`)
        return;
    }
    // here are the commands that have an argument - kill and restart
    const commandArray = data.split(' ');
    // assign the actual command on variable
    let command = commandArray[0];
    if (command === "kill") {
        // find the corresponding worker
        const filteredArr = Object.values(cluster.workers).filter((worker) => worker.process.pid === parseInt(commandArray[1]));
       // check if found
        if (filteredArr.length === 1) {
        // kill it
            filteredArr[0].kill("SIGTERM"); // emit a signal so the master //knows we killed it manually, so he will not restart it
            console.log(`\x1b[31mKilled worker ${filteredArr[0].process.pid} .\x1b[0m`);
        } else {
       // Display a friendly error message on bad PID entered
            console.log(`\x1b[31mWorker with PID ${commandArray[1]} does not found. Are you sure this is the PID?\x1b[0m`)
        }
    }
    // this command is quite like the kill, i think the explanation would 
    // be quite the same
    if (command === "restart") {
        const filteredArr = Object.values(cluster.workers).filter((worker) => worker.process.pid === parseInt(commandArray[1]));
        if (filteredArr.length === 1) {
            console.log(`\x1b[31mWorker ${filteredArr[0].process.pid} restarting\x1b[0m`)
            filteredArr[0].disconnect(); // this should be used to kill a process manually
            const nw = cluster.fork()
            console.log(`\x1b[32mWorker is up with new PID ${nw.process.pid}.\x1b[0m`)

        } else {
            console.log(`\x1b[31mWorker with PID ${commandArray[1]} does not found. Are you sure this is the PID?\x1b[0m`)
        }
    }
}

You can now use the CLI to view your workers (lsw), create workers (cw) and kill workers.
Remember, you can always use the -help command!

I hope you found this tutorial helpful and inspiring, as Node.js is great tech, and it is quite begginer friendly.Play around with the cli, explore the edge-cases and have fun!

Until next time,
Yoan