Why NodeJS? Demystifying threads, blocking, synchronicity, and callbacks.

Confusing statement #1: NodeJS is Javascript code run on a single thread.

Wait a minute, in this multi CPU core world, why would I ever want to use that? What makes NodeJS so popular?

Threads

NodeJS 19.6 released in Feb 2023 introduces worker threads that enable you to execute Javascript in parallel. BUT, that was only very recently. How could NodeJS have gotten so popular without this feature?

First some definitions.

Process - any program running on your operating system. Can execute one task at a time. Cannot access memory of other running programs.
Thread - Like processes, can execute one task at a time, but they have shared memory. Threads can communicate with other threads.

NodeJS has hidden threads: Under the hood Node uses the lubuv library which gives 4 extra threads to a NodeJS process. These are used with I/O operations such as reading a file or network requests. Node's V8 engine (written in C++) provides two more threads for handling garbage collection.

So to be more complete: NodeJS is Javascript code that runs your application on a single thread with 6 other helper threads available under the hood.

Synchronicity

Confusing statement #2:
NodeJS is a single-threaded non-blocking asynchronous concurrent language

Synchronous means coordinated in time. Asynchronous is the opposite. So in a programming world, synchronous means each line of code runs until it completes, then moves on to the next line.
Coding and debugging this way is very coordinated and predictable and probably how you learned to code.

Normally, programming languages are synchronous and some provide a way to manage asynchronicity in the language or through libraries. C, Java, C#, PHP, Go, Ruby, Swift, and Python are all synchronous by default. Some of them handle async operations by using threads, spawning a new process.

https://nodejs.dev/en/learn/javascript-asynchronous-programming-and-callbacks/#:~:text=C%2C%20Java%2C%20C%23%2C%20PHP,threads%2C%20spawning%20a%20new%20process

Most languages have the ability to run asynchronous and synchronous code, but the key here is that time consuming operations in NodeJS such as I/O events and I/O callbacks are asynchronous by default. This is what "asynchronous concurrent language means" in confusing statement #2.

Asynchronous code is running and completing independently and is thus uncoordinated. As you have probably learned if you have done much with asynchronous code, it is MUCH harder to debug than synchronous code!

Blocking

Blocking code is simply code that is slow. Obviously slow depends on the context. In the context of a desired Web UI refresh rate of ~ 16ms, typically something on the order of milliseconds is probably slow. "non-blocking" in the confusing statement #2 means exactly this. NodeJS has made blocking I/O functionality part of a default thread handled externally from the Node V8 engine allowing it to keep the web UI from getting sluggish.

Node JS motivations & history

Maybe we were doing I/O wrong and that maybe if we did everything in a non-blocking way, that we would solve a lot of the difficulties with programming. Like, perhaps we could forget about threads entirely and only use process abstractions and serialized communications."

Ryan Dahl - Creator of NodeJS.
https://mappingthejourney.com/single-post/2017/08/31/episode-8-interview-with-ryan-dahl-creator-of-nodejs/

He did later say:

And you can think through what you’re doing in many situations more easily if it’s blocking.

This is all motivated to allow web UIs to always be responsive to user input and not get blocked by anything such as database calls or loading events. Your web UI would like to be able to repaint the screen every 16.6ms. If it can't, and your data is changing, you may start to notice sluggish behavior.

How does this all work?

Excellent video that explains this: https://blog.risingstack.com/node-hero-async-programming-in-node-js/
Note: there are some great visuals and animations done in this link but the diagrams below are ones I made to make sure I understand it for myself.

Node V8 engine

Heap - memory allocations
Call Stack - one thread, one thing at a time, synchronous code execution

Provided by browser or C++ APIs
WebAPIs/C++ APIs (backend) - piece of the application that handles asynchronous code. This is WebAPIs on browser or C++ APIs on backend that the Javascript runtime sends to process its asynchronous requests. Asynchronous requests are submitted with synchronous code to be executed once completed.
Callback queue - receives synchronous callbacks from completed WebAPI async code that are now ready to execute. These sync callbacks are stored in order so the event loop can properly handle them.
Event Loop - Inserts synchronous callback code that is now ready to be executed since its associated async code has now completed. It does this by waiting for the stack to be empty and then pushing the next ready synchronous callback.

What does this all mean practically?

Ok, that's helpful.. kind of. But how do I know what code is asynchronous and what code is synchronous? And how should that affect how I code?

Is every line of NodeJS asynchronous? No.

Are all NodeJS functions with callbacks asynchronous? No.

Just because a function has a callback, doesn't mean it is being processed by the WebAPIs and has to go through the callback queue and event loop to be executed.

At this point, you may just be ready to give up. But here is the so what that has been a constant pain in my side since I have been coding in NodeJS.

If you treat an asynchronous callback like it is synchronous it will have potentially hazardous consequences.

Here is a basic synchronous example that works as expected.

function doWorkFirst(){
  console.log("do work first");
}
function doWorkSecond(){
  console.log("do work second")
}
doWorkFirst()
doWorkSecond()

//prints: do work first; do work second

And this is how you break it by handling async code as synchronous.

function doWorkFirst(){
  setTimeout(function(){
    console.log("do work first");
},0)

function doWorkSecond(){
  console.log("do work second")
}
doWorkFirst()
doWorkSecond()

//prints: do work second; do work first

Even with a value of 0 set in timeout's second parameter (setTimeout(fcn,timout_ms)) the doWorkFirst still completes second because setTimeout is an asynchronous callback. The v8 engine simply takes the setTimeout function and delegates it to the WebAPI and keeps moving. The WebAPI processes the setTimeout, and then passes the synchronous callback code to the callback queue, then the event loop, and finally back to the call stack. At this point the doWorkSecond has already completed and printed. So doWorkFirst prints second.

How would you fix this?

function doWorkFirst(){
    return new Promise(function(resolve,reject){
        setTimeout(function(){
            console.log("do work first")
            resolve();
       })
    })
}

function doWorkSecond(){
    console.log("doing work second")
}

doWorkFirst().then(function(){
    doWorkSecond()
})

//prints: do work first; do work second

I'm not going to go through a detailed explanation of promises here or how to properly do error handling (using reject and catch), but simply state: Promises are a good way to handle asynchronous code in Javascript.

Basically when the asynchronous code setTimeout completes, the .then executes whatever is inside it. In this case the doWorkSecond() is inside the .then and now doWorkSecond executes second as desired.

So how do we know what is asynchronous and what is synchronous? Unfortunately the answer is: It depends - read the documentation for everything you use for the answer.

Well that isn’t very helpful…. Ok, lets go over the common items.

Examples of Asynchronous:

• setTimeout
• setInterval
• database calls
• HTTP request
• WebSocket
• fs.readFile
• Function that returns a Promise

Examples of synchronous (most everything else):

• console.log
• functions
• object definitions
• variable assignments
• .forEach
• while loop
• for loop
• fs.readFileSync
• most if not all of the math library
• Number()
• .toString()
• ...

Hopefully this is a helpful starting list.

Next Steps

If you are totally new to all of this, you still don't really know how to write asynchronous code and will need to learn more about Promises and eventually the async keyword. I haven't ever really found any great reference that explains it all for me well. This link is probably a decent start https://exploringjs.com/es6/ch_promises.html

Two major things related to Promises that I want to point out now before I make an article on any of it:

• Promises start executing the instant you define them. If you want to put promises in an array to be executed later, you instead need to use an array of factory functions that all return a promise when executed. Here is a promiseFactory function:

function promiseFactory(delay_ms){
    return new Promise(function(){
        setTimeout(function(){
            console.log("I am delaying " + delay_ms + "ms");
        },delay_ms)
    })
}

• Eventually you will want to make sequential asynchronous code (or do sequential promises). In other words, if you have a series of tasks that are all asynchronous, you may want them to start one after the other. I want to write an article on this too, but this is a starting point: https://css-tricks.com/why-using-reduce-to-sequentially-resolve-promises-works/