Welcome back to the Event Loop series. So far in this event loop series, we discussed the event loop and its different phases, setImmediates, nextTicks, timers and in the last post, I/O. I believe you have a good understanding of the NodeJS event loop right now. Therefore, let’s talk about some best practices, Dos and Don’ts to get the best results and performance when you are writing Node applications. In the meantime, you can check out the previous articles of the series follows.
Post Series Roadmap
- Event Loop and the Big Picture
- Timers, Immediates and Next Ticks
- Promises, Next-Ticks, and Immediates
- Handling I/O
- Event Loop Best Practices (This article)
- New changes to timers and microtasks in Node v11
Most people fail their first few NodeJS apps merely due to the lack of understanding of the concepts such as the Event Loop, Error handling and asynchrony (I also discussed this in detail in a separate blog post). Now that you understand the event loop well, I believe you might already know and have understood most of the best practices I’m going to cover in this series. Let’s go one by one.
Avoid sync I/O inside repeatedly invoked code blocks
Always try to avoid sync I/O functions (fs.readFileSync, fs.renameSync etc.) inside repeatedly invoked code blocks such as loops and frequently called functions. This can reduce your application’s performance on a considerable scale because each time the synchronous I/O operation is executed, event loop will stay blocked until the completion. One of the safest use cases of these sync functions is to read configuration files during the application bootstrapping time.
Functions should be completely async or completely sync
Your application consists of small components called functions. In a NodeJS application, there will be two types of functions.
- Synchronous Functions — Most of the time returns the outputs using return keyword (e.g,
Mathfunctions,fs.readFileSyncetc.) or uses the Continuation-Passing style to return the results/perform an operation(e.g, Array prototype functions such asmap,filter,reduceetc.). - Asynchronous Functions — Returns the results deferred using a callback or a promise (e.g,
fs.readFile,dns.resolveetc.)
The rule of thumb is, the function you write should be,
- Completely synchronous — Behave synchronously for all the inputs/conditions
- Completely asynchronous — Behave asynchronously for all the inputs/conditions.
If your function is a hybrid of the above two and behaves differently for different inputs/conditions, it may result in unpredictable outcomes of your applications. Let’s see an example,
Now let’s write a small application using the above inconsistent function. For ease-of-read, let’s omit the error handling.
Now, if you run the letsRead function twice one after another, you will get the following output.
file read initiated
file read complete
file read complete
file read initiated
What’s happening here?
If you run letsRead for the first time, myfile.txt is not in the cache. Therefore, an async fs.readFile will be performed to access the file system. In this situation, the inconsistent readFile function behaves asynchronously printing file read initiated line first.
When the letsRead function runs for the second time, myfile.txt is now cached during the first read. Therefore, no need to access the file system and callback is immediately called synchronously. In this case, the inconsistent readFile function behaves synchronously printing file read complete before file read initiated.
When your application is getting complex, these kinds of inconsistent sync-async hybrid function can cause a lot of issues which are extremely hard to debug and fix. Therefore, it’s highly recommended to follow the above *sync or async rule * always.
So, how can we fix the above readFile function. We have two approaches:
Approach 1: Make the readFile function completely synchronous utilizing fs.readFileSync.
Approach 2: Make the readFile function completely asynchronous by invoking the callback asynchronously.
As we saw above, we know that it’s always good to call the async variant of a function inside a repeatedly called function. Therefore, we shouldn’t use the Approach 1 as it will have drastic performance issues. Then how can we implement the Approach 2, how can we invoke the callback asynchronously? It is simple! Let’s use process.nextTick.
process.nextTick will defer the execution of the callback by one phase of the event loop. Now, if you run letsRead function twice one after another, you will get a consistent output as follows:
file read initiated
file read complete
file read initiated
file read complete
You can also use setImmediate to achieve this, but I prefer using process.nextTick because nextTick queue is processed frequently than the immediates queue.
Too many nextTicks
While process.nextTick is very useful in many cases, recursively using process.nextTick can result in I/O starvation. This will enforce Node to execute nextTick callbacks recursively without moving to the I/O phase.
Ancient NodeJS versions (≤0.10) offered a way to set a maximum depth for nextTick callbacks which can be set using process.maxTickDepth. But this was ditched in NodeJS>0.12 with the introduction of setImmediate. Due to this, there is no way currently to limit nextTicks starving I/O indefinitely.
dns.lookup() vs dns.resolve*()
If you have gone through NodeJS docs for dns module, you might have seen that there are two ways to resolve a hostname to an IP address using dns module. They are either using dns.lookup or using one of the DNS resolve functions such as dns.resolve4, dns.resolve6 etc. While these two approaches seem to be the same, there is a clear distinction between them on how they work internally.
dns.lookup function behaves similarly to how ping command resolves a hostname. It calls the getaddrinfo function in operating system’s network API. Unfortunately, this call is not an asynchronous call. Therefore to mimic the async behavior, this call is run on libuv’s threadpool using the uv_getaddrinfo function. This could increase the contention for threads among other tasks that run on the threadpool and could result in a negative impact on the application’s performance. It is also important to revise that libuv threadpool contains only 4 threads by default. Therefore, four parallel dns.lookup calls can entirely occupy the threadpool starving other requests (file I/O, certain crypto functions, possibly more DNS lookups).
In contrast, dns.resolve() and other dns.resolve*() behave in a different way. Here is how dns.resolve* is described in official docs.
These functions are implemented quite differently than dns.lookup(). They do not use getaddrinfo(3) and they always perform a DNS query on the network. This network communication is always done asynchronously and does not use libuv's threadpool.
NodeJS provides the DNS resolving capabilities using a popular dependency called c-ares. This library does not depend on libuv’s threadpool and runs entirely on the network.
dns.resolve does not overload the libuv threadpool. Therefore, it is desirable to use dns.resolve instead of dns.lookup unless there’s a requirement to adhere to configuration files such as /etc/nsswitch.conf, /etc/hosts which are considered during getaddrinfo.
But there’s an even bigger problem!
Let’s say you are using NodeJS to make an HTTP request to www.example.com. First, it will resolve www.example.com into an IP address. Then it will use the resolved IP to set up the TCP connection asynchronously. So, sending an HTTP request is a two-step process.
Currently, Both Node http and https modules internally use dns.lookup to resolve a hostname to IP. During a failure of the DNS provider or a due to a higher network/DNS latency, multiple HTTP requests can easily keep the thread pool out-of-service for other requests. This has been a raised concern about http and https, but is still left as-is at the time of this writing, in order to stick to the native OS behavior. Making things worse, many userland http client modules such as request also use http and https under the hood and are affected by this issue.
If you notice a drastic performance drop in your application in terms of file I/O, crypto or any other threadpool-dependent task, there are few things you can do to improve your application’s performance.
- You can increase the capacity of the threadpool up-to 128 threads by setting UV_THREADPOOL_SIZE environment variable.
- Resolve hostname to IP address using
dns.resolve*function and use IP address directly. The following is an example of the same with request module.
Please note that the following script is unoptimized and merely a how-to reference. There are numerous other factors to consider for a more robust implementation. Also, the following code can be used for Node v8.0.0 onwards only because, lookup option is not available in early tls.connect implementations.
Concerns about the Threadpool
As we have seen throughout the series, libuv’s threadpool is used for many purposes other than file I/O and can be a bottleneck for certain applications. If you think your application seems to slow down in terms of file I/O or crypto operations than usual, consider increasing the threadpool size by setting UV_THREADPOOL_SIZE env variable.
Event loop monitoring
Monitoring the event loop for delays is crucial to prevent disruptions. This can also be leveraged to generate alerts, execute force restarts and scale up the service.
The easiest way to identify an event loop delay is by checking the additional time a timer takes to execute its callback. In simple terms, let’s say we schedule a timer for 500ms, if it took 550ms to execute the timer’s callback, we can deduce the event loop delay to be roughly 50ms. This additional 50ms should account for the time taken to execute events in other phases of the event loop. You don’t need to write the above from scratch, instead, you can use loopbench module which implements the same logic to accomplish the event loop monitoring. Let’s see how you can do this.
Once installed, you can use loopbench in your application with a few simple lines of code.
An interesting use case of this is, you can expose a health check endpoint exposing the above values so that you can integrate your application with an external alerting/monitoring tool.
An example response of the above API endpoint could be similar to the follows:
{
"message": "application is running",
"data": {
"loop_delay": "1.2913 ms",
"loop_delay_limit": "42 ms",
"is_loop_overloaded": false
}
}
With this implementation, you can return a 503 Service unavailable response in your health check API if the loop is overloaded to prevent further overloading. This will also help the load balancers to route the requests to other instances of your application if you have High Availability implemented.
That’s it. With this article, I’m concluding the Event Loop series. And I believe you might have learned some important key concepts of NodeJS by reading this series. If you have plans to upgrade your NodeJS versions to v11 or above, I recommend you read the additional article in this series which describes some important changes introduced to the execution order of timers and microtasks in NodeJS v11.
Medium
Further, If you need to learn how NodeJS works internally more in-depth, I suggest you read my Advanced NodeJS Internals post series. Thanks.
References:
- Designing APIs for asynchrony http://blog.izs.me/post/59142742143/designing-apis-for-asynchrony
- Loopbench https://www.npmjs.com/package/loopbench
- TLS Docs https://nodejs.org/api/tls.html
- NodeJS Design Patterns https://www.packtpub.com/web-development/nodejs-design-patterns-second-edition
Latest comments (0)