DEV Community: Altamash Ali

Learning Javascript Promise Patterns

Altamash Ali — Thu, 09 Jun 2022 05:19:20 +0000

Hello There!

Let's learn some advance Javascript promise techniques to write more efficient and reliable JS Code :)

1. Fetching multiple independent resources

consider you have function that fetches a post by ID.

const getPostById = async (id) => {
   try {
      // make an async call to fetch the post
      const post = await loadPost(id)
      ...
      return post;
   } catch(err) {
      // handle error
   }
}

Now if we have to fetch details of multiple posts:

const postIds = [1, 2, 3, 4, ...]

We can do something like this:

const getPosts = (postIds) => {
  const posts = [];
  for(const id of postIds){
    const post = await getPostById(id);
    // do processing
    ...
    posts.push(post);
  }
  return posts;
}

Wait a minute! There is a problem here. The problem is await keyword will pause the loop until it gets a response from getPostById(). Fetching each post by Id is an independent operation and result of multiple requests doesn't depend on each other's response. It doesn't make much sense to wait to fetch next post only after previous post has been fetched.

Let's talk how to resolve this issue. What we can do is make multiple requests concurrently and wait for all of them to get fetched or resolved.

Javascript provides two promise APIs to handle multiple requests concurrently:

Promise.all(...) and Promise.allSettled(...)

Using Promise.all(...)

const getPosts = (postIds) => {
  try {
    const postPromises = postIds.map(id => getPostById(id));
    const posts = await Promise.all(postPromises);
   // do processing
   ...
    return posts;
  } catch(err) {
     // handle error
  }
}

Now, good thing is we are not waiting for previous post request to finish to make request for next one instead now concurrent requests will be fired independent of each other and we are waiting until all posts has been fetched. But there is still one issue here. If one of the promises rejects, Promise.all(...) immediately rejects, causing every other post not to load. We can improvise it by using Promise.allSettled(...).

Promise.allSettled(...) returns a pending promise that resolves when all of the given promises have been settled either resolved or rejected. This behaviour is very useful to track multiple tasks that are not dependent on one another to complete.

const getPosts = (postIds) => {
  const postPromises = postIds.map(id => getPostById(id));
  const posts = await Promise.allSettled(postPromises);
    // outcome of each promise has a status property.
    // If success, it will have value property
    // If fails, it will have reason property
    return posts.reduce((result, post) => {
      if(post.status === 'fulfilled') {
        result.successfullyFetchedPosts.push(post.value)
      } else {
        result.failedPosts.push(post.reason)
     }
      return result;
    }, {
     successfullyFetchedPosts: [],
     failedPosts: [],
    })
}

// using this function
const {
  successfullyFetchedPosts, 
  failedPosts
} = await getPosts([...]);
...

Promise returned by Promise.allSettled(...) will almost always be fulfilled. The promise will only reject if we pass a value that is not iterable.

2. Avoiding single point of failure using Promise.any(...)

Sometimes, we have to fetch some critical resource like financial market data from external APIs. If the API is down, the app will stop working. The Promise.any(...) is extremely useful in this regard. It enables us to request data from multiple sources ( APIs) and use the result of the first successful promise.

Promise.any(...) returns a pending promise that resolves asynchronously as soon as one of the promises in the given iterable fulfils.

 const promises = [
    Promise.reject(),
    Promise.resolve(5),
    Promise.reject(),
 ];
 Promise.any(promises).then(console.log) // 5

Consider we have three APIs to fetch a resource. We can use Promise.any(...) like this:

const apis = [
  'https://api1/resource/10',
  'https://api2/resource/10',
  'https://api3/resource/10'
];

const fetchData = async api => {
  const response = await fetch(api);
  return response.ok ? response.json() : Promise.reject('some error');
}

const getResource = () => Promise.any(
  apis.map(api => fetchData(api))
);

getResource().then(response => {
  // process response
})

Promise.any(...) allows you to improve the performance of critical applications by using the data from the API that responds first. Also it allows you to improve reliability of the application as even if one of the APIs fails, it will continue working as expected. Promise.any(...) will only reject when all the promises passed as the argument reject.

3. Enforcing a time limit for async operations using Promise.race(...)

Suppose we are fetching some resource from an external API. User Interface will be in the loading state until we get response from the API. Sometimes, APIs take a lot of time to give response back to the client and user will be waiting for it , looking at the loading spinner like forever. A better user experience would be to timeout the request after a given number of milliseconds and show error in the UI like request time out. We can easily do this using Promise.race(...).

Promise.race(...) is somewhat similar to Promise.any(...) as both get settled whenever first promise in the iterable settles.

Promise.race(...) settles as soon as one of the promises rejects. .

Promise.any(...) resolves as soon as one of the promises fulfils.

Promise.race(...) rejects if the first promise that settles is rejected while Promise.any(...) rejects when all the given promises reject.

let's implement timeout of the request:

 const getData = async () => {
    const TIMEOUT_MS = 2000; // IN MILLISECONDS

    const request = fetch(API_URL); // actual request

    const timeout = new Promise((_, reject) => 
      setTimeout(() => reject(new Error('Request time out!')), TIMEOUT_MS)); // rejects after 2000 milliseconds

    return Promise.race([request, timeout];
 }

4. Batching async requests using Promise.race(...)

An interesting use case of Promise.race(...) is to batch async request. Here is the simple implementation:

/**
 * 
 * @param {{limit: number, concurrentBatches: number}} batchOptions 
 */
const performRequestBatching = async batchOptions => {

    const query = {
        offset: 0,
        limit: batchOptions.limit
    };
    let batch = [];
    let promises = [];
    do {
        batch = await model.findAll(query);
        query.offset += batchOptions.limit;
        if (batch.length) {
            const promise = performLongRequestForBatch(batch).then(() => {
                // remove the promise from promises list once it is resolved
                promises = promises.filter(p => p !== promise);
            });
            promises.push(promise);
            // if promises length is greater than provided max concurrent batches
            if (promise.length >= batchOptions.concurrentBatches) {
                // then wait for any promise to get resolved
                await Promise.race(promises);
            }
        }
    } while (batch.length)
    // wait for remaining batches to finish
    return Promise.all(promises);
}

// using batching
batchRequest({
  limit: 100,
  concurrentBatches: 5,
})

Conclusion

Hurray! That's it. We have learnt multiple patterns related to Javascript promise api like handling multiple request efficiently using Promise.all and Promise.allSettled, avoiding SPOF using Promise.any, timing out async request for better user experience and batching multiple requests using Promise.race.

Please like the post if you have learnt something new :). Also Feel free to point-out or provide suggestions in the comment section if there is some mistake in the post.

Happy Coding !

See you!

Node.js Event Loop In Action

Altamash Ali — Sun, 14 Nov 2021 07:20:22 +0000

In my last post(Deep dive into Node.js Architecture), we learnt about the internals of Node.js and how multiple asynchronous operations work efficiently in a single-threaded environment. We also talked about how event loop works and facilitates the event-driven architecture of Node.js. I would recommend going through my previous post before reading this one.

In this article, we are going to learn more about event loop and it's different phases along with a code example.

Before we start, you might ask why does a Node.js developer need to know about Event loop. The answer to this is:

event loop is the one that handles all the scheduling of your application code and any misconceptions regarding this can lead to poor performance and buggy code,
and it is a very important interview question if you are applying for Nodejs backend role.

so, Let's start :)

As we discussed earlier, Event loop is nothing but a loop. It just loops over the set of events sent by Synchronous Event Demultiplexer, trigger callbacks and move the application along.

Event Loop Phases

The event loop has several different phases to it and each one of these phases maintains a queue of callbacks that are to be executed. Callbacks are destined for different phases based on how they are used by the application.

Poll

The poll phase executes I/O-related callbacks.
Most of the application code executes in this phase.
Starting point of Node.js application

Check

In this phase, callbacks that are triggered via setImmediate() are executed.

Close

This phase executes callbacks triggered via EventEmitter close events.
For example, when a net.Server TCP server closes, it emits a close events that runs in this phase.

Timers

In this phase, callbacks triggered via setTimeout() and setInterval() are executed.

Pending

Special system events are run in this phase, like when a net.Socket TCP soccer throws an ECONNREFUSED error.

Apart from these, there are two special microtask queues that can have callbacks added to them while a phase is running.

The first microtask queue handles callbacks registered using process.nextTick().
The second microtask queues handles promises that reject or resolve.

Execution Priority and order

Callback in the microtask queues take priority over callbacks in the phase's normal queue.
Callbacks in the next tick microtask queue run before callbacks in the promise microtask queue.
When the application starts running, the event loop is also started and the phases are handled one at a time. Node.js adds callbacks to different queues as appropriate while the application runs
When the event loop gets to a phase, it will run all the callbacks in the phase's queue. Once all the callbacks in a given phase are executed, the event loop then moves on to the next phase.

Let's see one code example:

Output will be : 8, 3, 2, 1, 4, 7, 6, 5

Let's see what is happening behind the scene:

Code execution starts off executing line by line in the poll phase.
First, the fs module is required.
Next, the setImmediate() call is run and its callback is added to the check queue.
Next, the promise resolves, adding callback to the promise microtask queue.
Then, process.nextTick() runs next, adding its callback to the next tick microtask queue.
Next, the fs.readFile() tells Node.js to start reading the file, placing its callback in the poll queue once it is ready.
Finally console.log(8) is called and 8 is printed to the screen.

That's it for the current stack.

Now, the two microtask queues are consulted. The next tick microtask queue is always checked first, and callback 3 is called. Since, there is only one callback in the next tick microtask queue, the promise microtask queue is checked next and callback 2 is executed. That finished the two micro-task queues and the current poll phase is completed.
Now, the event loop enters the check phase. This phase has callback 1 in it, which is then executed. Both the microtask queues are empty at this point, so the check phase ends.
The close phase is checked next but is empty, so the loop continues. The same happens with the timers phase and the pending phase, and the event loop continues back around to the poll phase.

Once it is back in the poll phase, the application doesn't have much else going on, so it basically waits until the file has finished being read. Once that happens, the fs.readFile() callback is run.

The number 4 is immediately printed since it's the first line in the callback.
next, the setTimeout() call is made and callback 5 is added to the timers queue.
The setImmediate() call happens next, adding callback 6 to the check queue.
Finally, the process.nextTick() call is made, adding callback 7 to the next ticket microtask queue.

The poll phase is now finished and the microtask queues are again consulted.

Callback 7 runs from the next tick queue,
the promise queue is consulted and found empty, and the poll phase ends.
Again the event loop enters to the check phase where callback 6 is encountered. The number is printed and microtask queues are determined to be empty and the phase ends.
The close phase is checked again and found empty.
Finally, the timers phase is consulted and callback 5 is executed and prints 5 on the console.
Once that's done, the applications doesn't have any more work to do and it exits.

As we know, Node.js runtime environment is single-threaded. Running too much code in a single stack will stall the event loop and prevent other callbacks from firing. To prevent this event loop starving situation, you can break your CPU-heavy operations up across multiple stacks. For example, if you are processing 1000 data records, you can consider breaking down into 10 batches of 100 records, using setImmediate() at the end of each batch to continue processing the next batch. Another option is forking a new child process and offload processing to it. But never break up such work using process.nextTick(). Doing so will lead to a microtask queue that never empties and your application will be trapped in the same phase forever. The runtime won't throw any error instead it will remain a zombie process that eats through CPU.

That's all about event loop.

I hope you have enjoyed reading this article and found it interesting and useful :)

Thanks and see you later !

Reference

Distributed Systems with Node.js (Book)

Deep dive into Node.js Architecture

Altamash Ali — Sat, 06 Nov 2021 08:01:15 +0000

In this article, we are going to deep dive into Node.js architecture and understand the asynchronous nature of Node.js.

Let's dive into it.

Node.js is a single-threaded, asynchronous, event-driven runtime environment for running Javascript code on the server.

By Single-threaded means Javascript runtime executes only one piece of code( or statement) at any instance of time synchronously. It has only one call stack and one heap memory. But then how does runtime handles multiple asynchronous operations in an efficient way ? Node.js handles it efficiently using its event-driven approach. Don't worry about that now. We will come back to it soon :) .

I/O(input/output) is the slowest among the fundamental operations of a computer. It involves accessing data on the disk, reading and writing a file, waiting for user input, doing a network call, performing some database operation etc. It adds a delay between the moment the request is sent to the device and the moment the operation completes.

In tradition blocking I/O programming, the function call corresponding to an I/O request will block the execution of the thread until the operation completes. So, any web server that is implemented using blocking I/O will not be able to handle multiple connections in the same thread. Solution to this problem is using a separate thread( or process) to handle each concurrent connection.

Most modern operating systems support another mechanism to access resources which is called non-blocking I/O where the system call always return immediately without waiting for the I/O operation to complete. To handle concurrent non-blocking resources in an efficient way, it uses a mechanism called synchronous event demultiplexing or event notification interface. The synchronous event demultiplexing watches multiple resources and returns a new event (or set of events) when a read or write operation executed over one of those resources completes. The advantage here is that the synchronous event demultiplexer is synchronous so it blocks until there are new events to process.

Pseudocode of an algorithm that uses a generic synchronous event demultiplexer to read from two different resources:

Let's see what is happening in the above code snippet:

The resources are added to a data structure ( in our case watchedList), associating each one with a specific operation ( e.g. read)
The demultiplexer is set up with the group of resources to be watched. The call to demultiplexer.watch() is synchronous and blocks until any of the watched resources are ready for read. When this occurs, the event demultiplexer returns from the call and a new set of events is available to be processed.
Each event returned by the event demultiplexer is processed. At this point, the resource associated with each event is guaranteed to be ready to read and to not block during the operation. When all the events are processed, the flow will block again on the event demultiplexer until new events are again available to be processed. This is called the mysterious event loop.

You might notice with this pattern that we can handle several I/O operations inside a single thread. Reason we are talking about demultiplexing as using just a single thread, we can deal with multiple resources.

Multithreaded network applications handle the network load like this:

request ---> spawn a thread
---> wait for database request
----> answer request

so the thread spend most of their time using 0% CPU waiting for the database to return data. While doing so they have had to allocate memory required for a thread which includes a complete separate program stack for each thread etc. Also they would have to start a thread which while is not as expensive as starting a full process is still not exactly cheap.

Since, we spend most of our time using 0% CPU, why not run some code when we are not using CPU ? That way, each request will still get the same amount of CPU time as multithreaded applications but we don't need to start a thread. so this is what happens in a single threaded environment:

request -> make DB req
request -> make DB req
request -> make DB req
DB req complete -> send response
DB req complete -> send response
DB req complete -> send response

We can see that using only one thread doesn't impair our ability to run multiple I/O bound tasks concurrently. The tasks are spread over time, instead of being spread across multiple threads.

Let me now introduce the reactor pattern which is the heart of Node.js.

The main idea behind the reactor pattern is to have a handler associated with each I/O operation. A handler in Node.js is represented by a callback function.The handler will be invoked as soon as an event is produced and processed by the event loop. So, The reactor pattern handles I/O by blocking until new events are available from a set of observed resources and then reacts by dispatching each event to an associated handler.

The structure of the reactor pattern is shown below:

The application generates a new I/O operation and request will be submitted to Event Demultiplexer. The application also specifies a handler, which will be invoked when the operation completes. Submitting a new request to the Event Demultiplexer is a non-blocking operation and it returns control to the application immediately.
When a set of I/O operations completes, the Event Demultiplexer pushes a set of corresponding events into the Event Queue.
After receiving a set of events from Event Demultiplexer, event loop iterates over the items of the Event Queue.
Handler associated with each handler is invoked.
The handler which is part of the application code, gives control back to the Event loop when its execution completes(a).
While the handler execute, it can request new asynchronous operations, which in turn new items are added to the Event Demultiplexer(b).
When all the items in the Event Queue are processed, the Event loop blocks again on the Event Demultiplexer, which then triggers another cycle when a new event is available.

A Node.js application will exit when there are no more pending operations in the event demultiplexer and no more events to be processed inside the event queue.

Each OS has its own interface for the event demultiplexer and each I/O operation can behave quite differently depending on the type of resource, even within the same OS.

To handle these inconsistencies, Node.js core team created a native library called libuv which is written in C++.
Libuv represents the low-level I/O engine of Node.js. It is a higher-level abstraction for the OS event demultiplexer, which make Node.js compatible with all the major operation systems and normalise the non-blocking behaviour of the different types of resource.
It also implements the reactor pattern, thus providing an API for creating event loops, managing event queue, running asynchronous I/O operations and queuing other type of tasks.
Internally libuv maintains a thread pool for managing I/O operations as well as CPU-intensive operations like crypto and zlib. This is a pool of finite size where I/O operations are allowed to happen. If the pool only contains four threads, then only four files can be read at the same time.

The final high level architecture of Nodejs includes:

A set of bindings responsible for wrapping and exposing libuv and other low level functionalities to Javascript.
V8, the Javascript engine originally developed by Google for the Chrome browser. This is one of the reason why Node.js is so fast and efficient.
A core Javascript Library that implements the high-level Node.js API.

Conclusion
Node.js architecture is one of the hot topic for backend interviews. Having a deep understanding of Node.js asynchronous nature is a must for all Node.js devs for writing code efficiently. I really hope you have enjoyed reading this article. I would really recommend Node.js Design patterns book if you want to learn more about Node.js. In the next article, we will talk more about event loop.

References:

Node.js Design Patterns by Mario Casciaro and Luciano Mammino
Stack-overflow definitely :)

See you guys. Bye :)