Understanding Steams in Nodejs

Adarsh Kumar — Tue, 03 Jan 2023 15:15:22 +0000

NodeJs is known for its asynchronous non-blocking behavior. As a server it comes up with various functionality to handle-data efficiently. Streams is one of those concepts offered by Nodejs that help us to deal with memory and data effectively.

Let me give you an example,

Suppose client has requested 2Gb of data from the Database (Let say video file). In this case, if whole data is send to the server at once it will consume 2Gb of space on the server. If there will be 100Gb of data it is not possible for server to handle huge amount of data.

Here, concept of streams comes in. It enable the server to accept and send data part by part, this will save memory in the server and transfer data efficiently.

In real-life we all have experienced streams mechanism. On Youtube, or any other video streaming platform we have seen

As shown in the picture, the whole video is not loaded from the server at once, instead it loads the video chunk by chunk.

The main advantage of using Streams are :

Less Memory consumption.
Time Efficient

There are 4 types of Streams provided by NodeJs:

Readable
Writable
Duplex
Transform

Let us discuss each of these types in depth,

Readable - stream from which data can be read. (fs.createReadStream()) provided by file system module allow us to read the contents of file.

Writable - stream from which we can write data to the file.(fs.createWriteStream()) allow us to write in the file.

Duplex - it creates both readable and writable stream at once.(net.Socket()) supports two way operations.

Transform - It comprises of three steps :

Creation of read stream
Performing some operations to the data read from the file.(Transform)
Write the transformed data into a writable stream.

The above code example depicts how Readable and Writable Streams are created and used. Prerequisites to understand above example is to know about events and Event Emitter in NodeJs.
Here
You can check out this article for better understanding

This is how Duplex stream is implemented. PassThrough is a basic Duplex stream that act as a tunnel to pipe our Readable and Writable Stream.

It shows a transform function that will be implemented between read and write operations.

Bonus Topic - Pipeline

pipeline is a mechanism that guides the flow of data. It basically gets data from one stream and puts it into another stream.

Implementation of pipeline it take streams as arguments and lastly a callback function to catch the error.

Thats all about NodeJs streams.... Thank You for reading this Blog.

DEV Community: Adarsh Kumar

Understanding Steams in Nodejs

Bonus Topic - Pipeline