NodeJS Streams

In Node.js, streams are a way of handling data in chunks or pieces, rather than loading the entire file or data into memory all at once.

A stream is a sequence of data that is read from or written to over time. In other words, data is transferred from one end of the stream to the other in small pieces or chunks. There are two types of streams in Node.js: Readable streams and Writable streams.

A Readable stream is used for reading data, while a Writable stream is used for writing data. Readable streams are created from data sources such as files, network sockets, or HTTP requests, while Writable streams are created for data sinks like files, network sockets, or HTTP responses.

Streams can be used to handle large amounts of data without consuming too much memory, which makes them useful in situations where the entire dataset cannot be loaded into memory at once. For example, when streaming a large video file over the internet, the video can be sent in small chunks using a Readable stream, and the chunks can be displayed by the client application as they are received, without the need to download the entire file first.

Node.js provides a set of built-in modules for working with streams, such as "fs" for working with file streams, and "http" for working with network streams. Streams can also be piped together to allow data to flow from one stream to another, allowing for more efficient data processing.

Overall, streams in Node.js provide a flexible and efficient way of handling large amounts of data, allowing for more efficient and optimized data processing.

Example:-

const fs = require('fs');

// Create a readable stream from a file
const readableStream = fs.createReadStream('input.txt');

// Create a writable stream to a file
const writableStream = fs.createWriteStream('output.txt');

// Pipe the data from the readable stream to the writable stream
readableStream.pipe(writableStream);

// Log a message when the data has been written to the file
writableStream.on('finish', () => {
  console.log('Data has been written to the file');
});

// Log any errors that occur
readableStream.on('error', (err) => {
  console.error(err);
});

writableStream.on('error', (err) => {
  console.error(err);
});

In above example, we create a readable stream from a file called "input.txt" using the fs.createReadStream() method. We also create a writable stream to a file called "output.txt" using the fs.createWriteStream() method.

We then pipe the data from the readable stream to the writable stream using the readableStream.pipe(writableStream) method. This will transfer the data from the input file to the output file in chunks, without loading the entire file into memory at once.

Finally, we use the on('finish', () => {...}) event to log a message when the data has been written to the output file, and the on('error', (err) => {...}) events to log any errors that occur during the data transfer.