DEV Community: Krati Joshi

Redis for Node.js Backend Developers: Caching, Rate Limiting, and More

Krati Joshi — Wed, 12 Aug 2026 17:04:18 +0000

If you're preparing for a Node.js backend interview, Redis is one of those technologies you should understand beyond just knowing that it's "fast."

Redis is commonly used for caching, sessions, rate limiting, counters, Pub/Sub, distributed coordination, and fast data access.

In this article, I'll cover the Redis concepts that are especially important for backend development and interviews.

🚀 What is Redis?

Redis is an in-memory data store that provides several useful data structures and very fast operations.

Instead of primarily reading every value from disk like a traditional relational database, Redis keeps its working dataset in memory.

A common backend architecture looks like:

Client
   ↓
Node.js / Express API
   ↓
Redis
   ↓
Database

Redis can reduce the number of requests that reach the primary database.

⚡ Why is Redis so fast?

The major reason is that Redis is designed around in-memory data access.

Instead of:

Application → Disk-based Database → Response

we can have:

Application → Redis RAM → Response

This makes Redis especially useful when the same data is requested repeatedly.

📦 Redis Data Types

Redis isn't limited to simple key-value strings.

Some important data types are:

Data Type	Common Use
String	Cache, tokens, counters
Hash	User/object data
List	Queues, recent items
Set	Unique values
Sorted Set	Leaderboards/ranking
Stream	Event/message processing

String

SET user:1:name "Krati"
GET user:1:name

Hash

HSET user:1 name "Krati" role "Backend Developer"

Hashes are useful when storing multiple fields belonging to an object.

Set

SADD online_users 101
SADD online_users 102

Sets automatically maintain uniqueness.

Sorted Set

ZADD leaderboard 1000 user101
ZADD leaderboard 1500 user102

Useful for rankings and leaderboards.

⏳ TTL — Time To Live

When Redis is used as a cache, we usually don't want data to remain forever.

We can set an expiration time:

SET user:1 "some-data" EX 300

The key expires after 300 seconds.

Check the remaining time:

TTL user:1

TTL is important because it helps:

Remove old cache data
Control memory usage
Reduce stale data
Automatically clean temporary values

🧠 Cache-Aside Pattern

This is one of the most important Redis concepts for backend interviews.

Suppose we have:

GET /users/101

Instead of querying the database every time:

Request
   ↓
Database
   ↓
Response

we use Redis.

Request
   ↓
Redis
   ↓
Cache HIT → Return data

If Redis doesn't contain the data:

Request
   ↓
Redis
   ↓
CACHE MISS
   ↓
Database
   ↓
Store result in Redis
   ↓
Return response

This is called the Cache-Aside pattern.

The application is responsible for reading and populating the cache.

🎯 Cache Hit vs Cache Miss

Cache Hit

The requested data exists in Redis.

Request
 ↓
Redis
 ↓
HIT
 ↓
Return

The database doesn't need to be queried.

Cache Miss

The requested data isn't present.

Request
 ↓
Redis
 ↓
MISS
 ↓
Database
 ↓
Redis SET
 ↓
Return

A useful metric is:

Cache Hit Ratio =
Cache Hits / (Cache Hits + Cache Misses)

🗑️ Cache Invalidation

One of the hardest problems with caching is keeping cached data consistent with the database.

Suppose:

Database:
User name = Alice

Redis:
User name = Alice

Now the user changes their name:

Database:
Alice → Bob

But Redis still contains:

Alice

Now we have stale cache data.

A common solution is:

UPDATE Database
       ↓
DELETE Redis key

For example:

await db.user.update({
  where: { id },
  data: updateData
});

await redis.del(`user:${id}`);

The next request will cause a cache miss and fetch the latest data from the database.

🟢 Using Redis with Node.js

The official Node.js Redis client is node-redis.

Basic setup:

import { createClient } from "redis";

const redis = createClient({
  url: process.env.REDIS_URL
});

redis.on("error", (err) => {
  console.error("Redis error:", err);
});

await redis.connect();

Set a value:

await redis.set("name", "Krati");

Get it:

const name = await redis.get("name");

Set with TTL:

await redis.set("otp:123", "456789", {
  EX: 300
});

Delete:

await redis.del("otp:123");

🚦 Redis for Rate Limiting

Redis is also very useful for implementing API rate limiting.

Imagine:

POST /login

We want to allow only:

5 attempts / minute

We can maintain a counter in Redis.

Conceptually:

Request
   ↓
Redis INCR
   ↓
Check count
   ↓
Limit exceeded?
   ├── YES → Reject
   └── NO  → Continue

Example:

INCR login:user123
EXPIRE login:user123 60

Redis works well here because operations such as INCR are atomic.

It's also shared across multiple Node.js instances:

             ┌── Node Server 1
Client ──────┼── Node Server 2
             └── Node Server 3
                    ↓
                  Redis

All servers can use the same rate-limit state.

🔐 Redis for Sessions

When an application has multiple servers, storing sessions only in one server's memory can become problematic.

Instead:

                ┌── Server 1
Client ─────────┼── Server 2
                └── Server 3
                       ↓
                     Redis

All servers can access the same session information.

Redis documentation specifically describes Redis as a useful shared session store for stateless application servers.

🔄 Atomic Operations

An operation is atomic when it executes as one indivisible operation from the perspective of other Redis commands.

For example:

INCR counter

Instead of doing:

GET counter
   ↓
counter + 1
   ↓
SET counter

we can use:

INCR counter

This is especially useful for:

Counters
Rate limiting
Concurrent requests
Distributed applications

📢 Redis Pub/Sub

Redis can also provide a publish/subscribe mechanism.

Architecture:

Publisher
    ↓
Redis Channel
    ↓
Subscribers

Example:

PUBLISH notifications "Order created"

Another service can subscribe:

SUBSCRIBE notifications

Useful for:

Notifications
Real-time events
Lightweight service communication

Important interview point

Redis Pub/Sub is not a durable message queue.

If a subscriber is disconnected, it can miss messages.

For durable event processing, Redis Streams are a better concept to study. Redis Streams support persisted entries and consumer groups.

💾 Redis Persistence: RDB vs AOF

Redis is primarily memory-based, but it supports persistence.

Two important mechanisms are:

RDB

RDB = Redis Database snapshot

Redis periodically creates a point-in-time snapshot of the dataset.

Redis
  ↓
Snapshot
  ↓
Disk

Advantages:

Compact
Good for backups
Efficient snapshot-based recovery

Disadvantage:

Changes made after the latest snapshot may be lost after a failure.

AOF

AOF = Append Only File

Instead of only taking periodic snapshots, Redis records write operations in an append-only log.

SET user:1 Alice
INCR counter
DEL user:2
        ↓
       AOF
        ↓
       Disk

When Redis restarts, the recorded operations can be replayed to reconstruct the dataset.

AOF durability depends on its fsync configuration; stronger durability generally comes with more I/O cost.

Quick comparison

RDB	AOF
Snapshot	Write log
Point-in-time	Records changes
Usually more compact	Usually larger
Good for backups	Better durability options
Can lose recent changes	Less data loss depending on fsync

🧹 Redis Eviction

Redis data lives primarily in memory, so memory management is important.

If Redis reaches its configured memory limit, an eviction policy can determine what happens to keys.

Some policies include:

noeviction
allkeys-lru
volatile-lru
allkeys-lfu

Interview question:

What happens when Redis memory is full?

Answer:

Redis applies the configured memory policy. Depending on the policy, it may evict eligible keys or reject writes.

💥 Cache Stampede

Consider a popular API:

1000 requests
      ↓
Same cached key
      ↓
Cache expires
      ↓
1000 CACHE MISS
      ↓
1000 Database queries

The database can suddenly become overloaded.

This is called a cache stampede.

Possible solutions:

Distributed locking
Request coalescing
Background refresh
TTL jitter
Cache warming

🔒 Redis Distributed Lock

Redis can also be used for distributed coordination.

A basic locking concept is:

SET lock:payment 123 NX EX 30

Here:

NX → set only if the key doesn't exist
EX 30 → lock expires after 30 seconds

This can help ensure that multiple application instances don't simultaneously perform the same critical operation.

For production-grade distributed locking, however, you need to consider ownership, expiry, failures, and Redis topology rather than treating a single command as a complete locking solution.

🏗️ Redis in a Real Node.js Architecture

A typical backend might look like:

                     ┌──────────────┐
                     │   Client     │
                     └──────┬───────┘
                            ↓
                     ┌──────────────┐
                     │ Load Balancer│
                     └──────┬───────┘
                            ↓
                ┌───────────┴───────────┐
                ↓                       ↓
         Node.js Server 1       Node.js Server 2
                │                       │
                └───────────┬───────────┘
                            ↓
                         Redis
                            ↓
                         Database

Redis can handle:

Frequently accessed data
Sessions
Rate-limit counters
Temporary tokens
Distributed coordination
Real-time messaging

while the database remains the primary source of durable business data in many architectures.

🎯 Redis Interview Cheat Sheet

Before a Node.js backend interview, remember:

Redis
 ↓
In-memory data store
 ↓
Fast
 ↓
Cache
 ↓
TTL
 ↓
Cache Hit / Miss
 ↓
Cache-Aside
 ↓
Cache Invalidation
 ↓
Rate Limiting
 ↓
Sessions
 ↓
Atomic Operations
 ↓
Pub/Sub
 ↓
Streams
 ↓
RDB / AOF
 ↓
Eviction
 ↓
Cache Stampede
 ↓
Distributed Lock

Questions you should be able to answer

1. Why is Redis fast?

Because it is designed around in-memory data access and efficient data structures.

2. Redis vs PostgreSQL?

Redis is commonly used for fast access, caching, sessions and transient/shared state; PostgreSQL is generally used for durable relational business data.

3. What is Cache-Aside?

Application checks Redis → on miss queries DB → stores result in Redis → returns response.

4. How do you handle stale cache?

Invalidate or update the cache when the underlying database data changes.

5. Why Redis for rate limiting?

Fast shared storage plus atomic operations such as INCR.

6. RDB vs AOF?

RDB = snapshots.
AOF = append-only write log.

7. What happens if Redis goes down?

For cache use cases, the application should ideally fall back to the database where appropriate, with proper timeouts, error handling and monitoring.

💡 Final Takeaway

Redis isn't just a "cache."

For a backend developer, think of Redis as a high-speed shared data layer that can solve several problems:

Caching       → Reduce DB load
TTL           → Automatically expire data
Rate limiting → Control requests
Sessions      → Share session state
Counters      → Atomic increments
Pub/Sub       → Lightweight messaging
Streams       → Durable event processing
Locks         → Distributed coordination
Persistence   → RDB / AOF

If you're preparing for a 2+ YOE Node.js interview, focus less on memorizing commands and more on being able to explain why Redis is used, where it fits in the architecture, cache invalidation, failure scenarios, and the trade-offs involved.

📚 Further Reading

Redis Documentation: https://redis.io/docs/latest/
Redis Session Store: https://redis.io/docs/latest/develop/use-cases/session-store/
Redis Streams: https://redis.io/docs/latest/develop/data-types/streams/
Redis Distributed Locks: https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/

nodejs #redis #backend #javascript #webdevelopment #systemdesign #interview

Full CheatSheet

Postgresql Simplified

Krati Joshi — Mon, 10 Aug 2026 18:15:49 +0000

PostgreSQL Backend Developer Essentials: The Concepts I Learned Beyond CRUD

When working with Node.js backend applications, knowing SQL CRUD operations is only the beginning.

As backend developers, we eventually need to understand transactions, indexes, concurrency, query optimization, JSONB, connection pooling, and how PostgreSQL actually executes our queries.

Today I explored some of the PostgreSQL concepts that matter most for backend development.

🐘 What is PostgreSQL?

PostgreSQL is an open-source object-relational database management system (ORDBMS).

It provides traditional relational database features such as tables, rows, columns, primary keys, foreign keys, and SQL, while also supporting advanced capabilities such as JSONB, arrays, custom types, functions, extensions, and specialized indexes.

A typical Node.js backend flow looks like:

Client
   ↓
Node.js / Express
   ↓
Prisma / PostgreSQL driver
   ↓
PostgreSQL

🔑 Primary Key vs Foreign Key

A primary key uniquely identifies a row.

id UUID PRIMARY KEY

A foreign key creates a relationship between tables.

user_id UUID REFERENCES users(id)

The foreign key also helps maintain referential integrity.

A simple way to remember:

Primary Key → identifies
Foreign Key → relates

🔒 Transactions and ACID

A transaction groups multiple database operations into one logical unit.

For example, a money transfer requires both:

Account A → -₹1000
Account B → +₹1000

If one operation fails, we don't want only half of the transaction to be committed.

That's where ACID comes in:

A → Atomicity
C → Consistency
I → Isolation
D → Durability

Transactions help make database operations reliable.

🔄 MVCC

One PostgreSQL concept I found particularly important is MVCC — Multi-Version Concurrency Control.

PostgreSQL maintains different row versions/snapshots so concurrent transactions can work with consistent views of data.

The goal is to allow reads and writes to happen concurrently with less blocking than a simple locking model would provide.

For a backend developer, the key takeaway is:

MVCC is one of the mechanisms PostgreSQL uses to provide concurrency and transaction isolation.

🧩 JSONB

PostgreSQL supports JSONB for storing semi-structured data.

For example:

CREATE TABLE users (
    id UUID PRIMARY KEY,
    preferences JSONB
);

We could store:

{
  "theme": "dark",
  "language": "en",
  "notifications": true
}

Then query a property:

SELECT preferences->>'theme'
FROM users;

JSONB becomes particularly powerful when combined with appropriate indexing, such as a GIN index.

However, JSONB shouldn't automatically replace relational columns. Frequently queried, constrained, and relational data is often better represented using normal columns.

📊 Indexes

Indexes can significantly improve query performance by providing a faster access path to relevant rows.

For example:

CREATE INDEX idx_users_email
ON users(email);

But indexes aren't free.

They consume storage and add overhead to INSERT, UPDATE, and DELETE operations because the index also needs to be maintained.

So:

Don't create indexes blindly. Create them based on actual query patterns and execution plans.

🔍 EXPLAIN and EXPLAIN ANALYZE

When a query becomes slow, we need to understand how PostgreSQL is executing it.

EXPLAIN
SELECT *
FROM users
WHERE email = 'test@example.com';

EXPLAIN shows the planner's estimated execution plan.

For actual execution information:

EXPLAIN ANALYZE
SELECT *
FROM users
WHERE email = 'test@example.com';

EXPLAIN ANALYZE actually executes the query and reports actual runtime statistics.

For deeper investigation:

EXPLAIN (ANALYZE, BUFFERS)
SELECT ...

This can help identify CPU and I/O-related bottlenecks.

Some important things to look for include:

Sequential Scan
Index Scan
Actual execution time
Estimated vs actual rows
Loops
Buffer hits
Buffer reads

A Sequential Scan isn't automatically bad. If a query needs a large percentage of the table, PostgreSQL may correctly decide that scanning the table is cheaper than using an index.

🧠 CTEs

CTE stands for Common Table Expression.

It allows us to define a named intermediate query:

WITH active_users AS (
    SELECT *
    FROM users
    WHERE status = 'active'
)
SELECT *
FROM active_users;

CTEs can make complex SQL easier to read and can also be useful for recursive queries and multi-step data processing.

📈 Window Functions

Window functions allow calculations across related rows without collapsing the result into one row per group.

For example:

SELECT
    name,
    salary,
    RANK() OVER (ORDER BY salary DESC) AS rank
FROM employees;

Common window functions include:

ROW_NUMBER()
RANK()
DENSE_RANK()
LAG()
LEAD()
SUM() OVER()
AVG() OVER()

This is an important distinction:

GROUP BY
→ reduces/groups rows

Window Function
→ keeps rows + calculates across them

🧱 Other PostgreSQL Features

Some other concepts worth knowing as a backend developer are:

Arrays

PostgreSQL can store arrays directly:

skills TEXT[]

UUID

UUIDs provide globally unique identifiers and can be useful in distributed systems.

Views

A view is a saved query that behaves like a virtual table.

Functions and Procedures

PostgreSQL allows reusable logic to execute inside the database.

Extensions

Extensions add additional functionality to PostgreSQL.

Examples include:

pgcrypto
pg_trgm
PostGIS
citext

🚀 PostgreSQL in a Node.js Backend

A typical architecture can look like:

Client
   ↓
Express API
   ↓
Controller
   ↓
Service
   ↓
Prisma
   ↓
Connection Pool
   ↓
PostgreSQL

Connection pooling is important because creating a new database connection for every request is expensive.

A connection pool allows the application to reuse a controlled number of database connections.

🧪 A Real Query Optimization Example

One practical query I work with filters a large consumer table using conditions such as:

WHERE DIV_CODE IN (...)
  AND BILL_CYC_CD = 'SBM'
  AND CON_STATUS IN (...)
  AND SUPPLY_TYPE NOT BETWEEN 50 AND 59

Instead of immediately creating an index, the better approach is:

EXPLAIN
   ↓
EXPLAIN ANALYZE
   ↓
EXPLAIN (ANALYZE, BUFFERS)
   ↓
Inspect execution plan
   ↓
Check existing indexes
   ↓
Optimize
   ↓
Measure again

This taught me an important backend lesson:

Performance optimization should be measurement-driven, not guess-driven.

🎯 Key Takeaways

The PostgreSQL concepts I consider most important for backend interviews are:

ACID
MVCC
Indexes
EXPLAIN ANALYZE
Connection Pooling
JSONB
JOINs
CTEs
Window Functions
Constraints

Knowing CRUD tells us how to interact with a database.

Understanding transactions, concurrency, indexing, query plans, and connection management helps us understand how to build reliable and performant backend systems.

That's the difference I'm aiming for: not just knowing how to write a query, but understanding what PostgreSQL is doing underneath it.

PostgreSQL #NodeJS #BackendDevelopment #Database #SQL #WebDevelopment #Programming

🚀 Day 14 of My Node.js Learning Journey: Understanding the HTTP Module in Node.js

Krati Joshi — Tue, 04 Aug 2026 11:41:50 +0000

As I continue my Node.js learning journey, today's focus was on one of the most fundamental concepts of backend development—the HTTP Module.

Before learning frameworks like Express.js, it's important to understand how Node.js handles HTTP requests and responses under the hood. Since Express is built on top of the HTTP module, mastering this concept makes debugging and backend development much easier.

🌐 What is HTTP?

HTTP (HyperText Transfer Protocol) is the protocol that enables communication between a client (browser, mobile app, or Postman) and a server.

Every time you visit a website, the following happens:

Browser
   │
HTTP Request
   │
   ▼
Node.js Server
   │
HTTP Response
   ▼
Browser

The client sends a request, the server processes it, and then returns a response.

📦 What is the HTTP Module?

The HTTP module is a built-in Node.js module that allows us to create web servers and handle HTTP requests without installing any external packages.

Importing the module is simple:

const http = require("http");

Since it's a core module, no installation is required.

🛠️ Creating Your First HTTP Server

One of the first things I learned was how easy it is to create a basic HTTP server.

const http = require("http");

const server = http.createServer((req, res) => {
    res.end("Hello, Node.js!");
});

server.listen(3000, () => {
    console.log("Server is running on port 3000");
});

After running this program and opening http://localhost:3000, the browser displays the response sent by the server.

🔍 Understanding `req` and `res`

The callback inside createServer() receives two important objects:

`req` (Request Object)

Contains information sent by the client, such as:

Requested URL
HTTP Method
Headers
Client IP

Some commonly used properties are:

req.url
req.method
req.headers

`res` (Response Object)

Used to send data back to the client.

Some frequently used methods include:

res.write()
res.end()
res.setHeader()
res.writeHead()

One important thing I learned today:

Always call res.end() to complete the response. Without it, the client keeps waiting because the server doesn't know the response has finished.

🧭 Simple Routing Without Express

Even without Express.js, we can create basic routes using conditions.

if (req.url === "/") {
    res.end("Home Page");
} else if (req.url === "/about") {
    res.end("About Page");
} else {
    res.statusCode = 404;
    res.end("Page Not Found");
}

This helped me understand how frameworks like Express simplify routing.

📌 Common HTTP Methods

Every API uses HTTP methods to perform different operations.

Method	Purpose
GET	Retrieve data
POST	Create new data
PUT	Replace existing data
PATCH	Update part of existing data
DELETE	Remove data

These methods form the foundation of REST APIs.

📄 HTTP Status Codes

Servers use status codes to indicate the result of a request.

Some commonly used ones are:

✅ Success

200 — OK
201 — Created

❌ Client Errors

400 — Bad Request
401 — Unauthorized
403 — Forbidden
404 — Not Found

🔥 Server Errors

500 — Internal Server Error

Learning these status codes is essential for backend interviews and API development.

📦 Headers and MIME Types

HTTP headers carry metadata about requests and responses.

One commonly used response header is:

res.setHeader("Content-Type", "application/json");

This tells the browser that the response contains JSON data.

Some common MIME types include:

application/json
text/html
text/plain
image/png
application/pdf

⚡ HTTP Module vs Express.js

One question that often comes up is:

"If Express exists, why learn the HTTP module?"

The answer is simple.

The HTTP module provides the core functionality, while Express builds on top of it by adding features like routing, middleware, and request parsing.

Understanding the HTTP module gives you a much clearer picture of what happens behind the scenes whenever an Express application receives a request.

🎯 Key Takeaways

Today I learned:

HTTP enables communication between clients and servers.
Node.js provides a built-in HTTP module for creating servers.
createServer() creates an HTTP server.
req contains request information.
res is used to send responses.
res.end() is mandatory to complete the response.
Routing can be implemented without Express.
HTTP methods define CRUD operations.
Status codes communicate the outcome of a request.
Express.js is built on top of the HTTP module.

💭 Final Thoughts

Today's session helped me understand what happens behind every API request.

Whenever we use Express.js, call an API, or open a webpage, the HTTP module is working behind the scenes. Learning it first makes backend concepts much easier to understand and gives a stronger foundation for building scalable server-side applications.

Every day of learning brings me one step closer to becoming a better backend developer. Looking forward to exploring more Node.js concepts in the coming days!

If you're also learning Node.js, I'd love to hear your thoughts or discuss backend concepts in the comments. Let's keep learning together! 🚀

#NodeJS #JavaScript #BackendDevelopment #ExpressJS #WebDevelopment #RESTAPI #100DaysOfCode #LearningInPublic #Programming #DevCommunity

🚀 Day 13 of My Node.js Learning Journey: Understanding Buffers in Node.js

Krati Joshi — Mon, 03 Aug 2026 17:07:52 +0000

Today, I learned one of the core concepts behind how Node.js handles binary data—Buffers.

When working with files, images, videos, PDFs, or network communication, Node.js doesn't process everything as normal JavaScript strings. Instead, it uses Buffers to efficiently store and manipulate raw binary data.

🤔 What is a Buffer?

A Buffer is a built-in Node.js class used to temporarily store raw binary data in memory.

Think of it as a temporary container that holds small chunks of data before they're processed or transferred.

Simple definition: A Buffer is a fixed-size memory allocation used to store binary data efficiently.

💡 Why Do We Need Buffers?

JavaScript mainly works with:

Strings
Numbers
Objects

But backend applications often deal with:

📄 PDF files
🖼️ Images
🎥 Videos
🎵 Audio files
🌐 Network packets

Since these are binary data, Node.js uses Buffers to handle them efficiently without converting everything into strings.

⚡ Where Are Buffers Used?

Buffers are used in many backend operations, including:

File System (fs)
Streams
HTTP requests and responses
TCP sockets
WebSockets
Image processing
Video and audio streaming
Encryption and compression

If you're building backend applications, you're already using Buffers—even if you don't realize it.

🧠 Interesting Interview Fact

One thing that surprised me today is that Buffers are not stored inside the V8 JavaScript heap.

Instead, they are allocated in Native (C++) Memory, which helps:

Reduce Garbage Collection overhead
Improve performance
Efficiently manage large binary data

🛠️ Creating Buffers

Create from a String

const buffer = Buffer.from("Hello");

console.log(buffer);

Allocate Safe Memory

const buffer = Buffer.alloc(10);

Fixed size
Filled with zeros
Safe to use

Allocate Fast Memory

const buffer = Buffer.allocUnsafe(10);

Faster allocation
Memory isn't initialized
Should always overwrite before using

🔥 Common Buffer Methods

Some methods I learned today:

Buffer.from()
Buffer.alloc()
Buffer.allocUnsafe()
toString()
write()
copy()
slice()
Buffer.concat()

These methods are frequently used while working with files and streams.

📦 Buffer vs Stream

This was the most important takeaway for me.

Buffer	Stream
Stores binary data temporarily	Transfers data continuously
Fixed-size memory	Chunk-by-chunk data flow
Holds data	Moves data
Used internally by Streams	Uses Buffers internally

Easy way to remember:

Buffer stores. Stream transports.

🎯 Key Takeaways

Buffers store raw binary data
They are fixed-size and mutable
Buffers are stored in Native (C++) Memory
Streams use Buffers internally
They improve performance while working with files and network communication
Essential for backend development and frequently asked in Node.js interviews

📚 What I Learned Today

Today's topic helped me understand what actually happens behind the scenes when Node.js reads a file or receives data over the network.

Instead of loading everything into memory at once, Node.js processes data efficiently using Buffers, making applications faster and more memory-efficient.

Every new Node.js concept makes the backend ecosystem feel more connected. Understanding Buffers also makes it much easier to understand Streams, file handling, and network programming.

Thanks for reading! 🙌

I'm currently documenting my Node.js interview preparation journey by sharing what I learn every day. If you're preparing for backend interviews or starting with Node.js, feel free to follow along.

#NodeJS #JavaScript #BackendDevelopment #WebDevelopment #Programming #100DaysOfCode #Coding #OpenSource #LearningInPublic #DevCommunity

🚀 Day 12 of My Node.js Learning Journey: Mastering Streams & Backpressure 📂⚡

Krati Joshi — Sun, 02 Aug 2026 11:07:22 +0000

One of the biggest strengths of Node.js is its ability to process large amounts of data efficiently. Today, I explored Streams, one of the most important concepts for backend development and a frequently asked topic in Node.js interviews.

💡 What are Streams?

A Stream is a way to process data chunk by chunk instead of loading the entire file into memory.

Instead of reading a 2 GB file at once, Node.js reads small chunks (typically 64 KB for fs.createReadStream()), making applications much more memory-efficient and scalable.

Why use Streams?

✅ Lower memory usage

✅ Faster processing

✅ Better performance

✅ Ideal for large files

✅ Non-blocking data transfer

📦 What is a Chunk?

A chunk is a small piece of data transferred by a stream.

Instead of:

2 GB File
      ↓
Load everything into RAM

Streams work like this:

2 GB File
      ↓
64 KB
      ↓
64 KB
      ↓
64 KB
      ↓
...

This is why Node.js can efficiently handle file uploads, downloads, video streaming, and large datasets.

🌊 Types of Streams

Node.js provides four types of streams:

📖 Readable Stream

Used to read data from a source.

Examples:

fs.createReadStream()
HTTP Request
process.stdin

✍️ Writable Stream

Used to write data to a destination.

Examples:

fs.createWriteStream()
HTTP Response
process.stdout

🔄 Duplex Stream

Can both read and write data.

Examples:

TCP Sockets
Network connections

⚙️ Transform Stream

A special type of Duplex Stream that modifies data while passing it through.

Examples:

Compression (zlib.createGzip())
Encryption
Decryption

🚀 `pipe()` — The Easiest Way to Transfer Data

Instead of manually reading and writing chunks:

const fs = require("fs");

const readStream = fs.createReadStream("input.txt");
const writeStream = fs.createWriteStream("output.txt");

readStream.pipe(writeStream);

Benefits of `pipe()`

Less code
Automatic chunk transfer
Handles flow efficiently
Automatically ends the destination stream
Manages backpressure internally

🌊 Understanding Backpressure

One of the most interesting concepts I learned today was Backpressure.

Imagine:

📥 Producer can generate 100 MB/sec
📤 Consumer can process only 20 MB/sec

Without flow control, data would keep accumulating in memory.

Backpressure solves this by pausing the producer until the consumer catches up, preventing unnecessary memory growth and improving stability.

Node.js handles this automatically when using pipe().

💧 `highWaterMark`

highWaterMark defines the buffer threshold that helps determine when backpressure should start.

Some useful defaults:

fs.createReadStream() → 64 KB
Most Readable Streams → 16 KB
Most Writable Streams → 16 KB

It's important to remember that highWaterMark is a threshold, not a hard memory limit.

🔄 Flowing Mode vs Paused Mode

Flowing Mode

Uses the data event
Data flows automatically
Most commonly used

Paused Mode

Uses the readable event
Data is read manually using stream.read()
Gives more control over reading

📦 Buffer vs Stream

One interview question that helped me understand the difference:

Buffer

Temporary memory
Stores binary data
Holds one chunk at a time

Stream

Transfers data continuously
Processes multiple chunks
Better suited for large files

A simple way to remember it:

Buffer stores data. Stream moves data.

🚀 `pipeline()` vs `pipe()`

Although pipe() is simple and powerful, Node.js provides pipeline() for production-ready applications.

Why use `pipeline()`?

Better error handling
Automatically cleans up streams
Prevents resource leaks
Recommended for production code

🎯 Interview Takeaways

Today's most important interview concepts:

What is a Stream?
Why Streams are better than fs.readFile()
Readable vs Writable Stream
Duplex vs Transform Stream
pipe()
Backpressure
drain event
pause() and resume()
highWaterMark
Flowing vs Paused Mode
Buffer vs Stream
pipeline()

📚 Key Learning

Streams are one of the core reasons why Node.js is highly efficient for backend development. They allow applications to process large files with minimal memory usage while maintaining excellent performance and scalability.

Understanding Streams, Backpressure, and pipe() has given me a much deeper appreciation of how Node.js handles data under the hood.

Looking forward to learning the HTTP module next! 🚀

If you have any interview tips or real-world use cases for Streams, I'd love to hear them in the comments.

NodeJS #JavaScript #Backend #WebDevelopment #Programming #Coding #100DaysOfCode #OpenSource #Developer #Streams #LearningInPublic

🚀 Day 11 of My Node.js Learning Journey: Understanding Events & EventEmitter

Krati Joshi — Sat, 01 Aug 2026 15:30:42 +0000

One of the biggest reasons Node.js is so powerful is its event-driven architecture. Today, I learned how Node.js uses events to build scalable and loosely coupled applications.

💡 What are Events?

An event is an action or occurrence that happens during the execution of a program. Instead of constantly checking whether something has happened, Node.js waits for an event and reacts when it occurs.

Examples:

User logs in
HTTP request reaches the server
File reading completes
Database connection is established
Timer finishes

📌 EventEmitter

Node.js provides the built-in EventEmitter class from the events module to create and handle custom events.

const EventEmitter = require("events");

const emitter = new EventEmitter();

emitter.on("greet", () => {
  console.log("Hello, Developer!");
});

emitter.emit("greet");

Output:

Hello, Developer!

🔥 Common EventEmitter Methods

✅ `on()`

Registers a listener for an event.

emitter.on("login", () => {
  console.log("User Logged In");
});

✅ `emit()`

Triggers an event and executes all registered listeners.

emitter.emit("login");

✅ `once()`

Runs the listener only the first time the event is emitted.

emitter.once("welcome", () => {
  console.log("Welcome!");
});

✅ Passing Data with Events

emitter.on("login", (username) => {
  console.log(`Welcome ${username}`);
});

emitter.emit("login", "Krati");

✅ Multiple Listeners

A single event can notify multiple independent listeners.

emitter.on("orderPlaced", () => console.log("Inventory Updated"));
emitter.on("orderPlaced", () => console.log("Email Sent"));
emitter.on("orderPlaced", () => console.log("Invoice Generated"));

emitter.emit("orderPlaced");

⚠️ The Special `error` Event

The "error" event is treated specially in Node.js. If it is emitted without an error listener, the application may terminate.

emitter.on("error", (err) => {
  console.error(err.message);
});

emitter.emit("error", new Error("Something went wrong"));

Always register an "error" listener when emitting error events.

🌍 Real-World Example

Imagine an e-commerce application:

Order Placed
      │
      ├── Update Inventory
      ├── Send Confirmation Email
      ├── Generate Invoice
      ├── Notify Admin
      └── Reward Loyalty Points

Instead of calling every function manually, the application emits a single orderPlaced event, and different modules respond independently. This makes the system easier to maintain and extend.

🎯 Interview Takeaways

EventEmitter is provided by the built-in events module.
on() registers a listener.
emit() triggers an event.
once() executes only once.
One event can have multiple listeners.
EventEmitter listeners execute synchronously by default.
The "error" event should always have an error listener.
EventEmitter is different from the Event Loop.
Streams, HTTP servers, and the process object all use EventEmitter internally.

📚 What I Learned Today

✔️ What events are in Node.js
✔️ How EventEmitter works
✔️ Difference between on(), emit(), and once()
✔️ Passing data through events
✔️ Handling multiple listeners
✔️ Why the error event is important
✔️ Real-world use cases of EventEmitter
✔️ Why Node.js follows an event-driven architecture

💭 Final Thoughts

Events are one of the core concepts that make Node.js efficient and scalable. By using EventEmitter, different parts of an application can communicate without being tightly connected, making code cleaner, more modular, and easier to maintain.

I'm excited to continue exploring more Node.js concepts in the coming days! 🚀

#NodeJS #JavaScript #BackendDevelopment #WebDevelopment #100DaysOfCode #OpenSource #Coding #EventEmitter #LearningInPublic #DevCommunity

🚀 Day 10 of My Node.js Learning Journey – Mastering the Path & OS Modules

Krati Joshi — Fri, 31 Jul 2026 14:42:50 +0000

One thing I really like about Node.js is that it provides several built-in modules that solve common backend problems without requiring any external packages.

Today I learned two of those modules: Path and OS.

At first, these modules might look “too basic” or even optional—but in reality, they are fundamental building blocks of backend development. We don’t just learn them for interviews; we learn them because they solve real problems that every backend application eventually faces.

🤔 Why do we even need to learn Path & OS modules?

Before jumping into methods, it’s important to understand why these modules matter in the first place.

✅ 1. Because backend apps deal with files and servers constantly

In real-world Node.js applications, we often:

Upload files (images, PDFs, videos)
Store logs
Read configuration files
Serve static assets
Deploy apps on different servers

All of these involve file paths and system-level information.

Without the path module, we would be manually handling strings like:

uploads/images/profile.png

This becomes risky because different operating systems handle paths differently.

✅ 2. Because Node.js runs on different operating systems

Your code might run on:

Windows (development)
Linux (production servers)
macOS (team machines)

Each OS behaves differently in terms of:

File paths
CPU architecture
Memory availability
System uptime

The os module helps us write code that adapts to the environment instead of breaking in production.

✅ 3. Because production systems need reliability

Small mistakes like:

wrong file path
incorrect directory reference
assuming server specs

can cause real production bugs.

These modules help prevent that.

✅ 4. Because interviews expect this understanding

Companies don’t just ask “what is path.join()?”

They want to know:

Do you understand file system behavior?
Do you know cross-platform issues?
Can you write production-safe backend code?

So yes—this is also high-frequency interview knowledge, but more importantly, it’s real engineering knowledge.

📂 The Path Module

The path module helps us work with file and directory paths in a cross-platform way.

Why is this important?

Windows and Linux use different path formats:

Windows

C:\Users\Krati\Documents\resume.pdf

Linux/macOS

/home/krati/Documents/resume.pdf

If we manually concatenate strings, our application may break on different systems. The path module solves this by generating correct paths automatically.

Key Methods I Learned

✅ path.join()

Safely joins multiple path segments.

path.join("uploads", "images", "profile.png");

Best for building folder structures dynamically.

✅ path.resolve()

Converts a relative path into an absolute path.

Useful when a library or API requires a full file location.

✅ path.basename()

Extracts the filename from a full path.

/users/docs/resume.pdf → resume.pdf

✅ path.dirname()

Returns the parent directory of a file.

✅ path.extname()

Returns file extension like .pdf, .jpg, .js.

Very useful for file validation during uploads.

⭐ My Biggest Learning Today

`path.join()` vs `path.resolve()`

Before today, I thought they were interchangeable.

Now I understand:

path.join() → just combines path segments
path.resolve() → builds an absolute path from the current working directory

This difference matters a lot in real backend projects.

📁 Another Important Concept

`__dirname` vs `process.cwd()`

This is a classic Node.js interview question—but also a real-world debugging tool.

__dirname → directory of the current file
process.cwd() → directory where the Node app was started

This difference becomes important when:

loading config files
serving static files
handling uploads

💻 The OS Module

The os module gives us information about the operating system.

Instead of guessing server details, we can directly access them using Node.js.

Useful methods I explored:

os.platform() → OS type (win32, linux, etc.)
os.arch() → CPU architecture
os.hostname() → machine name
os.cpus() → CPU core details
os.totalmem() → total RAM
os.freemem() → available RAM
os.uptime() → system running time

🌍 Real-World Backend Usage

These modules are not theoretical—they are used in production systems daily.

Path module is used for:

File uploads
Static file serving
Configuration management
Cross-platform file handling

OS module is used for:

Health check APIs
Monitoring dashboards
Performance tracking
Scaling decisions (based on CPU cores)
Logging system environment details

🎯 Interview Takeaways

Today’s key concepts:

Why we should never manually build file paths
Difference between path.join() and path.resolve()
Difference between __dirname and process.cwd()
Why OS-level information matters in backend systems
Real-world use cases of Path and OS modules

💡 What I Learned Today

The biggest realization today is this:

Backend development is not just about writing APIs—it’s about understanding the environment your code runs in.

The path module ensures our file handling is safe and cross-platform, while the os module helps our application understand the system it is running on.

These modules may look small, but they are core to building reliable, production-ready backend systems.

Looking forward to continuing my Node.js journey and exploring more modules that power real-world applications! 🚀

#NodeJS #JavaScript #BackendDevelopment #WebDevelopment #100DaysOfCode #LearningInPublic #ExpressJS #SoftwareEngineering #Coding #SystemDesign

🚀 fs Module=> Loaded

Krati Joshi — Thu, 30 Jul 2026 17:16:14 +0000

Today I explored one of the most fundamental Node.js modules—the File System (fs) module. It's the module that enables Node.js applications to interact with files and directories, making it essential for backend development.

💡 Key Takeaways

✅ fs is a built-in module – No installation required.

✅ Three ways to work with files

Callback API
Promise API (fs/promises)
Synchronous API (readFileSync, writeFileSync)

✅ Asynchronous operations are preferred

Don't block the Event Loop.
File operations are handled by libuv's thread pool, allowing the server to continue processing other requests.

✅ Common File Operations

fs.readFile() → Read a file
fs.writeFile() → Create or overwrite a file
fs.appendFile() → Add content to an existing file
fs.rename() → Rename a file
fs.unlink() → Delete a file
fs.mkdir() → Create a directory
fs.readdir() → Read directory contents
fs.stat() → Get file metadata

✅ Always handle errors
Every file operation can fail (missing file, permissions, etc.), so proper error handling is essential.

✅ Use Streams for large files
fs.readFile() loads the entire file into memory, whereas streams process data in chunks, making them much more memory-efficient.

🎯 Interview Nuggets

There is no fs.createFile() method. Use fs.writeFile() to create a new file.
fs.writeFile() overwrites existing content, while fs.appendFile() preserves it and adds new data.
fs.readFile() is asynchronous and doesn't block the Event Loop.
Avoid synchronous (Sync) methods in production request handlers because they block the Event Loop.

📚 Example

const fs = require("fs");

fs.readFile("users.txt", "utf8", (err, data) => {
  if (err) {
    console.error(err);
    return;
  }

  console.log(data);
});

Every backend developer works with files at some point—whether it's reading configuration files, generating reports, processing uploads, or maintaining logs. Understanding the fs module is a foundational step toward building robust Node.js applications.

✅ Complete. On to Path & OS Module next! 🚀

NodeJS #JavaScript #Backend #WebDevelopment #100DaysOfCode #OpenToWork #LearningInPublic #FullStack #SoftwareEngineer #Developers

🚀 Node.js Streams & Buffers Explained: Handle Large Files Without Crashing Your Server

Krati Joshi — Wed, 29 Jul 2026 17:39:42 +0000

If you've ever used fs.readFile() to load a large file, you might have noticed high memory usage or even application crashes.

The reason? fs.readFile() loads the entire file into memory before your application can process it.

Node.js solves this problem with Streams and Buffers.

Let's understand why they are one of the most important concepts for every backend developer.

📌 What is a Stream?

A Stream is a way to process data chunk by chunk instead of loading everything into memory at once.

Think of watching a movie on Netflix. The entire movie isn't downloaded before playback starts—you receive small chunks continuously.

That's exactly how Streams work.

Why use Streams?

✅ Memory Efficient
✅ Better Performance
✅ Handles Large Files
✅ Ideal for Network Communication
✅ Used in File Uploads & Downloads

📦 What is a Buffer?

A Buffer is a temporary memory area that stores raw binary data before it's processed or transmitted.

Whenever Node.js reads data from a file or receives data from a network, it first stores it in a Buffer.

Buffer = Temporary Binary Storage

🌊 Types of Streams

Node.js provides four types of Streams:

1️⃣ Readable Stream

Reads data from a source.

Examples:

fs.createReadStream()
HTTP Request

2️⃣ Writable Stream

Writes data to a destination.

Examples:

fs.createWriteStream()
HTTP Response

3️⃣ Duplex Stream

Supports both reading and writing.

Example:

TCP Socket

4️⃣ Transform Stream

Reads, modifies, and writes data.

Examples:

Compression
Encryption
Decompression

🔄 `pipe()` — Connecting Streams

One of the most useful Stream methods is pipe().

Instead of manually reading chunks and writing them yourself, you can directly connect a Readable Stream to a Writable Stream.

This keeps the code simple and automatically handles data flow efficiently.

🚦 Backpressure

Imagine a fast producer sending data to a slow consumer.

Without flow control, memory usage keeps increasing.

Node.js solves this using Backpressure.

When the Writable Stream becomes slower, Node pauses the Readable Stream until the destination catches up.

Backpressure = Flow Control

💧 What is `highWaterMark`?

highWaterMark defines the maximum amount of data that can be buffered before backpressure is applied.

Think of it as a water level limit.

Once the internal buffer reaches this limit:

Reading pauses
Writable Stream processes buffered data
Reading resumes automatically

📊 Stream Flow

Large File
     │
Readable Stream
     │
   Chunks
     │
Internal Buffer
     │
HighWaterMark Reached
     │
Backpressure
     │
Writable Stream
     │
Output

💡 Key Interview Takeaways

Streams process data chunk by chunk.
Buffers temporarily store binary data.
pipe() connects Readable and Writable Streams.
Backpressure prevents memory overflow.
highWaterMark controls when backpressure starts.
Streams are ideal for large files, uploads, downloads, and video streaming.

🎯 Final Thoughts

Streams and Buffers are at the heart of Node.js performance.

If you're building APIs that handle file uploads, large downloads, logs, media, or network communication, understanding these concepts will help you write scalable and memory-efficient applications.

What real-world use case have you used Streams for? Share it in the comments! 👇

#NodeJS #JavaScript #Backend #WebDevelopment #Programming #Streams #Buffer #Performance #100DaysOfCode

🚀 Understanding Async Patterns in Node.js: From Callbacks to Async/Await

Krati Joshi — Tue, 28 Jul 2026 14:13:02 +0000

JavaScript is single-threaded, but Node.js can efficiently handle thousands of concurrent operations. The secret lies in its asynchronous programming model.

Whether you're reading files, querying databases, calling APIs, or communicating with Redis or Kafka, understanding async patterns is essential for writing scalable Node.js applications.

Let's explore how asynchronous programming in Node.js has evolved over time.

Why Asynchronous Programming?

Imagine your application needs to read a large file.

A synchronous approach blocks the entire program until the file is read.

const data = fs.readFileSync("data.txt", "utf8");
console.log(data);
console.log("Finished");

Nothing else executes until the file operation completes.

Now compare that with an asynchronous approach:

fs.readFile("data.txt", "utf8", (err, data) => {
    console.log(data);
});

console.log("Finished");

Output:

Finished
<File Content>

Instead of waiting, Node.js continues executing other code while the file is being read in the background.

That's the power of asynchronous programming.

The Evolution of Async Patterns

Over time, Node.js introduced better ways to manage asynchronous code.

Callbacks
      ↓
Callback Hell
      ↓
Promises
      ↓
Async/Await

Each step improved readability, maintainability, and error handling.

1. Callbacks

A callback is simply a function passed to another function that executes after an asynchronous task completes.

function greet(callback) {
    console.log("Hello");
    callback();
}

greet(() => {
    console.log("Node.js");
});

Output

Hello
Node.js

Callbacks were the foundation of asynchronous programming in Node.js.

The Problem: Callback Hell

As applications grow, callbacks become deeply nested.

login(() => {
    fetchProfile(() => {
        fetchOrders(() => {
            sendEmail(() => {
                logout(() => {

                });
            });
        });
    });
});

This creates the infamous "Pyramid of Doom."

Problems include:

❌ Hard to read
❌ Difficult to debug
❌ Complex error handling
❌ Poor maintainability

2. Promises

Promises solve callback hell by representing the eventual result of an asynchronous operation.

A Promise has three states:

Pending
Fulfilled
Rejected

Creating a Promise:

const promise = new Promise((resolve, reject) => {

    const success = true;

    if (success) {
        resolve("Success");
    } else {
        reject("Failed");
    }

});

Using a Promise:

promise
.then(result => {
    console.log(result);
})
.catch(error => {
    console.error(error);
});

Promises make asynchronous code much easier to understand.

Promise Chaining

Instead of nesting callbacks, Promises allow chaining.

login()
.then(fetchProfile)
.then(fetchOrders)
.then(sendEmail)
.then(logout)
.catch(console.error);

The execution flow becomes much cleaner.

Useful Promise Methods

`Promise.all()`

Runs multiple asynchronous operations in parallel.

await Promise.all([
    getUser(),
    getOrders(),
    getNotifications()
]);

If one promise rejects, the entire operation fails.

`Promise.allSettled()`

Waits for every promise to finish, regardless of success or failure.

Useful when every result matters.

`Promise.race()`

Returns whichever promise settles first.

Perfect for implementing request timeouts.

`Promise.any()`

Returns the first successfully resolved promise while ignoring rejected ones.

Useful when multiple sources can provide the same data.

3. Async/Await

Async/Await is built on top of Promises and makes asynchronous code look almost synchronous.

async function getUser() {

    const user = await fetchUser();

    console.log(user);

}

Much cleaner than multiple .then() calls.

Error Handling

One of the biggest advantages of Async/Await is cleaner error handling.

async function registerUser() {

    try {

        const user = await createUser();

        console.log(user);

    } catch (error) {

        console.error(error);

    }

}

Instead of attaching .catch() everywhere, a single try...catch block handles failures gracefully.

Sequential vs Parallel Execution

Sequential execution:

await fetchUser();
await fetchOrders();
await fetchProducts();

Each operation waits for the previous one.

Parallel execution:

await Promise.all([
    fetchUser(),
    fetchOrders(),
    fetchProducts()
]);

Independent tasks execute simultaneously, significantly reducing response time.

Rule of thumb: Use Promise.all() whenever tasks don't depend on each other.

Common Mistakes

Forgetting `await`

const user = fetchUser();

console.log(user);

Output:

Promise { <pending> }

Correct:

const user = await fetchUser();

Using `await` Outside an Async Function

Incorrect:

await fetchUser();

Correct:

async function main() {
    await fetchUser();
}

Ignoring Errors

Always handle rejected promises.

try {
    await fetchUser();
} catch (err) {
    console.error(err);
}

Real-World Example

A user registration API typically performs multiple asynchronous operations.

async function registerUser(req, res) {

    try {

        const hashedPassword = await hashPassword(req.body.password);

        const user = await saveUser(hashedPassword);

        await sendWelcomeEmail(user.email);

        res.status(201).json(user);

    } catch (err) {

        res.status(500).json({ error: err.message });

    }

}

This pattern is common in modern Express.js applications.

Key Takeaways

Node.js uses asynchronous programming to avoid blocking the Event Loop.
Callbacks introduced async programming but often led to callback hell.
Promises improved readability and simplified error handling.
Async/Await provides the cleanest syntax for asynchronous code.
Use Promise.all() for independent operations to improve performance.
Always wrap asynchronous code in try...catch.
Choose Async/Await for modern Node.js applications whenever possible.

Final Thoughts

Asynchronous programming is one of the biggest reasons Node.js excels at building scalable backend services.

Mastering callbacks, Promises, and Async/Await isn't just about passing interviews—it's about writing cleaner, faster, and more maintainable production code.

What async pattern do you use most in your projects—Callbacks, Promises, or Async/Await? Let me know in the comments! 🚀

NodeJS #JavaScript #Backend #AsyncProgramming #Promises #AsyncAwait #WebDevelopment #100DaysOfCode #DevCommunity

🚀 Backend Internals #7: libuv — The Real Reason Node.js Is Asynchronous

Krati Joshi — Mon, 27 Jul 2026 18:15:04 +0000

By Krati Joshi

Most developers say:

"Node.js is single-threaded."

That's true—but it's also incomplete.

If JavaScript runs on a single thread, then who reads files?
Who handles thousands of HTTP connections?
Who performs DNS lookups and cryptographic operations without freezing your application?

The answer is libuv.

Understanding libuv is the moment Node.js starts making sense.

Let's dive in.

The Problem

Imagine this code:

const fs = require("fs");

const data = fs.readFileSync("movie.mp4");

console.log("Done");

If movie.mp4 is 5 GB, your application waits until the file is completely read.

Nothing else executes.

This is called blocking I/O.

Now imagine an Express server handling 500 users.

If every request performs blocking file operations, your server quickly becomes unresponsive.

Clearly, Node.js needed a better approach.

Enter libuv

libuv is a cross-platform C library used internally by Node.js.

Its job is to provide:

Asynchronous I/O
Event Loop implementation
Worker Thread Pool
Timers
File System APIs
Networking abstraction
DNS operations

Without libuv, Node.js would not be able to perform asynchronous operations efficiently.

Where Does libuv Fit?

Every Node.js program roughly follows this architecture:

JavaScript Code
        │
        ▼
     V8 Engine
        │
        ▼
Node.js Native C++ Bindings
        │
        ▼
      libuv
   ┌────┴────┐
   │         │
   ▼         ▼
Thread Pool  Operating System
             Async I/O APIs

Each component has a different responsibility.

V8

Executes JavaScript
Performs JIT compilation
Manages memory and garbage collection

Node.js Native Bindings

Expose APIs like:

fs
http
crypto
dns

to JavaScript.

libuv

Coordinates asynchronous work with the operating system.

Why JavaScript Can't Read Files

JavaScript was originally designed for browsers.

It has no built-in ability to:

Read files
Create sockets
Open network connections
Access operating system APIs

When you write:

fs.readFile("notes.txt", callback);

JavaScript doesn't read the file itself.

Instead, the request travels through Node.js into libuv, which performs the work using the operating system or its worker thread pool.

The Flow of `fs.readFile()`

Internally, the request looks like this:

JavaScript

↓

V8 Engine

↓

Node.js Native Bindings

↓

libuv

↓

Worker Thread

↓

Operating System

↓

File Read Complete

↓

Completion Queue

↓

Event Loop

↓

JavaScript Callback

Notice something important:

The JavaScript thread never waits.

It simply continues executing other work until the callback is ready.

That's the foundation of Node.js's non-blocking model.

The Thread Pool

Some operations cannot rely entirely on the operating system's asynchronous APIs.

For these, libuv maintains a Worker Thread Pool.

By default:

4 Worker Threads

Typical operations that use the thread pool include:

File System (fs)
Cryptography (crypto)
Compression (zlib)
Some DNS lookups (dns.lookup())

The pool size can be increased when appropriate:

UV_THREADPOOL_SIZE=8 node app.js

This can improve throughput for workloads dominated by thread-pool operations like hashing or compression.

Not Every Async Operation Uses the Thread Pool

One of the biggest interview misconceptions is:

"All asynchronous work uses the thread pool."

It doesn't.

Uses Thread Pool

fs.readFile()
crypto.pbkdf2()
crypto.scrypt()
zlib
dns.lookup()

Doesn't Use Thread Pool

HTTP requests
TCP sockets
Timers
Most network I/O

Networking is usually handled using efficient operating-system mechanisms, while libuv waits for readiness notifications and schedules callbacks.

Why Can Node.js Handle Thousands of Connections?

Unlike many traditional server models, Node.js does not create one thread per connection.

Instead:

Thousands of Clients

↓

Operating System

↓

libuv Event Loop

↓

JavaScript Callback

Most connections spend their time waiting for network events.

Because JavaScript isn't blocked waiting for I/O, one thread can coordinate a huge number of concurrent connections.

Event Loop vs Thread Pool

These two concepts are often confused.

Event Loop	Thread Pool
Executes callbacks	Performs supported blocking work
Runs on the JavaScript thread	Runs on worker threads
Single-threaded	Multi-threaded
Schedules completed tasks	Executes background operations

The Event Loop doesn't read files.

The Thread Pool doesn't execute JavaScript.

They work together.

Common Interview Questions

Why is Node.js called single-threaded?

Because JavaScript execution happens on a single main thread.

Node.js itself also uses libuv, worker threads, and operating-system asynchronous I/O.

Does every async API use the thread pool?

No.

Only certain APIs such as fs, crypto, zlib, and some DNS operations use it.

Most networking operations rely on the operating system's asynchronous I/O facilities.

Why is libuv written in C?

Because it needs direct access to low-level operating system APIs while providing a consistent interface across Linux, Windows, and macOS.

Key Takeaways

JavaScript execution is single-threaded.
libuv powers Node.js asynchronous behavior.
libuv implements the Event Loop and manages the worker thread pool.
File system, crypto, compression, and some DNS operations use the thread pool.
HTTP requests and most networking do not use the thread pool.
Understanding libuv helps explain why Node.js scales well for I/O-heavy applications.

If you've ever wondered why Node.js can serve thousands of concurrent requests without creating thousands of threads, the answer isn't magic—it's the combination of libuv, the operating system, and the Event Loop working together.

In the next article, we'll explore how the Node.js HTTP module works internally and see what actually happens when a browser sends a request to your server.

🚀 Backend Internals #6: The Node.js Event Loop Explained — Why Async Code Doesn't Block

Krati Joshi — Sun, 26 Jul 2026 08:58:43 +0000

By Krati Joshi

If you've ever wondered how Node.js can handle thousands of requests while running on a single JavaScript thread, the answer is the Event Loop.

Many developers know what the Event Loop is, but understanding how it works can completely change the way you write and debug asynchronous code.

Let's break it down.

🤔 The Problem

JavaScript is single-threaded, which means it can execute only one task at a time.

Imagine reading a large file or waiting for a database query. If JavaScript had to wait for these operations to finish, the entire application would freeze.

That's where Node.js shines.

⚡ How Node.js Solves This

Node.js delegates long-running operations to libuv, which works with the operating system to handle asynchronous tasks.

While those tasks are running, JavaScript continues executing other code.

When an operation completes, its callback is placed in a queue, waiting for the Event Loop to execute it.

JavaScript Code
      │
      ▼
  Call Stack
      │
      ▼
Async Operation (libuv)
      │
      ▼
Callback Queue
      │
      ▼
  Event Loop
      │
      ▼
  Call Stack

The Event Loop constantly checks:

"Is the Call Stack empty?"

If the answer is Yes, it moves the next callback to the Call Stack.

🧠 A Simple Example

console.log("Start");

setTimeout(() => {
  console.log("Timeout");
}, 0);

console.log("End");

Output:

Start
End
Timeout

Even with a delay of 0, the callback cannot execute until the current synchronous code finishes and the Call Stack becomes empty.

🔄 The Event Loop Phases

The Node.js Event Loop consists of several phases:

Timers

Executes setTimeout() and setInterval() callbacks.

Pending Callbacks

Executes certain deferred system callbacks.

Poll

Processes completed I/O operations such as file reads, network requests, and database responses.

Check

Executes callbacks scheduled with setImmediate().

Close Callbacks

Handles cleanup events like socket close callbacks.

The Poll phase is where most asynchronous I/O callbacks are processed, making it one of the most important phases to understand.

⚔️ `process.nextTick()` vs Promise vs Timers

Node.js executes asynchronous callbacks in a specific order.

console.log("Start");

process.nextTick(() => console.log("nextTick"));

Promise.resolve().then(() => console.log("Promise"));

setTimeout(() => console.log("Timeout"), 0);

setImmediate(() => console.log("Immediate"));

console.log("End");

Typical output:

Start
End
nextTick
Promise
Timeout
Immediate

Priority Order

Current Synchronous Code
        ↓
process.nextTick()
        ↓
Promise Microtasks
        ↓
Timers
        ↓
Poll
        ↓
Check (setImmediate)

Understanding this execution order helps explain many "unexpected" behaviors in asynchronous JavaScript.

💡 Key Takeaways

JavaScript is single-threaded.
Node.js remains non-blocking by delegating asynchronous work to libuv.
The Event Loop moves completed callbacks to the Call Stack when it's empty.
process.nextTick() executes before Promise callbacks.
Promise callbacks execute before timer callbacks.
setImmediate() executes during the Check phase.

🎯 Final Thoughts

The Event Loop is the engine behind Node.js's scalability. Once you understand how callbacks, microtasks, and Event Loop phases interact, asynchronous code becomes much easier to reason about.

Mastering this concept is essential for writing efficient Node.js applications and performing well in backend interviews.

What's the most confusing Event Loop behavior you've encountered? Drop it in the comments, and let's discuss! 👇

NodeJS #JavaScript #Backend #EventLoop #AsyncProgramming #WebDevelopment #Coding #Programming #100DaysOfCode

DEV Community: Krati Joshi

Redis for Node.js Backend Developers: Caching, Rate Limiting, and More

🚀 What is Redis?

⚡ Why is Redis so fast?

📦 Redis Data Types

String

Hash

Set

Sorted Set

⏳ TTL — Time To Live

🧠 Cache-Aside Pattern

🎯 Cache Hit vs Cache Miss

Cache Hit

Cache Miss

🗑️ Cache Invalidation

🟢 Using Redis with Node.js

🚦 Redis for Rate Limiting

🔐 Redis for Sessions

🔄 Atomic Operations

📢 Redis Pub/Sub

Important interview point

💾 Redis Persistence: RDB vs AOF

RDB

AOF

Quick comparison

🧹 Redis Eviction

💥 Cache Stampede

🔒 Redis Distributed Lock

🏗️ Redis in a Real Node.js Architecture

🎯 Redis Interview Cheat Sheet

Questions you should be able to answer

💡 Final Takeaway

📚 Further Reading

nodejs #redis #backend #javascript #webdevelopment #systemdesign #interview

Postgresql Simplified

PostgreSQL Backend Developer Essentials: The Concepts I Learned Beyond CRUD

🐘 What is PostgreSQL?

🔑 Primary Key vs Foreign Key

🔒 Transactions and ACID

🔄 MVCC

🧩 JSONB

📊 Indexes

🔍 EXPLAIN and EXPLAIN ANALYZE

🧠 CTEs

📈 Window Functions

🧱 Other PostgreSQL Features

Arrays

UUID

Views

Functions and Procedures

Extensions

🚀 PostgreSQL in a Node.js Backend

🧪 A Real Query Optimization Example

🎯 Key Takeaways

PostgreSQL #NodeJS #BackendDevelopment #Database #SQL #WebDevelopment #Programming

🚀 Day 14 of My Node.js Learning Journey: Understanding the HTTP Module in Node.js

🌐 What is HTTP?

📦 What is the HTTP Module?

🛠️ Creating Your First HTTP Server

🔍 Understanding req and res

req (Request Object)

res (Response Object)

🧭 Simple Routing Without Express

📌 Common HTTP Methods

📄 HTTP Status Codes

📦 Headers and MIME Types

⚡ HTTP Module vs Express.js

🎯 Key Takeaways

💭 Final Thoughts

🚀 Day 13 of My Node.js Learning Journey: Understanding Buffers in Node.js

🤔 What is a Buffer?

💡 Why Do We Need Buffers?

⚡ Where Are Buffers Used?

🧠 Interesting Interview Fact

🛠️ Creating Buffers

Create from a String

Allocate Safe Memory

Allocate Fast Memory

🔥 Common Buffer Methods

📦 Buffer vs Stream

🔍 Understanding `req` and `res`

`req` (Request Object)

`res` (Response Object)

🚀 `pipe()` — The Easiest Way to Transfer Data

Benefits of `pipe()`

💧 `highWaterMark`

🚀 `pipeline()` vs `pipe()`

Why use `pipeline()`?

✅ `on()`

✅ `emit()`

✅ `once()`

⚠️ The Special `error` Event

`path.join()` vs `path.resolve()`

`__dirname` vs `process.cwd()`