DEV Community: Fahim Hasnain Fahad

CAP Theorem Explained Like You’re Ordering Pizza

Fahim Hasnain Fahad — Sun, 15 Jun 2025 14:54:05 +0000

What is the CAP Theorem? 🤔

Imagine you're running a popular pizza restaurant chain 🍕 with multiple locations across the city. You want three things:

Consistency - All locations have the same menu and prices
Availability - Every location is always open and serving customers
Partition Tolerance - Locations can still operate even if phone lines or internet go down

The CAP Theorem tells us something surprising: you can only guarantee 2 out of these 3 at any given time!

This same principle applies to distributed databases and systems.

Breaking Down the CAP 📊

C - Consistency 🔄

What it means: All nodes in your system show the same data at the same time.

Real-life example:
Think of your bank account 💳. When you check your balance on your phone, ATM, or online banking, you expect to see the exact same amount everywhere. If you just deposited $100, that update should be visible across all platforms immediately.

In tech terms: Every read receives the most recent write or an error.

A - Availability 🟢

What it means: Your system is always up and responding to requests.

Real-life example:
Amazon's website 🛒 during Black Friday. Even with millions of users shopping simultaneously, the site stays responsive. You can browse, add items to cart, and checkout without getting error messages like "Service Unavailable."

In tech terms: Every request gets a response (success or failure) without guaranteeing the data is the latest.

P - Partition Tolerance 🌐

What it means: The system continues working even when network connections fail between parts of the system.

Real-life example:
Imagine a library system 📚 with multiple branches. If the internet connection between branches goes down, each branch should still be able to check out books to customers using their local systems, even if they can't sync with other branches immediately.

In tech terms: The system continues operating despite network failures that prevent communication between nodes.

The Hard Choice: Pick Two! ⚖️

Here's where it gets interesting. According to CAP theorem, when network partitions happen (and they will!), you must choose between Consistency and Availability.

Scenario 1: Choosing Consistency + Partition Tolerance (CP) 🔒

Example: Banking systems during network issues
When your bank's network has problems, they'll often make ATMs unavailable rather than risk showing incorrect account balances. Better to be temporarily unavailable than to accidentally let someone overdraw their account!

Real tech example: Traditional SQL databases like PostgreSQL

Scenario 2: Choosing Availability + Partition Tolerance (AP) 📱

Example: Social media feeds during network issues

Facebook or Twitter will keep showing you posts even if some servers are disconnected. You might see slightly outdated content, but the app keeps working. They'll sync up the latest posts once the network is stable.

Real tech example: NoSQL databases like MongoDB, Cassandra

What about CA (Consistency + Availability)? 🤷‍♂️

This combination assumes no network partitions occur. In the real world of distributed systems, network issues are inevitable, so this isn't practical for multi-node systems.

Example: Single-server databases can be CA, but they're not truly distributed.

Real-World CAP Decisions 🌍

Netflix 🎬 - Chooses AP

Why: Users expect to watch movies anytime
Trade-off: Your "Continue Watching" list might be slightly out of sync across devices
Result: You can always stream, even if some data is temporarily inconsistent

Banking Apps 💰 - Choose CP

Why: Money accuracy is non-negotiable
Trade-off: ATM might be temporarily unavailable during network issues
Result: When available, your balance is always accurate

WhatsApp 💬 - Chooses AP

Why: People expect messaging to always work
Trade-off: Messages might arrive out of order during network issues
Result: You can always send messages, they'll sync up eventually

The Restaurant Chain Example 🍕

Let's revisit our pizza chain to see CAP in action:

Normal Day (No Network Issues):

All locations sync prices and inventory ✅
All locations serve customers ✅
Network is working fine ✅

Network Goes Down Between Locations:

Option 1 - Choose Consistency:

Close all locations until network is fixed 🚫
Ensures all locations have same prices when open
Customers can't get pizza 😢

Option 2 - Choose Availability:

Keep all locations open 🟢
Let each location operate independently
Risk: One location might run out of ingredients or have different prices
Customers can still get pizza 😊

Practical Takeaways 👨‍💻

When to Choose CP:

Financial systems 💳
Inventory management 📦
Any system where incorrect data is worse than temporary downtime

When to Choose AP:

Social media platforms 📱
Content delivery networks 🌐
Any system where user experience is prioritized over perfect data consistency

Key Questions to Ask:

Can your business tolerate temporary inconsistency?
Is downtime worse than showing slightly outdated data?
How critical is real-time accuracy for your users?

Common Misconceptions ❌

Myth: "CAP means you can only have 2 out of 3, ever!"
Reality: You can have all 3 during normal operations, but must choose 2 during network partitions.

Myth: "NoSQL is always AP, SQL is always CP"

Reality: It depends on configuration. MongoDB can be configured for stronger consistency, PostgreSQL can be set up for higher availability.

5 Hard Truths I Learned From The Boss I Hated Most

Fahim Hasnain Fahad — Wed, 11 Jun 2025 21:09:16 +0000

For over six years in the software industry — two onsite and four remote — I've written thousands of lines of code, deployed scalable systems, and architected features across stacks. But the biggest merge conflicts weren’t in Git — they were with a boss who was sharp, relentless, and, honestly, hard to deal with.

He wasn’t toxic. Just… tough. And here’s the surprising part: I’m grateful.

Below are five life-changing lessons I learned from working under pressure, and how they transformed me technically and personally.

🧠 1. “Don’t Bring Me Problems — Bring Me Options”

Before: I’d report issues like, “Hey, the deployment failed.”
After: I’d say, “The deployment failed due to X. We have three paths: rollback, hotfix, or isolate the failing module. I recommend rollback.”

💡 Why it matters:

Without Solutions	With Solutions
Reactive	Proactive
Wastes time	Saves time
Looks unprepared	Shows leadership

Engineering takeaway:
Learn to pre-diagnose, research workarounds, and own problems. It’s a muscle every senior dev needs to build.

🛠️ 2. “Build Like You Won’t Be Here Tomorrow”

He once said this after I hardcoded a value and walked away.

🔐 Key habits I developed:

Writing self-documenting, clean code 🧼
Creating bulletproof onboarding docs 📄
Avoiding tribal knowledge 🤐
Making PRs that tell a story 🧵

Why it’s life-changing:
When you code as if someone else will maintain it — they’ll love you. Sometimes, that person is future you.

🚨 3. “Deadlines Aren’t Suggestions”

At first, I thought he was overreacting when I missed a backend delivery by a day. His words stuck with me:

“If the backend is late, frontend stops. QA stops. The client waits. You don’t miss a sprint. You miss a chain.”

📅 I learned to:

Break tasks into atomic commits 📦
Time-box debugging ⏱️
Communicate blockers early 🗣️
Add 20% buffer to every estimate 📊

Result? Fewer late nights. More trust. Greater responsibility.

🌱 4. “Comfort Kills Growth”

I once asked, “Why do you always assign me the toughest tickets?”
He replied, “Because you’re smart enough to figure them out.”

🤯 That flipped my mindset:

Before	After
Avoided unknowns	Embraced the hard stuff 💪
Focused on speed	Focused on learning 🧠
Did just enough	Delivered with intention 🎯

Technically: I grew from debugging UI to owning CI/CD pipelines, DevOps workflows, and designing RESTful APIs.

❤️ 5. “Respect Is Earned in Reviews, Not Meetings”

He was brutal in code reviews. But he always had a point. He wanted quality, not comfort.

🔍 What I learned:

Every review is an opportunity to level up
Defensive coding ≠ clean coding
Feedback is a mirror, not an attack

Now, I seek out review-heavy teams — it’s where I grow the fastest.

🏁 Final Thoughts

That boss wasn’t easy. But the lessons he hammered into me — responsibility, resilience, and ownership — made me more than a better engineer.

He made me indispensable.

10 Battle-Tested Node.js Microservice Patterns for 99.9% Uptime

Fahim Hasnain Fahad — Fri, 06 Jun 2025 03:56:30 +0000

Today, I'm sharing the 10 battle-tested patterns that took us from constant firefighting to 99.9% uptime.

Pattern 1: The Circuit Breaker 🔌

Remember playing musical chairs as a kid? The Circuit Breaker pattern works similarly! When a service starts failing, we stop sending it requests temporarily.

// Simple circuit breaker with opossum
const circuitBreaker = new CircuitBreaker(callAPI, { failureThreshold: 50 });

This prevents cascading failures across your entire system. When one service gets overwhelmed, it doesn't bring down everything else with it.

Pattern 2: Health Checks & Auto-healing 🩺

Just like regular health check-ups prevent serious illness, implementing service health checks keeps your system robust.

app.get('/health', (req, res) => res.status(isHealthy ? 200 : 500).send());

Pattern 3: API Gateway Pattern 🚪

Think of this as your system's receptionist. It directs traffic and keeps the chaos out of your internal services.

Pattern 4: Service Discovery ️🔍

Imagine trying to call a friend whose phone number keeps changing! That's what services feel like without service discovery.

Pro Tip ⭐: Use etcd or Consul with Node.js for lightweight service discovery instead of heavyweight solutions.

Pattern 5: Event-Driven Architecture 📬

Services communicate through events, like colleagues using a message board instead of interrupting each other constantly.

// Publish event with Kafka
producer.send({ topic: 'user-created', messages: [{ value: JSON.stringify(user) }] });

Common Microservice Patterns Comparison

Pattern	Complexity	Best Used For	Impact on Uptime
Circuit Breaker	Medium	Preventing cascading failures	⬆️⬆️⬆️
Health Checks	Low	Early detection of issues	⬆️⬆️
API Gateway	Medium	Traffic management	⬆️⬆️
Service Discovery	Medium	Dynamic environments	⬆️⬆️
Event-Driven	High	Loose coupling	⬆️⬆️⬆️

Pattern 6: Bulkhead Pattern 🚢

Named after ship compartments, this pattern isolates components so one failure doesn't sink the whole ship.

Pattern 7: Retry with Exponential Backoff ⏱️

Like a polite person who knocks louder each time if no one answers, this pattern increases wait time between retries.

Pattern 8: Asynchronous Communication 📨

Not everything needs an immediate response. Some tasks can happen in the background, just like you might send an email instead of calling.

Pattern 9: Database-per-Service 💾

Each service gets its own database - no more fighting over the same resources!

Pattern 10: Monitoring & Observability 📊

You can't fix what you can't see. Proper monitoring is like having security cameras throughout your building.

Technology Stack Recommendations

Component	Technology	Why It Works
API Gateway	Express Gateway	Lightweight, Node.js native
Service Mesh	Istio/Linkerd	Advanced traffic management
Messaging	Kafka/RabbitMQ	Reliable async communication
Databases	MongoDB/PostgreSQL	Flexible schemas or ACID compliance
Containerization	Docker + Kubernetes	Consistent environments, auto-scaling

Key Takeaways 🔑

🚀 Start with health checks and circuit breakers for quick wins
🧩 Event-driven architecture enables loose coupling between services
📈 Monitoring is not optional - it's essential for high uptime
🔄 Implement graceful degradation so failures in one area don't affect others
💡 Microservices aren't just about splitting code - they're about resilience patterns

Remember: perfect uptime is a journey, not a destination. Start small, measure everything, and keep improving! 💪

Scaling to 10M Users with Event-Driven Architecture

Fahim Hasnain Fahad — Wed, 04 Jun 2025 15:32:12 +0000

🌟 The Day Our Server Crashed 🌟

Remember that feeling when your application suddenly crashes? That's exactly what happened to me last month. 😱

I was sipping my morning coffee ☕ when my phone exploded with notifications. Our startup's Node.js app had completely crashed after a marketing campaign brought in 50,000 new users overnight. What should have been a celebration turned into a nightmare.

"But I built this with Express! It should handle the load!" I remember telling my senior developer, Maria.

She smiled knowingly. "Time to learn about event-driven architecture, my friend."

🔍 Understanding the Problem

Our monolithic Express app was processing everything synchronously - user signups, email notifications, analytics tracking, and data processing were all happening in a single request-response cycle:

app.post('/signup', async (req, res) => {
  // This does EVERYTHING in one request! 😵
  await createUser(req.body);
  await sendWelcomeEmail(req.body.email);
  await trackSignupAnalytics(req.body);
  res.send({ success: true });
});

🧩 Enter Event-Driven Architecture

Maria explained that event-driven architecture is like a relay race 🏃‍♀️ where each runner (service) does their part and passes the baton (event) to the next runner, rather than having one person run the entire race.

🔥 The Core Concept

Instead of doing everything at once, we:

Publish events when something happens
Let other parts of the system subscribe to events they care about
Process those events asynchronously

⚡ Transforming Our App

We refactored our code to use events:

// Now we just publish an event! 🎉
app.post('/signup', async (req, res) => {
  await createUser(req.body);
  eventEmitter.emit('user.created', req.body);
  res.send({ success: true });
});

🎯 The Architecture Evolution

Before	After
Synchronous processing	Asynchronous event handling
Single point of failure	Distributed resilience
Tight coupling	Loose coupling
Linear scaling	Horizontal scaling
5-second response times	200ms response times

📊 Results After 30 Days

Metric	Before	After	Improvement
Max concurrent users	5,000	250,000	50x
Average response time	2,500ms	120ms	20x faster
Server costs	$1,200/mo	$750/mo	37% cheaper
Uptime	97.2%	99.99%	2.79% increase

🛠️ Implementation Steps

Our journey to scaling involved these key steps:

Identify bottlenecks in our monolithic app
Decouple components using an event bus
Deploy microservices for heavy processing tasks
Implement circuit breakers for resilience

🔧 Technology Stack We Used

Node.js + Express for API services
Kafka for event streaming at scale
Redis for pub/sub in smaller services
MongoDB for document storage
PostgreSQL for relational data

💡 Pro Tip: Start Simple!

You don't need Kafka right away! Begin with Node.js's built-in EventEmitter:

const EventEmitter = require('events');
const eventBus = new EventEmitter();

This gives you the event-driven pattern without the infrastructure complexity. Later, you can swap in more robust solutions without changing your core logic!

🚀 Scaling to 10M Users

The most beautiful part? The same architecture that handled 250,000 users scaled effortlessly to 10M when our app went viral six months later.

Event-driven architecture gave us three superpowers:

Elasticity: Services scale independently based on event volume
Resilience: System degraded gracefully under load, never crashed
Maintainability: New team members could contribute to isolated services

🌈 Key Takeaways

Start with events early - retrofitting is much harder!
Simple events first - use Node's built-in EventEmitter before complex message brokers
Think in asynchronous flows - design for eventual consistency
Decouple ruthlessly - services should know as little as possible about each other
Monitor everything - events give amazing visibility into system performance

Remember, scaling isn't just about handling more users—it's about building systems that can evolve with your business needs. By embracing event-driven architecture early, we set ourselves up for sustainable growth. 🚀

What scaling challenges are you facing with your Node.js apps? I'd love to hear about them in the comments! 👇

Securing Node.js: 10 Auth Practices for 99.9% Protection

Fahim Hasnain Fahad — Tue, 27 May 2025 15:04:48 +0000

The Night Everything Broke 🌙

Remember that Tuesday night when I deployed our startup's new user dashboard? I was so proud! 🚀 Three months of work, finally live. I treated myself to pizza and fell asleep watching Netflix.

At 3:27 AM, my phone exploded with notifications. 📱 Our database was being drained, user accounts compromised, and someone had changed our homepage to a dancing banana GIF. 🍌

"But I implemented authentication!" I protested to our CTO, on our emergency Zoom call.

"You implemented something," she sighed. "Let's fix this properly."

The Authentication Journey Begins 🧭

The next morning, our CTO sketched out a security roadmap. I realized my "authentication" was like using a paper lock on a bank vault. 😱

"Authentication isn't a feature, it's the foundation everything else stands on." - My wiser, post-incident self

The 10 Practices That Saved My Career ⚔️

1. Hash Those Passwords! 🧂

NEVER store plain-text passwords. Ever! I had stored them with basic encryption (rookie mistake).

// DON'T do this
user.password = req.body.password;

// DO this instead
user.password = await bcrypt.hash(req.body.password, 10);

2. JWT: Your Digital ID Badge 🪪

JSON Web Tokens became my new best friends. They're like digital ID badges that expire!

Pro Tip: 💡 Store JWTs in HttpOnly cookies, not localStorage, to protect against XSS attacks.

res.cookie('token', token, { httpOnly: true, secure: true });

3. Embrace Environment Variables 🌍

My database credentials were basically public before. Moving them to environment variables was like putting money in a safe instead of leaving it on the sidewalk.

4. Rate Limiting: The Digital Bouncer 💪

Adding rate limiting was like hiring a bouncer for our API:

app.use(rateLimit({ windowMs: 15 * 60 * 1000, max: 100 }));

5-10: The Complete Protection Package 🛡️

Practice	What It Does	Implementation Difficulty
🔒 2FA Authentication	Adds a second verification layer	Medium
🚫 CSRF Protection	Prevents cross-site request forgery	Easy
👮 Input Validation	Ensures data matches expected format	Easy
⏱️ Token Expiration	Makes stolen tokens useless after time	Easy
🔍 Audit Logging	Records who did what, when	Medium
🚪 Secure Password Reset	Prevents account takeover via resets	Medium

Security vs. Developer Experience ⚖️

I worried implementing all this would make our codebase complex. Turns out, good security can be clean:

Approach	Security Level	Developer Friendliness
Manual Implementation	🔴 Variable	🟠 Complex
Passport.js	🟢 High	🟢 Simple
Auth0/Firebase	🟢 Very High	🟢 Very Simple

The Happy Ending 🌈

Two weeks later, I deployed our properly secured application. That night, I slept peacefully. No alerts, no dancing bananas.

When our startup got acquired six months later, the security audit passed with flying colors. The acquiring company's CISO actually said, "Whoever set up your auth system knew what they were doing."

I may have teared up a little. 🥹

Quick Win: The One-Line Security Boost ⚡

Add this single line to your Express app to prevent common attacks:

app.use(helmet());

This one package adds 12 middleware security enhancements instantly!

Key Takeaways 🎯

🔐 Proper authentication is essential, not optional
🧂 Always hash passwords, never store them as plaintext
🍪 Store JWTs in HttpOnly cookies for better security
🔢 Implement 2FA for critical operations
📝 Validate all inputs, no exceptions
🕰️ Set appropriate token expiration times
📊 Log authentication events for auditing
🛡️ Use established libraries instead of rolling your own auth

Remember: Security isn't about being unhackable (nothing is); it's about making the effort required greater than the reward. Make your app the digital equivalent of a house with good locks, an alarm system, and no valuables visible from the windows! 🏠🔒

Mastering Node.js Error Handling: From Crashes to 99.9% Uptime

Fahim Hasnain Fahad — Sat, 24 May 2025 07:16:04 +0000

Mastering Node.js Error Handling: From Crashes to 99.9% Uptime

The Great Server Meltdown of 2023 🔥

It was 3 AM when my phone erupted with notifications. Our production server had crashed—again. 😱 As I groggily logged in, the dreaded UnhandledPromiseRejection error greeted me like an old enemy.

"Why didn't anyone catch this exception?" I muttered while frantically restarting services.

Sound familiar? Six months ago, I was that junior developer whose Node.js applications resembled a house of cards—one unexpected error and everything came tumbling down. Today, our uptime is 99.9%, and I sleep through the night. Let me tell you how I turned things around! 🚀

The Error-Handling Journey 🗺️

Phase 1: The Naive Days 🙈

Like many developers, I started with the classic "hope nothing breaks" approach:

app.get('/users', (req, res) => {
  const data = someRiskyOperation(); // What could go wrong?
  res.json(data);
});

One database hiccup later, and our entire API was down. Users were angry, managers were angrier.

Phase 2: Try-Catch Everything! 🤔

My first solution? Wrap EVERYTHING in try-catch blocks:

try {
  // 500 lines of business logic
} catch (error) { console.error("Something went wrong", error); }

Better than nothing, but still missing the mark. It was like putting a single fire extinguisher in a skyscraper and calling it "fire safety."

The Error Handling Toolkit ⚒️

As I learned more, I developed a systematic approach to handling errors in Node.js. Here's what changed everything:

1. Categorizing Errors 📊

Not all errors are created equal!

Error Type	Description	Example	How to Handle
Operational Errors	Expected problems during normal operation	Network timeout, Invalid user input	Catch and handle gracefully
Programmer Errors	Bugs in the code	Trying to read property of undefined	Fix the bug!
Unrecoverable Errors	System-level failures	Out of memory, Uncaught exceptions	Crash and restart safely

2. The Error Handling Hierarchy 🏰

I implemented a layered approach:

Layer	Responsibility	Implementation
Function Level	Handle known errors specific to function	try/catch, return Result objects
Module Level	Handle domain-specific errors	Custom error classes, error mappers
Process Level	Catch uncaught exceptions	Global handlers, graceful shutdown
Infrastructure Level	Ensure system resilience	Process managers, health checks

The Game-Changers ⚡

Custom Error Classes 🔍

Creating custom error types completely transformed our debugging experience:

class DatabaseConnectionError extends Error {
  constructor(message) { super(`Database connection failed: ${message}`); }
}

Async/Await Error Handling 🧩

Promise rejections were our #1 source of crashes until I discovered this pattern:

const result = await asyncOperation().catch(err => ({ error: err }));
if (result.error) return handleError(result.error);

💡 Pro Tip: Express Error Middleware

The single biggest improvement came from centralizing error handling:

app.use((err, req, res, next) => {
  res.status(err.statusCode || 500).json({ message: err.message });
});

This simple middleware saved us hours of debugging time and prevented countless crashes!

Real-World Results 📈

Before implementing these techniques, we had:

2-3 production crashes weekly 😓
Hours spent debugging cryptic errors
Angry customers and stressed developers

After:

99.9% uptime for 6 months straight 🎉
Clear error messages that point to root causes
A team that sleeps through the night!

Key Takeaways 🎯

Don't just catch errors—categorize them and handle each type appropriately
Build an error handling strategy that works at multiple levels of your application
Use custom error classes to make debugging faster and easier
Implement central error handling middleware in Express applications
Treat error handling as a first-class feature, not an afterthought
Log strategically—too little information is useless, too much is noise

Remember: In Node.js, good error handling isn't just about preventing crashes—it's about building resilient systems that fail gracefully and recover automatically. Now go forth and catch those errors! 🦸‍♂️

10X Your NodeJS: Performance Tricks for 100ms Responses

Fahim Hasnain Fahad — Mon, 19 May 2025 09:23:59 +0000

🚀 The Wake-Up Call

It was 2 AM when my phone rang. The CEO's voice was tense. "The app is crawling. Users are complaining. Fix it."

I still remember that night vividly. Our NodeJS app that handled customer orders was buckling under traffic. Response times had gone from snappy to sluggish—2-3 seconds per request. 😱

Sound familiar? If you're a junior developer working with NodeJS, you might have faced (or will face) this scenario. Let me share how I transformed our application from a tortoise to a cheetah! ⚡

🕵️ Diagnosing the Problem

The next morning, bleary-eyed but determined, I started investigating. Our stack looked pretty standard:

Technology	Purpose
NodeJS/Express	API server
MongoDB	Database
Redis	Caching
AWS EC2	Hosting

The app was structured like most Express applications, with routes handling requests and controllers processing business logic. But something was clearly wrong.

🔍 The Bottlenecks

I discovered three main issues that were throttling our performance:

Database queries were inefficient 🐢
No caching strategy 💾
Blocking operations in the event loop ⛓️

🛠️ The Transformation Journey

Step 1: Optimizing MongoDB Queries ⚡

I found queries like this throughout our codebase:

const users = await User.find({});

This was loading ALL users into memory before filtering on the application side!

I replaced them with properly indexed, filtered queries:

const users = await User.find({ active: true }).select('name email').limit(10);

Quick Win 🏆: Adding appropriate indexes reduced query times by 80%!

Query Type	Before	After	Improvement
User Search	1200ms	95ms	92% faster
Product Listing	1800ms	120ms	93% faster
Order History	2100ms	150ms	93% faster

Step 2: Implementing Redis Caching 🚀

For frequently accessed data that rarely changes (like product categories), I implemented Redis:

const data = await redisClient.get('categories') || await fetchAndCacheCategories();

This simple pattern eliminated repetitive database queries for common data. Think of Redis as your refrigerator - why go grocery shopping (query the database) every time you want a snack? 🍎

Step 3: Non-Blocking Operations 🔄

I discovered we were reading files synchronously in some routes:

// Before: Blocking the event loop 😖
const template = fs.readFileSync('./templates/email.html');

// After: Non-blocking 😊
const template = await fs.promises.readFile('./templates/email.html');

Remember: NodeJS is like a single-lane highway. One slow truck (blocking operation) causes traffic for everyone!

🎯 The Memory Leak Mystery

Our server would occasionally crash after running for a few days. Using node --inspect and Chrome DevTools, I discovered we were accumulating references to large objects.

The culprit? Event listeners that weren't being properly removed:

// Fixed by adding proper cleanup
emitter.removeListener('data', handleData);

⚡ Pro Tip: Async/Await Patterns

One pattern that significantly improved our code readability and performance:

// Use Promise.all for parallel operations
const [users, products, categories] = await Promise.all([
  fetchUsers(),
  fetchProducts(),
  fetchCategories()
]);

This runs all three operations concurrently rather than sequentially, cutting response time by ~66%!

📊 The Results

Metric	Before	After
Avg Response Time	2300ms	95ms
Server Memory	1.2GB	450MB
Crash Frequency	Every 2-3 days	None in 3 months

🎯 Key Takeaways

Index everything that you query frequently in MongoDB 🔍
Cache aggressively with Redis for data that changes infrequently 💾
Avoid blocking the event loop - use async operations wherever possible 🔄
Monitor memory usage to catch leaks before they become problems 📈
Use Promise.all for concurrent operations ⚡
Profile regularly - what's fast today might be slow tomorrow 🕒

The CEO called me a week later: "Whatever you did, it worked. The app feels lightning fast."

Sometimes the biggest performance gains come from the simplest changes. Happy optimizing! 🚀

Scale to 10M Users: CQRS in NestJS for API Performance

Fahim Hasnain Fahad — Sat, 17 May 2025 11:07:39 +0000

Scale to 10M Users: CQRS in NestJS for API Performance

In today's digital landscape, building applications that can handle millions of users requires thoughtful architecture decisions. Command Query Responsibility Segregation (CQRS) is one such pattern that helps maintain performance at scale. Let's explore!

What is CQRS and Why Use It?

CQRS splits your application into two models:

Command model: Handles create, update, and delete operations
Query model: Manages read operations

This separation addresses the reality that most applications have asymmetric read/write loads—typically with reads far outnumbering writes.

Setting Up CQRS in NestJS

First, install the required packages:

npm install @nestjs/cqrs uuid kafkajs redis mongodb pg

Directory Structure

src/
├── commands/
│   ├── handlers/
│   ├── impl/
├── queries/
│   ├── handlers/
│   ├── impl/
├── events/
├── models/
├── controllers/
└── app.module.ts

Basic Implementation

Let's start with our app.module.ts:

import { Module } from '@nestjs/common';
import { CqrsModule } from '@nestjs/cqrs';
import { MongooseModule } from '@nestjs/mongoose';
import { TypeOrmModule } from '@nestjs/typeorm';
import { CommandHandlers } from './commands/handlers';
import { QueryHandlers } from './queries/handlers';
import { EventHandlers } from './events/handlers';
import { Controllers } from './controllers';

@Module({
  imports: [
    CqrsModule,
    MongooseModule.forRoot('mongodb://localhost:27017/cqrs_read'),
    TypeOrmModule.forRoot({
      type: 'postgres',
      host: 'localhost',
      port: 5432,
      username: 'postgres',
      password: 'password',
      database: 'cqrs_write',
      entities: [__dirname + '/**/*.entity{.ts,.js}'],
      synchronize: true,
    }),
  ],
  controllers: [...Controllers],
  providers: [
    ...CommandHandlers,
    ...QueryHandlers,
    ...EventHandlers,
  ],
})
export class AppModule {}

Now, let's define a command:

// commands/impl/create-user.command.ts
export class CreateUserCommand {
  constructor(
    public readonly email: string,
    public readonly name: string,
  ) {}
}

And its handler:

// commands/handlers/create-user.handler.ts
import { CommandHandler, ICommandHandler, EventBus } from '@nestjs/cqrs';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository } from 'typeorm';
import { CreateUserCommand } from '../impl/create-user.command';
import { User } from '../../models/user.entity';
import { UserCreatedEvent } from '../../events/impl/user-created.event';

@CommandHandler(CreateUserCommand)
export class CreateUserHandler implements ICommandHandler<CreateUserCommand> {
  constructor(
    @InjectRepository(User)
    private userRepository: Repository<User>,
    private eventBus: EventBus,
  ) {}

  async execute(command: CreateUserCommand): Promise<User> {
    const { email, name } = command;

    const user = new User();
    user.email = email;
    user.name = name;

    const savedUser = await this.userRepository.save(user);

    // Publish event for read model synchronization
    this.eventBus.publish(new UserCreatedEvent(savedUser.id, email, name));

    return savedUser;
  }
}

Query Side with MongoDB

For read operations, we'll use MongoDB for its superior query performance:

// queries/impl/get-user.query.ts
export class GetUserQuery {
  constructor(public readonly id: string) {}
}

// queries/handlers/get-user.handler.ts
import { QueryHandler, IQueryHandler } from '@nestjs/cqrs';
import { InjectModel } from '@nestjs/mongoose';
import { Model } from 'mongoose';
import { GetUserQuery } from '../impl/get-user.query';
import { UserReadModel } from '../../models/user.read-model';

@QueryHandler(GetUserQuery)
export class GetUserHandler implements IQueryHandler<GetUserQuery> {
  constructor(
    @InjectModel(UserReadModel.name)
    private userModel: Model<UserReadModel>,
  ) {}

  async execute(query: GetUserQuery): Promise<UserReadModel> {
    return this.userModel.findOne({ userId: query.id }).exec();
  }
}

Synchronizing Read/Write Models with Kafka

To keep our read model updated, we'll use Kafka for event sourcing:

// events/handlers/user-created.handler.ts
import { EventsHandler, IEventHandler } from '@nestjs/cqrs';
import { Inject } from '@nestjs/common';
import { ClientKafka } from '@nestjs/microservices';
import { UserCreatedEvent } from '../impl/user-created.event';

@EventsHandler(UserCreatedEvent)
export class UserCreatedHandler implements IEventHandler<UserCreatedEvent> {
  constructor(
    @Inject('KAFKA_SERVICE') private kafkaClient: ClientKafka
  ) {}

  handle(event: UserCreatedEvent) {
    this.kafkaClient.emit('user-created', {
      id: event.id,
      email: event.email,
      name: event.name,
      timestamp: new Date().toISOString()
    });
  }
}

And our consumer service:

// services/read-model-sync.service.ts
import { Injectable, OnModuleInit } from '@nestjs/common';
import { InjectModel } from '@nestjs/mongoose';
import { Model } from 'mongoose';
import { Consumer, Kafka } from 'kafkajs';
import { UserReadModel } from '../models/user.read-model';

@Injectable()
export class ReadModelSyncService implements OnModuleInit {
  private consumer: Consumer;

  constructor(
    @InjectModel(UserReadModel.name)
    private userModel: Model<UserReadModel>,
  ) {
    const kafka = new Kafka({
      clientId: 'read-model-sync',
      brokers: ['localhost:9092'],
    });
    this.consumer = kafka.consumer({ groupId: 'read-model-sync-group' });
  }

  async onModuleInit() {
    await this.consumer.connect();
    await this.consumer.subscribe({ topic: 'user-created', fromBeginning: true });

    await this.consumer.run({
      eachMessage: async ({ topic, message }) => {
        const eventData = JSON.parse(message.value.toString());

        if (topic === 'user-created') {
          await this.userModel.updateOne(
            { userId: eventData.id },
            {
              userId: eventData.id,
              email: eventData.email,
              name: eventData.name,
              updatedAt: new Date()
            },
            { upsert: true }
          );
        }
      },
    });
  }
}

To further enhance read performance, we can add Redis Caching for Read Queries.

Breaking the Scale Barrier: 1 Million Messages with NodeJS and Kafka

Fahim Hasnain Fahad — Wed, 15 Jan 2025 15:17:24 +0000

Apache Kafka is a distributed event streaming platform which can be used as a Pub-Sub broker in NodeJS microservices. Recently, I worked in a project where publisher needed to send messages to 1 million users and give the response back. Here, a cool thing can be done by Message Queue and we can send 1 million messages in a queue and it will send the messages to the users one by one. But, a pub-sub can broadcast to all instances of a consumer which are subscribed to a certain topic.

The Magic of Message Queues vs. Pub-Sub in Kafka

When faced with such a massive scale of messaging, two approaches often come to mind:

Message Queue: Send 1 million messages to a queue, where each message is processed one by one by consumers.
Pub-Sub: Broadcast a single message to all instances of consumers subscribed to a specific topic.

For our use case, Kafka’s flexibility in handling both patterns made it the ideal choice. Let’s dive into how to implement a Producer-Consumer microservice system with Kafka.

Check out the demo code: https://github.com/fahadfahim13/producer-consumer-kafka.git

Architecture

Producer Microservice:

Let's create a Producer microservice and a Consumer microservice.
Code in producer/index.js will be like this:

const { Kafka } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'my-producer',
  brokers: [process.env.KAFKA_BROKER],
});

const producer = kafka.producer();

(async () => {
  await producer.connect();
  for (let i = 0; i < 1000000; i++) {
    await producer.send({
      topic: process.env.TOPIC_NAME,
      messages: [{ value: `Message ${i}` }],
    });
    console.log(`Sent message ${i}`);
  }
  await producer.disconnect();
})();

Here, we can see the producer is producing 1 million messages and sending into a Kafka topic.

Consumer Microservice

Code in consumer microservice's index.js file will be like this:

const { Kafka } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'my-consumer',
  brokers: [process.env.KAFKA_BROKER],
});

const consumer = kafka.consumer({ groupId: 'test-group' });

(async () => {
  await consumer.connect();
  await consumer.subscribe({ topic: process.env.TOPIC_NAME, fromBeginning: true });

  await consumer.run({
    eachMessage: async ({ topic, partition, message }) => {
      console.log(`Received message: ${message.value} in topic: ${topic} at partition: ${partition}`);
    },
  });
})();

The consumers will be subscribed to a Kafka topic and receive messages asynchronously and process those messages.

Docker-compose File

The docker-compose.yml file looks like this which will run zookeeper, kafka and producer & consumer services in different containers.

version: '3.8'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - "22181:2181"
    networks:
      - kafka-network
    volumes:
      - kafka_data:/var/lib/kafka/data

  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - "29092:29092"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    networks:
      - kafka-network

  producer:
    build: 
      context: ./producer
      dockerfile: Dockerfile
    container_name: kafka-producer
    depends_on:
      - kafka
    networks:
      - kafka-network
    volumes:
      - ./producer:/app
    environment:
      KAFKA_BOOTSTRAP_SERVERS: kafka:9092
      KAFKA_BROKER: kafka:9092
      TOPIC_NAME: my-topic

  consumer:
    build: 
      context: ./consumer
      dockerfile: Dockerfile
    container_name: kafka-consumer
    depends_on:
      - kafka
    networks:
      - kafka-network
    volumes:
      - ./consumer:/app
    environment:
      KAFKA_BOOTSTRAP_SERVERS: kafka:9092
      KAFKA_BROKER: kafka:9092
      TOPIC_NAME: my-topic

networks:
  kafka-network:
    driver: bridge


volumes:
  kafka_data:

Optimizing Kafka Performance with Partitions

To handle high throughput, it’s essential to increase the number of partitions in our Kafka topic. More partitions allow parallel processing, enabling producers and consumers to scale horizontally. For example:

kafka-topics --create \
  --zookeeper localhost:2181 \
  --replication-factor 1 \
  --partitions 10 \
  --topic my-topic

Each partition acts as a separate queue, distributing the load among consumers.

Check out the demo code: https://github.com/fahadfahim13/producer-consumer-kafka.git

Full Stack in a Container: NodeJS, NextJS, and MongoDB with Docker

Fahim Hasnain Fahad — Fri, 10 Jan 2025 20:20:15 +0000

Recently, I had to create a NodeJS project for backend and a NextJS project for Frontend and connect MongoDB local database in NodeJS.
I tried to run everything locally but using Docker gave me the most optimal way of running everything together inside different containers.

I have run all the projects and DB inside different containers. The Frontend will run at 3000 port, Backend will run at 8000 port and DB will run at 27017 port of localhost.
The Project structure was something like this:

project/
│
├── docker-compose.yml     # Where all the containers are declared
│
├── backend/                  
│   ├── package.json           
│   ├── Dockerfile            
│   └── index.js            # Where server runs
│
│
├── frontend/                  
│   ├── package.json           
│   ├── Dockerfile            
│   └── layout.jsx           # Where home page runs
│

Now, we need to run the frontend and declare the Dockerfile for it. In order to run these setups in production as well, I've kept the build commands.
Frontend/Dockerfile looks like this:

# Use the official Node.js image as the base image
FROM node:20

# Set the working directory
WORKDIR /app

# Copy package.json and package-lock.json
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy the rest of the application code
COPY . .

# Build the application
RUN npm run build

# Expose the port the app runs on
EXPOSE 3000

# Start the application
CMD ["npm", "start"]

Similarly, the backend Dockerfile looks like this:

# Dockerfile

# Use the official Node.js image as a base image
FROM node:18

# Set the working directory inside the container
WORKDIR /app

# Copy package.json and package-lock.json
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy the rest of the application code
COPY . .

# Build the application
RUN npm run build

# Expose the port that the app runs on
EXPOSE 8000

# Command to run the application
CMD ["npm", "start"]

Now, In order to connect these containers with DB container, we need to create a docker-network and keep all the containers under the same network.

The root project folder has the docker-compose.yml file which will define all the containers, DB volume and the docker network.

version: '3.8'

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      - MONGODB_URI=mongodb://mongo:27017/db
      - PORT=8000
    depends_on:
      - mongo
    networks:
      - project-network

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    environment:
      - NEXT_PUBLIC_WEBSOCKET_URL=http://localhost:8000
      - NEXT_PUBLIC_HTTP_URL=http://localhost:8000
    depends_on:
      - backend
    networks:
      - project-network

  mongo:
    image: mongo:latest
    ports:
      - "27017:27017"
    volumes:
      - mongo-data:/data/db
    networks:
      - project-network

networks:
  project-network:
    driver: bridge

volumes:
  mongo-data:

This config file has 3 containers: frontend, backend & mongo. The backend container depends on mongo container, and the frontend container depends on the backend container. All 3 containers has connected into the project-network.

Now if we run docker-compose up -d in the root folder to run it in detached mode, in the Docker Desktop, we can see the logs for all containers.

To see the full project code, visit now: https://github.com/fahadfahim13/httpbin-monitor.git

Node.js Meets PostgreSQL and MongoDB in Docker: Docker Diaries

Fahim Hasnain Fahad — Tue, 07 Jan 2025 20:36:33 +0000

Recently, I had to create a NodeJS application where I had to connect to 2 different database in PostgreSQL and MongoDB. For application scaling, I had to use Docker and Kubernetes. Here's how I connected all these inside docker.

In order to run this whole thing using docker-compose, visit this repo: https://github.com/fahadfahim13/node-pg-mongo-docker.git

Project Setup

Run the following commands with your preferred project name:

mkdir node-postgres-mongo
cd node-postgres-mongo
npm init -y

The install these dependencies:

npm install express pg mongoose dotenv

Inside this project, create an index.js file and a .env file.
Also, create a config folder, and inside this folder, create mongodb.js and postgres.js files for configuration.

The index.js file should look like this:

const express = require('express');
const pgPool = require('./config/postgres');
const connectMongoDB = require('./config/mongodb');
require('dotenv').config();

const app = express();
const port = process.env.PORT || 8088;

app.get('/', (req, res) => {
    res.send('Hello World!');
});

app.listen(port, async () => {
    try {
        // Initialize database connections
        await connectMongoDB();
        await pgPool.connect();
        console.log('PostgreSQL connected successfully');

        console.log(`Server is running at http://localhost:${port}`);
    } catch (error) {
        console.error('Error during startup:', error);
        process.exit(1);
    }
});

The config/postgres.js should look like this:

const { Pool } = require('pg');
require('dotenv').config();

const pgPool = new Pool({
    host: process.env.POSTGRES_HOST || 'postgres-db', // PostgreSQL container name
    port: 5432,
    user: process.env.POSTGRES_USER || 'admin',
    password: process.env.POSTGRES_PASSWORD || 'admin123',
    database: process.env.POSTGRES_DB || 'testdb',
  });


module.exports = pgPool;

The config/mongodb.js should look like this:

const mongoose = require('mongoose');
require('dotenv').config();

const connectMongoDB = async () => {
    try {
        const mongoURI = `mongodb://${process.env.MONGO_INITDB_ROOT_USERNAME || 'root'}:${process.env.MONGO_INITDB_ROOT_PASSWORD || 'root123'}@${process.env.MONGO_INITDB_ROOT_CONTAINER || 'mongo-container'}:27017/testdb?authSource=admin`;
        mongoose.connect(mongoURI, { useNewUrlParser: true, useUnifiedTopology: true })
        .then(() => console.log('Connected to MongoDB'))
        .catch(err => console.error('MongoDB connection error:', err));
        console.log('MongoDB connected successfully');
    } catch (error) {
        console.error('MongoDB connection error:', error);
        process.exit(1);
    }
};

module.exports = connectMongoDB;

Docker Setup

Create Dockerfile

Create a file named Dockerfile inside root directory of the nodejs project and paste this code:

FROM node:18

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 8088

CMD ["npm", "start"]

Network creation

In order to connect all containers, we need one docker network by which all containers can connect. Run the following commands:

docker network create node-postgres-mongo # Creates a network named node-postgres-mongo

docker network ls # Lists all networks available in docker to check

PostgreSQL container setup

docker pull postgres # Pulls image if not available in local
docker run --name postgres-db --network node-postgres-mongo -e POSTGRES_USER=admin -e POSTGRES_PASSWORD=admin123 -e POSTGRES_DB=testdb -p 5432:5432 -d postgres # Runs the DB in a container named postgres-db

After this, if you run docker ps, you should see a container named postgres-db.

MongoDB container setup

docker pull mongo # Pulls if image is not available in local
docker run --name mongo-db --network node-postgres-mongo -e MONGO_INITDB_ROOT_USERNAME=root -e MONGO_INITDB_ROOT_PASSWORD=root123 -p 27017:27017 -d mongo # Creates a container in the same network named mongo-db

NodeJS container run

docker build -t node-pg-mongo . # Build the image with name node-pg-mongo in the root directory


docker run --name node-pg-mongo --network node-postgres-mongo -p 8088:8088 -d node-pg-mongo # Run the image into container named node-pg-mongo in the same network

All Done! Now if we run docker logs node-pg-mongo, we can see the logs of the running application and it can connect to both mongodb and postgres:

In order to run this whole thing using docker-compose, visit this repo: https://github.com/fahadfahim13/node-pg-mongo-docker.git

Scaling Node.js: Handling 1 Million Requests Like a Pro

Fahim Hasnain Fahad — Fri, 03 Jan 2025 16:03:34 +0000

JavaScript is a single threaded, synchronous language. This means when users request to a NodeJS server, it runs on one Core of CPU of the server and doesn't use other cores.

Requirements

Node application that can handle millions of requests in a very short time.
Each request's response time shouldn't go over a threshold of 500ms.
System must be fault tolerant.

I found lots of ways to handle this but found really a simple solution with NodeJS's own Cluster module.

It uses all cores of the CPU of server and launches multiple instances of Node application in the same port. There is one main cluster which forks child processes (instances in other cores) and channels requests to the child processes. The child processes handles these requests as a standalone instance.

Code Implementation

Create Express application and install dependencies:

cd project-name
npm init -y 
npm i express mongoose nodemon dotenv

General Approach

Then in the index.js file, create an usual express application.

require('dotenv').config();
const express = require('express');
const connectDB = require('./config/database');

const app = express();

app.use(express.json());

connectDB();

app.use("/api/items", require("./routes/items"));

app.get("/", (req, res) => {
  res.json({ message: "Welcome to the API" });
});

const PORT = process.env.PORT || 8000;
app.listen(PORT, () => {
  console.log(`Worker ${process.pid} started - Server running on port ${PORT}`);
});

Now, run the application using nodemon index.js command.

Then, install loadtest package globally for load testing:

npm install -g loadtest

Now run the loadtest on the local server using this command:

loadtest -n 1000000 --rps 10000 http://localhost:8000 #use your port number

This command creates 1M reuests with maximum 10K rps (request per second).

This is the result:

Target URL:          http://localhost:8000
Max requests:        1000000
Target rps:          10000
Concurrent clients:  3805
Running on cores:    6
Agent:               none

Completed requests:  1000000
Total errors:        43446
Total time:          539.523 s
Mean latency:        97.7 ms
Effective rps:       1853

Percentage of requests served within a certain time
  50%      10 ms
  90%      265 ms
  95%      543 ms
  99%      1450 ms
 100%      4509 ms (longest request)

   -1:   43446 errors

We can see, almost 44K requests failed to respond.

Cluster Module

Let's use NodeJS Cluster Now.

In the index file, change the code in Cluster:

require('dotenv').config();
const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
const express = require('express');
const connectDB = require('./config/database');

if (cluster.isMaster) {
    console.log(`Master ${process.pid} is running`);
    console.log(`Number of CPUs: ${numCPUs}`);

    // Fork workers
    for (let i = 0; i < numCPUs; i++) {
        cluster.fork();
    }

    cluster.on('exit', (worker, code, signal) => {
        console.log(`Worker ${worker.process.pid} died`);
        cluster.fork(); // Replace the dead worker
    });
} else {
    const app = express();

    // Middleware
    app.use(express.json());

    // Connect to MongoDB
    connectDB();

    // Routes
    app.use('/api/items', require('./routes/items'));

    // Basic route
    app.get('/', (req, res) => {
        res.json({ message: 'Welcome to the API' });
    });

    const PORT = process.env.PORT || 8000;
    app.listen(PORT, () => {
        console.log(`Worker ${process.pid} started - Server running on port ${PORT}`);
    });
}

Here the main Cluster forks the child cluster and uses all available Cores of the CPU.
When we run the application, we can see:

We can see it uses all the cores of the server.

Now, let's run the loadtest. We'll see the result like this:

We can see, it got 100% success rate which is crazy! Also, the mean latency came down to 9.9ms from 97.7ms which is another crazy efficiency. Also, 99% of the requests responded success under 200ms where it took 1.5s previously.

Find the whole codebase in this repo: [https://github.com/fahadfahim13/node-scale.git](https://github.com/fahadfahim13/node-scale.git)