DEV Community: Nitin Malviya

How Kafka Saved a Failing System -The Story of Arjun & Meera

Nitin Malviya — Mon, 24 Aug 2026 14:04:56 +0000

Modern systems run on data in the form of fast, real-time, continuous streams.

But what happens when your system grows faster than your architecture can handle?

This is the story of Arjun, a backend engineer from Indore, and Meera, a DevOps engineer — and how Kafka rescued their collapsing system while teaching them how real-time streaming actually works.

It's a story, but it's also a complete walkthrough: why queues break, what Kafka replaces them with, the code, the bugs, and the cases where Kafka is genuinely the wrong choice.

Chapter 1: The Breaking Point

SwiftKart, a fast-growing hyperlocal delivery startup, was booming.

Orders jumped from 4,000 to 40,000 per day in just two months.

But growth brought chaos. Arjun, the Node.js backend engineer, was drowning:

Notifications delayed by 15 minutes
Redis queue overflowing
Analytics dashboard freezing
Duplicate messages
Missing events
Workers crashing randomly

Meanwhile, Meera's monitoring showed red spikes everywhere.

Arjun said:

"Meera, the system can't breathe. The Redis queue is suffocating!"

Their stack was the classic startup design:

Node.js microservices
MySQL
Redis job queues
Background workers

Perfect at small scale. Completely helpless for high-volume real-time streaming.

Chapter 2: Why Their Old System Failed

As traffic increased, the cracks became craters.

1. Queue overflow. Redis had a single-lane queue → bottlenecks everywhere.

2. No durability. If Redis restarts, messages disappear.

3. No replay. Consumed once = gone forever. If analytics breaks, that data is lost permanently.

4. Slow consumers collapsed the system. Events arrived faster than workers could drain them, and back-pressure took everything down with it.

5. No multi-consumer support. Analytics, notifications, fraud detection — sabko same event chahiye tha. Redis sirf ek consumer ko deta tha. Baaki sab bhookhe.

Arjun finally said:

"We need a real streaming system. Not this jugaad."

Chapter 3: The Monday Meeting — Kafka Enters

The CTO walked into the meeting room and declared:

"We are moving to Kafka."

Arjun frowned. "Isn't Kafka too complex? Why do we need it?"

Meera smiled. "Kafka isn't overkill. It's exactly what we need."

What Kafka gives you:

Pro	What it means in practice
High throughput	Millions of events/sec on modest hardware
Durability	Events written to disk and replicated — a broker dying loses nothing
Scalability	Add partitions and consumers instead of rewriting the system
Replay	Reset an offset and reprocess history
Fault tolerance	Replicas take over automatically
Decoupled architecture	Producers don't know or care who consumes

And the cons, honestly:

Complex to operate
Resource-heavy
Genuine overkill for small apps

But SwiftKart had moved well beyond "small."

Chapter 4: Kafka Explained Like Arjun Understood It

Meera drew a diagram on the whiteboard and started with the sentence that reframed everything:

Kafka is not a queue. Kafka is a distributed, durable, replayable event log.

A queue deletes a message when someone reads it. A log doesn't. Readers just move a bookmark forward, and every reader has their own bookmark.

Then the vocabulary:

Term	What it is
Topic	A category of events, e.g. `order.events`
Partition	Parallel lanes inside a topic
Offset	A message's index within a partition
Producer	A service that sends events
Consumer	A service that reads events
Consumer group	A team of consumers sharing the load of one topic
Broker	A Kafka server that stores data
Cluster	A collection of brokers
Replication	Copies of data across brokers
Retention	How long data stays before deletion

The one that took Arjun longest: consumer group.

Consumers in the same group split partitions between them — that's how you scale. Consumers in different groups each get their own copy of every message — that's how notifications, analytics and fraud detection all read the same event stream without fighting over it.

That single distinction is what Redis could never do.

Chapter 5: The Migration Plan — Nobody Turns Redis Off on a Friday

Arjun's first instinct was to rip out Redis in one weekend.

Meera stopped him.

"Agar tumne ek hi raat mein switch kiya, aur Kafka mein koi bug nikla, toh 40,000 orders ka data kahan jayega? Migration is not a deploy. It's a slow, boring, reversible process."

They agreed on a three-week plan.

Week 1 — Dual write. Every service that pushed to Redis now also published the same event to Kafka. Redis stayed the source of truth; Kafka just quietly collected events. Zero risk — if Kafka broke, nothing broke.

Week 2 — Shadow consumers. New Kafka consumers ran alongside the old Redis workers, doing the same work but writing to a separate analytics table. Every morning Meera compared both tables. When the counts matched for seven days straight, they had proof.

Week 3 — Cutover, one consumer at a time. Notifications first (least damage if it broke). Analytics second. Fraud detection last. The Redis workers were disabled, not deleted, for another two weeks.

Then came the part Arjun underestimated: designing the topics.

His first draft had one topic called events. Meera rejected it immediately.

"One topic for everything means one retention policy for everything, one partition count for everything, and no way to give a team access to only what it needs."

The final design:

Topic	Partitions	Retention	Key
`order.events`	12	7 days	`orderId`
`payment.events`	6	30 days	`orderId`
`notification.requests`	6	1 day	`userId`
`order.events.dlq`	3	30 days	`orderId`

Two rules drove every choice.

1. The key decides the order.

Kafka guarantees ordering inside a partition, not across a topic. Keying by orderId means every event for order #5512 — CREATED, PACKED, PICKED_UP, DELIVERED — lands in the same partition and is read in sequence. Key it by something random and you will eventually deliver an order before you accept it.

2. Partition count is your maximum parallelism.

A consumer group can never have more active consumers than partitions. 12 partitions = up to 12 notification workers, and the 13th sits idle. You can add partitions later, but adding them rehashes the keys, so old and new events for the same order can land in different partitions. Over-provision a little on day one.

Chapter 6: The Code Arjun Actually Wrote

They used KafkaJS, since the whole backend was Node.js.

The event envelope

Before writing a single producer, Meera made him standardise the shape of every event:

// events/envelope.js
import { randomUUID } from "crypto";

export function buildEvent({ type, version = 1, orderId, payload }) {
  return {
    eventId: randomUUID(),   // for consumer-side deduplication
    type,                    // "ORDER_CREATED"
    version,                 // schema version — you WILL need this
    orderId,
    occurredAt: new Date().toISOString(),
    payload,
  };
}

"Version field abhi useless lagta hai," she said. "Six months baad jab tum payload change karoge aur purane consumers crash honge, tab samajh aayega."

The producer

// kafka/producer.js
import { Kafka, Partitioners, CompressionTypes } from "kafkajs";

const kafka = new Kafka({
  clientId: "order-service",
  brokers: process.env.KAFKA_BROKERS.split(","),
  ssl: true,
  sasl: {
    mechanism: "scram-sha-512",
    username: process.env.KAFKA_USER,
    password: process.env.KAFKA_PASS,
  },
  retry: { retries: 8, initialRetryTime: 300 },
});

const producer = kafka.producer({
  idempotent: true,              // no duplicates on internal retries
  maxInFlightRequests: 5,
  createPartitioner: Partitioners.DefaultPartitioner,
});

await producer.connect();

export async function publishOrderEvent(event) {
  await producer.send({
    topic: "order.events",
    acks: -1,                    // wait for all in-sync replicas
    compression: CompressionTypes.GZIP,
    messages: [
      {
        key: event.orderId,      // ← this is what guarantees ordering
        value: JSON.stringify(event),
        headers: {
          eventType: event.type,
          version: String(event.version),
        },
      },
    ],
  });
}

Three lines do the heavy lifting:

idempotent: true — when the producer retries after a network blip, the broker recognises the duplicate and drops it. Without this, retries create duplicate events.
acks: -1 — the send only succeeds once every in-sync replica has the message. acks: 1 is faster and loses data when a leader dies mid-write.
key: event.orderId — ordering, as above. Omit it and Kafka round-robins your events across partitions.

The consumer

// consumers/notification.consumer.js
const consumer = kafka.consumer({
  groupId: "notification-service",  // each service = its own group
  sessionTimeout: 30000,
  heartbeatInterval: 3000,
});

await consumer.connect();
await consumer.subscribe({ topic: "order.events", fromBeginning: false });

await consumer.run({
  autoCommit: false,                // commit only after real success
  eachMessage: async ({ topic, partition, message, heartbeat }) => {
    const event = JSON.parse(message.value.toString());
    const nextOffset = (BigInt(message.offset) + 1n).toString();

    try {
      if (await alreadyProcessed(event.eventId)) return;  // dedupe
      await sendNotification(event);
      await markProcessed(event.eventId);
    } catch (err) {
      await sendToDLQ(event, err);  // park it, don't block the partition
    }

    await consumer.commitOffsets([{ topic, partition, offset: nextOffset }]);
    await heartbeat();              // long jobs: prove you're still alive
  },
});

process.on("SIGTERM", async () => {
  await consumer.disconnect();      // commit in-flight work, leave the group cleanly
  process.exit(0);
});

Two details worth pausing on. You commit the next offset to read, not the one you just processed — hence the + 1. And offsets are int64, which is why KafkaJS hands them to you as strings; BigInt keeps that safe once your topics get busy.

And here's the part that made Arjun finally get it:

Notifications, analytics and fraud each ran the same subscribe call — same topic, different groupId.

All three received every single event, independently, at their own pace. Analytics could be an hour behind and notifications wouldn't care. Redis could never do this. That was the whole point.

Chapter 7: The Four Bugs Nobody Warns You About

Kafka went live. Then Kafka started teaching.

Bug 1 — Duplicates came back anyway

"Meera, maine idempotent producer lagaya tha. Phir bhi ek user ko do SMS gaye!"

Producer idempotence only prevents duplicates on the write side. On the read side Kafka is at-least-once: if a consumer processes a message and dies before committing the offset, the next consumer reads it again.

The fix isn't in Kafka — it's in your consumer. Give every event a unique eventId, store processed IDs behind a unique constraint, and skip anything you've seen. That's the alreadyProcessed() call above.

Assume every message will arrive twice. Design for it.

Bug 2 — The rebalancing death spiral

One notification took 45 seconds (a slow third-party SMS gateway). sessionTimeout was 30 seconds. Kafka assumed the consumer was dead, evicted it, and triggered a rebalance. The rebalance paused every other consumer. Those consumers then timed out too.

Three fixes, in order of preference: call heartbeat() during long work, move the slow call off the consumer path entirely, or raise sessionTimeout as a last resort.

A consumer that goes quiet is a consumer Kafka will evict.

Bug 3 — One bad message froze an entire partition

A malformed payload threw on every retry. Kafka doesn't skip it — it can't, because skipping would break ordering. So the consumer retried forever and every message behind it waited.

That's what order.events.dlq is for: catch, park the poison message in a dead-letter topic with the error attached, commit the offset, move on. Fix it offline.

Without a DLQ, one bad record can stall a partition indefinitely.

Bug 4 — Consumer lag was invisible until it wasn't

Meera added lag monitoring on day three:

kafka-consumer-groups.sh --bootstrap-server $BROKER \
  --describe --group notification-service

The LAG column is the single most important number in a Kafka system — it tells you how far behind reality your consumers are. They alerted at lag > 10,000 and scaled consumers (never beyond the partition count) when it fired.

And the security bits Arjun almost shipped without

The brokers were exposed with PLAINTEXT initially. They moved to SASL/SCRAM over TLS — the config in the producer above.
Every service got its own credentials and ACLs. The notification service can read order.events and nothing else; it has no business writing to payment.events.
No raw PII in event payloads. Kafka retains data for days and every consumer group reads everything. Send userId and let the consumer look up the phone number. A topic is not a place to store card numbers.

Chapter 8: Two Months Later

Arjun opened the dashboard on a Monday morning. Orders had crossed 90,000/day.

Nothing was on fire.

	Before (Redis queue)	After (Kafka)
Notification delay	up to 15 min	under 2 sec
Peak throughput	~1,200 events/min	40,000+ events/min
Broker restart	messages lost	messages survive
Replay after a bug	impossible	reset offset, reprocess
Consumers per event	1	unlimited groups
Slow consumer	took down the system	falls behind, alone

The moment that convinced the CTO wasn't the throughput number.

Analytics had a bug for six days — a wrong tax calculation on every order. Under Redis, that data was consumed and gone; six days of reporting would have been permanently wrong. With Kafka, Meera reset the analytics consumer group's offset to a week earlier and reprocessed everything in forty minutes.

"Kafka isn't just faster," she said. "It gave you a time machine for your data."

But please don't copy this blindly

Kafka is the wrong answer for a lot of teams, and Arjun is very clear about that now:

Under ~10,000 events/day? Redis, BullMQ or a database-backed queue will serve you better at a fraction of the operational effort.
Need per-message delay, priority or scheduling? That's a job queue's job. Kafka has no concept of "run this one first" or "run this in 10 minutes."
Nobody to own it? Kafka isn't a library you install, it's infrastructure someone maintains — partitions, retention, rebalances, lag, upgrades. (Modern Kafka runs on KRaft, so at least ZooKeeper is no longer part of the deal.)
One consumer and no replay requirement? Then you're paying Kafka's complexity tax for features you don't use.

Kafka solved SwiftKart's problem because SwiftKart had exactly the problem Kafka is built for: high volume, multiple independent consumers, and a hard need for durability and replay.

If that's not your problem, it's not your tool.

If you're running Kafka in production, I'd love to hear how you sized your partitions — it's the one decision that's genuinely hard to undo.

JS + DSA Roadmap (3–4 Months)

Nitin Malviya — Mon, 27 Oct 2025 06:50:58 +0000

Month 1: Foundation & Recursion

Topics:

Recursion basics (factorial, Fibonacci, sum of array, reverse string, power of number)
Time & Space Complexity basics (O(1), O(n), O(n²), O(log n))
Iterative vs Recursive solutions comparison

Practice:

Solve 10–12 recursion problems
Analyze TC & SC for each
Dry run + recursion stack understanding

Month 2: Arrays & Linked Lists

Topics:

Arrays (search, sort, sliding window, two pointers)
Linked List: Single, Doubly, Circular
Operations: Insert, Delete, Reverse, Merge, Detect Loop

Practice:

5 problems per topic (Array + Linked List)
TC/SC analysis for each solution
Pattern recognition (sliding window, fast/slow pointers)

Month 3: Stack, Queue, Hash Table, Trees

Topics:

Stack (LIFO, parenthesis check, min stack)
Queue (FIFO, circular queue, deque)
Hash Table (frequency map, two sum)
Trees: Binary Tree, Binary Search Tree
Tree traversal: Inorder, Preorder, Postorder, Level Order

Practice:

5 problems per topic
TC/SC analysis + recursion where applicable
Patterns: recursion in trees, hash map tricks

Month 4: Advanced DS & Algorithms

Topics:

Heap (min/max heap, priority queue)
Suffix Tree, Trie, Disjoint Set
Graphs: BFS, DFS, shortest path
Sorting: Merge, Quick, Heap
Greedy, Backtracking, Dynamic Programming basics

Practice:

5 problems per topic
Analyze TC/SC + identify patterns (divide & conquer, memoization, greedy choice)
Start with easy → medium → hard problems

Daily/Weekly Practice Suggestion

Daily: 1–2 problems + TC/SC analysis
Weekly: 1 topic summary + patterns + mini project if possible
Monthly: 2–3 mock problem sets (Leetcode/Hackerrank)

Extra Tips

Use JS throughout (Node.js environment)
Dry run solutions on paper before coding
Note patterns + tricks in Notion for future reference
Start integrating DSA in small projects (e.g., book management features)

Frontend & Java script Interview Questions

Nitin Malviya — Mon, 27 Oct 2025 06:47:16 +0000

Core JavaScript

Difference between var, let, and const.
Explain hoisting in JavaScript.
Difference between == and ===.
Explain this keyword in different contexts (global, function, class, arrow function).
Difference between call, apply, and bind.
Event bubbling vs event capturing.
What is closure? Give example.
Difference between null and undefined.
Difference between map, forEach, filter, reduce.
What is IIFE (Immediately Invoked Function Expression)?
Difference between synchronous and asynchronous JavaScript.
What is the event loop? Explain microtasks and macrotasks.
Difference between shallow copy and deep copy — how to implement.
Difference between Object.assign() and spread operator.
Explain prototype and prototype chain.
Explain ES6 modules and CommonJS modules difference.
What are Promises and how do they work?
Difference between async/await and .then/.catch().
How does garbage collection work in JS?
Difference between for…in and for…of.

Advanced / Scenario-Based JS

Implement debounce and throttle functions — explain use case.
Difference between mutable and immutable objects in JS.
Explain memory leak scenarios in JS and how to prevent.
Implement a function to deep clone nested objects.
Difference between setTimeout(fn, 0) and Promise.resolve().then(fn).
Explain Symbol and its usage.
Difference between generator function and async generator.
How to handle large array processing without blocking UI (chunking).
Explain WeakMap and WeakSet.
Difference between Object.freeze() and const in objects.

Tricky JavaScript Questions (Advanced)

What's the output?

javascript

console.log(typeof typeof 1); console.log(0.1 + 0.2 === 0.3); console.log([] == ![]); console.log(+'10' + 5);

Explain the execution order:

javascript

console.log('1'); setTimeout(() => console.log('2'), 0); Promise.resolve().then(() => console.log('3')); console.log('4');

Temporal Dead Zone (TDZ) — explain with example and why it exists.
What happens when you do this?

javascript

const obj = { a: 1 }; const arr = [obj, obj]; arr[0].a = 2; console.log(arr[1].a); *// ?*

Explain currying — implement a curry function that works for any function.
What's wrong with this code?

javascript

for (var i = 0; i < 3; i++) { setTimeout(() => console.log(i), 100); }

How to fix it? (Multiple solutions)

Implement a pipe/compose function — explain the difference.
Explain Proxy and Reflect API — practical use case (e.g., validation).
What is the difference between Object.create(null) and {}?
Implement a function that executes promises in sequence (not parallel).
Explain the difference between:

javascript

`function foo() { return { bar: "hello" }; }
function foo() { 
  return 
  { bar: "hello" }; 
}`

What is the new keyword doing internally? Implement your own new operator.
Explain tail call optimization — does JS support it?
Implement a memoization function that works for any function with any number of arguments.
What's the output and why?

javascript

`const promise = new Promise((resolve) => {
  console.log(1);
  resolve();
  console.log(2);
});
promise.then(() => console.log(3));
console.log(4);`

How do you detect if a function is called with new?
Implement Promise.all, Promise.race, and Promise.allSettled from scratch.
What is the difference between Promise.then() chaining and async/await error propagation?
Explain the "double equals quirks":

javascript

`[] == ![] *// true*
{} == !{} *// false*
[] == [] *// false*`

What's the difference between arguments and rest parameters? Why avoid arguments?

Advanced Memory & Performance Questions

Memory leak scenario: Event listeners not removed — how to detect and fix?
Explain V8's garbage collection algorithms: Mark-and-sweep, Scavenge, Major GC.
How to profile memory usage in Node.js? Tools and approaches.
What causes memory leaks in closures? Give specific example.
Explain hidden classes and inline caching in V8 — how to write optimized code?
What is the -max-old-space-size flag? When to use it?
How do you handle 10GB file processing in Node.js without loading it all into memory?
Implement a LRU (Least Recently Used) cache with O(1) get and put operations.
Explain memory fragmentation — how does it affect Node.js applications?
How to detect and fix event loop blocking in production?

React / Next.js / State Management Questions

React Core

Difference between class components and functional components.
What is virtual DOM and how React uses it?
Explain React lifecycle methods (class) and hooks equivalents.
Explain useState and useEffect — common pitfalls.
Explain useMemo and useCallback — when and why to use.
Difference between controlled and uncontrolled components.
Explain React context API and its use case.
How to prevent unnecessary re-renders in React.
Difference between key props in lists — why important.
Explain reconciliation in React.

React Advanced / Optimization

Explain code splitting and lazy loading in React.
Difference between React.memo, PureComponent, and useMemo.
How to optimize React app performance — list techniques.
Explain event delegation in React.
How to handle large forms efficiently.
How to handle error boundaries in React.
What is Suspense and Concurrent Mode in React.
Difference between client-side rendering and server-side rendering.

React Advanced & Tricky Questions

What's wrong with this code?

javascript

function Component() { const [count, setCount] = useState(0); const handleClick = () => { setCount(count + 1); setCount(count + 1); setCount(count + 1); }; return <button onClick={handleClick}>{count}</button>; }

How to fix it? Explain batching.

Explain the difference between:

javascript

useEffect(() => { */* ... */* }, []); useEffect(() => { */* ... */* }); useLayoutEffect(() => { */* ... */* }, []);

Why is this an infinite loop?

javascript

useEffect(() => { setData(fetchData()); }, [data]);

Implement a custom hook for debounced value.
What's the difference between:

javascript

<Component data={data} /> <Component data={[...data]} />

Explain stale closures in React hooks — give example and solution.
How to share stateful logic between components without Context or Redux? (Custom hooks)
What's the purpose of the key prop in React? Why not use index as key?
Implement useReducer from scratch — explain when to use it over useState.
Explain React Fiber architecture — how it enables concurrent rendering.
How does React decide when to batch state updates? Automatic batching in React 18.
What's the difference between flushSync and normal state updates?
How to prevent child component re-render when parent re-renders? (Multiple techniques)
Explain React's reconciliation algorithm — Diff algorithm in detail.
What happens when you call setState in render method? Why is it dangerous?
Implement a custom usePrevious hook that returns previous value of a state.
How to handle race conditions in useEffect with API calls?
Explain ref forwarding and useImperativeHandle — when to use?
What's the difference between synthetic events and native events in React?
How to optimize context to prevent unnecessary re-renders? Split context technique.

Next.js Advanced Questions

Difference between SSR, SSG, ISR, and CSR in Next.js.
Explain getServerSideProps vs getStaticProps.
Explain API routes in Next.js.
How to implement dynamic routing in Next.js.
Explain Next.js image optimization.
How to handle authentication in Next.js (JWT, cookies).
Explain incremental static regeneration (ISR).
How to optimize Next.js app performance (bundle splitting, caching).

Next.js Advanced & Tricky

How does Next.js handle automatic code splitting? Can you control it?
Explain the difference between next/link and regular anchor tags — prefetching behavior.
How to implement middleware in Next.js 13+? Use cases and limitations.
Explain App Router vs Pages Router — migration strategy.
How to handle environment variables securely in Next.js? NEXT_PUBLIC_ prefix.
What's the difference between:

javascript

`export const revalidate = 60; *// ISR*
export const dynamic = 'force-dynamic'; *// SSR*
export const dynamic = 'force-static'; *// SSG*`

How to implement edge functions in Next.js? Difference from serverless.
Explain streaming SSR in Next.js 13+ — Suspense boundaries.
How to optimize fonts in Next.js? next/font system.
What happens when you use use client directive? Bundle implications.
How to handle SEO for dynamic routes? generateMetadata function.
Explain parallel and intercepting routes in App Router.

Redux / State Management

Difference between Redux and React context.
Explain Redux flow — actions, reducers, store.
How to avoid unnecessary re-renders with Redux.
Difference between middleware in Redux — thunk, saga.
How to structure Redux state for large apps.
Explain Redux Toolkit and why it's preferred.
How to handle async API calls in Redux.
How to persist Redux state in localStorage/sessionStorage.

Redux Advanced & Tricky

Why is Redux state immutable? What happens if you mutate?
Explain Redux middleware — implement a custom logger middleware.
What's the difference between:

javascript

`useSelector(state => state.user)
useSelector(state => ({ user: state.user }))`

Re-render implications.

How does Redux Toolkit's createSlice work under the hood? Immer integration.
Explain Redux Saga effects: call, put, take, fork, spawn.
How to handle optimistic updates in Redux?
What's the difference between combineReducers and a single reducer?
How to implement time-travel debugging? Redux DevTools internals.
Explain selectors and reselect library — memoization benefits.
How to handle authentication state in Redux? Token refresh flow.
What's the difference between Redux Thunk and Redux Saga? When to use each?
How to test Redux reducers and actions? Best practices.

Backend Interview Questions

Nitin Malviya — Mon, 27 Oct 2025 06:45:32 +0000

Core Node.js Questions

Node.js single-threaded hai — phir bhi multiple requests efficiently kaise handle karta hai? Explain event loop and non-blocking I/O.
Difference between process.nextTick() and setImmediate().
How does Node.js handle 1000 concurrent requests? Explain async nature and event loop.
Explain Event Loop phases.
What are microtasks and macrotasks in Node.js? Give examples.
How does Node.js handle CPU-intensive tasks?
How do you debug performance issues (CPU/memory) in Node.js?
Difference between require and import in Node.js.
Explain Streams in Node.js and give a scenario where they are useful.
Difference between fs.readFile and fs.createReadStream.

Node.js Advanced & Tricky Questions

Explain the 6 phases of Node.js event loop in detail:
- Timers
- Pending callbacks
- Idle, prepare
- Poll
- Check
- Close callbacks
What's the order of execution?

javascript

`setImmediate(() => console.log('immediate'));
process.nextTick(() => console.log('nextTick'));
setTimeout(() => console.log('timeout'), 0);
Promise.resolve().then(() => console.log('promise'));
console.log('sync');`

How do you handle CPU-intensive tasks without blocking the event loop?
- Worker threads
- Child processes
- Clustering
Explain the difference between fork(), spawn(), exec(), execFile() in child_process.
What is libuv? Role in Node.js architecture.
How does Node.js utilize multiple CPU cores? Cluster module.
Explain backpressure in streams — how to handle it?
What's the difference between readable, writable, duplex, and transform streams?
Implement a custom transform stream that converts input to uppercase.
How do you prevent thread starvation in Node.js?
Explain Node.js module resolution algorithm — node_modules lookup.
What's the purpose of package-lock.json? Difference from package.json.
How to implement graceful shutdown in Node.js? Handle ongoing requests.
Explain the uncaughtException and unhandledRejection events — best practices.
What's the difference between Buffer and Uint8Array?
How to detect memory leaks in production Node.js apps? Tools: heapdump, clinic.
Explain Node.js security best practices:
- Helmet
- Rate limiting
- Input validation
- HTTPS
- Dependency scanning
What is N-API? Native addons in Node.js.
How do you monitor Node.js application performance? Metrics to track.
Explain the -inspect flag — debugging production apps.

Express.js Questions

What is middleware in Express? Types and examples.
Explain error-handling middleware in Express.
How to handle async errors in Express routes?
How does Express handle routing internally?
Difference between app.use() and router.use().
How do you handle file uploads in Express?
Explain rate limiting in Express — scenario where multiple requests come simultaneously.
How to implement CORS in Express?
How to secure Express app against common vulnerabilities (XSS, CSRF, SQL injection)?
How do you implement logging in Express applications?

Express.js Advanced & Tricky

What's the order of middleware execution? How to control it?
How do you handle async errors without try-catch in every route?

javascript

`const asyncHandler = (fn) => (req, res, next) => {
  Promise.resolve(fn(req, res, next)).catch(next);
};`

Explain the request-response cycle in Express — detailed flow.
How to implement custom middleware for authentication? JWT verification.
What's the difference between app.route() and router.route()?
How to handle multipart/form-data? Multer internals.
Implement a middleware that logs request time for every route.
How to implement CSRF protection? Token-based approach.
Explain the difference between:

javascript

`app.use('/api', router);
app.use(router);`

How to implement request validation? Joi, express-validator.
What happens if you call next() multiple times? Common mistakes.
How to implement compression in Express? gzip, brotli.
Explain the res.locals object — use cases.
How to implement API versioning? Multiple approaches.
What's the difference between res.send(), res.json(), res.end()?

MongoDB / Mongoose Questions

Difference between find(), findOne(), and findById().
Explain aggregation pipeline with an example.
What is indexing? How does it improve query performance?
What is populate() in Mongoose? Give example.
Difference between embedded documents and references.
How to design schema for a blog or e-commerce app?
How to handle large collections efficiently?
Explain transactions in MongoDB and when to use them.
How to implement pagination in MongoDB?
How to handle schema validation in Mongoose?

MongoDB Advanced & Tricky Questions

Explain MongoDB ACID properties — transaction support.
What's the difference between:

javascript

`Model.find({})
Model.find({}).lean()
Model.find({}).cursor()`

How does MongoDB indexing work internally? B-tree structure.
Explain compound indexes — order matters?
What's the difference between covered query and indexed query?
Implement text search in MongoDB — full-text search indexes.
How to handle schema versioning? Migration strategies.
Explain sharding in MongoDB — when to use it?
What's the difference between replica set and sharding?
How to optimize aggregation pipeline performance? $match early, indexes.
Explain the aggregation stages:
- $match, $group, $project, $sort, $limit, $skip
- $lookup (joins), $unwind, $addFields
How to implement geospatial queries? 2dsphere indexes.
What's the difference between:

javascript

`{ $set: { name: 'John' } }
{ name: 'John' }`

In update operations.

Explain write concerns in MongoDB — w, j, wtimeout.
How to handle large file storage? GridFS.
What's the difference between updateOne, updateMany, replaceOne?
Implement soft delete with Mongoose — plugin approach.
How to implement data auditing? Timestamp tracking, change history.
Explain the Mongoose middleware — pre/post hooks.
What's the difference between:

javascript

`schema.virtual()
schema.method()
schema.static()`

Authentication / Authorization Questions

Explain JWT authentication flow.
Difference between access token and refresh token.
How to secure API routes in Express using JWT?
How to implement role-based access control (RBAC) in Express?
How to store JWT securely on client-side?

Auth Advanced & Tricky

What's inside a JWT token? Header, payload, signature — explain each.
How to handle token refresh without logout? Refresh token rotation.
What's the difference between stateless and stateful authentication?
Implement OAuth 2.0 flow — authorization code grant.
How to prevent JWT replay attacks? Nonce, jti claim.
Explain the difference between httpOnly and secure cookies.
How to implement multi-factor authentication (MFA)? TOTP algorithm.
What's the difference between authentication and authorization?
How to implement API key authentication? Rate limiting per key.
Explain session-based authentication — vs JWT comparison.
How to handle password reset securely? Token expiration, one-time use.
What is bcrypt and why use it? Salting, hashing rounds.
Implement permission-based access control — granular permissions.
How to secure refresh tokens? Rotation, revocation strategies.
What's the difference between Bearer token and API key?

Scenario-Based / Practical Questions

Async/Await vs Promises in error handling — reusable async wrapper.
MongoDB slow query on 1 million+ documents — how to optimize?
User token expired — how to handle refresh token scenario?
Multiple users uploading large files simultaneously — backend crash scenario, optimization.
Aggregation: Top 3 users with highest total orders in last month.
Public API exposed — prevent DDOS / brute force attacks using rate limiting.
Express app running slow under load — how to debug performance issues.
You need to migrate MongoDB schema without downtime — approach?
Implement search functionality in MongoDB with text indexes.
How to implement soft delete (logical delete) in MongoDB.
Implement caching in Node.js for frequently accessed DB data (Redis / memory).
Scenario: Real-time notifications using Node.js and MongoDB — approach.
Scenario: File upload + image processing pipeline — handle asynchronously.
Scenario: Multiple microservices interacting with one MongoDB — how to maintain consistency?
Scenario: Multi-tenancy in Node.js + MongoDB application.

Extreme Scenario-Based Questions (Real Production Issues)

Your API suddenly returns 502 Bad Gateway for all requests — where do you start debugging?
Memory usage keeps increasing in production Node.js app — hits 2GB and crashes. How to debug?
Database connection pool exhausted — 100 concurrent users, all waiting. What's wrong?
Race condition in checkout system — two users bought last item. How to prevent?
MongoDB aggregation taking 30+ seconds on 10M documents — optimize without changing schema.
JWT token size causing performance issues — payload too large. Solutions?
API rate limiter blocking legitimate users — shared IP (corporate network). How to handle?
Bulk email sending causing Express app to freeze — 10,000 emails. Architecture solution?
File upload fails for files > 50MB — timeout errors. Complete solution?
Redis cache hit rate is 5% — supposed to be 80%+. What's wrong?
Mongoose query returns all 1M documents instead of paginated results — catastrophic error. Why?
WebSocket connections keep dropping every 60 seconds — load balancer issue?
Database indexes not being used — explain query plan shows COLLSCAN. Debug process?
Concurrent API calls updating same document cause data loss — optimistic locking solution.
Authentication fails randomly for some users — JWT verification errors. Debugging approach?
Image processing queue backed up — 10,000 pending jobs. How to handle gracefully?
Circular dependency in Node.js modules — causing undefined exports. How to resolve?
API returns stale data despite database updates — caching issue. Multi-layer cache invalidation strategy?
Server crashes when processing CSV with 5M rows — memory-efficient streaming solution.
N+1 query problem in GraphQL — 1000 database queries for single request. Solutions?
WebSockets not scaling horizontally — sticky sessions, Redis adapter solution.
Transaction deadlock in MongoDB — two operations waiting for each other. Prevention?
CDN serving old JavaScript bundle — aggressive caching. Cache busting strategies?
Automated tests passing but production failing — environment differences. Investigation steps?
API response time increased from 100ms to 5s overnight — no code changes. What to check?

System Design & Architecture Questions

Design a URL shortener (like bit.ly) — complete backend architecture.
Design a rate limiter — support distributed systems.
Design a real-time chat application — WebSockets, scaling strategy.
Design a file storage system (like Dropbox) — chunking, deduplication.
Design a notification system — email, SMS, push notifications.
Design an e-commerce cart system — handle race conditions, inventory management.
Design a job queue system — Bull, Redis, failure handling.
Design an analytics dashboard — handling time-series data efficiently.
Design a payment gateway integration — security, idempotency.
Design a microservices architecture for a blog platform — service boundaries, communication.

Performance & Optimization Questions

How to optimize a Next.js app that has 3-second initial load time? Step-by-step approach.
API endpoint taking 10 seconds to respond — profiling and optimization strategy.
React app re-rendering 100 times on single user action — identify and fix.
MongoDB query using 4GB of RAM — aggregation optimization techniques.
Node.js app using 100% CPU — identify bottleneck without production downtime.
Implement caching strategy for REST API — cache invalidation, TTL strategies.
How to reduce Docker image size from 2GB to 200MB? Best practices.
Optimize bundle size in React app — tree shaking, code splitting, dynamic imports.
Database connection pool sizing — how to determine optimal pool size?
Implement database read replicas — read/write split in Node.js application.

DevOps & Deployment Questions

Explain CI/CD pipeline for Node.js application — GitHub Actions, Jenkins.
Zero-downtime deployment strategy — blue-green, canary deployment.
How to handle environment variables securely? Secrets management.
Explain Docker multi-stage builds — optimize image layers.
How to monitor Node.js app in production? PM2, New Relic, Prometheus.
Implement health check endpoint — readiness vs liveness probes.
How to handle database migrations in production? Rollback strategy.
Explain horizontal vs vertical scaling — when to use each?
How to implement logging in distributed systems? Centralized logging (ELK stack).
Container orchestration — Kubernetes basics for Node.js apps.

Code Review & Best Practices Questions

Review this code — find all issues:

javascript

app.get('/users/:id', (req, res) => { User.findById(req.params.id).then(user => { res.json(user); }); });

What's wrong with this Mongoose schema?

javascript

const userSchema = new Schema({ email: String, password: String });

Review this React component:

javascript

function List({ items }) { return items.map(item => <div>{item.name}</div>); }

What's the security issue here?

javascript

app.get('/search', (req, res) => { const query = req.query.q; db.collection('posts').find({ $where: query }); });

Identify the performance problem:

javascript

useEffect(() => { const data = expensiveOperation(); setData(data); });

Testing Questions

How to test async Express routes? Jest, Supertest.
Difference between unit, integration, and E2E tests — examples for each.
How to mock MongoDB in tests? mongodb-memory-server.
Testing React components — React Testing Library best practices.
How to test private functions? Should you test them?
Implement test coverage — what percentage is ideal?
How to test error scenarios? Error boundaries, API failures.
Testing WebSocket connections — approach and tools.
How to handle flaky tests? Identification and fixes.
Test database transactions — rollback after each test.

Bonus: Algorithm & Data Structure Questions

Implement LRU Cache — O(1) get and put.
Two Sum problem — optimal solution.
Reverse a linked list — iterative and recursive.
Validate balanced parentheses — stack approach.
Find duplicate in array — without extra space.
Implement debounce and throttle — from scratch.
Binary search — find element in sorted array.
Merge two sorted arrays — in-place.
Implement a trie (prefix tree) — autocomplete use case.
Detect cycle in linked list — Floyd's algorithm.