System Design: A popular cache key expires and 40,000 requests hit your database. How do you fix it?

Gaurav Sharma — Wed, 12 Aug 2026 19:49:37 +0000

You are in a Staff Engineer system design interview. The interviewer draws a simple architecture on the whiteboard: an API, a Redis cache, and a Postgres database.

Then they drop the scenario:

"A highly popular key reaches its TTL (Time-To-Live) and expires. In the exact same second, 40,000 requests arrive looking for that key. What happens, and what do you change to prevent the database from melting?"

Most engineers immediately start talking about scaling the database or adding read replicas. But if you say that, you've missed the hidden signal the interviewer is actually scoring you on.

Here is exactly how a Staff Engineer tackles the Cache Stampede (also known as a Thundering Herd) problem.

1. Do the arithmetic before you name a fix

Take the premise at 40,000 requests a second and assume building the cache value takes 200 milliseconds (a complex join or calculation).

Every single request that arrives while the rebuild is in-flight will register as a cache miss, because nothing has been written back to Redis yet. That is 8,000 requests in the 200ms gap, and every one of them starts its own copy of the exact same query against the database.

The database is not serving 8,000 different questions. It is serving one question 8,000 times.

State that number out loud in an interview. It changes the shape of the answer, because it shows the problem is duplication rather than volume. You do not need a bigger database; you need 7,999 of those requests to stop asking.

(Note: The same arithmetic tells you when this is not worth solving. If the rebuild takes 2 milliseconds, the gap holds 80 requests. 80 duplicate queries is a spike your database won't even notice. Cache stampede is a function of rebuild latency, not popularity alone).

2. The Fix: Serialise or serve stale

There are two families of fixes, and the choice between them is a product decision rather than a technical one.

Option A: Single-flight (Serialise)

The first request to miss takes a short-lived lock on the key in Redis, does the DB query, writes the value, and releases the lock. Every other request finds the lock held and waits, then reads the freshly-written value.

In Redis, this looks like SET key:lock token NX EX 5 (create only if absent, expire after 5 seconds).

The Trade-off: Everybody gets fresh data, but everyone also pays the full rebuild latency. Your p99 latency for that second spikes to 200 milliseconds.

Option B: Stale-while-revalidate

Store the value with its own logical "freshness" time and keep it in the cache past that point. A reader arriving past the freshness time returns the old bytes immediately to the user, but triggers an asynchronous refresh in the background.

The Trade-off: Latency stays completely flat for everyone, but readers see data up to one rebuild cycle (200ms) old.

Say which you are choosing and why: "This is a product listing, so I serve stale for 200 milliseconds; but if this were an account balance, I’d use the lock." That single sentence does more for you in an interview than reciting every technique in existence.

3. The stampede you cause next week

Fixing the rebuild doesn't fix the pattern that produced it. If every key is written with the exact same lifetime by the same warm-up job, they all expire together. The expiry itself becomes a synchronised event.

You must add jitter at write time so the herd disperses.

import random

# A nominal five-minute life, spread over the last 10 per cent of it.
# Ten thousand keys written together will no longer expire together.
BASE_TTL = 300
ttl = BASE_TTL - random.randint(0, BASE_TTL // 10)
cache.set(key, value, ex=ttl)

4. What happens when the lock fails?

The single-flight lock answer is only complete if you explain what happens when the rebuilding request dies.

Suppose the single request holding the lock times out against the database. The lock is still held in Redis until its own TTL expires. Every waiter is blocked on a value that is never coming. If they wait naively, you've converted a stampede into a stall, which is harder to diagnose because the database looks perfectly healthy!

To survive this:

The lock must carry a short expiry so it cannot outlive a dead holder.
Waiters must have their own deadline, shorter than the lock's, after which they return an error rather than waiting indefinitely.

Know the Rubric. Pass the Interview.

This is just one of over 900 highly-detailed technical scenarios we've built at Preptima.

Most interview platforms just give you a model answer. Preptima gives you the hidden grading rubrics that Staff and Principal engineers actually use to score you.

If you are preparing for a backend, data, or system design interview, stop guessing what the interviewer wants. Check out the full Preptima curriculum here and get the scorecard.

Designing a Practical IIoT Architecture

Gaurav Sharma — Tue, 31 Mar 2026 16:25:27 +0000

Introduction

Industrial IoT (IIoT) architectures often look clean in diagrams but become messy when implemented in real systems.

While exploring this space, I tried to structure a more practical reference architecture that balances clarity with real-world concerns like scalability, reliability, and multi-tenancy.

This is a work in progress, and I’d love feedback from others building similar systems.

The Problem with Typical IoT Architectures

Most reference architectures:

stay too high-level
don’t separate concerns clearly (telemetry vs command)
ignore multi-tenancy
don’t address edge vs cloud trade-offs

This creates challenges when moving from POC → production.

Proposed Architecture Overview

At a high level, the system is divided into four layers:

1. Device & Controller Layer

Sensors, controllers, PLCs
Communicate via MQTT or similar lightweight protocols
Separate channels for telemetry and command

2. Edge Layer

Local processing and filtering
Handles intermittent connectivity
Reduces latency for critical operations

3. Cloud Ingestion Layer

MQTT broker → event streaming pipeline
Separation of:
- telemetry (data ingestion)
- command (control path)

4. Application Layer

Multi-tenant SaaS architecture
Monitoring dashboards
Command and control APIs

Key Design Considerations

Telemetry vs Command Separation

Combining both leads to complexity. Keeping them separate improves scalability and clarity.

Edge vs Cloud Responsibilities

Edge: real-time, low-latency decisions
Cloud: aggregation, analytics, orchestration

Multi-Tenancy

Needs to be built-in from the start (not bolted on later).

Open Questions / Trade-offs

Some areas I’m still exploring:

How to design reliable command execution (ACK/NACK, retries, idempotency)
Where to draw the boundary between edge and cloud
Best approaches for tenant isolation at scale

Reference

I’ve put together a more detailed version (with diagrams) here:

👉 https://gauravs19.github.io/iiot-reference-architecture/

Looking for Feedback

If you’ve worked on IoT/IIoT systems:

What patterns have worked well for you?
Where do architectures typically break?
What would you change here?

Open to suggestions and alternative approaches.

DEV Community: Gaurav Sharma