You are in a Staff Engineer system design interview. The interviewer draws a simple architecture on the whiteboard: an API, a Redis cache, and a Postgres database.
Then they drop the scenario:
"A highly popular key reaches its TTL (Time-To-Live) and expires. In the exact same second, 40,000 requests arrive looking for that key. What happens, and what do you change to prevent the database from melting?"
Most engineers immediately start talking about scaling the database or adding read replicas. But if you say that, you've missed the hidden signal the interviewer is actually scoring you on.
Here is exactly how a Staff Engineer tackles the Cache Stampede (also known as a Thundering Herd) problem.
1. Do the arithmetic before you name a fix
Take the premise at 40,000 requests a second and assume building the cache value takes 200 milliseconds (a complex join or calculation).
Every single request that arrives while the rebuild is in-flight will register as a cache miss, because nothing has been written back to Redis yet. That is 8,000 requests in the 200ms gap, and every one of them starts its own copy of the exact same query against the database.
The database is not serving 8,000 different questions. It is serving one question 8,000 times.
State that number out loud in an interview. It changes the shape of the answer, because it shows the problem is duplication rather than volume. You do not need a bigger database; you need 7,999 of those requests to stop asking.
(Note: The same arithmetic tells you when this is not worth solving. If the rebuild takes 2 milliseconds, the gap holds 80 requests. 80 duplicate queries is a spike your database won't even notice. Cache stampede is a function of rebuild latency, not popularity alone).
2. The Fix: Serialise or serve stale
There are two families of fixes, and the choice between them is a product decision rather than a technical one.
Option A: Single-flight (Serialise)
The first request to miss takes a short-lived lock on the key in Redis, does the DB query, writes the value, and releases the lock. Every other request finds the lock held and waits, then reads the freshly-written value.
In Redis, this looks like SET key:lock token NX EX 5 (create only if absent, expire after 5 seconds).
- The Trade-off: Everybody gets fresh data, but everyone also pays the full rebuild latency. Your p99 latency for that second spikes to 200 milliseconds.
Option B: Stale-while-revalidate
Store the value with its own logical "freshness" time and keep it in the cache past that point. A reader arriving past the freshness time returns the old bytes immediately to the user, but triggers an asynchronous refresh in the background.
- The Trade-off: Latency stays completely flat for everyone, but readers see data up to one rebuild cycle (200ms) old.
Say which you are choosing and why: "This is a product listing, so I serve stale for 200 milliseconds; but if this were an account balance, Iād use the lock." That single sentence does more for you in an interview than reciting every technique in existence.
3. The stampede you cause next week
Fixing the rebuild doesn't fix the pattern that produced it. If every key is written with the exact same lifetime by the same warm-up job, they all expire together. The expiry itself becomes a synchronised event.
You must add jitter at write time so the herd disperses.
import random
# A nominal five-minute life, spread over the last 10 per cent of it.
# Ten thousand keys written together will no longer expire together.
BASE_TTL = 300
ttl = BASE_TTL - random.randint(0, BASE_TTL // 10)
cache.set(key, value, ex=ttl)
4. What happens when the lock fails?
The single-flight lock answer is only complete if you explain what happens when the rebuilding request dies.
Suppose the single request holding the lock times out against the database. The lock is still held in Redis until its own TTL expires. Every waiter is blocked on a value that is never coming. If they wait naively, you've converted a stampede into a stall, which is harder to diagnose because the database looks perfectly healthy!
To survive this:
- The lock must carry a short expiry so it cannot outlive a dead holder.
- Waiters must have their own deadline, shorter than the lock's, after which they return an error rather than waiting indefinitely.
Know the Rubric. Pass the Interview.
This is just one of over 900 highly-detailed technical scenarios we've built at Preptima.
Most interview platforms just give you a model answer. Preptima gives you the hidden grading rubrics that Staff and Principal engineers actually use to score you.
If you are preparing for a backend, data, or system design interview, stop guessing what the interviewer wants. Check out the full Preptima curriculum here and get the scorecard.
Top comments (0)