Every design round asked me about caching. Few of them asked directly.
The question arrives as latency, cost or load, and the answer is almost always a cache plus a story about what happens when it is wrong.
Written after a design round that stalled on invalidation. Updated October 2026. 8 min read
One request, two outcomes
client
cache
database
only on a miss
HIT, most of the time
2 ms
MISS, occasionally
90 ms
The short version: adding a cache is easy and interviewers know it. The marks are in what you say about staleness, eviction, what happens when many requests miss the same key at once, and how you would know the cache is helping. Say the hit rate you expect and what you would do if it came back at fifty percent.
The numbers in the illustration are typical shapes, not measurements of any specific system.
The follow up that ended a round
"What happens when the data changes?"
I had drawn a clean design with a cache in front of the database and felt good about it. The interviewer asked how a price update reaches a user who already has a cached page, and I produced the word invalidation and then nothing useful.
Anybody can add a cache. The interview is about what you do when it is wrong.
The four things to say
In this order.
1
What you cache
The exact key, and
why that
granularity.
2
How it is filled
Cache aside, read
through, or write
through.
3
How it goes stale
Time to live,
explicit
invalidation, or
both.
4
What breaks
Stampedes, hot keys,
memory pressure.
Four sentences that cover most caching follow ups.
Staleness is a product decision
Say that out loud.
How stale is acceptable depends on what the data is. A product description can be minutes old. A bank balance cannot. Asking what the acceptable staleness window is, before choosing a strategy, shows you treat correctness as a requirement rather than a default.
Usually fine when stale
Product descriptions
Search result lists
Recommendations
Public profile pages
Usually not fine
Account balances
Inventory at checkout
Permissions and roles
Anything used for a safety decision
Eviction and the failure modes
The vocabulary interviewers expect.
Least recently used
A sensible default when recent access predicts future access.
Time to live
Simple, and it bounds how wrong you can be.
Cache stampede
Many misses on one hot key at once. Coalesce requests or stagger expiry.
Hot key
One key taking a disproportionate share of traffic, which can overwhelm a single node.
How would you know it is working
The question nobody prepares for.
Name the metrics before being asked: hit rate, latency at the ninety fifth percentile with and without the cache, and database load. Then say what each number would make you do. A hit rate of fifty percent usually means your key is wrong or your time to live is too short, not that caching does not help here.
How I practised design follow ups
Design it, then attack it.
My routine was to sketch a design on paper and then have it attacked rather than extended. I used Phantom Code AI[ in its slower reasoning mode for that, describing my cache placement and asking specifically what breaks when a key goes hot or when the data changes mid request. The invalidation story was reliably my weak point. Because it also handles coding, SQL and behavioral practice, the same evening habit carried the rest of the loop.
Common questions
Short answers.
Main strategies?
Why is invalidation hard?
Eviction policies?
What is a stampede?
Cache answers are about being wrong well
Adding the box is the trivial part of a design. This question survives in every loop because the interesting part is what happens at the edges.
Take any design you have sketched and write one sentence about what happens when the underlying data changes.
Written after an invalidation question I now enjoy
What is the worst cache bug you have seen?
Mine served one customer another customer's dashboard.
Top comments (1)
The dashboard example at the end makes the cache-key question very concrete: which tenant, user and permission-dependent dimensions belong in the key? A high hit rate wouldn't make cross-user leakage acceptable.
For a smaller coding exercise alongside the design discussion, Databricks guide has a worked persistent cache with overwrite, eviction and restart traces.