Recap
Day 1 was the big picture (client-server, scaling, latency vs. throughput). Day 2 went into databases (SQL vs. NoSQL, replication, sharding). Today's topic was one I thought I already understood: caching. Turns out I only understood the easy 20% of it.
Why Caching Exists
The core idea is simple: fetching data is expensive (database queries, network calls, computation), so you store a copy of the result somewhere faster and cheaper to access, and serve from that copy instead of redoing the work every time.
But "store a copy somewhere faster" hides a lot of decisions: where do you store it, how long do you keep it, and how do you know when that copy is no longer accurate?
Where Caches Live
I mapped out the different layers where caching shows up in a typical system, and it's more places than I expected:
- Client-side caching — browser cache, mobile app cache. Avoids hitting the network at all.
- CDN caching — caching static assets (images, scripts, videos) geographically close to the user.
- Application-layer caching — an in-memory store like Redis or Memcached sitting between your app server and your database.
- Database caching — query result caching or buffer pools built into the database itself.
The pattern I noticed: the closer the cache is to the user, the faster it is, but the more stale it's likely to become and the harder it is to invalidate reliably.
Caching Strategies
A few patterns kept coming up:
- Cache-aside (lazy loading) — the application checks the cache first; on a miss, it fetches from the database and writes the result to the cache. Simple, and only caches what's actually requested, but the first request always pays the full cost.
- Write-through — data is written to the cache and the database at the same time. Keeps the cache consistent, but adds latency to every write.
- Write-behind (write-back) — data is written to the cache immediately and to the database asynchronously later. Fast writes, but riskier if the cache fails before the write reaches the database.
None of these is universally "correct." Cache-aside is the one I'd reach for by default, but write-through makes sense when staleness is unacceptable, and write-behind makes sense when write throughput matters more than durability guarantees.
Cache Invalidation: The Actually Hard Part
This is where the famous line clicked for me: there really are only two hard problems in computer science — cache invalidation, naming things, and off-by-one errors (the joke itself is a joke about how hard naming and counting things are). Invalidation is hard because you're trying to answer "has the source of truth changed?" without constantly re-checking the source of truth, which would defeat the point of caching in the first place.
The strategies I looked at:
- TTL (time-to-live) — the cache entry just expires after a fixed time. Simple, predictable, but you're always trading off freshness against cache-hit rate.
- Explicit invalidation — when the underlying data changes, the application actively tells the cache to drop or update that entry. More accurate, but adds coupling between every write path and the cache layer.
- Event-based invalidation — changes publish an event (e.g., through a message queue), and any cache watching that event invalidates itself. More scalable across multiple services, but adds infrastructure and a new failure mode: what if the event never arrives?
Cache Stampede — A Failure Mode I Hadn't Considered
One thing that stood out: what happens when a popular cache entry expires and a huge number of requests hit the database at the exact same moment because they all missed the cache simultaneously? This is called a cache stampede (or thundering herd), and it's a good example of how a cache — a tool meant to protect your database — can end up hammering it hardest at the worst possible moment if you're not careful. Mitigations like staggered expiration times or "locking" so only one request repopulates the cache while others wait came up as common fixes.
What Clicked Today
Caching looks like a performance optimization on the surface, but it's really a consistency problem wearing a performance-optimization costume. Every caching decision is really a decision about how stale you're willing to let your data be, and for how long, in exchange for speed.
That's the same trade-off shape I kept running into on Day 1 (CAP theorem) and Day 2 (replication). It's starting to feel like most of system design is a small number of trade-offs showing up over and over in different clothes.
Tomorrow
Next up: load balancing — how traffic actually gets distributed across servers, different load balancing algorithms, and where load balancers themselves become a single point of failure if you're not careful.
If you've been burned by a cache invalidation bug in production, I'd genuinely love to hear the story — those tend to be the most memorable ones.
This is part of a 30-day series on learning system design from scratch. Catch up on Day 1 and Day 2 if you missed them.
Top comments (0)