DEV Community: Muralidharan Lakshmanan

Designing a Production-Ready Caching Strategy

Muralidharan Lakshmanan — Fri, 28 Aug 2026 00:43:09 +0000

This series started with a simple question: why do we need caching? The simple answer — "it makes applications faster" — is true, but fourteen parts later it's clear that's not the whole story. Caching touches application performance, database capacity, scalability, consistency, availability, cost, security, and resilience all at once. Which is exactly why the most common mistake is opening a caching discussion with "should we use Redis?"

That's not the first question. The better one: where is our system actually spending time and resources, and can caching safely reduce that cost? Let's build the practical framework this whole series has been pointing toward.

1. Start with the problem, not the technology

If a database is becoming a bottleneck behind a fleet of application instances, the instinct is to reach for Redis immediately. Resist it. First understand what's actually slow, what's expensive, what's repetitive, what's genuinely frequent. If 60% of database queries turn out to be hitting the same 5% of data, now caching has a clear justification — not because caching is generally good, but because this specific pattern is exactly what it's built to solve.

2. Find the hot data

Not all data is equally worth caching. If a database holds 10 million products but traffic concentrates on 50,000 of them, those 50,000 are the hot data — and caching just those instead of attempting to cache everything cuts memory consumption, cost, evictions, and operational complexity all at once. A good caching strategy starts from actual traffic patterns, not from an assumption that more coverage is automatically better.

3. Measure before you cache

Before touching Redis, measure the system as it exists: API latency, database latency, database QPS, CPU, memory, connection pools, slow queries, request frequency, error rate. If GET /products/{id} averages 220ms with 180ms of that spent in the database at 5,000 requests/sec, the opportunity is obvious — caching could plausibly bring that down to 10–20ms. But that's a hypothesis until it's measured, not a promise that exists just because Redis is available.

4. Decide whether the data is actually cacheable

Five questions, in order: is it expensive to retrieve? Is it requested repeatedly? Does it change less often than it's read? Can stale data be tolerated? Is it sensitive or personalized? The first two determine whether caching would help at all. The third is one of the strongest signals that exists. The fourth is where real risk lives. The fifth determines whether a shared cache is even safe to use.

5. Think about staleness before anything else

A product description changing every few weeks can tolerate an hour-long TTL without anyone noticing. An account balance cannot tolerate the same treatment for even a few minutes. Picture this as a spectrum: images, product descriptions, and country lists sit on the low-freshness-requirement end; account balances, payment status, inventory, and authorization sit on the high end. The further right something sits, the more careful the caching strategy around it needs to be — this was the same axis Part 13 plotted in detail, and it's worth carrying forward as the first filter on every new caching decision.

6. Choose the right layer — not just Redis

Assuming everything belongs in Redis is one of the most common architectural mistakes in this whole space. There's a real hierarchy: browser cache, CDN or edge cache, application-local cache, distributed cache, and the database underneath all of it. Each layer exists to solve a different problem, and picking the wrong one for a given piece of data means paying for infrastructure that isn't actually earning its place.

7. Browser cache

Best for images, CSS, JavaScript, fonts, and other static assets — a hit here means the request never reaches your infrastructure at all, which is about as cheap as a cache hit gets.

8. CDN cache

Excellent for images, video, static assets, public API responses, and anything worth serving from a location physically closer to the user than your origin. Instead of every request from Atlanta, California, and London all traveling to one backend, each can be served from an edge location near it instead.

9. Local application cache

For data an application checks constantly and that barely changes — feature flags, configuration, country codes, static business rules — an in-memory local cache means no network call at all. The trade-off: every application instance holds its own copy, which is exactly the consistency question Part 5's invalidation techniques exist to answer.

10. Distributed cache

For data that needs to be shared identically across every application instance — sessions, product data, frequently accessed query results, shared configuration, recommendations, rate-limiting counters — a distributed cache like Redis is where that consistency actually gets solved centrally.

11. Multi-level caching, used deliberately

Sometimes the right answer combines layers: check a local cache first, fall through to Redis on a miss, fall through to the database on a miss from there. This can meaningfully cut network traffic to Redis while keeping the shared cache's consistency guarantees. But every additional layer is additional complexity — add one because a measured problem calls for it, not because the architecture diagram has room for another box.

12. Choose the pattern deliberately

Cache-Aside remains the strongest default: the application checks the cache, falls back to the database on a miss, and populates the cache for next time. It's simple, flexible, and easy to reason about during an incident — which matters more than it sounds like it should. Write-Through is worth reaching for when the write path itself needs the cache updated as part of the transaction, at the cost of more coordination complexity. Neither is universally correct; the choice follows your read/write ratio, consistency requirements, failure behavior, and who actually owns the data.

13. Design cache keys with real care

product:12345 looks simple, but product:v2:12345, customer:12345:profile, and search:v3:headphones:page:2 show how much a key needs to carry as a system grows. Good keys are predictable, unique, consistent, easy to debug, and versionable — and versioning specifically pays off the moment a cached object's shape changes: bumping product:v1:123 to product:v2:123 means the application never has to understand two incompatible formats living side by side.

14. Choose TTL deliberately, and don't make it uniform

TTL should follow the data, not a company-wide default: configuration might reasonably get an hour, product descriptions thirty minutes, recommendations five minutes, search results two. A single global TTL applied everywhere is usually the tell that caching was bolted on rather than actually designed around the data underneath it.

15. Plan invalidation before you need it

TTL-based expiration, explicit invalidation on write, and event-driven invalidation each solve the same underlying problem differently. The genuinely hard part was never deleting one key — it's knowing every cached representation a single database change actually touches. One product update might need to invalidate the product itself, several search result pages, a recommendations cache, and a trending list. Cache design has to account for those relationships up front, not discover them during an incident.

16. Write staleness tolerance down as a real requirement

Not as a vague feeling, but as an actual number per data type: product description, an hour; recommendations, ten minutes; search results, two minutes; inventory, seconds; account balance, near real-time; static images, days. Writing this down turns an abstract debate into something engineers can actually build and test against.

17. Plan for cache failure before it happens in production

What happens when Redis is completely unavailable? The honest menu of options — database fallback, serving a local stale copy, graceful degradation, rejecting the request, or returning a sensible default — and the right one depends entirely on what that specific cache was doing. This is the whole subject of the previous part in this series, and it deserves to be decided at design time, not discovered during the outage itself.

18. Protect the database as a first-class design concern

At a 98% hit ratio and 50,000 requests/sec, the database normally sees roughly 1,000/sec. If the cache disappears, it could suddenly face all 50,000 — a 50x jump. Connection limits, concurrency limits, timeouts, circuit breakers, and load shedding aren't optional extras bolted onto a caching layer; caching and database capacity planning are the same planning exercise, not two separate ones.

19. Monitor the cache — and its relationship to everything downstream

Hit ratio, miss ratio, latency, error rate, evictions, memory, connection count, CPU, network traffic, hot keys, expired keys — all worth tracking. But the metric that tells the real story is the relationship: cache hit ratio against database QPS against database CPU against application latency, watched together rather than in isolation.

20. A high hit ratio isn't automatically a healthy system

At 100 million requests and a 95% hit ratio, the remaining 5% is still 5 million database requests — potentially enormous depending on what each one costs. The right question isn't "is our hit ratio impressive" — it's "how much actual load is the cache removing from the system." A 90% hit ratio can be fantastic in one system and genuinely inadequate in another, depending entirely on what that other 10% costs to serve.

21. Watch for hot keys specifically

A single key responsible for 30% of all cache traffic can become a bottleneck even while every aggregate metric looks completely healthy — the exact trap Part 11 walked through with cluster averages hiding a single overloaded node. Local caching, replication, request coalescing, precomputation, and sharding strategy all apply here, and hot-key analysis becomes more important, not less, the larger a system gets.

22. Plan capacity ahead of the wall, not into it

Keys times average object size times overhead, plus replication, plus expected growth, plus deliberate headroom — worked through with real numbers rather than waiting for a memory alert at 99%. The exact formula depends on the technology, but the discipline is universal: size the cache before it's full, not in response to it being full.

23. Eviction is a policy decision, not a safety net

"Redis will just evict something if it fills up" is true and also not reassuring on its own — what gets evicted determines whether that's fine or a production incident. If a genuinely hot key gets evicted, the result is repeated misses landing straight on the database. LRU, LFU, TTL-based, and random eviction each encode a different assumption about access patterns, and the right one should be chosen deliberately and watched afterward, not left on whatever the default happened to be.

24. Security is part of cache design, not an afterthought

Could one user ever receive another user's cached data? A cache key like profile instead of profile:user:123 is exactly how that happens — User A's request and User B's request colliding on the same shared entry. For personalized data, key isolation, authorization, encryption, the shared-vs-private distinction, and CDN caching rules all need deliberate attention. Performance should never be purchased at the cost of data isolation.

25. Don't cache everything

Maybe the single most important lesson in this entire series. Caching adds real, ongoing complexity: expiration, invalidation, consistency, monitoring, and failure handling, all layered on top of the application that already existed without it. If a query takes 2ms and runs 10 times a second, a caching layer around it is very likely adding more complexity than it's returning in value.

26. Caching has a real cost, and it belongs in the decision

Memory, infrastructure, replication, network traffic, operations, monitoring, development effort, and debugging complexity are all real costs, not hypothetical ones. Cache value isn't "cache = faster" — it's performance improvement plus database savings plus scalability, minus infrastructure cost, minus operational complexity. The goal was never maximum caching. It's maximizing that whole equation, including the parts that subtract.

27. A production caching architecture, assembled

Putting it together for something like an e-commerce platform: a CDN in front for public and static content, a local cache for extremely hot and stable data, Redis for shared application data, and the database underneath as the one source of truth. Each layer earns its specific job rather than one technology trying to solve every problem at once — which is a far more effective default than reaching for a single caching tool and stretching it to cover everything.

28. The full decision pipeline

"Let's put Redis in front of the database" skips every step in that pipeline after the first box. Screening a candidate is the easy 20% of the work; choosing the layer, the pattern, the invalidation strategy, the failure behavior, and the monitoring is the harder 80% that actually determines whether the caching decision holds up in production.

The production checklist

Before calling a caching strategy complete, it should have real answers across eight areas: what's being cached and why; which architectural layer it belongs in; which pattern governs how the application talks to it; whether the keys are unique, predictable, versioned, and properly isolated per user; whether TTL, jitter, and invalidation are all defined; what happens on a Redis outage, a timeout, a retry storm, or a stampede; whether memory sizing, growth, replication, and eviction are planned; and whether hit ratio, latency, errors, evictions, memory, hot keys, and database impact are all actually observable. Answering all of these is the difference between having added a cache and having designed a caching strategy.

What fifteen parts actually taught

We opened this series with "Caching Is Simple… Until It Isn't," and fifteen parts later that title still holds up exactly. The idea starts simple: don't calculate or retrieve the same thing twice. At production scale, that simple idea becomes an entire architectural discipline — covering why caching matters, how it actually works, where it should live, the patterns for reading and writing through it, the hardest problem in caching (invalidation), Redis versus Memcached, the specific ways caching goes wrong under load, what a production-ready architecture looks like, building one for real with Spring Boot, proving it works through testing, scaling it past a single node, reaching beyond Redis into CDNs and HTTP caching, matching real scenarios to real strategies, designing for the moment it fails — and now, pulling every one of those threads into one practical framework.

Caching was never really a technology decision. Redis is a technology. Memcached is a technology. A CDN is a technology. Browser caching is a technology. But deciding what should be cached, where it should live, how long it should survive, how it gets invalidated, and what happens when it fails — that's architecture, and it's where good caching design actually begins.

The one-line version

If this whole series had to compress into a single sentence, it would be this: don't cache because you can — cache because you understand the problem you're solving, and you understand exactly what happens when the cache is wrong, stale, full, or completely gone.

That's the mindset that turns caching from a performance trick into a production-grade architectural capability. Thanks for reading all the way through — it's been a genuinely long build.

Of everything covered across this series, which part changed how you'd actually design a cache the most? I'm curious which one lands differently once you've seen the whole arc.

When the Cache Goes Down: Designing for Failure

Muralidharan Lakshmanan — Wed, 26 Aug 2026 14:43:00 +0000

Most of this series has been about what happens when caching works: request, hit, fast response. But production doesn't stay in that state forever. Redis becomes unavailable, a network path breaks, a node crashes, connections exhaust, a deployment clears the cache, a config mistake causes mass expiration, a region goes dark. Eventually, cache goes down.

The question that actually matters is what happens to the application next. A poorly designed system turns a Redis outage into database overload into an application outage, in seconds. A well-designed one turns it into degraded performance, a protected database, and an application that stays up. That gap is the entire subject of this post.

1. The most important principle

A cache should usually be an optimization, not the source of truth. If Redis disappears and the database still holds the authoritative data, Application → Database remains a valid path even with Redis gone. That's the theory. The practice is harder: what happens when 100,000 requests/sec were relying on Redis catching most of them, and it suddenly can't?

2. The cache failure avalanche

Under normal conditions, 100,000 requests/sec against a 95% hit ratio means the database sees roughly 5,000/sec — comfortable. The instant Redis fails, all 100,000 requests fall through, because there's no cache left to catch any of them. That's not a proportional increase. It's the full, unfiltered load the cache was built specifically to prevent, arriving all at once.

3. Why the database can take the whole system down with it

If the database comfortably handles 10,000 requests/sec but Redis was normally absorbing enough that the database only saw a fraction of 100,000 total traffic, a sudden 100,000-request flood is 10x what the database can actually process. From there the chain writes itself: the database slows down, application requests queue, connection pools fill, threads wait, timeouts rise, clients retry, and the retries add more load on top of an already-struggling system. This is a cascading failure — the cache didn't just fail, its failure propagated.

4. Retries make this worse, not better

A request that times out against Redis and retries twice more has turned one logical request into three actual ones. Multiply that across 100,000 requests and you've turned 100,000 requests into 300,000 — against a dependency that was already failing before the retries even started. Retries sound protective. Against an already-overloaded system, they're an amplifier.

5. Timeouts decide how long the damage takes to start

A 10-second Redis timeout sounds conservative, but it means 100,000 requests can all be waiting on Redis simultaneously, holding threads and connections the whole time. A short timeout — 50ms, not 10 seconds — lets a request fail fast and move to its fallback instead of sitting there. The principle holds regardless of the exact number: a dependency that's already failing shouldn't be allowed to hold your application hostage while it does.

6. Fail open vs. fail closed

When Redis is unavailable, should the application keep going without it, or refuse the operation entirely? Both are legitimate answers, and which one is correct depends entirely on what the cache was actually doing.

A product description can fail open — skip the cache, hit the database, the user gets a slightly slower but correct response. Rate limiting is the opposite case: if you can't verify whether a user has exceeded their limit, silently allowing every request through removes a protection you were relying on. Failing closed there — rejecting or restricting the request — is the safer default. Neither choice is universally right; the purpose behind the cache is what decides.

7. Cache as optimization vs. cache as infrastructure

This distinction matters more than it first appears. When Redis is purely making reads faster, its disappearance still leaves Application → Database intact. But Redis is often doing more than caching — sessions, rate limiting, distributed locks, queues, coordination. At that point it's not a cache anymore, it's infrastructure, and its failure needs to be handled with the seriousness of losing infrastructure, not the shrug you'd give a slow cache. Don't assume every Redis dependency has a database fallback waiting behind it — some of them don't have one at all.

8. Circuit breakers

Sending every one of 10,000 failing requests to a Redis that's already down accomplishes nothing except adding load to something already struggling. A circuit breaker recognizes the failure pattern and stops sending traffic to Redis at all, routing everything straight to the fallback instead.

CLOSED is business as usual. OPEN means Redis isn't contacted at all — every request goes straight to its fallback. HALF-OPEN is the recovery test: one request is allowed through after a cooldown period, and its outcome decides whether the circuit reopens fully or closes back to normal. The point of the whole mechanism isn't detecting that Redis failed — it's not re-detecting the same failure 100,000 times a second while Redis is trying to recover.

9. Graceful degradation

The system doesn't need to be perfect. It needs to remain useful. If Redis goes down and an e-commerce homepage normally shows product info, search, checkout, recommendations, and personalized offers, taking down the entire page because recommendations can't load is a self-inflicted wound. Product information, search, and checkout can stay up; recommendations and personalized offers can show a fallback or simply not render. The user still has a working site.

10. Sorting data into critical, important, and optional

This classification should directly shape your fallback strategy. Critical data — authorization, payment validation, certain security controls — means the operation genuinely cannot proceed safely without it. Important data — recommendations, personalization, analytics — means the operation continues with reduced functionality. Optional data — a trending widget, recently viewed items, decorative content — the user may not even notice its absence. Knowing which bucket a piece of data falls into, before the incident happens, is what makes graceful degradation possible instead of theoretical.

11. Serving stale data on purpose during an outage

If a local, slightly-old copy exists, serving it can beat going all the way to the database. Trending products that are 10 minutes stale — the user probably never notices. An account balance served 10 minutes stale is a completely different kind of problem. The decision about which data gets this treatment during an outage is the same staleness-tolerance question from Part 13, just applied under failure conditions instead of normal ones.

12. Cache miss and cache failure are not the same thing

A miss means the cache is working correctly and simply doesn't have this key — GET product:123 returning nil is expected, routine behavior. A failure means you couldn't reliably talk to the cache at all — a connection timeout is not the same category of event, and treating it identically to a miss hides a real problem. A miss needs a database query. A failure may need a fallback, a circuit breaker trip, an alert, and traffic protection — a much bigger response than "just go to the database this once."

13. Don't keep trying to write to a cache that's already down

Here's a subtle waste: Redis is unhealthy, the application correctly falls back to the database, and then still tries Redis SET to repopulate the cache — which also fails, on every single request, for no benefit. Once the system knows Redis is unhealthy, it can stop attempting cache writes entirely until the circuit breaker's test request confirms Redis is actually back.

14. Protecting the database is the actual priority during an outage

Connection pool limits, concurrency limits, request queues, rate limiting, circuit breakers, load shedding, and fallback responses all exist to keep the database from being the thing that turns a cache incident into a full outage. Deliberately capping how many requests reach the database — say, 10,000 out of 100,000 — and returning a controlled response to the rest sounds harsh compared to "let everything through." But 10,000 succeeding while 90,000 get a controlled failure is a materially better outcome than all 100,000 requests taking down the database and leaving zero requests succeeding.

15. Load shedding is a deliberate design choice, not a failure

If database capacity is 10,000 req/sec and incoming traffic is 50,000, deliberately routing 10,000 through and rejecting or deferring the other 40,000 is what keeps the system available at all. A resilient architecture treats "no" as a legitimate, planned response — not a symptom of something broken.

16. Recovery can recreate the exact problem you just survived

Redis comes back online with an empty cache, and every application instance immediately tries to refill it — which, at scale, is just the original stampede happening again, this time triggered by recovery instead of failure. Cache warming, request coalescing, jittered expiration, and rate-limited rebuilding — the Part 7 stampede defenses — apply just as much to the moment a cache comes back as to the moment a popular key expires.

17. Redis Cluster failure and its own subtleties

Replication lets a cluster fail over from a failed primary to a replica, reducing downtime — but failover isn't instantaneous or invisible. There can be a failover delay, transient connection errors, temporarily unavailable keys, and replication lag even after the new primary takes over. Applications still need to handle these as real, if brief, failure conditions rather than assuming clustering makes failure disappear.

18. Network failure can look exactly like Redis failure

Redis can be perfectly healthy while your application simply can't reach it — a network partition looks identical to a dead Redis from the application's point of view. This is why monitoring Redis server health alone isn't sufficient: Redis server health: GOOD and Application-observed Redis latency: BAD can both be true simultaneously, and only the application's own perspective catches the second one.

19. Run the capacity math before an incident forces you to

At 50,000 requests/sec and a 98% hit ratio, the database normally sees roughly 1,000/sec — well within a 5,000/sec capacity. If Redis fails, the database faces the full 50,000/sec: 50 times its normal load, and 10 times more than it can actually process. This is exactly the kind of calculation that belongs in an architecture review, done deliberately with real numbers, rather than discovered for the first time during the incident itself.

20. What an incident actually looks like end to end

Watched together rather than separately, these five numbers tell a single causal story: cache hit ratio collapses, database traffic spikes in direct response, CPU saturates under that load, and application latency follows immediately after. Knowing "Redis is at 80% CPU" in isolation tells you far less than seeing this whole chain — which is why cache miss rate correlated against database QPS is one of the most valuable signals a caching system can expose.

21. Don't retry everything, indefinitely

The failure mode to avoid: retry, fail, retry, fail, retry, fail, with no bound and no growing delay. The alternative is bounded retries, short timeouts, exponential backoff, jitter on that backoff, and a circuit breaker that eventually stops the attempts altogether. The goal was never "keep trying until it works" — it's "give the dependency one reasonable chance without piling onto its failure or ours."

22. Cache failure needs to actually be tested

Most test suites, per Part 10, cover the cache working. Far fewer cover it not working: Redis unavailable, Redis slow, connection exhaustion, a node failing outright, elevated network latency, an empty cache, mass expiration, a slow database, an unavailable database. The question every one of those scenarios needs answered isn't "does this pass" — it's "does the application remain useful."

23. Chaos testing, for systems that can support it

Mature systems can go further and deliberately inject a Redis failure in a controlled environment, then observe: does traffic actually move to the database, does the database survive it, do circuit breakers open when they should, do alerts fire, does recovery behave the way it's supposed to. The goal was never "prove nothing fails" — that's not a realistic goal for any real system. It's "know exactly how the system behaves when something does," ahead of the moment you need that knowledge.

The golden rule

Never assume a cache outage is harmless just because the cache isn't your source of truth. The cache may be absorbing 95%, 98%, or more of your total traffic — remove it suddenly, and the database inherits all of it at once. The question to ask when designing any cache is "what happens if this disappears right now" — and the answer should come from a calculation, not a guess.

The bigger lesson

Caching gets introduced to make a system faster. Once a system becomes dependent on that cache, caching also becomes a resilience question — and the failure chain (cache failure → misses → database overload → application slowdown → timeouts → retries → more load → cascading failure) is entirely predictable in shape, even if the trigger varies.

A resilient system breaks that chain with timeouts, circuit breakers, fallbacks, concurrency limits, load shedding, graceful degradation, cache warming, and real observability — not as separate features bolted on, but as one coordinated resilience strategy. The goal was never making the cache impossible to fail. That's not achievable. The goal is making the application capable of surviving the failure when it happens — and that distinction is what separates a caching implementation from a production-grade caching architecture.

Has your system been through a real cache outage — and looking back, was the actual failure mode the one you'd already planned for, or something the runbook hadn't considered?

Real-World Caching Patterns: Choosing the Right Strategy

Muralidharan Lakshmanan — Wed, 26 Aug 2026 02:51:21 +0000

This series has covered cache-aside, read-through and write-through, TTL, eviction, invalidation, stampedes, distributed caching, Redis, local caching, CDNs, edge caching, and scaling. But the question that actually matters when you sit down to design a system is simpler than any of that: what should I cache, where should I cache it, how long should I keep it, and what happens if the cached value is wrong?

There's no single answer. Let's walk through real scenarios and see how the answer changes each time.

1. E-commerce product catalog

A store with 10 million products, all getting hit with GET /products/12345 repeatedly. Thousands of users requesting the same product is exactly the shape caching exists for — one database query can serve all of them through Redis.

But should every field on that product be cached the same way? A description and an image rarely change; a rating changes periodically; a price can change often; inventory changes constantly. Treating all of them identically wastes the opportunity that caching offers — the strategy should follow each field's actual volatility, not a single TTL applied to the whole object.

2. Inventory is a different animal entirely

"Only 1 left!" and two customers clicking "buy" at nearly the same instant is exactly the scenario where a cached inventory count can sell something that's no longer there. That's a materially different kind of wrong than a stale product description — one is cosmetic, the other loses money and trust. A common answer: cache the product information normally, but keep inventory tied to an authoritative source, or use a very short-lived cache combined with an authoritative check at the actual moment of checkout. Not all data on the same object deserves the same consistency guarantee.

3. Banking customer profile

GET /customer/profile doesn't change often, which makes it look cacheable — but it's private, personalized, and potentially sensitive, so it can never casually land in a shared CDN cache the way a product image could. A private cache keyed by identity — customer:12345:profile — keeps one customer's data from ever being served to another. This is the Part 12 public/private distinction showing up with real stakes attached.

4. Account balance is where caching should back off

GET /account/balance is the sharper version of the same problem. A stale product description is mildly annoying. A stale balance showing $10,500 when the real number is $10,000 is a different category of wrong entirely. This is often a case for going straight to an authoritative service rather than aggressively caching — caching isn't automatically good just because something is read frequently. Correctness comes first, and for this kind of data, it isn't a close call.

5. Social media feed — precompute instead of cache-on-request

A feed is expensive to generate, requested constantly, personalized, and constantly changing — a genuinely hard caching problem, because the usual "compute once, serve many times" logic doesn't hold when the result is different for every user. The answer often isn't caching after the fact — it's precomputing before anyone asks: a new post triggers fan-out processing that updates a feed cache directly, so a user's request reads an already-assembled result instead of triggering a live rebuild.

6. The precompute pattern, generalized

The general version of that idea: instead of "store the result after someone asks for it," it's "calculate the result before anyone asks." Trending products recalculated every five minutes and written to Redis means 100,000 users read the same precomputed value instead of triggering the computation 100,000 times. This pattern earns its keep for leaderboards, trending content, recommendations, reports, dashboards, and aggregated statistics — anywhere the expensive part can happen once, on a schedule, ahead of demand.

7. Search results — repetition matters more than volume

Search is expensive, but many users searching "wireless headphones" means genuine repetition exists to exploit — cache the result, and check the cache before hitting the search engine on a miss.

8. Cache key explosion is the trap hiding behind that idea

Search results depend on query, filters, sorting, pagination, and sometimes location or personalization — combine enough of those dimensions and the number of distinct cache keys can explode into millions of low-value entries that are each requested once and never again. Before caching search results, it's worth actually asking how often the exact same query repeats. If nearly every query is unique, caching provides close to nothing — the infrastructure cost without the benefit.

9. Configuration — one of the best caching candidates that exists

Feature flags, application settings, supported countries, UI configuration: read constantly, changed rarely. This is about as clean a caching candidate as exists, and it's worth loading at application startup directly into memory rather than hitting Redis on every request — thousands of requests can then be served from a local copy without a network call at all.

10. But configuration has a real invalidation problem

If enableNewPaymentFlow flips from false to true and application instances refresh their local copies at different times, some instances run the new flow while others run the old one simultaneously — genuinely inconsistent behavior across a fleet that's supposedly running the same code. Event-driven refresh, the pattern from Part 5, is the common fix: a configuration-changed event fans out to every instance, and each one refreshes its local cache on receipt rather than waiting for its own independent TTL to expire.

11. Session data

session:abc123 holding user ID, login state, preferences, and expiration is the classic reason to keep sessions in a shared store like Redis rather than pinned to one application server — any instance behind the load balancer can serve any user's session, which is what makes horizontal scaling behind a load balancer actually work cleanly.

12. API response caching — sometimes the CDN is simply the right layer

GET /api/countries changes once every few weeks but gets hit constantly. Public, shared, relatively static, and naturally an HTTP response — that's the CDN's exact sweet spot from Part 12, and routing this through Redis instead would be solving a problem the CDN already solves for free.

13. Images and static content — the easiest call in this entire post

Images, CSS, JavaScript, fonts, video: a CDN is the obvious answer, and the application doesn't need to participate in serving any of it. This is a large part of how modern web architectures serve enormous traffic volumes without correspondingly enormous application-server fleets.

14. Expensive computation, not just expensive database reads

Caching isn't only for database results. If calculateCustomerRisk() takes 800ms and the underlying inputs only change hourly, caching the result of the computation — not a database row — is exactly as valid a caching decision. The same logic applies to recommendation scores, ML predictions, pricing calculations, analytics, reports, and complex business rules: anywhere the expensive part is CPU time rather than a database round trip.

15. Caching a specific query result, not a whole object

Sometimes what's worth caching isn't a business object at all but one specific expensive query — SELECT COUNT(*) FROM orders WHERE customer_id = 12345 cached as customer:12345:order-count. The invalidation question shows up immediately: every new order changes the answer, so either invalidate on order creation or deliberately accept some staleness. Which one is right depends entirely on how much that count is allowed to lag reality.

16. Rate limiting isn't caching, but it's the same muscle

Tracking user:12345:requests against a 100-requests-per-minute limit via a Redis counter isn't "cache the database result" in the traditional sense — there's no database result being cached at all. It's worth including here anyway, because it demonstrates something broader: a fast, shared, in-memory store solves problems well beyond caching once it exists in your architecture.

17. Feature flags, and designing the fallback deliberately

isNewCheckoutEnabled() checked hundreds of times a second shouldn't hit a remote configuration service on every call — a local cache refreshed periodically gives low latency, resilience, and less network traffic. The question worth asking explicitly: what happens if the cached flag value is wrong? For some flags the safe default is off; for others it's on. That fallback behavior needs to be a deliberate choice per flag, not an accident of whatever the cache happened to return.

18. Recommendations tolerate more staleness than they get credit for

A recommendation engine combining user history, the product catalog, an ML model, and business rules can easily take hundreds of milliseconds to compute. Caching recommendations:user:12345 for five minutes (or longer) raises a genuinely useful question: does a recommendation actually need to change every second? Usually not — and once that's acknowledged, a five-minute-old recommendation stops looking like a compromise and starts looking like the obviously correct design.

19. Stock market data — one domain, four completely different strategies

AAPL isn't one caching decision — it's at least four. The live price genuinely shouldn't be cached at all; it belongs in a streaming system built for continuous updates. Historical prices are effectively immutable once recorded, so they cache indefinitely. Company information changes maybe once a year and can sit in a long-lived cache. News needs a short TTL measured in minutes, not hours. Same ticker symbol, four different answers to "how should this be cached" — because the question was never really about the ticker, it was about each piece of data behind it.

The real question, restated

Every scenario above resolves to the same underlying question, and it's worth seeing them plotted against it directly rather than as a list.

Access frequency tells you whether caching would help. Staleness tolerance tells you whether it's safe. Those are different questions, and a data type can score high on the first and low on the second — account balances are read constantly, which says nothing about how safe they are to cache.

A framework for the decision itself

Before caching anything, it's worth running through this in order:

A "no" on being frequently read, or a "no" on being expensive to retrieve, just means caching wouldn't help much — a mild missed opportunity at worst. A "no" on tolerating staleness is a different kind of answer entirely: it means caching could make the system actively wrong, not just occasionally slow, and the better move is going to the authoritative source directly rather than trying to cache carefully around the problem.

Don't forget the failure path

Every design above has a HIT path and a MISS path. Production also needs a defined answer for what happens when the cache itself is unavailable — Redis ERROR needs its own branch, not an assumption that it collapses into MISS behavior automatically. We'll go deeper on this specific failure mode next.

One application, many strategies at once

The biggest practical lesson across every scenario above: there's no requirement to standardize on one mechanism company-wide.

Different data has different requirements, and using a different mechanism for each is completely normal — not a sign the architecture is inconsistent, but a sign it's actually matched to the problem in each case.

The architect's actual starting question

Not "should we use Redis?" — that question skips past everything that determines the right answer. The better starting point: which requests are expensive, repetitive, and safe to serve from a cached representation? From there: what's the data, how often is it read, how often does it change, how stale can it safely be, who's allowed to see it, where should it live, how does it get invalidated, and what happens when the cache fails? That sequence, worked through deliberately, is a far stronger design process than picking a technology first and looking for places to apply it.

The bigger lesson

A product catalog, a bank balance, an image, a session, a recommendation, a search result, and a stock price can all legitimately need completely different caching approaches — and that's not a failure to standardize, it's the correct outcome of actually looking at each one. The question was never "can I cache this?" It's "what happens if I serve an old value?" If the honest answer is "nothing important," that's very likely a great caching candidate. If the honest answer is "we could lose money, expose private data, or make an incorrect business decision," caching needs far more careful design — or, in some cases, shouldn't be used there at all.

Which of these scenarios hits closest to something you've actually built — and did the staleness question get asked before or after something went wrong in production?

Caching Beyond Redis: CDN, HTTP Cache & Edge Caching

Muralidharan Lakshmanan — Tue, 25 Aug 2026 02:29:40 +0000

Most of this series has lived inside one architecture: user, application, Redis, database. Redis is genuinely good at keeping database load down. But there's a question worth asking that Redis can't answer: what if the request never reached Redis either? Or further still — what if it never reached the application at all?

That's HTTP caching, browser caching, CDNs, and edge caching — and they lead to one of the most useful ideas in performance engineering: the fastest request is the one you never have to process.

1. Browser caching

Start at the very edge: the user's own browser. A site's logo.png, styles.css, and app.js don't need to be downloaded on every single visit. The browser can keep a local copy, and the next request becomes a local cache hit — no network request, no CDN request, no application request, no database request. It's about as cheap as a cache hit gets.

2. How the browser knows what to cache

Cache-Control headers make the decision. Cache-Control: max-age=3600 tells the browser the response is fresh for an hour. For static content that rarely changes, something more aggressive is common: Cache-Control: public, max-age=31536000, immutable.

That raises an obvious question: if the browser holds onto a file for a year, how does it ever get a new version? Through the filename itself — app.abc123.js becomes app.def456.js when the content changes, the browser sees a different URL, and it downloads the new file without needing to be told the old one expired. This is cache busting, and it's the foundation for a technique we'll come back to.

3. Cache-Control decides where a response is allowed to live, not just for how long

This is easy to undersell. Cache-Control: public, max-age=3600 means shared caches like CDNs are allowed to store the response for everyone. Cache-Control: private, max-age=3600 means only the individual browser may keep it — a CDN sitting in between must not. Cache-Control: no-store means don't keep this anywhere, period.

This distinction is a security boundary, not just a performance knob. Getting it backwards — marking personalized data public by accident — doesn't just cause staleness. It can mean one user's cache serves another user's data.

4. What actually belongs in the browser

Good candidates: JavaScript, CSS, images, fonts, static HTML, public assets, and some genuinely public API responses — a public product catalog, public configuration, public content. Bad candidates: account information, private messages, financial details, anything personalized, anything authentication-related. The browser isn't only a performance layer — it's a data boundary, and "it's just cached" is never a reason to assume something is safe to cache.

5. Enter the CDN

If the application lives in Virginia and a user is in Tokyo, every request without a CDN makes that full round trip. With a CDN, the Tokyo user hits a nearby edge location instead, and the response comes from far closer to home. A Content Delivery Network maintains many geographically distributed edge locations — sometimes called points of presence, or PoPs — each holding copies of content that originates from one authoritative source.

The CDN cache-hit and cache-miss shape is identical to the Redis pattern from earlier in this series, just relocated: a hit returns the cached resource without touching the origin; a miss fetches from the origin, returns it, and typically caches it there for the next request.

6. The layered picture, and what it actually saves

Put browser, CDN, application, Redis, and database in one request path, and every layer that answers stops the request from going any deeper.

That's not a hypothetical ratio — it's the realistic shape of a well-cached system under real traffic. Each layer isn't competing with the others; it's catching what the layer before it missed.

7. ETags and conditional requests

Here's a different mechanism worth knowing well: the browser already has product.json, but instead of re-downloading it wholesale on every request, the server can hand back an identifier for the current version — ETag: "abc123". The next time the browser asks for that resource, it includes If-None-Match: "abc123". If nothing changed, the server replies 304 Not Modified and sends no body at all.

For a large response, this is a real saving — bandwidth, transfer time, and server-side work, all avoided on a request that still technically happened but never had to pay for the expensive part.

8. CDN and Redis solve related but different problems

Redis lives close to the application and excels at application data, sessions, computed results, and frequently accessed database records. A CDN lives close to the user and excels at images, JavaScript, CSS, video, static files, and cacheable public HTTP responses. Neither replaces the other — most real systems that need both use both, each doing the job it's actually good at.

9. Edge caching isn't only for static files anymore

The instinct is "CDN = static assets," but modern edge platforms can cache dynamic HTTP responses too. If GET /products/popular returns an identical response for a large share of users, caching that response at the edge turns 100,000 application requests into a small number of them — occasionally, on a miss, one request reaches the application, Redis, and the database, and the CDN caches the result for everyone else. This can change the entire scalability profile of an API without touching the backend at all.

10. Personalized data breaks the pattern completely

GET /account/profile cannot be cached and served identically to everyone — Alice's balance is not Bob's balance, and caching that response globally means Alice eventually sees Bob's data. Every caching decision needs to ask: is this response public, personalized, or sensitive, and can it tolerate staleness at all? Getting this wrong isn't a performance bug.

11. Cache keys and the Vary header

For shared HTTP caches, what makes two requests "the same" is genuinely more subtle than the URL alone. GET /products?category=laptops and GET /products?category=phones need separate cache entries even though they hit the same endpoint. Headers can matter too — a response might legitimately differ for Accept-Language: en versus Accept-Language: fr. The Vary header tells caches this explicitly: Vary: Accept-Language means the cache needs to keep a separate representation per language rather than assuming one response fits everyone.

12. Invalidation shows up here too — and gets solved cleanly

Part 5 called invalidation the hardest problem in caching, and it doesn't get easier once a resource is cached in a browser, three CDN regions, and who knows where else. Waiting for TTL works but is slow. Explicitly purging every CDN edge works but requires reaching every single one. There's a third option that sidesteps the problem almost entirely.

app.8f31c2.js becomes app.a91d22.js when the content changes. Nobody has to tell any cache anywhere to forget the old file — the old URL just stops being referenced, and its cached copies become permanently irrelevant rather than dangerously stale. This is one of the cleanest invalidation techniques that exists, and the same trick applies to Redis keys directly: product:v1:101 becoming product:v2:101 sidesteps needing to invalidate every old representation, the versioning technique from Part 9.

13. Stale-while-revalidate, at the HTTP layer

We covered this pattern in Part 5 and Part 7 as a stampede defense; HTTP has a first-class way to express it directly: Cache-Control: max-age=60, stale-while-revalidate=300. For the first 60 seconds the response is fresh. For the next 300 seconds after that, a cache may serve the stale value immediately while refreshing it in the background. If generating a response normally costs 500ms, the user never pays that cost directly — they get the existing value immediately while the refresh happens behind them. This only works where staleness is genuinely acceptable: product recommendations, probably fine; an account balance, not even close.

14. What edge caching costs you

None of this is free. Freshness suffers by definition — users can see slightly old data. Invalidation needs a real strategy, whether that's purging or the versioning technique above. Query parameters and headers can fragment one logical resource into many cache entries. Personalized responses must never leak into a shared cache. And debugging gets genuinely harder — a response might have passed through a browser cache, a CDN, an API gateway, the application, and Redis before you ever see it, and figuring out which layer actually served a given response becomes its own skill.

X-Cache: HIT or X-Cache: MISS headers, which most CDNs expose in some form, are what make that debugging tractable — being able to see "Browser: MISS, CDN: HIT" versus "Browser: MISS, CDN: MISS, Redis: HIT" turns a guessing game into a quick trace.

15. A practical decision framework

Before deciding where something belongs, it's worth running through a short list of questions: Is the data static, or close to it? Is it public and genuinely shared across users, or personalized? Is it application data that fits naturally in Redis? Is it hot enough to justify a local cache? Can it tolerate being briefly stale? How often does it actually change? How expensive is it to generate in the first place? The answers point toward a layer — browser, CDN, Redis, database, or in some cases, deliberately, nowhere at all.

The bigger lesson

The goal was never "cache everything, everywhere" — that's how systems become fragile and hard to reason about. The goal is the minimum work required to produce a correct response, and sometimes that minimum genuinely is "don't cache this." Caching, across this whole series, has really been one continuous idea at different distances from the user: CPU cache, memory, a local application cache, Redis, a CDN, the origin, the database — different costs, different consistency guarantees, different failure modes, same underlying question of how close a copy of the truth needs to sit to where it's actually needed.

A high-performance system doesn't necessarily need a bigger database cluster. Sometimes the biggest win is simply making sure most requests never reach the database — and sometimes, further still, making sure they never reach the application at all.

Which layer in your own stack is doing the most invisible work right now — browser cache, CDN, or something further back — and would you actually know if it stopped?

Caching at Scale: When Millions of Requests Hit Your System

Muralidharan Lakshmanan — Mon, 24 Aug 2026 03:23:37 +0000

Every architecture in this series so far has had roughly the same shape: application, Redis, database. That works beautifully for a lot of real systems. But traffic grows — 1,000 requests/sec becomes 10,000, then 100,000, then over a million — and at some point the question stops being "should we use a cache?" and becomes "how do we make the cache itself scale?"

That's a genuinely different problem, and it's what this post is about.

1. Scaling changes the question

The simple picture — application, Redis, database — assumes roughly one application talking to one Redis. Once there are hundreds of application instances behind a load balancer, all of them are hitting the same Redis, and Redis itself becomes the scalability boundary the rest of the system depends on. It stops being "the fast layer" and starts being "the thing everything else is now downstream of."

2. Can one Redis node handle it?

Sometimes, yes — a single modern Redis instance handles a lot, especially with simple operations. But it runs into real limits eventually: CPU, memory, network bandwidth, connection count, command throughput, dataset size. A Redis instance sitting at 90% memory, 85% CPU, and 80% network isn't going to absorb indefinitely more traffic just because it's "just a cache." At some point the workload needs to be distributed rather than pushed harder into one place.

3. Redis Cluster and sharding

The common answer is a Redis Cluster: instead of one node taking 100% of traffic, keys get spread across several nodes. This is sharding — dividing the dataset so no single node holds all of it, and no single node absorbs all the traffic either.

The mechanism is hashing: hash(key) → slot → node. The application or Redis client can work out which node owns a given key without needing a central lookup table. The cache stops being one big bucket and becomes a distributed collection of smaller ones — which solves the capacity problem, but introduces a new one.

4. The hidden problem: hot keys again

Hashing distributes keys evenly, but it says nothing about traffic being even. If homepage:trending gets 40% of all requests, hashing still sends every one of those requests to the same single shard — the exact hot-key problem from Part 7, just relocated to a cluster instead of a single instance. One node ends up carrying a wildly disproportionate share of load while the other three sit comfortably under capacity.

5. Averages hide this

This is the metric trap worth naming explicitly: a cluster-wide average can look completely healthy while one node is already in trouble.

A 45% cluster average is genuinely meaningless if it's made up of three nodes at 30% and one node quietly sitting at 95%. That fourth node becomes the system's bottleneck long before the aggregate number would ever suggest a problem — which is why per-node metrics, not cluster averages, are what actually catch this in time.

6. Local caching helps, and complicates things

One response to hot keys: put a local, in-process cache in front of Redis, so an extremely popular value can be served from application memory without a network hop at all. It's dramatically faster — there's no network round trip, which matters more at scale than it sounds like it should.

But this reintroduces the Part 5 invalidation problem at a larger scale. If four application instances each have their own local copy of product:101 and the product changes, all four copies need to be told, not just Redis. More locality generally buys speed at the cost of more consistency complexity — the same trade-off from Part 8, just compounding as you add layers.

7. The multi-level cache, formalized

L1 in application memory, L2 in a Redis cluster, L3 the database — the goal is for most requests to resolve at L1, a smaller share at L2, and very few ever reaching the database. This is the same shape from Part 8's reference architecture, just under enough load that each layer is now load-bearing rather than optional.

8. Connection pooling becomes an architectural concern

At scale, Redis isn't only a server-side problem. If 500 application instances each open hundreds of Redis connections, that's potentially 100,000+ connections — a real operational number that needs managing through maximum connections, pooling, timeouts, idle-connection handling, and watching for leaks. A cache can be fast on paper while the application performs badly anyway, purely because of how it's managing its connections to that cache.

9. Network latency and serialization both start to matter

"Redis is in-memory, so it's basically instant" undersells the actual request path: application → network → Redis → network → application. At high request rates those hops add up, which is another reason local caching earns its place. Serialization has the same story — at 500,000 cache requests/sec, the JSON encode/decode on both ends can become a genuinely significant CPU cost. When something feels slow at scale, the bottleneck is often not where "Redis is slow" would suggest — measure application CPU, serialization CPU, network, Redis CPU, and Redis latency separately rather than assuming.

10. Replication solves a different problem than sharding

Sharding distributes data. Replication answers a different question: what happens when a node fails? A primary with a replica means a failure doesn't have to mean data loss — but replication isn't free. Writes need to propagate, which costs network traffic, memory, and operational complexity, and replicas can lag behind the primary. For a cache, some lag is often tolerable. Whether it's tolerable for your cache depends entirely on how much staleness the application can absorb — the same freshness question from Part 2 and Part 5, now with a replication delay added to the TTL.

11. Multi-region caching, and why it's not a small step

If users are spread across North America, Europe, and Asia, and Redis only lives in North America, every Asian user pays a full round trip to another continent even though the cache itself responds in milliseconds. Regional caching — a cache per region, each close to its users — fixes the latency problem. It does not simplify anything else.

Every layer from a single-region architecture now exists per region — and invalidation, which used to be one DELETE against one Redis, now has to reach every region before a change is actually reflected everywhere. Multi-region caching is a legitimate answer to a real latency problem, but it should be reached for because the latency problem is real and measured, not because global feels more impressive on an architecture diagram.

12. Consistency gets structurally harder as each layer is added

At the start, "is the cache correct" was a one-line answer: delete the Redis key after the database updates. Once there's a Redis cluster, local caches on top of it, and multiple regions on top of that, the question stops being "is the cache correct" and becomes "how quickly does a change propagate to every copy, everywhere" — a meaningfully harder problem with no single clean answer.

13. Event-driven invalidation, at this scale

Publishing an event on every data change — Product Updated fanning out to L1 consumers, Redis in Region A, Redis in Region B — keeps the producer decoupled from needing to know who's listening, the same pattern from Part 5. At this scale, the event system itself becomes something to manage: duplicate events, ordering, retries, delayed delivery, lost events, and idempotency all become real operational concerns rather than edge cases. It's trading one kind of complexity (manual multi-region invalidation) for a different, more distributed one.

14. Cache warming, and the deployment-specific version of it

For a known traffic spike — a major sale, a product launch, a sporting event — preloading the keys you already know will be hot avoids a wave of simultaneous first-time misses. There's a deployment-specific version of this worth calling out: deploying a new version means every application instance starts with an empty local cache, and if hundreds of instances all reach for Redis at once — especially if Redis itself was also just restarted — that's the exact shape of a self-inflicted stampede from Part 7. A controlled warm-up on deploy is cheap insurance against a problem you'd otherwise cause yourself, on a schedule you control.

15. The database still matters — read replicas included

"We have Redis, so the database doesn't matter" is wrong in a way that tends to surface at the worst moment: the database is still the source of truth, and every miss eventually reaches it. At high read volume, database read replicas can absorb cache-miss traffic that a single primary couldn't. But replicas introduce their own lag, so a value read from cache and then re-fetched from a replica on a miss can still be stale relative to a very recent write — performance bought with yet another consistency trade-off, the pattern this whole post keeps returning to.

16. Backpressure becomes non-optional

At a million requests/sec, an unprotected system facing a slow Redis looks like: requests wait, threads get consumed, connection pools exhaust, the application fails. The alternative is deliberate backpressure — queuing, limits, bounded concurrency — that protects Redis and the database by refusing some work rather than accepting all of it and failing everything. A high-performance system isn't one that accepts unlimited traffic; sometimes the highest-performing thing it can do is say no to some of it on purpose.

17. Capacity planning is not just "value size times key count"

10 million objects at 5KB average looks like roughly 50GB of raw data — but that's not the real Redis memory requirement. Keys, metadata, data structure overhead, replication, fragmentation, and operational headroom all add to that number, often substantially. Size a cache from measured actual memory consumption, not from multiplying the serialized value size by the key count.

18. Eviction policy is a decision about access patterns, not a default

When memory fills up, something has to go — LRU, LFU, and TTL-based policies all exist for a reason, and the right one depends on the actual access pattern rather than a general recommendation. If a small number of keys are extremely popular, LFU may outperform simple recency-based eviction for that specific workload. The point was never "always use LFU" — it's that the eviction policy should be chosen deliberately, the same principle from Part 2, now with real operational stakes attached.

19. Observability has to go deeper as scale grows

A single dashboard is plenty at 1,000 requests/sec. At a million, cluster-level visibility needs node health, CPU, memory, network, connections, commands/sec, latency, evictions, replication lag, and hot keys — and the application side needs L1 hit ratio, L2 hit ratio, misses, Redis errors, database fallback, P95/P99, and request volume. The goal is being able to trace a problem across the whole system, not just see that "something" is elevated somewhere.

20. Don't build any of this because it looks impressive

Multi-level caching, distributed Redis, replication, regional deployment, read replicas, and event-driven invalidation are each a real answer to a real problem — and together they're a lot of operational surface area. At 500 requests/sec, none of this is warranted. At 500,000+ requests/sec with strict latency requirements, the conversation genuinely changes. Architecture should follow scale, not the other way around — the same "earn the complexity" principle from Part 8, just at a size where the stakes of getting it wrong are much higher.

The biggest lesson: the bottleneck doesn't disappear, it moves

Every fix in this post follows the same pattern. Add caching, and the database bottleneck eases — but Redis is now the thing under load. Add clustering, and Redis's aggregate capacity problem eases — but now one hot node is the constraint. Add local caching to relieve that node, and the constraint moves again, this time to keeping every local copy consistent. The bottleneck was never eliminated at any step. It relocated, and the job was to notice where it went next.

That's the actual discipline behind caching at scale: not implementing every technique in this post preemptively, but continuously asking "where is the bottleneck now?" and adding exactly the piece that answers it — no more, no earlier.

What this means for the series so far

Caching at small scale can be Application → Redis → Database. At real scale it can become load balancers, application instances, local caches, a Redis cluster, regional caching, and a database layer with read replicas behind it. More infrastructure was never automatically a better system — the actual skill is knowing when each layer earns its place and why.

Caching was never really a product decision. Redis is a tool. Memcached is a tool. Local memory is a tool. A CDN is a tool. The architecture comes from understanding your workload, your consistency requirements, your failure modes, and your actual scale — not from which tool has the most features.

If you've operated a Redis cluster at real scale, what was the metric that actually caught your first hot-key incident — was it a per-node dashboard, or did it take an outage to teach you to look there?

Testing Your Cache: Proving It Actually Works

Muralidharan Lakshmanan — Sat, 22 Aug 2026 21:15:50 +0000

Part 9 built a Cache-Aside implementation with Spring Boot and Redis: TTL, structured keys, serialization, invalidation, database fallback. At that point it's tempting to declare victory — "the cache works." But what does "works" actually mean? Does it work when Redis is empty, or unavailable, or when a thousand requests land at the same instant, or when a cached value expires while the database keeps moving underneath it? And underneath all of that: did the cache actually make anything faster, or does it just look like it should have?

That's what this post tests.

1. Don't just test the happy path

The test almost everyone writes first is Request → Redis → HIT → success. That's useful, but it's one of three paths a cache actually has in production.

The HIT path is the one every test suite has. The ERROR path — Redis throwing a connection exception mid-request — is the one that's usually missing, and it's the one that determines whether an incident stays contained or spreads.

2. Test scenarios worth defining upfront

For a product API, a reasonably complete list looks like: cache hit (don't call the database), cache miss (call the database and populate the cache), TTL expiration (fetch fresh data once the entry ages out), an update (invalidate the cache), Redis unavailable (fall back appropriately), concurrent requests (avoid redundant database load), database unavailable (fail predictably), a serialization mismatch (fail safely), and a direct performance comparison (cache should measurably reduce latency). Naming these up front turns "test the cache" into something concrete enough to actually write.

3. Test #1 — cache hit

Suppose Redis already has product:101 cached, and a request comes in for it. The database should never be touched.

@Test
void shouldReturnProductFromCache() {

    Product product =
        new Product(101L, "MacBook Pro", 1999);

    when(redisTemplate.opsForValue()
        .get("product:101"))
        .thenReturn(product);

    Product result =
        productService.getProduct(101L);

    assertEquals("MacBook Pro", result.getName());

    verify(productRepository, never())
        .findById(101L);
}

The assertion that matters most here isn't assertEquals — it's verify(repository, never()). That's the line proving the database wasn't touched, not just that the right value came back.

4. Why that second assertion matters

It's possible for a cache to be returning correct data while still quietly querying the database on every request — Redis HIT → database query anyway → return. The application looks like it works, the response is correct, and none of that tells you the cache isn't providing its actual benefit. A real cache-hit test checks two separate things: correctness (the value is right) and efficiency (the database wasn't touched to get it). Either one alone can pass while the other silently fails.

5. Test #2 — cache miss

Now Redis returns null. The application needs to query the database, return the result, and populate Redis — the full Cache-Aside loop from Part 4:

@Test
void shouldLoadFromDatabaseOnCacheMiss() {

    Product product =
        new Product(101L, "MacBook Pro", 1999);

    when(redisTemplate.opsForValue()
        .get("product:101"))
        .thenReturn(null);

    when(productRepository.findById(101L))
        .thenReturn(Optional.of(product));

    Product result =
        productService.getProduct(101L);

    assertEquals(101L, result.getId());

    verify(productRepository)
        .findById(101L);

    verify(redisTemplate.opsForValue())
        .set(
            eq("product:101"),
            eq(product),
            any(Duration.class)
        );
}

This one verifies the whole loop, not just the read.

6. Test #3 — TTL

TTL is easy to get right in code and wrong in configuration, so it's worth verifying explicitly rather than assuming. For a unit test, you don't need to wait ten minutes — just confirm the cache write was configured with the TTL you expect:

verify(redisTemplate.opsForValue())
    .set(
        eq("product:101"),
        eq(product),
        eq(Duration.ofMinutes(10))
    );

For an integration test, a short TTL — two seconds — lets you actually wait for expiration and confirm the entry disappears when it should.

7. Unit tests and integration tests catch different bugs

A unit test mocks Redis; an integration test runs against a real instance and exercises real serialization, real TTL behavior, real key formatting, and a real connection. Integration tests catch classes of bugs mocks structurally can't — misconfigured connection settings, a serialization format that doesn't round-trip the way you assumed, a TTL that isn't actually being applied. Both layers earn their place; neither substitutes for the other.

8. Test #4 — cache invalidation

Suppose Redis has product:101 at $1,999, and an update changes the database price to $1,899. The application should delete the cache entry:

@Test
void shouldInvalidateCacheAfterUpdate() {

    Product product =
        new Product(101L, "MacBook Pro", 1899);

    when(productRepository.save(product))
        .thenReturn(product);

    productService.updateProduct(product);

    verify(productRepository)
        .save(product);

    verify(redisTemplate)
        .delete("product:101");
}

This test exists specifically to catch the most common caching bug there is: the database changed, and the cache didn't hear about it.

9. Test #5 — Redis is unavailable

Simulate the failure directly and confirm the application falls back to the database rather than failing the request outright:

when(redisTemplate.opsForValue()
    .get("product:101"))
    .thenThrow(new RedisConnectionFailureException(
        "Redis unavailable"
    ));

then verify productRepository.findById(101L) gets called. This is the try/catch fallback from Part 9, and this test is what proves it's actually wired up.

10. But be careful what this test does and doesn't prove

If Redis is down and the application is taking 10,000 requests/sec, every one of them now falls through to the database — and this test will still pass, because "fallback works" was never in question. Whether the database survives 10,000 requests/sec it wasn't sized for is a completely different question, and this test doesn't answer it. Functional correctness and resilience are separate properties. A unit test proves the fallback path exists; only a load test proves the system survives actually using it — which is exactly what Part 6 and Part 7 covered from the failure-mode side.

11. Test #6 — concurrent requests

This is arguably the single most valuable test in the whole suite. If a popular key expires and a thousand requests arrive at nearly the same moment, unprotected Cache-Aside turns that into a thousand simultaneous database queries — the cache stampede from Part 7. With request coalescing in place, it should look closer to one request loading the database and the other 999 reading the result once it lands.

12. How to actually test that

Fire concurrent calls from a thread pool:

ExecutorService executor =
    Executors.newFixedThreadPool(20);

List<Callable<Product>> tasks =
    IntStream.range(0, 100)
        .mapToObj(i ->
            () -> productService.getProduct(101L))
        .toList();

Execute them together, then check how many times the database was actually called. With coalescing working, 100 concurrent application requests should collapse into something close to a single database call — not exactly one, depending on timing and the coalescing strategy, but dramatically fewer than 100. This is a test an ordinary unit test simply cannot catch, because the bug only exists under concurrency.

13. Test #7 — database failure

Flip the failure around: Redis misses cleanly, but the database itself is unavailable. The API shouldn't return a bare 500 with no useful information — depending on the application, something like a 503 with enough context to be debuggable later is the better shape. The property that actually matters here isn't the exact status code; it's that the failure is predictable, observable, and bounded rather than an unhandled exception bubbling up as a mystery.

14. Test #8 — serialization

Suppose the cached bytes for product:101 don't match what the current application expects — an old serialized shape sitting next to new deserialization code, the exact versioning problem from Part 9. Deserialization can fail outright here, and a good integration test verifies the application doesn't just crash on it. A reasonable strategy: catch the deserialization failure, delete the bad entry, fall through to the database, and repopulate the cache correctly. That turns a corrupted cache entry into a self-healing miss instead of an application error.

15. Test #9 — measure before and after

The question that actually matters outside the engineering team: did this caching work improve anything? "Redis is fast" isn't evidence. Real numbers are — average latency, P95 latency, and database CPU, measured with caching off and then on, under the same load.

Caching isn't valuable because Redis looks good on an architecture diagram — it's valuable because these specific numbers moved, on this specific workload.

16. What to actually measure

Two scenarios to benchmark: without the cache (application straight through to the database) and with it (application through Redis, database only on a miss). Track throughput, average latency, P95, P99, database queries per second, CPU, memory, and network traffic — not just the average, which brings us to the next point.

17. Why P95 and P99 matter more than the average

A 20ms average latency sounds great in isolation. If P95 is 200ms and P99 is 2 seconds, a meaningful slice of real users are having a genuinely bad experience that the average is quietly hiding. For production systems, tail latency is usually the number that determines whether users notice a problem — a caching benchmark that only reports the average is reporting the least informative number available.

18. A simple load test

Tools like JMeter, Gatling, or k6 can drive traffic — say, 1,000 virtual users hitting GET /products/101 for ten minutes, run once with caching disabled and once enabled:

	No cache	With cache
Throughput	5K/sec	20K/sec
P95	350 ms	45 ms
DB CPU	85%	30%
DB queries/sec	5K/sec	200/sec

The exact numbers depend entirely on your application and environment — what matters is running this methodology against your own workload, not adopting someone else's numbers as a benchmark for your system.

19. Test the worst case, not just the best one

Testing cache hit, cache hit, cache hit, cache hit will always look fantastic — of course it does, that's the easy path. The tests that actually tell you something: an empty cache, expiration under load, a Redis restart, Redis being fully unavailable, a hot key, a slow database, a traffic spike, concurrent requests racing each other. Production doesn't grade on the good day. It grades on the bad one.

20. Test cache recovery

Redis restarts, and every cached entry is gone at once. The application needs to survive the climb back from MISS → database → Redis SET → HIT repeating across a cold cache under live traffic. Worth measuring explicitly: how fast the cache repopulates, how much database load the recovery window generates, what application latency looks like during that window, and whether a stampede happens on the way back up. A cache restart is a routine operational event — it shouldn't be able to take the application down with it.

21. Test a hot key deliberately

Manufacture the scenario from Part 7 on purpose: send a large share of traffic — a million requests, say — at a single key like product:popular, and watch Redis CPU, Redis network, application latency, and database traffic while it happens. This test earns its keep specifically for applications with popular products, trending content, homepage data, shared configuration, or anything else likely to concentrate traffic on one key.

22. Testing isn't only about code

A cache can pass every unit test in the suite and still fail in production, because production has things a test environment doesn't fully replicate: real traffic patterns, real concurrency, real network latency, real failure timing, real data shapes, real deployment behavior. That gap is exactly why a single layer of testing was never going to be enough.

23. Four levels of cache testing

Unit tests: cache hit, cache miss, invalidation, error handling — fast, isolated, and there should be a lot of them. Integration tests: Spring Boot against a real Redis, real serialization, real TTL and key behavior. Load tests: thousands of requests, concurrent users, hot keys, sustained misses — this is where you learn how the system behaves under pressure rather than in isolation. Failure tests: Redis unavailable, Redis restarting, a slow database, a database that's fully down, network faults — this is where you learn whether the architecture is actually resilient or just untested.

24. What to still monitor in production

Testing builds confidence before release; production observability is what tells you whether that confidence held up. At minimum: cache hit ratio, miss ratio, cache latency, Redis memory and CPU, evictions, connections, errors, hot keys, database fallback traffic, and application latency — and critically, correlated with each other rather than watched in isolation. A hit ratio drop that's followed by rising database traffic, rising database CPU, and rising application P95 tells a complete story that no single metric tells on its own.

25. The metric that matters more than hit ratio

A 99.9% hit ratio at 10,000 requests/sec still means 10 requests/sec are hitting the database — and if each of those misses is a genuinely expensive 2-second query, that 0.1% is not a rounding error, it's very possibly your entire P99 problem. The question worth asking alongside "what's our hit ratio" is "what does a miss actually cost us" — the two questions together tell you something neither one tells you alone.

26. What "the cache works" should actually mean

Put together, a production-ready cache should demonstrate correctness (the right data comes back), performance (hits are measurably faster than the database path), efficiency (database traffic is genuinely reduced), resilience (behavior stays predictable when Redis fails), scalability (concurrency and traffic spikes don't cause a stampede), recoverability (the cache can rebuild itself after a failure), and observability (you can see when it's helping, and when it's quietly not). That's a considerably more useful bar than "the demo worked."

The bigger lesson

Testing a cache was never really about proving SET key value works — Redis already knows how to do that. It's about proving that Application → Cache → Database behaves correctly across everything working, something going wrong, traffic increasing, and data changing, all at once and in combination. That's the actual difference between code that works and a system that can survive contact with production.

What's the test in your suite that actually caught a real caching bug before it shipped — was it a unit test, or did it take a load test or a chaos experiment to surface?

Let's Build a Cache: Redis + Spring Boot

Muralidharan Lakshmanan — Fri, 21 Aug 2026 02:13:40 +0000

The first eight parts of this series covered why caching improves performance, how hits and misses work, TTL and eviction, cache-aside and the other patterns, invalidation, Redis vs. Memcached, stampedes and hot keys, and how to design a production-ready cache architecture. Now let's actually build one.

The stack is one a lot of enterprise Java teams already run: Spring Boot, Redis, and a database behind it. The goal isn't a caching framework — it's something simple enough to fully understand and realistic enough to use as a real starting point.

1. What we're building

A simple product service: GET /products/{id}, backed by a database row like { "id": 101, "name": "MacBook Pro", "price": 1999.00 }. Without caching, every request — even a thousand requests for the same product — goes all the way to the database. With caching, it's Cache-Aside, the pattern from Part 4: check Redis first, fall back to the database on a miss, populate Redis for next time.

2. The stack

Java, Spring Boot, Spring Data Redis, Redis itself, JPA, and a relational database — Postgres or MySQL, it doesn't matter much for this example. The request path is client → Spring Boot → Redis → (on a miss) → database.

3. Start with the database

A plain Product entity and repository, nothing caching-specific yet:

@Entity
public class Product {

    @Id
    private Long id;

    private String name;

    private BigDecimal price;

    // getters and setters
}

public interface ProductRepository
        extends JpaRepository<Product, Long> {
}

Without caching, the service just goes straight through:

@Service
public class ProductService {

    private final ProductRepository repository;

    public ProductService(ProductRepository repository) {
        this.repository = repository;
    }

    public Product getProduct(Long id) {
        return repository.findById(id)
                .orElseThrow(() ->
                    new ProductNotFoundException(id));
    }
}

Every request hits the database. Now let's introduce Redis.

4. Add Redis

Spring Data Redis is the dependency:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

and the connection is standard Spring Boot configuration:

spring:
  data:
    redis:
      host: localhost
      port: 6379

5. The simplest Cache-Aside implementation

Using RedisTemplate directly:

@Service
public class ProductService {

    private final ProductRepository repository;
    private final RedisTemplate<String, Product> redisTemplate;

    public ProductService(
            ProductRepository repository,
            RedisTemplate<String, Product> redisTemplate) {

        this.repository = repository;
        this.redisTemplate = redisTemplate;
    }

    public Product getProduct(Long id) {

        String key = "product:" + id;

        Product cachedProduct =
                redisTemplate.opsForValue().get(key);

        if (cachedProduct != null) {
            return cachedProduct;
        }

        Product product = repository.findById(id)
                .orElseThrow(() ->
                    new ProductNotFoundException(id));

        redisTemplate.opsForValue()
                .set(key, product);

        return product;
    }
}

That's Cache-Aside, fully implemented. On the first request for product:101, Redis misses, the database gets queried, and the result gets written back to Redis before returning. On the second request for the same ID, Redis hits and the database never gets touched — that gap between the two is where the entire performance benefit comes from.

6. Add a TTL

Right now we're storing values indefinitely, which means a stale product can live in the cache forever if invalidation ever fails. Add an expiration:

redisTemplate.opsForValue()
        .set(
            key,
            product,
            Duration.ofMinutes(10)
        );

The entry now expires automatically after ten minutes — a safety net that holds even when explicit invalidation doesn't fire, which we'll get to shortly.

Resist hardcoding Duration.ofMinutes(10) everywhere it's needed, though. Once you have a product TTL, a user-profile TTL, a recommendations TTL, and a configuration TTL scattered through the codebase, tuning any of them means a code change and a redeploy. Externalizing them instead —

cache:
  product:
    ttl: 10m

— means freshness can be tuned from configuration, which matters more than it looks like it should the first time you need to change one in production without shipping code.

7. Cache keys matter

We used product:101 rather than a bare 101, and that's deliberate — a bare numeric key can collide across object types (101 the product, 101 the customer, 101 the order all fighting over the same slot). Namespacing avoids that: product:101, customer:101, order:101. Going one step further, product:v1:101 leaves room to version the cached shape later, the same technique covered in Part 8.

8. Serialization — the part that's easy to skip past

Redis stores bytes, not Java objects, so something has to convert between them. JSON is the common choice — readable, easy to inspect, and it round-trips cleanly:

Java Object → JSON → Redis
Redis → JSON → Java Object

It isn't free, though — serialization costs CPU, memory, and bandwidth, and changing the format later can break compatibility with whatever's already sitting in the cache. Treat the serialization format as an architectural decision made once, not a configuration line picked without much thought.

9. Cache invalidation on updates

Now the write path. PUT /products/101 updates the price — the database gets the update, and the cache entry gets deleted rather than updated in place:

public Product updateProduct(Product product) {

    Product updated = repository.save(product);

    redisTemplate.delete(
        "product:" + product.getId()
    );

    return updated;
}

UPDATE → DELETE CACHE, and the next read rebuilds the entry from the now-current database row. This is the same pattern from Part 5: deleting is simpler than updating in place because the cache is a temporary copy, and when the source of truth changes, the cleanest move is to stop trusting the old copy rather than try to patch it.

10. What if cache invalidation itself fails?

The real production scenario: the database update succeeds, but the Redis delete fails. Now the database says $1,899 and Redis still says $1,999. This is exactly the dual-write problem from Part 5, and the TTL from step 6 is what keeps it from being permanent — the stale entry eventually expires on its own. TTL isn't a substitute for invalidation here; it's the safety net underneath it.

11. What if Redis is completely down?

The read path shouldn't treat Redis as a hard dependency. A reasonable Cache-Aside implementation catches a connection failure and falls through to the database rather than failing the request outright:

try {
    Product cached = redisTemplate
            .opsForValue()
            .get(key);

    if (cached != null) {
        return cached;
    }

} catch (RedisConnectionFailureException ex) {
    // Log and continue to database
}

return repository.findById(id)
        .orElseThrow(...);

That's the right instinct, but it isn't the whole answer — if Redis is down and traffic is high, this fallback alone can be exactly the flood that overwhelms the database, the failure mode from Part 6 and Part 7. That's what request coalescing, rate limiting, circuit breakers, local caching, and bounded concurrency are for; this try/catch is necessary, not sufficient.

12. Add metrics

An unmeasured cache is hard to operate. At minimum:

meterRegistry.counter("cache.hit").increment();
meterRegistry.counter("cache.miss").increment();

plus cache.error and cache.latency. From hits and misses you get the hit ratio — 9,900 hits against 100 misses is 99% — but a high ratio on its own doesn't mean the system is healthy. A 99% hit ratio is meaningless if the 1% of misses are the expensive ones.

13. The N+1 cache problem

Here's a subtler issue that shows up once caching is actually working. Suppose an endpoint returns an order with a customer and four products:

One API request just became five cache operations. Each one is individually cheap, but at real scale that adds up, and it's easy to miss because no single call looks expensive in isolation. Caching doesn't remove an N+1 access pattern — it just makes each hop in it cheaper. Measure the whole request path, not just the cache calls that look slow on their own.

14. Don't cache everything

Good candidates share a shape: frequent reads, expensive computation or database access, infrequent changes, and staleness the business can tolerate. Poor candidates: highly volatile data, sensitive data without real access controls, data nobody actually requests often, data that's cheap to fetch anyway, and anything where a stale value is genuinely unacceptable.

A cache should exist because it solves a demonstrated performance problem — the same principle from Part 8's "earn the complexity" argument, just applied at the level of an individual field instead of an architecture layer. It shouldn't be a box ticked on an architecture diagram because caching is what's done around here.

15. Putting the read and write paths together

This is a simple, understandable caching architecture — which is exactly the point. Nothing here should surprise anyone reading it for the first time during an incident.

16. What we've built

Working through the steps above, this implementation now has Cache-Aside on the read path, a TTL so entries expire even if invalidation fails, explicit invalidation on writes, structured and versioned cache keys, a deliberate serialization format, basic hit/miss/error observability, and a Redis connection that isn't a hard dependency for every request. The database stays authoritative throughout — Redis is a fast, disposable copy of it, never the other way around.

17. Is this production-ready yet?

Not quite — this is a solid foundation, but a high-traffic production system needs more of what Part 8 covered in full: Redis high availability and clustering, connection pooling, timeouts, circuit breakers, request coalescing, hot-key protection, cache warming, TTL jitter, security, monitoring, and alerting.

There's also a question this post hasn't touched at all: testing. How do you actually prove caching improved anything? How do you test a cache hit, a cache miss, Redis being unavailable, a stale entry, concurrent requests racing each other, expiration timing, and a database failure underneath all of it?

That's genuinely a different skill from writing the cache-aside logic itself, and it's where the next part of this series is headed.

If you've built something close to this, what was the first thing that broke once it hit real traffic — the invalidation path, a serialization mismatch, or something in the failure handling you hadn't tested?

Designing a Production-Ready Cache

Muralidharan Lakshmanan — Thu, 20 Aug 2026 02:25:56 +0000

Across this series we've covered why caching exists, how memory and TTL work, where a cache can live, cache-aside and the other patterns, invalidation, Redis vs. Memcached, and the specific ways caching goes wrong under load. Now let's put it together.

Imagine someone asks: "We have a high-traffic backend, the database is becoming a bottleneck — can we add Redis?" The easy answer is "yes, put Redis in front of the database." The architect's answer is closer to: "Maybe — let's understand the workload, the consistency requirements, the failure scenarios, and what happens when Redis isn't there."

Adding a cache isn't the architecture. Designing what happens around the cache is the architecture.

1. Start with the problem

Picture a typical backend: users, a load balancer, application servers, a database. It's receiving 20,000 requests/sec, but a lot of those requests are asking for the same handful of things over and over — the same product pages, the same lookups. The database keeps redoing work it already did a second ago. Database CPU sits at 85%, connections at 90%, average latency 250ms, P95 latency 600ms.

That's a good candidate for caching. But before reaching for Redis, it's worth asking: what data are we actually caching, how often does it change, how stale can it be, how much memory do we need, what happens if the cache disappears, and what happens during a traffic spike? Those five questions shape the actual design far more than the decision to add a cache in the first place.

2. Cache-aside is still the right starting point

For most applications, Cache-Aside is where I'd start — it's the pattern from Part 4, and it holds up here. The read path checks the cache, falls back to the database on a miss, and populates the cache for next time:

Product product = cache.get(key);

if (product == null) {
    product = database.findProduct(id);
    cache.put(key, product, ttl);
}

return product;

It's easy to understand and easy to troubleshoot, and that second property matters more than it sounds like it should — production systems benefit enormously from designs an engineer can reason about quickly at 2am during an incident.

The write path is just as simple: update the database, then delete the cache entry, and let the next read rebuild it.

database.update(product);
cache.delete("product:" + product.getId());

Simple. But production systems aren't always simple, and the rest of this post is about what "simple" needs around it before it's actually production-ready.

3. What if Redis is down?

This is one of the most important questions in the whole design, and we covered the mechanics in Part 6 and Part 7 — worth restating the number here because it's the one that should drive the rest of this architecture. If Redis is normally absorbing 99,000 of every 100,000 requests/sec, and it suddenly disappears, all 100,000 fall straight through to the database. That's a 100x increase, and most databases were never sized for it.

Cache failure handling is not an afterthought here — it's as important as the cache design itself.

4. Don't let cache failure become database failure

A local, in-process application cache is one of the simpler ways to blunt that failure. Frequently accessed data can be served from application memory without ever reaching Redis, which also cuts network latency on the common path. The cost is a second copy of the data and a second invalidation problem — use it deliberately, not by default.

Layered together, this becomes a multi-level cache: L1 in application memory, L2 in Redis, L3 the database itself, each layer catching what the one before it missed. The closer the data sits to the application, the faster it responds — but every additional layer is more complexity, so don't add L1 just because you can; add it because you've measured Redis taking more traffic than it should.

5. Redis needs high availability too

Once Redis is load-bearing, a single Redis instance is a single point of failure sitting right where you least want one. Production deployments typically add replication, automatic failover, clustering, or a managed Redis offering across multiple availability zones — the exact topology depends on your provider, but the principle doesn't: don't create a single point of failure in the component you're depending on for performance.

High availability isn't the same as zero downtime, though. A successful failover can still involve connection failures, a short window of elevated latency, retries, and topology changes. The application needs to handle a Redis reconnect gracefully — retry with backoff, then continue — rather than have a hundred threads hammer the reconnect at once.

6. TTL strategy, and why jitter matters at scale

We covered TTL as a freshness question back in Part 2 and Part 5: not "what TTL should I use" but "how stale can this specific data safely be." A product description tolerates an hour; an exchange rate tolerates almost nothing; inventory depends entirely on the business.

The scale-specific risk shows up on deploys. If an application restart loads 500,000 cache entries with a flat 60-minute TTL, a large fraction of them can expire together an hour later — a self-inflicted cache avalanche, the failure mode from Part 7. Adding jitter (TTL = 60 minutes + random(0–5 minutes)) spreads that expiration across a window instead of a single instant. It's a small decision that prevents a surprisingly large spike.

The same logic from Part 7 applies here too: preventing cache stampede with request coalescing, and watching for hot keys where traffic concentrates on one popular value regardless of how many total keys exist. Both are things this architecture needs to have an answer for, not things to solve for the first time when they show up in production.

7. Cache warming

Rather than letting a cold cache meet its first real traffic spike head-on, preload the data you already know will be popular — before Black Friday, before a product launch, before a ticket sale goes live. You don't want your first million users of the day to be the ones warming the cache for everyone after them.

8. What if the database is also slow?

A scenario worth designing for explicitly: Redis is healthy, but the database itself is slow, so a cache miss now takes five seconds instead of five milliseconds. Hundreds of application threads can end up waiting on that single slow path simultaneously, the connection pool exhausts, requests queue, and then they start timing out anyway.

This is why a production cache architecture still needs database connection limits, request timeouts, circuit breakers, bounded concurrency, and backpressure. Caching reduces how often you hit the database — it doesn't remove the need for the database path itself to fail gracefully when it's slow.

9. Bringing it together

Here's what all of the above looks like assembled into one system, rather than as separate techniques:

This is no longer "we added Redis." It's a system with a specific, considered answer for what happens when any single layer of it fails.

10. But don't build all of this on day one

Looking at that diagram, it's tempting to implement everything at once — cluster, local cache, distributed locks, event-driven invalidation, warming, circuit breakers, multiple TTL strategies, full monitoring — right from the start. Don't.

Start with the simplest design that solves the actual problem: application, Redis, database. Then measure. If Redis is taking too much traffic, consider a local cache. If you're seeing stampedes, consider request coalescing. If invalidation is noticeably delayed, consider making it event-driven. Add complexity in response to a demonstrated problem — not a hypothetical one.

11. Observability: watch more than the hit ratio

The most common gap in caching implementations isn't a missing technique — it's not watching the right things. A 99% hit ratio looks great in isolation and tells you almost nothing about what happens during the other 1%, or what that number does under stress.

Every metric on that board earns its place, but the one worth watching above the rest is database fallback traffic — because it's the one that tells you, immediately, whether a Redis latency blip is staying contained or turning into database load, application latency, and eventually timeouts. Knowing "Redis is at 80% CPU" on its own tells you far less than knowing what that 80% is doing to everything downstream of it.

12. Security is part of the cache design, not separate from it

A cache holds application data, and "it's internal infrastructure" isn't a reason to skip authentication, encryption in transit, access control, network isolation, credential management, or tenant isolation. The specific trap worth naming: caching data without considering who's allowed to see it. A key like user:123:profile is not the same thing as profile:123 the moment the underlying data carries user-specific authorization — a cache key design mistake can quietly become a security bug.

13. Cache keys are part of the architecture, not an implementation detail

A good key is deterministic, unique, easy to read, and scoped appropriately: product:v1:123 beats a bare 123, because it leaves room for versioning and avoids collisions. User-specific data gets scoped explicitly (user:123:recommendations); tenant-specific data gets scoped explicitly too (tenant:45:product:123). Sloppy key design causes collisions, stale reads, security issues, and invalidation that's harder than it needs to be.

Versioning the key itself is a small trick that pays off repeatedly: when a cached object's shape changes — say, a currency field gets added to a product payload — bumping product:v1:123 to product:v2:123 means old and new cached shapes never collide, and a deploy doesn't need to worry about what the previous version of the code left behind in the cache.

14. What if the cache is completely lost?

This is the real test of whether something is "just a cache." If Redis disappears entirely, the application should be able to rebuild from the database as the authoritative source — the open question is whether the database can survive the temporary surge while that rebuild happens. Request coalescing, rate limiting, local caching, gradual warming, connection limits, and circuit breakers are all protections aimed at exactly this moment.

The goal is cache failure leading to degraded performance and then recovery — not cache failure leading to database overload, then application failure, then everything going down together.

15. The production checklist

Before I'd call a caching implementation production-ready, I'd want answers across seven areas: what we're caching and what the source of truth is; the expected hit ratio, latency, and memory footprint; the TTL for each data type and whether it's jittered; what happens if Redis is unavailable and whether the database can absorb the fallback; whether a stampede is possible and whether coalescing is in place; how invalidation works and what happens if it fails; whether Redis can scale horizontally and what happens at memory exhaustion; and finally, whether there are metrics, alerts, and a way to spot hot keys and cache-driven database spikes before a person has to find out the hard way.

If you can't answer most of those, the architecture probably isn't finished — even if it's already handling production traffic today.

The biggest lesson from this series

Caching was never really about Redis, or Memcached, or any particular technology. At its core, it's a trade-off.

We trade freshness for performance, and we accept some complexity as the price of doing that trade deliberately instead of by accident. The best caching architecture was never the one with the highest hit ratio on a dashboard — it's the one that finds the right balance of performance, consistency, reliability, and operational simplicity for what the business actually needs.

What's next

That closes out the conceptual side of this series — why caching exists, how it works, where it lives, how to read and write through it, how to keep it honest, what it's built out of, how it breaks, and how to design around all of that on purpose.

If you've taken a cache from "we added Redis" to something closer to this reference architecture, what was the first piece you added — and was it because you measured the problem, or because you got paged for it?

When Caching Goes Wrong: Cache Stampede, Hot Keys & Thundering Herds

Muralidharan Lakshmanan — Tue, 18 Aug 2026 23:05:31 +0000

Caching is supposed to make systems faster. Here's the surprising part: a cache can sometimes make a system slower — and in extreme cases, bring it down entirely.

Picture this. Your application normally handles 10,000 requests/sec, and your database comfortably handles 500 queries/sec, because the cache is absorbing almost everything. Then one popular cache entry expires. Suddenly all 10,000 requests miss at once, all 10,000 become database queries, and the database was never sized for that. CPU climbs, connection pools fill up, response times rise, requests start timing out — and then clients retry, which sends the database more traffic, not less.

That's how a small cache event turns into a production incident. This post covers the failure modes behind it: cache stampede, thundering herd, hot keys, cache penetration, cache avalanche, and the retry storms that make all of them worse — plus the standard defenses for each.

1. Cache stampede

Suppose product:123 has a 10-minute TTL and thousands of users are requesting it. For ten minutes, every request is a hit. Then the TTL expires, and if 5,000 requests arrive in roughly the same instant, every single one of them sees a miss and goes straight to the database.

The cache was protecting the database. Then, for one bad moment, it stopped.

2. Why this is so dangerous

Numbers make this concrete. Say normal traffic is 20,000 requests/sec with a 99% hit ratio — the database sees roughly 200 requests/sec, which is nothing.

Now one popular key expires. The same 20,000 requests/sec are still arriving, but now all of them miss — the database goes from 200 requests/sec to roughly 20,000. Nothing about the traffic changed. One cache entry going cold multiplied database load a hundred times over, and that's exactly the kind of jump that turns a healthy database into a struggling one within seconds.

3. The thundering herd problem

Thundering herd is the general name for this shape of problem: a large number of requests are waiting on the same resource, and when it becomes available (or in this case, unavailable), they all rush it simultaneously. The database isn't necessarily slow — the problem is that far too many requests are doing the exact same expensive work at the exact same moment.

4. The simple fix: let one request do the work

If a thousand requests miss the same key at once, there's no reason for a thousand identical database queries. Instead, let one request load the data while everyone else waits for that single result.

This is usually called request coalescing or single flight. It's a genuinely simple idea with an outsized effect: we're not making the database faster, we're just stopping 999 requests from doing work that's already being done.

5. Locking

One way to implement coalescing is a distributed lock: on a miss, try to acquire a lock for that key; whoever gets it queries the database, populates the cache, and releases the lock; everyone else waits, then reads the now-populated cache once it's released.

It works well, but a distributed lock brings its own list of things to get right — lock expiration, deadlocks, a process crashing while holding the lock, who owns the lock, and what retry behavior looks like for the requests waiting on it. Don't reach for distributed locking as a default; reach for it once you've confirmed you need it.

6. Randomized TTL — add some jitter

Here's a much cheaper technique for a related problem. If 100,000 cache entries were all created around the same time with a flat one-hour TTL, a lot of them expire together an hour later. Add a small random offset instead:

TTL = 60 minutes + random(0–5 minutes)

Now expirations spread across a window instead of landing on a single instant. This is TTL jitter — simple to add, and surprisingly effective at preventing synchronized expiry from becoming synchronized load.

7. Hot keys

Different problem, same root cause. Imagine 10 million cached keys — plenty of spread on paper — but 80% of traffic is for exactly one of them, say product:iphone. That single key is now a hot key, and it doesn't matter how many other keys exist; the traffic isn't evenly distributed across them.

8. Why hot keys are dangerous

If a single cache node handles 50,000 requests/sec but one key alone is getting 200,000 requests/sec, that key becomes a bottleneck regardless of how much headroom the rest of the cluster has.

This matters most in distributed caches specifically, because adding more nodes doesn't fix it — the hot key still maps to the same node it always did. You've spread the average load across the cluster; the one key that's actually the problem never moved.

The usual responses: keep the hottest data in application-local memory too, so it doesn't have to reach the distributed cache on every request; replicate the hot key across multiple nodes instead of pinning it to one; apply request coalescing specifically to that key; or, if the data doesn't change often, simply give it a longer TTL to reduce how often it needs refreshing at all.

9. Cache penetration

A different failure shape entirely. Suppose users request product:999999999 — a product that doesn't exist. Every request misses the cache, queries the database, gets NOT FOUND, and the next request does the exact same round trip. The cache isn't helping, because there's nothing valid to cache.

This is cache penetration, and it's particularly nasty when it's driven by scraping, enumeration, or a bug generating IDs that were never real to begin with.

10. Cache the "not found" result

The fix is almost too simple: cache the negative result too.

product:999999999 → NOT_FOUND
TTL = 60 seconds

Now the next request for that ID is a cache hit, even though the answer is "it doesn't exist" — no database call needed. The one thing to watch: don't cache a negative result for hours, since the object might legitimately get created shortly afterward. Negative caching generally wants a short TTL specifically because "not found today" and "will never exist" are different claims.

11. Cache avalanche

Now combine the timing problem with scale. If thousands of entries were all loaded around the same time with the same TTL, they can all expire together — a cache avalanche. It can also be triggered by a cache cluster failure, a mass invalidation, an application restart, a deployment, or a network blip.

Whatever the trigger, the shape is the same: cache failure leads to a cache-miss explosion, which leads to a database traffic explosion, which leads to database slowdown, application slowdown, timeouts, retries, and then even more traffic than before. That's a cascading failure, and it's the same underlying mechanics as a stampede — just triggered by breadth instead of one popular key.

12. The retry problem makes it worse

Here's the part that turns a bad moment into a genuine outage: when a request times out, the client retries. Retry, timeout, retry, timeout — instead of reducing load on a struggling system, the system generates more of it.

The database didn't fail once here. It failed, and then got asked to fail again, faster, by the same clients that just timed out. This is why caching problems and retry storms are dangerous specifically together — a resilient system needs bounded retries, exponential backoff, jitter on the backoff itself, timeouts, circuit breakers, and rate limits. Caching is only one part of the resilience story; the retry behavior around it is just as load-bearing.

13. Cache warming

Rather than letting the application start with a cold, empty cache, you can proactively load the data you already know will be popular. Before Black Friday, preload the popular products. Before a major product launch, ticket sale, or marketing campaign, warm the keys you already know will spike. You don't want your first million users of the day to be the ones who happen to warm your cache for you.

14. Stale-while-revalidate, again

We covered this in Part 5 as an invalidation technique — it's just as useful here as a stampede defense. Instead of deleting an expired value immediately and forcing every subsequent request to wait on the database, serve the existing value while refreshing it in the background:

User → stale value → response immediately
              ↓
       background refresh → cache updated

This trades a small amount of freshness for a large amount of availability and latency stability. As always, whether that trade is acceptable is a business question, not a technical one.

15. Don't treat the cache as a black box

The single biggest observability mistake is only watching the hit ratio. A 99% hit ratio looks great in isolation, but it doesn't tell you what happens during the other 1%, or what happens to that number under stress.

Also watch: miss rate, eviction rate, memory usage, latency, key distribution, hot keys specifically, connection count, errors, and timeouts. Above all, watch what happens to database traffic when cache performance degrades — that's usually the single most important signal in the whole system, because it's the one that tells you whether your cache is a performance optimization or a hidden single point of failure.

16. A more useful mental model

Most people design a cache by thinking Request → Cache → Database and stop there. The more useful model asks what happens at the miss branch specifically — is there a lock, a coalescing layer, a TTL strategy, something standing between "everyone missed at once" and "everyone queries the database at once"?

The goal was never just "make the cache fast." It's "make the system behave predictably when the cache is slow, empty, overloaded, or unavailable." That reframing is worth more than any individual technique in this post.

17. Five problems worth remembering

If you keep nothing else from this post, keep these five, matched to their standard fix:

Cache stampede — many requests miss the same key at once. Fix with request coalescing, locking, or stale-while-revalidate.
Hot key — one key receives disproportionate traffic. Fix with local caching, replication, a longer TTL, or coalescing.
Cache penetration — requests repeatedly ask for data that doesn't exist. Fix with negative caching, input validation, or Bloom filters at large scale.
Cache avalanche — many keys expire or disappear together. Fix with TTL jitter, cache warming, staggered expiration, and a resilient fallback path.
Retry storm — failures cause aggressive client retries. Fix with exponential backoff, jitter, bounded retries, circuit breakers, and rate limiting.

The bigger lesson

"Put frequently used data in memory and everything gets faster" is true — right up until it isn't. At production scale, the real questions are what happens when the cache expires, what happens when 10,000 requests miss at the same instant, what happens when one key gets disproportionately popular, what happens when the cache goes down entirely, and what happens when clients start retrying into all of that.

Those questions are what separate a cache that works in a demo from a caching architecture that survives production. And the principle underneath all of them is the same one this whole series keeps returning to: a cache should protect your database, not become another single point of failure.

Has your system ever hit one of these five in production — and which one? Stampede and retry storms tend to travel together in my experience; curious if that matches what others have seen.

Redis vs. Memcached: What Are We Really Getting?

Muralidharan Lakshmanan — Tue, 18 Aug 2026 00:05:44 +0000

By now, we've covered why caching improves performance, how caches work, where they can live, common caching patterns, and why invalidation is hard. Let's make things concrete.

When someone says "let's add a distributed cache," two names almost always come up: Redis and Memcached. Both are extremely fast, both store data primarily in memory, and both can dramatically reduce database load.

So the obvious question is which one to use. But the answer isn't "Redis is better" — that's too simplistic. The better question is: what problem are you trying to solve?

1. What is Memcached?

Memcached is essentially a high-performance distributed key-value cache. You give it a key, it gives you back a value:

user:123 → {name: "John", age: 42}

GET user:123

That's the basic idea, and Memcached keeps it deliberately simple. Its job is to store temporary data in memory and retrieve it very quickly. That simplicity is one of its biggest strengths.

2. What is Redis?

Redis also started as a fast in-memory key-value store, but it's evolved into something broader. It supports several data structures — strings, hashes, lists, sets, sorted sets, and streams — so instead of storing a flat JSON blob under user:123, you can store a structured hash with individual fields. Redis also gets used for counters, leaderboards, queues, distributed coordination, rate limiting, session storage, pub/sub, and event streams.

The simplest way to hold the distinction: Memcached says "I need a really fast temporary cache." Redis says "I need a really fast in-memory data platform that can also act as a cache." Memcached focuses on simplicity. Redis gives you significantly more capability — for a price we'll get to.

3. Redis can be more than a cache — and that's a warning too

Imagine your architecture starts simple: Redis as a cache. Then someone notices Redis can also handle session data, so it does. Then rate limiting gets added. Then someone wires up Redis Streams for events. Each step is individually reasonable.

Here's the architectural lesson: the more responsibilities you put into Redis, the less comfortable you should be treating it as disposable cache infrastructure. If Redis is only a cache, you can usually rebuild it. If it contains important state, losing it becomes a much more serious conversation.

4. Persistence

Redis can be configured with persistence so data survives a restart — useful once Redis is holding more than disposable cache entries. Memcached is much more cache-oriented: if the server restarts, you generally assume the cached data is gone, and that's fine as long as the architecture was designed with that assumption in mind. A cache should usually be rebuildable — persistence is what you reach for once something stops being "just a cache."

5. If Redis goes down, what happens?

This is a question every architect should ask before it happens in production, not after.

Ideally, Redis becoming unavailable means a clean fallback: cache miss, go to the database. But if Redis normally absorbs 99% of traffic and it suddenly disappears, all of that traffic lands on the database at once — overload, slow responses, timeouts, and the application degrading along with it.

Redis isn't dangerous because it's slow. It's dangerous because it's so effective at absorbing traffic that the rest of the system may never have been tested without it. We'll dig into this failure mode properly in the next part of this series.

6. Scaling

Both Redis and Memcached scale horizontally, but the shape differs. Memcached typically uses a straightforward distributed model — keys spread across nodes with a simple hashing scheme. Redis also supports clustering, but Redis clustering brings additional considerations: partitioning, replication, failover, topology, and resharding.

The lesson isn't that one scales and the other doesn't — both do. The real question is how much operational complexity you're willing to take on.

7. Performance

You'll hear "Redis is faster" and "Memcached is faster" from different people, and in practice, both can be extremely fast for a basic GET/SET. For most applications the difference won't matter — your actual bottleneck is more likely to be the network, serialization, application processing, connection pooling, or the database itself.

Don't pick a caching technology off a microbenchmark. Ask instead: does it meet my application's actual latency and throughput requirements?

8. Memory efficiency and eviction

Memcached's simplicity gives it relatively straightforward memory management, which matters if you're storing millions of tiny objects and every byte counts. Redis's richer data structures can carry more memory overhead depending on which ones you use. The right answer depends on key size, value size, entry count, structure choice, and metadata overhead — measure your actual workload rather than assuming.

Both support eviction policies (Redis offers LRU, LFU, TTL-based, and no-eviction configurations; Memcached has its own caching-oriented mechanism). We covered why eviction matters back in Part 2 — the architectural lesson there still applies: know what happens when your cache runs out of memory before production finds out for you.

9. Where Redis pulls ahead: counters, rate limiting, leaderboards

A few examples where Redis's extra structure earns its complexity. For rate limiting, an atomic counter per user makes a request-per-minute limit straightforward to implement correctly under concurrency — something Memcached can approximate but wasn't built around. For leaderboards, Redis Sorted Sets are purpose-built for maintaining and efficiently querying ranked data; Memcached's flat key-value model isn't designed for that kind of query at all.

These aren't caching problems in the traditional sense — they're state problems that happen to want the same speed a cache provides.

10. When Memcached makes perfect sense

Redis gets most of the attention, but that doesn't make Memcached obsolete. If your requirements are simple key-value caching, temporary data, a large number of entries, straightforward scaling, no need for rich data structures, and a cache that can be completely rebuilt from scratch — Memcached's simplicity is a feature, not a limitation. If all you need is GET, SET, and DELETE, there's no reason to carry the operational weight of streams, sorted sets, pub/sub, and persistence you'll never use.

11. A practical comparison

Capability	Redis	Memcached
Basic key-value caching	Yes	Yes
In-memory performance	Excellent	Excellent
Rich data structures	Yes	Limited
Persistence options	Yes	Primarily cache-oriented
Counters	Yes	Yes
Sorted sets	Yes	No
Streams / pub-sub	Yes	No
Operational simplicity	Moderate	Very good
Cache-only workloads	Excellent	Excellent

Don't turn this into a checklist exercise — the right choice depends on your workload, not on how many rows favor one column.

12. A real-world decision

Say you're designing a simple product catalog service: GET product:123, returning a small JSON object that changes occasionally. No queues, no streams, no leaderboards, no counters, no persistence requirement. Either Redis or Memcached would work fine here — there's no reason to reach for Redis just because it has more features sitting unused.

Now imagine the same platform also needs rate limiting, session storage, counters, leaderboards, and some distributed coordination. Redis becomes much more compelling, because now you actually need the capabilities it provides beyond caching.

13. Don't ask "which is better?"

This is the biggest lesson in this comparison. Don't ask "Redis vs. Memcached, which is better?" Ask what capabilities does my system actually need? A simple cache points toward Memcached being enough. A richer in-memory platform points toward Redis being the better fit. And sometimes the honest answer is neither — a managed cloud caching service, a CDN, a database-level cache, or an application-local cache might solve the actual problem better.

Technology should follow the requirement, not the other way around.

14. The architect's checklist

Six questions worth running through before committing to either: What's the data model — plain key-value, or something richer? What's the durability requirement — can this data simply disappear, or does it need a persistence strategy? What's the availability story — what happens to the application when the cache is unreachable? What's the expected scale — keys, requests per second, total data size? Who owns the operational complexity — monitoring, patching, scaling, recovery, troubleshooting? And most importantly, what's the failure behavior — what does the application actually do when the cache is gone?

That last question usually matters more than which cache wins a benchmark.

The bigger lesson

Redis and Memcached are both excellent technologies, and the interesting difference isn't "newer vs. older" or "better vs. worse." It's focused caching vs. caching plus a broader in-memory toolbox — and there's a wider principle underneath that distinction: don't introduce complexity unless you actually need the capability that comes with it. A simple cache that does exactly what you need can beat a powerful platform your team doesn't need yet.

What's next

We've now covered why caching exists, how it works, where it lives, the patterns for using it, how to keep it honest, and two of the most common technologies that implement it.

But we still haven't answered the question this post kept circling back to: what actually happens to your system when the cache goes down? That's next.

If you've run both in production, what tipped the decision for you — was it the data model, or was it really about who was going to operate it at 2am?

The Hardest Problem in Caching: Invalidation

Muralidharan Lakshmanan — Mon, 17 Aug 2026 01:28:56 +0000

When we started this series, caching sounded simple: put the data somewhere faster, get a faster response. But there's a question we haven't fully answered yet.

What happens when the data in the database changes?

Say the database has product:123 at name = "Laptop", price = $999. We put it in the cache. Everything works perfectly. Then the business changes the price to $899 — but the cache still has $999.

This is stale data, and it leads us to one of the most famous lines in software engineering — often attributed to Phil Karlton:

"There are only two hard things in Computer Science: cache invalidation and naming things."

The reason it's hard is simple: a cache creates another copy of your data, and once you have multiple copies, keeping them synchronized becomes a problem.

1. What does "invalidation" actually mean?

Cache invalidation simply means making sure an old cached value is no longer used. There are several ways to do it: delete the cached value and let the next request reload it, update the cached value directly, wait for the TTL to expire it, or invalidate it in response to a published event. Each has trade-offs. Let's look at them.

2. Strategy #1 — TTL

The simplest solution is to not worry about invalidation at all: just let the cache expire.

product:123
TTL = 10 minutes

After ten minutes, the entry expires and the next request becomes a miss that repopulates it from the database. TTL is popular because it's simple — you don't need to know exactly when the database changes. The cache eventually becomes invalid on its own.

3. TTL is a trade-off

We covered this in Part 2, so briefly: a short TTL gives fresher data at the cost of more misses and more database traffic. A long TTL gives better cache efficiency at the cost of potentially stale data.

So the useful question was never "what TTL should I use?" It's "how stale can this data safely be?" — a much more concrete engineering question, and one we'll come back to near the end of this post.

4. Strategy #2 — explicit invalidation

Instead of waiting for the cache to expire, we can invalidate it the moment the data changes:

UPDATE database
       ↓
DELETE product:123 from cache

The next request becomes a cache miss, reads $899 from the database, and repopulates the cache.

Why delete rather than update in place? Sometimes updating is fine, but deleting is often simpler — the next read just goes and gets the authoritative value, so fewer things need to stay in sync. That gives us Write → Database → Delete cache rather than Write → Database → Update cache.

5. The dual-write problem

Here's the catch: you're now writing to two systems, and either one can fail independently of the other.

If you update the database first and the cache delete fails, the cache keeps serving $999 until the TTL eventually clears it — the database is correct, the cache is just temporarily behind. If you update the cache first and the database write fails, the cache is now ahead of the source of truth, showing a value the database never actually confirmed.

Whenever you're writing to two systems independently, you need to think through what happens if one succeeds and the other doesn't.

6. Strategy #3 — update the cache directly

Another approach updates both values in the same operation: Database: $999 → $899 and Cache: $999 → $899, avoiding the temporary miss entirely. The next request immediately sees $899.

The cost is coordination. The application has to make sure the cache update actually happens — and if it fails, do you retry, log it, send an event, or fall back to just deleting the entry instead? The more sophisticated the strategy, the more failure scenarios you need to have an answer for.

7. Strategy #4 — event-driven invalidation

Instead of every component directly coordinating the cache update, the system can publish an event and let interested consumers react to it.

The product service doesn't need to know every consumer that cares about the change — cache invalidation, search indexing, and analytics can all react independently. That's a more loosely coupled architecture. The price is another distributed component: the event system itself, and events can fail, be delayed, be duplicated, or arrive out of order.

If an invalidation event is lost, the cache consumer never fires and the stale entry just stays — which is exactly why production systems often combine event-driven invalidation with a reasonable TTL. The event gives fast invalidation; the TTL is the safety net that cleans up if the event never arrives.

8. Cache-Aside + invalidation

Connecting this back to Part 4: if you're using Cache-Aside, the read path checks the cache and falls back to the database on a miss, and the write path is just:

updateProduct(product);
cache.delete("product:" + product.getId());

Update the database, delete the cache entry, let the next read rebuild it. This is one of the simplest practical caching designs, and it's often easier to reason about than trying to manually keep every cached representation in sync.

9. But what if multiple caches exist?

Recall Part 3: a request can pass through a browser cache, a CDN, a local application cache, and a distributed cache before it ever reaches the database. Now the product price changes — which of those do you invalidate?

It's no longer DELETE key from Redis. It's an architectural question, and it's one reason systems try to avoid caching the same mutable data at too many layers: every additional copy is another invalidation problem.

10. The cache invalidation race condition

Here's a subtler problem. Suppose two requests happen at almost the same time: Request A is updating the price, and Request B is reading it.

The delete itself wasn't wrong — it happened exactly when it should have. The problem is timing: Request B had already read the old value before Request A's write landed, and Request B's write to the cache arrived after the delete. The stale value gets written back in, and now the cache is wrong until the TTL expires it.

This is why cache consistency isn't just a matter of adding a cache.delete() call. Concurrency and ordering matter too.

11. Invalidation vs. refresh

These are related but different. Invalidation removes the value — DELETE, then let the normal read path rebuild it. Refresh replaces the old value directly with the new one, avoiding a cache miss but requiring the system to know how to fetch and apply the latest value. Invalidation is simpler; refresh can be faster for the next reader. Which one fits depends on how expensive a miss is for that particular key.

12. What about stale-while-revalidate?

A cleverer option: if the cached value is technically expired, serve it anyway while refreshing it in the background, rather than making the user wait on a database call.

User → Cache → serve the slightly stale value
                       ↓
                background refresh → Database

This can give excellent user-facing latency. But it's deliberately serving stale data on purpose, so it only works where the business can tolerate that — which brings us to the real question underneath all of this.

13. The real question: how fresh is fresh enough?

Consider three systems. On a blog, if a reader sees a just-published article's old version for 30 seconds, that's almost certainly fine. On an e-commerce site, a customer seeing a stale price for 30 seconds might be acceptable, or might be a real business problem, depending on context. On a financial system, a user seeing yesterday's account balance is unacceptable for most operations.

The same caching technique cannot be blindly applied everywhere. This is the question every strategy in this post is really answering, just from a different angle.

14. A practical starting point

For many backend systems, a reasonable default looks like this:

WRITE                          READ
  ↓                              ↓
Database                      Cache
  ↓                          /      \
Invalidate cache          HIT        MISS
  ↓                         ↓          ↓
Set a TTL too            Return    Database
                                       ↓
                                 Populate cache

Fast reads, relatively simple invalidation, TTL as a safety mechanism, and the database as the one source of truth. It isn't perfect — but it's understandable, and understandability is a real production advantage.

15. Six rules worth keeping in mind

Know your source of truth. Usually the database is authoritative and the cache is a temporary copy — don't accidentally let the cache become the source of truth unless your architecture explicitly calls for it.
Define acceptable staleness deliberately. Don't pick a TTL at random; ask how long this specific data can safely be wrong.
Prefer simple invalidation when it's enough. UPDATE DB, DELETE cache beats a complicated synchronization mechanism more often than it seems like it should.
Have a safety net. TTL keeps protecting you even when explicit invalidation fails.
Think about failures explicitly. What happens if the database succeeds but the cache invalidation fails? What happens if the cache is unavailable altogether?
Don't create unnecessary copies. Every additional caching layer is another invalidation problem, not just another performance win.

The bigger lesson

Caching is usually introduced as a performance optimization. But the moment you start caching mutable data, you're actually managing multiple copies of reality — which means thinking about performance, freshness, consistency, failure handling, and operational complexity all at once.

The fastest cache isn't necessarily the best one. A cache that's always perfectly fresh may be too expensive to run. A cache that's extremely cheap may serve data that's too stale to trust. The engineering work is finding the right balance for each piece of data — not once, but deliberately, the way we did with the product example back in Part 4.

What's next

We've now covered why caching exists, how it works, where it lives, the patterns for reading and writing through it, and how to keep it from lying to you.

Next, we'll get more concrete. There are two names you're almost guaranteed to run into when working with distributed caching: Redis and Memcached. But the interesting question was never "which one is better?" — it's "what kind of caching problem are we actually trying to solve?"

Has a cache invalidation bug ever made it to production on something you built? The delete-vs-update-order kind of bug is usually invisible until exactly the wrong moment — curious what triggered yours.

Cache Patterns Every Engineer Should Know

Muralidharan Lakshmanan — Sat, 15 Aug 2026 22:33:53 +0000

In the previous parts of this series, we answered two questions: why do we need caching, and where should the cache live?

Now we need a more practical one:

How should the application actually interact with the cache?

This is where caching patterns come in. A cache is not just a box where we put data. We need a strategy for reading data, handling misses, writing data, updating cached values, and dealing with staleness. Different patterns solve these problems differently. Let's look at the most common ones.

1. Cache-Aside — the most common pattern

Let's start with Cache-Aside, also called lazy loading. The basic idea: the application is responsible for checking the cache and loading data on a miss.

GET /products/123

Application → Cache → HIT → Product. Done.

Application → Cache → MISS
                        ↓
                    Database
                        ↓
                  Store in cache
                        ↓
                    Response

Product product = cache.get("product:123");

if (product == null) {
    product = database.findProduct(123);
    cache.put("product:123", product);
}

return product;

The next request can be served from the cache.

Cache-Aside is popular because it's simple. The application explicitly decides what to cache, when to cache it, what TTL to use, what to do on a miss, and when to invalidate it. It also works with almost any cache technology.

The downside is that the application now carries caching logic. Developers need to remember the check-fallback-populate sequence, and if several services do this independently, that logic tends to get duplicated. Still, Cache-Aside is often the best starting point.

2. Read-Through — let the cache do the loading

Read-Through moves that responsibility away from the application. Instead of the application saying "if the cache misses, I'll query the database," it simply says "give me the data." The cache handles the miss itself.

The application code gets simpler:

product = cache.get("product:123");

If the cache doesn't have it, the cache layer knows how to retrieve it. The application doesn't need to know as much about the underlying data source, which can produce cleaner application code.

The catch is that not every cache supports this natively — you need a caching layer or framework that knows how to load the missing data, which means more abstraction and configuration. Read-Through can be elegant, but Cache-Aside is usually easier to understand and implement directly.

3. Write-Through — write to the cache and database together

So far we've mostly talked about reads. But what happens when data changes? Suppose a customer updates their address. Now the database and the cache both need to reflect the new value.

With Write-Through, a write goes through the cache, and the cache updates the underlying data store as part of the same operation:

Update customer address
          ↓
       Cache
          ↓
      Database

The main benefit is freshness — the cache is updated as part of the write path, so the system reduces the chance of serving an old value. The cost is that writes become more expensive: instead of Application → Database, a write now involves Application → Cache → Database. We're trading write performance and complexity for better cache freshness.

4. Write-Behind — make writes fast

Now the opposite approach. What if writes are extremely frequent — say, 50,000 updates per second? Writing every one immediately to the database may be expensive.

With Write-Behind, the cache accepts the update first and the write returns immediately. The database catches up later, asynchronously.

This can make writes extremely fast. But there's a serious trade-off: what happens if the cache crashes before the update reaches the database? Potentially, data loss.

That's why Write-Behind shouldn't be treated as simply "a faster Write-Through." It's a fundamentally different consistency and durability model. It works best when eventual persistence is acceptable, the cache has reliable durability mechanisms of its own, updates can be replayed or recovered, and extreme write performance genuinely matters.

5. Refresh-Ahead — don't wait for the cache to expire

Here's another problem. Suppose product:123 is cached with a TTL of 10 minutes, and thousands of users are requesting it. At minute 9, every request is a hit. At minute 10, the cache expires — and thousands of requests can arrive at the database at the same instant.

We touched on this earlier in the series. It's commonly called a cache stampede or thundering herd.

Refresh-Ahead tries to avoid it. Instead of waiting for the value to expire, the system refreshes it shortly beforehand, while the existing cached value keeps serving requests in the meantime.

Imagine a flight-search application where a popular route is requested thousands of times. Instead of letting the cache expire completely, the system refreshes the data once it has, say, 30 seconds left. The next request doesn't have to wait for a database call — it just gets served from an already-fresh cache.

6. Comparing the patterns

Pattern	Who handles the miss?	How writes work	Main benefit
Cache-Aside	Application	Application manages writes	Simple and flexible
Read-Through	Cache	Depends on implementation	Cleaner application code
Write-Through	Cache layer	Cache and database together	Better freshness
Write-Behind	Cache	Database updated later	Very fast writes
Refresh-Ahead	Cache / background process	Usually a normal write strategy	Reduces cold misses

There is no universally "best" pattern. The workload determines the answer.

7. A real-world example

Let's imagine an e-commerce product with a name, description, price, inventory count, and reviews. Should all of it use the same caching pattern? Probably not.

The description changes rarely, so plain Cache-Aside with a one-hour TTL is fine. The price changes more often, so we pair Cache-Aside with explicit invalidation the moment it changes. Inventory is more sensitive — a stale number could let a customer buy something that's no longer available, so a much shorter TTL, or skipping the cache in some parts of the workflow, makes more sense. Recommendations can be expensive to calculate, which makes them a good candidate for Cache-Aside plus Refresh-Ahead.

That's an important lesson: don't choose one caching pattern for your entire system. Choose the pattern based on the behavior of the data.

8. The hidden complexity: cache invalidation

Suppose the database has product:123 at $899, but the cache still has $999. A caching pattern doesn't automatically solve this — you still need to decide what happens to the cache when the database changes.

The common answers are to delete the entry and let the next request reload it, update the cached value directly, let the TTL expire naturally, or invalidate the cache in response to a published event when the database changes.

This is a big enough topic that the next part of this series is devoted entirely to it.

9. Which pattern should you start with?

If you're designing a new application and aren't sure what to use, don't reach for the most sophisticated pattern first. For many read-heavy applications, plain Cache-Aside in front of the database is an excellent starting point.

Then measure. Is the hit ratio good? Is database load actually reduced? Are misses expensive? Are hot keys causing problems? Is stale data acceptable? Are writes becoming a bottleneck? Only once you have answers should you introduce something more sophisticated.

10. The engineer's mental model

Here's a short way to hold all five patterns in your head at once:

Cache-Aside — "I'll manage the cache myself."
Read-Through — "The cache will load missing data for me."
Write-Through — "When I write, update the cache and the source together."
Write-Behind — "I'll make the cache the fast write point and persist later."
Refresh-Ahead — "Don't let popular data go cold if I can refresh it proactively."

Once these five ideas are second nature, most caching architectures become much easier to reason about.

What's next

We've now covered why caching exists, how it actually works, where it should live, and the patterns that govern how an application talks to it.

But there is one caching problem that almost every engineer eventually runs into:

How do you make cached data disappear when the real data changes?

That's cache invalidation — often called the hardest problem in caching — and it's next.

Which pattern is running in your production system right now, and did your team choose it deliberately or inherit it? Curious to hear in the comments.