Every architecture in this series so far has had roughly the same shape: application, Redis, database. That works beautifully for a lot of real systems. But traffic grows — 1,000 requests/sec becomes 10,000, then 100,000, then over a million — and at some point the question stops being "should we use a cache?" and becomes "how do we make the cache itself scale?"
That's a genuinely different problem, and it's what this post is about.
1. Scaling changes the question
The simple picture — application, Redis, database — assumes roughly one application talking to one Redis. Once there are hundreds of application instances behind a load balancer, all of them are hitting the same Redis, and Redis itself becomes the scalability boundary the rest of the system depends on. It stops being "the fast layer" and starts being "the thing everything else is now downstream of."
2. Can one Redis node handle it?
Sometimes, yes — a single modern Redis instance handles a lot, especially with simple operations. But it runs into real limits eventually: CPU, memory, network bandwidth, connection count, command throughput, dataset size. A Redis instance sitting at 90% memory, 85% CPU, and 80% network isn't going to absorb indefinitely more traffic just because it's "just a cache." At some point the workload needs to be distributed rather than pushed harder into one place.
3. Redis Cluster and sharding
The common answer is a Redis Cluster: instead of one node taking 100% of traffic, keys get spread across several nodes. This is sharding — dividing the dataset so no single node holds all of it, and no single node absorbs all the traffic either.
The mechanism is hashing: hash(key) → slot → node. The application or Redis client can work out which node owns a given key without needing a central lookup table. The cache stops being one big bucket and becomes a distributed collection of smaller ones — which solves the capacity problem, but introduces a new one.
4. The hidden problem: hot keys again
Hashing distributes keys evenly, but it says nothing about traffic being even. If homepage:trending gets 40% of all requests, hashing still sends every one of those requests to the same single shard — the exact hot-key problem from Part 7, just relocated to a cluster instead of a single instance. One node ends up carrying a wildly disproportionate share of load while the other three sit comfortably under capacity.
5. Averages hide this
This is the metric trap worth naming explicitly: a cluster-wide average can look completely healthy while one node is already in trouble.
A 45% cluster average is genuinely meaningless if it's made up of three nodes at 30% and one node quietly sitting at 95%. That fourth node becomes the system's bottleneck long before the aggregate number would ever suggest a problem — which is why per-node metrics, not cluster averages, are what actually catch this in time.
6. Local caching helps, and complicates things
One response to hot keys: put a local, in-process cache in front of Redis, so an extremely popular value can be served from application memory without a network hop at all. It's dramatically faster — there's no network round trip, which matters more at scale than it sounds like it should.
But this reintroduces the Part 5 invalidation problem at a larger scale. If four application instances each have their own local copy of product:101 and the product changes, all four copies need to be told, not just Redis. More locality generally buys speed at the cost of more consistency complexity — the same trade-off from Part 8, just compounding as you add layers.
7. The multi-level cache, formalized
L1 in application memory, L2 in a Redis cluster, L3 the database — the goal is for most requests to resolve at L1, a smaller share at L2, and very few ever reaching the database. This is the same shape from Part 8's reference architecture, just under enough load that each layer is now load-bearing rather than optional.
8. Connection pooling becomes an architectural concern
At scale, Redis isn't only a server-side problem. If 500 application instances each open hundreds of Redis connections, that's potentially 100,000+ connections — a real operational number that needs managing through maximum connections, pooling, timeouts, idle-connection handling, and watching for leaks. A cache can be fast on paper while the application performs badly anyway, purely because of how it's managing its connections to that cache.
9. Network latency and serialization both start to matter
"Redis is in-memory, so it's basically instant" undersells the actual request path: application → network → Redis → network → application. At high request rates those hops add up, which is another reason local caching earns its place. Serialization has the same story — at 500,000 cache requests/sec, the JSON encode/decode on both ends can become a genuinely significant CPU cost. When something feels slow at scale, the bottleneck is often not where "Redis is slow" would suggest — measure application CPU, serialization CPU, network, Redis CPU, and Redis latency separately rather than assuming.
10. Replication solves a different problem than sharding
Sharding distributes data. Replication answers a different question: what happens when a node fails? A primary with a replica means a failure doesn't have to mean data loss — but replication isn't free. Writes need to propagate, which costs network traffic, memory, and operational complexity, and replicas can lag behind the primary. For a cache, some lag is often tolerable. Whether it's tolerable for your cache depends entirely on how much staleness the application can absorb — the same freshness question from Part 2 and Part 5, now with a replication delay added to the TTL.
11. Multi-region caching, and why it's not a small step
If users are spread across North America, Europe, and Asia, and Redis only lives in North America, every Asian user pays a full round trip to another continent even though the cache itself responds in milliseconds. Regional caching — a cache per region, each close to its users — fixes the latency problem. It does not simplify anything else.
Every layer from a single-region architecture now exists per region — and invalidation, which used to be one DELETE against one Redis, now has to reach every region before a change is actually reflected everywhere. Multi-region caching is a legitimate answer to a real latency problem, but it should be reached for because the latency problem is real and measured, not because global feels more impressive on an architecture diagram.
12. Consistency gets structurally harder as each layer is added
At the start, "is the cache correct" was a one-line answer: delete the Redis key after the database updates. Once there's a Redis cluster, local caches on top of it, and multiple regions on top of that, the question stops being "is the cache correct" and becomes "how quickly does a change propagate to every copy, everywhere" — a meaningfully harder problem with no single clean answer.
13. Event-driven invalidation, at this scale
Publishing an event on every data change — Product Updated fanning out to L1 consumers, Redis in Region A, Redis in Region B — keeps the producer decoupled from needing to know who's listening, the same pattern from Part 5. At this scale, the event system itself becomes something to manage: duplicate events, ordering, retries, delayed delivery, lost events, and idempotency all become real operational concerns rather than edge cases. It's trading one kind of complexity (manual multi-region invalidation) for a different, more distributed one.
14. Cache warming, and the deployment-specific version of it
For a known traffic spike — a major sale, a product launch, a sporting event — preloading the keys you already know will be hot avoids a wave of simultaneous first-time misses. There's a deployment-specific version of this worth calling out: deploying a new version means every application instance starts with an empty local cache, and if hundreds of instances all reach for Redis at once — especially if Redis itself was also just restarted — that's the exact shape of a self-inflicted stampede from Part 7. A controlled warm-up on deploy is cheap insurance against a problem you'd otherwise cause yourself, on a schedule you control.
15. The database still matters — read replicas included
"We have Redis, so the database doesn't matter" is wrong in a way that tends to surface at the worst moment: the database is still the source of truth, and every miss eventually reaches it. At high read volume, database read replicas can absorb cache-miss traffic that a single primary couldn't. But replicas introduce their own lag, so a value read from cache and then re-fetched from a replica on a miss can still be stale relative to a very recent write — performance bought with yet another consistency trade-off, the pattern this whole post keeps returning to.
16. Backpressure becomes non-optional
At a million requests/sec, an unprotected system facing a slow Redis looks like: requests wait, threads get consumed, connection pools exhaust, the application fails. The alternative is deliberate backpressure — queuing, limits, bounded concurrency — that protects Redis and the database by refusing some work rather than accepting all of it and failing everything. A high-performance system isn't one that accepts unlimited traffic; sometimes the highest-performing thing it can do is say no to some of it on purpose.
17. Capacity planning is not just "value size times key count"
10 million objects at 5KB average looks like roughly 50GB of raw data — but that's not the real Redis memory requirement. Keys, metadata, data structure overhead, replication, fragmentation, and operational headroom all add to that number, often substantially. Size a cache from measured actual memory consumption, not from multiplying the serialized value size by the key count.
18. Eviction policy is a decision about access patterns, not a default
When memory fills up, something has to go — LRU, LFU, and TTL-based policies all exist for a reason, and the right one depends on the actual access pattern rather than a general recommendation. If a small number of keys are extremely popular, LFU may outperform simple recency-based eviction for that specific workload. The point was never "always use LFU" — it's that the eviction policy should be chosen deliberately, the same principle from Part 2, now with real operational stakes attached.
19. Observability has to go deeper as scale grows
A single dashboard is plenty at 1,000 requests/sec. At a million, cluster-level visibility needs node health, CPU, memory, network, connections, commands/sec, latency, evictions, replication lag, and hot keys — and the application side needs L1 hit ratio, L2 hit ratio, misses, Redis errors, database fallback, P95/P99, and request volume. The goal is being able to trace a problem across the whole system, not just see that "something" is elevated somewhere.
20. Don't build any of this because it looks impressive
Multi-level caching, distributed Redis, replication, regional deployment, read replicas, and event-driven invalidation are each a real answer to a real problem — and together they're a lot of operational surface area. At 500 requests/sec, none of this is warranted. At 500,000+ requests/sec with strict latency requirements, the conversation genuinely changes. Architecture should follow scale, not the other way around — the same "earn the complexity" principle from Part 8, just at a size where the stakes of getting it wrong are much higher.
The biggest lesson: the bottleneck doesn't disappear, it moves
Every fix in this post follows the same pattern. Add caching, and the database bottleneck eases — but Redis is now the thing under load. Add clustering, and Redis's aggregate capacity problem eases — but now one hot node is the constraint. Add local caching to relieve that node, and the constraint moves again, this time to keeping every local copy consistent. The bottleneck was never eliminated at any step. It relocated, and the job was to notice where it went next.
That's the actual discipline behind caching at scale: not implementing every technique in this post preemptively, but continuously asking "where is the bottleneck now?" and adding exactly the piece that answers it — no more, no earlier.
What this means for the series so far
Caching at small scale can be Application → Redis → Database. At real scale it can become load balancers, application instances, local caches, a Redis cluster, regional caching, and a database layer with read replicas behind it. More infrastructure was never automatically a better system — the actual skill is knowing when each layer earns its place and why.
Caching was never really a product decision. Redis is a tool. Memcached is a tool. Local memory is a tool. A CDN is a tool. The architecture comes from understanding your workload, your consistency requirements, your failure modes, and your actual scale — not from which tool has the most features.
If you've operated a Redis cluster at real scale, what was the metric that actually caught your first hot-key incident — was it a per-node dashboard, or did it take an outage to teach you to look there?
Top comments (0)