This is Part 6 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear.
Let's pick up exactly where we left off.
In Part 5, we solved the server bottleneck by adding more application servers behind a load balancer. Traffic that used to crush a single server now gets spread across many. On paper, that feels like a win.
But we ended on an uncomfortable observation.
No matter how many application servers we add, they all talk to the same database. The servers stopped being the problem. The database didn't go anywhere.
And as more servers send more requests, all of them still end up in the same place the database's doorstep.
So we asked a question and left it hanging:
If thousands of users request the same information repeatedly, do we really need to ask the database every single time?
Today, we answer that question properly. Not by naming a technology. By actually thinking it through.
--
Section 1: The Question We Left With
Let's make this concrete, because "database bottleneck" is still a vague phrase until you see it happen.
Imagine you're running a news website. It's a slow Tuesday morning, nothing unusual, until one article about a major event starts trending. Within the next five minutes, fifty thousand people open your homepage.
Every single one of them is asking your application the same question:
"What are the top 10 trending articles right now?"
Your application, being obedient, does exactly what it was built to do. It takes that question and forwards it to the database.
User 1 → App Server → Database: "Give me top 10 trending articles"
User 2 → App Server → Database: "Give me top 10 trending articles"
User 3 → App Server → Database: "Give me top 10 trending articles"
...
User 50,000 → App Server → Database: "Give me top 10 trending articles"
Fifty thousand separate trips to the database. Fifty thousand times the database has to scan through rows, sort by trending score, and assemble a result.
Now here's the part that should genuinely bother you.
The answer was identical every single time.
Nothing changed between request #1 and request #50,000. The database wasn't computing fifty thousand different answers. It was doing the exact same work, fifty thousand times in a row, to produce the exact same result.
If a coworker asked you the same question fifty thousand times in five minutes, you wouldn't recalculate the answer from scratch each time. You'd remember what you said the first time, and just repeat it. Your database doesn't get that luxury , it has no concept of "I already answered this." Every query looks brand new to it, even if it's identical to the one from half a second ago.
So the problem was never that the database is slow, or poorly designed. The problem is that we keep making it repeat work it has already done.
--
Section 2: Do We Really Need To Ask Again?
Let's slow down and really sit with this, because the instinct to "just ask the database" is so automatic that it's worth questioning directly.
Why does the application ask the database every time?
Because that's the default behavior we build into applications. A request comes in, the application has no memory of anything, so it goes and fetches the data fresh. It doesn't matter if the same data was fetched one second ago. The application doesn't remember that. It just knows how to ask.
But think about what "asking the database" actually costs.
The database has to:
- Receive the query over the network
- Parse and plan how to execute it
- Read data from disk or its own memory
- Filter, sort, or join tables if needed
- Package the result and send it back over the network Every one of those steps takes time. Even a "fast" query that takes 20 milliseconds is still 20 milliseconds of real work , CPU cycles, disk reads, network round trips.
Now multiply that by 50,000 identical requests. That's 50,000 times the CPU work, 50,000 times the disk activity, 50,000 times the network traffic , for a result that never changed.
This is the moment where the solution should start to feel obvious, even before we name it. If the answer doesn't change, and we already computed it once, why compute it again?
Right now, every request takes the same path, no matter how many times it's been taken before:
Without a cache:
User → Application → Database → Response
Every single request, without exception, walks all the way down to the database. There's no shortcut, no memory of what was already asked.
Now picture inserting one extra stop along that path , something that can answer instantly if it already knows the answer, and only lets the request continue to the database if it doesn't:
With a cache:
User → Application → Cache → Database (only if needed) → Response
The database's job is to be the source of truth. It should be trusted to hold the correct data. But that doesn't mean it has to be asked every single time someone wants to read that data. What if, somewhere between the user and the database, we kept a copy of recent answers? A copy that's fast to check, and that we hand out instead of bothering the database again?
That idea — keeping a copy of an answer so you don't have to redo the work to get it again — is the entire foundation of caching. And it's worth being precise about what's actually being saved here. Caching isn't really about storing data. It's about avoiding expensive work. The data is just the byproduct of that work , a database scan, a sort, a computation. The cache exists so we never have to pay for that work twice when the answer hasn't changed.
--
Section 3: Meet The Cache
Let's define it the way it actually emerges, not the way a textbook would state it upfront.
A cache is a place where we store the result of an expensive operation, so that the next time someone asks for the same thing, we can hand them the stored result instantly , without redoing the expensive operation. The point was never the storage itself. The point is dodging the work that produced it.
That's it. That's the whole idea. Everything you'll learn about caching in the coming weeks - Redis, TTLs, invalidation strategies, cache layers is just refinement on top of this one sentence.
Let's walk through how this changes our trending-articles example.
The first user opens the homepage. The application has never answered this question before, so it goes to the database, gets the top 10 trending articles, and sends them back to the user.
But this time, before responding, the application does one more thing. It takes that result and stores it somewhere fast to access — let's just call it "the cache" for now with a label like trending_articles.
User 1 → App Server → Database: "Give me top 10 trending articles"
← Database returns result
App Server → Cache: "Save this as trending_articles"
App Server → User 1: here's your result
Now user 2 shows up, asking for the same thing. This time, the application does something different. Before going anywhere near the database, it checks the cache first.
User 2 → App Server → Cache: "Do you have trending_articles?"
← Cache: "Yes, here it is."
App Server → User 2: here's your result
No database involved. No query planning, no disk reads, no sorting. Just a quick lookup in a place designed to answer fast.
User 3, user 4, all the way to user 50,000 , same story. As long as the cached result is still considered valid, they all get served from the cache. The database, which used to handle 50,000 identical queries, now handles just one.
That single shift checking a fast, temporary storage location before going to the slow, authoritative one — is what a cache does. It sits between your application and your database (or any expensive computation, really) and intercepts repeat requests before they become repeat work.
--
Section 4: Cache Hit vs Cache Miss
Once you accept the basic idea, two very natural situations start to matter, and it's worth naming them clearly because you'll see these terms everywhere from now on.
Cache hit: the application checks the cache, and the data it's looking for is already there. It hands that data back immediately, without touching the database.
Cache miss: the application checks the cache, and the data isn't there. Maybe nobody has asked for it yet, or maybe it was stored earlier but has since been removed. In this case, the application has no shortcut. It has to go to the database, get the answer the normal way, and importantly store it in the cache before responding, so that the next request becomes a hit.
Let's trace through both cases with a slightly different example: user profile pages.
Imagine a user named Aisha visits her own profile page for the first time today.
Request: GET /profile/aisha
App Server checks cache for key "profile:aisha"
Cache: "I don't have that." → Cache Miss
App Server → Database: "Get profile data for aisha"
Database → App Server: returns profile data
App Server → Cache: "Store this as profile:aisha"
App Server → User: here's the profile
That was a miss. Slower path, database involved, but the cache is now "warmed up" with Aisha's data.
Now, thirty seconds later, Aisha refreshes the page, or a friend visits her public profile.
Request: GET /profile/aisha
App Server checks cache for key "profile:aisha"
Cache: "Yes, here it is." → Cache Hit
App Server → User: here's the profile
That was a hit. No database involved at all. Just a lookup.
This is the pattern that repeats constantly in real systems: the first request for something is always a miss, because nothing has been cached yet. But every subsequent request for that same thing as long as it stays in the cache becomes a hit, and hits are dramatically cheaper than misses.
This is also why caching helps most for data that's read far more often than it changes. Trending articles, product listings, user profiles, popular search results these are all things that thousands of people might request in a short window, while the underlying data itself barely changes minute to minute. That's exactly the kind of workload where caching turns a database from "constantly overwhelmed" into "occasionally consulted."
--
Section 5: Why Is Cache So Fast?
At this point you might be wondering something completely reasonable: if a cache is just "storage" and a database is also "storage," why is one so much faster than the other?
The answer comes down to where the data physically lives.
A traditional database, especially for data that doesn't all fit comfortably in memory, keeps a lot of its data on disk. Even fast solid-state disks are still slower than the alternative we're about to talk about, because reading from disk involves physically locating and retrieving data through a storage controller.
A cache, on the other hand, almost always stores its data in RAM the computer's main memory.
Here's the difference that matters: RAM is designed for extremely fast access by the CPU, while persistent storage is optimized for keeping data safely over time, not for handing it back instantly. That's why reading from RAM is significantly faster than reading from disk nanoseconds instead of fractions of a millisecond. That might sound like a small difference on paper, but at the scale of thousands of requests per second, it's the difference between a database that's gasping for breath and one that barely notices the traffic.
There's a rough mental model worth keeping:
CPU register access → fastest (fractions of a nanosecond)
RAM access → very fast (nanoseconds)
SSD disk access → fast, but much slower than RAM
Network round trip → slower still
Traditional spinning disk → slowest of the common options
A cache deliberately trades one thing for the speed it gains: RAM is more expensive per gigabyte than disk, and it's volatile if the machine loses power, whatever's in RAM disappears. That's precisely why we don't use a cache as our source of truth. The database still holds the real, permanent data on disk, safely persisted. The cache just holds a temporary, disposable copy of the most frequently requested answers, sitting in memory where it can be handed out almost instantly.
This is also why a cache being "wrong" is a survivable problem. If the cache is lost a server restarts, the cache is cleared, whatever — nothing is actually lost. The application just experiences a wave of cache misses, falls back to the database, and starts rebuilding the cache from scratch. The database is still there as the safety net.
That single property cache is fast but temporary, database is slower but permanent is the core tradeoff that everything else in caching design is built around.
--
Section 6: Redis — The Most Popular Cache
So far we've talked about "the cache" as a concept, without naming any specific technology. That was deliberate. The idea needed to make sense on its own before attaching a name to it.
In practice, when engineers build this "fast, in-memory storage layer sitting in front of the database" idea, one of the most popular tools they reach for is Redis.
Redis is, at its core, an in-memory data store. It keeps data in RAM, which as we just established is exactly what makes it fast enough to serve as a cache. It's typically used as a separate service that your application servers talk to, sitting between them and the database.
Here's roughly how our trending-articles flow looks with Redis in the picture:
┌───────────────┐
Request ───► │ App Server │
└──────┬────────┘
│
check cache first
│
┌──────▼────────┐
│ Redis │ ← fast, in-memory
└──────┬────────┘
│ (only on a miss)
┌──────▼────────┐
│ Database │ ← slower, on disk
└───────────────┘
Redis supports simple commands that map naturally to what we've been describing storing a value under a key, and retrieving a value by that key. Something conceptually like:
SET trending_articles "[...serialized list of articles...]"
GET trending_articles
That SET is what happens on a cache miss, right after the application gets fresh data from the database. That GET is what happens on every request afterward, checking whether a cached answer already exists.
It's worth being honest here: Redis isn't the only way to build a cache, and it does a lot more than just basic caching once you dig into it. But for a beginner building a mental model of System Design, the important thing isn't memorizing Redis's full feature set. It's understanding that Redis is simply one popular, well-built tool for doing the thing we just spent this entire article reasoning our way toward — storing answers in fast memory so you don't have to keep recomputing them.
Once you understand why caching exists, learning any specific caching tool becomes a matter of syntax, not concept.
--
Conclusion
Let's retrace the path we took today.
We started with a database drowning under repeated, identical queries. We asked whether it made sense to keep asking the same question over and over, and realized it didn't the answer wasn't changing, so the work was wasted.
That led us to the idea of a cache: a fast, temporary storage layer that holds onto previously computed answers so future requests can be served instantly, without touching the database at all.
We learned to tell a cache hit from a cache miss, and saw why the very first request for something is always a miss, while everything after tends to become a hit as long as the cached data is still there.
We looked at why caches are fast in the first place: RAM instead of disk, nanoseconds instead of milliseconds, at the cost of being temporary rather than permanent.
And finally, we named Redis as one of the most common tools engineers reach for to build exactly this kind of caching layer in real systems.
But notice something we conveniently avoided this entire article.
We stored trending_articles in the cache. We stored profile:aisha in the cache. But what happens when Aisha updates her profile picture five minutes later? The cache still confidently hands out the old profile data, because as far as it knows, nothing changed. It has no idea the underlying data in the database was just updated.
The cache doesn't automatically know when the real data changes. It just keeps serving whatever it was told to remember until something tells it otherwise.
So the question we're left with is this:
How does a cache know when the data it's holding is no longer correct?
That question is exactly where Part 7 begins.
Top comments (0)