If you have worked on software systems for a while, you have probably heard this advice:
"Just add a cache."
It sounds simple.
- Your application is slow? Add a cache.
- Your database is getting too many requests? Add a cache.
- Your API needs to handle more users? Add a cache.
And surprisingly often, it works.
But caching is one of those things that looks incredibly simple when you first learn it — and becomes much more interesting once you use it in a real production system.
Because the moment you add a cache, you are no longer dealing with just one question: "How can I make this faster?"
You also have to think about what to cache, how long to keep it, what happens when the data changes, what happens when the cache is unavailable, how much memory it will use, and what happens when thousands of users request the same uncached data.
That's when caching stops being simple.
What problem does caching actually solve?
Imagine you have an application that needs to display a user's profile.
User → Application → Database → Application → User
The application receives the request, queries the database, gets the data, and sends it back.
GET /users/123
SELECT * FROM users WHERE id = 123;
Nothing is wrong with this. But what happens if 10,000 requests ask for the same information?
You could end up executing thousands of database queries for data that hasn't changed.
That's where caching comes in. Instead of asking the database every time, we keep frequently requested data in a faster storage layer.
User → Application → Cache → Database (only when necessary)
If the data is already in the cache, we don't need to contact the database. That is the basic idea.
A real-world analogy
Think about a restaurant. Imagine you are a waiter and customers frequently ask for the restaurant's menu.
Approach 1: Go to the kitchen every time.
Every time someone asks, "Can I see the menu?", you walk to the kitchen, find the menu, bring it back, and hand it over. Now imagine doing that 500 times. It would be ridiculous.
Approach 2: Keep the menu at the table.
Instead, you keep a copy of the menu on every table. When someone asks for it, you simply point to the menu already sitting there. Much faster.
| In the restaurant | In your system |
|---|---|
| The customer | The user |
| The waiter | The application |
| The kitchen | The database |
| The menu on the table | The cache |
We keep frequently needed information closer to where it is needed.
Latency: why does caching make things faster?
One of the biggest reasons to use caching is latency — the time it takes for something to respond.
| Operation | Example latency |
|---|---|
| Cache request | ~2 ms |
| Database request | ~80 ms |
The numbers above are illustrative, not universal benchmarks. The important point is that an in-memory cache can often respond much faster than a database path that involves network communication, query processing, indexes, joins, disk I/O, connection management, and other work.
When you multiply that difference by thousands or millions of requests, the impact becomes significant.
Caching also reduces database load
Speed is only part of the story. Caching can also protect your database.
Without caching: 100,000 requests → potentially 100,000 database queries
With caching: 100,000 requests → cache → perhaps only a handful of queries
Imagine an e-commerce site where a popular product is requested thousands of times. Product name, description, specifications, and other relatively stable information may be good candidates for caching.
The database gets fewer requests. The application gets faster responses. The system can potentially handle more traffic.
Cache hit vs. cache miss
Two terms appear everywhere when talking about caching: cache hit and cache miss.
Cache hit
A cache hit happens when the data you are looking for is already in the cache.
Application → Cache → "Found it!" → Return data
The database is not needed for that request.
Cache miss
A cache miss happens when the requested data is not in the cache.
Application → Cache → "Not found" → Database → Cache → Return data
The first request is slower, but the result can be stored in the cache so future requests can be faster.
This is the cache-aside pattern, and it's worth noticing who is in charge: the application checks the cache, the application falls back to the database, and the application writes the result back. The cache itself knows nothing about your database.
So why not cache everything?
This is where things get interesting. If caching makes things faster, why not simply cache everything?
Because caching has a cost. A cache requires memory, infrastructure, monitoring, maintenance, and additional application logic. And perhaps the biggest problem is stale data.
Imagine you cache this product:
{ "product": "Laptop", "price": "$999" }
The cache keeps it for 30 minutes. Five minutes later, the real price changes from $999 to $899. The cache still contains $999.
Now the application may show the user the wrong price.
This is the classic caching problem: how do you keep cached data reasonably fresh?
Possible approaches include short expiration times, explicit invalidation, event-driven updates, versioned cache keys, and other strategies. Each one introduces additional design decisions.
Caching can actually make a system worse
Caching isn't automatically good. A poorly designed cache can create new problems.
For example, imagine your application caches millions of objects without considering memory usage:
Cache memory → 100% → Evictions → Cache misses → More database requests
Or imagine the cache suddenly becomes unavailable:
Normal: 10,000 req/sec → Cache → Database sees a manageable load
Failure: 10,000 req/sec → Database
If the database is suddenly exposed to traffic that the cache normally absorbs, the database can become the next bottleneck.
Other problems include cache stampede, hot keys, invalidation issues, inconsistent data, memory pressure, network latency, cache availability, and serialization/deserialization overhead.
We'll explore these problems later in the series.
The important question isn't "should I use caching?"
A better question is:
"What should I cache, why should I cache it, and what happens when the cache is wrong or unavailable?"
That shift in thinking is important.
Often good candidates:
- Frequently requested data
- Relatively stable data
- Expensive database queries
- Expensive computations
- External API responses
- Configuration or reference data
Often poor candidates:
- Data that changes constantly
- Data that requires strict real-time consistency
That second list isn't a hard "never" — it may just require a more carefully designed caching strategy. There is no universal answer.
Caching is a trade-off
| Performance | ↔ Complexity |
| Freshness | ↔ Speed |
| Memory cost | ↔ Database cost |
| Availability | ↔ Consistency |
That is the real story of caching. A cache can make a system faster and reduce database pressure, but it also introduces another component and another set of failure modes.
And that's why caching is simple... until it isn't.
What's next
Next in this series, we'll go one level deeper and look at how caching actually works: where cached data lives, memory vs. disk, TTL (time to live), cache keys, eviction strategies such as LRU and LFU, and what happens when a cache runs out of memory.
Once you understand those fundamentals, technologies such as Redis and Memcached become much easier to understand.
Because before learning which caching technology to use, it's worth understanding what problem we're actually trying to solve.
What's the worst caching bug you've shipped? I'm collecting war stories for a later post in this series — drop yours in the comments.
Top comments (0)