DEV Community: Ammar Eyad

Common Cache Production

Ammar Eyad — Wed, 09 Sep 2026 07:07:54 +0000

Cache Stampede, Hot Keys, and Cache Penetration

Caching is one of the most common ways to improve application performance and reduce database load.

But as traffic grows, caching introduces its own problems.

Three common ones are:

Cache Stampede
Hot Keys
Cache Penetration

Let's understand each one with simple examples.

1. Cache Stampede

A cache stampede (also called the thundering herd or dogpiling problem) happens when a popular cached value expires and many requests try to fetch the same data from the database at the same time.

Instead of protecting the database, the cache suddenly becomes the reason the database gets overloaded.

When and Why Does It Happen?

Normally, we put a cache such as Redis in front of our database:

Client
   ↓
Application
   ↓
  Cache
   ↓
 Database

Cached data usually has a Time To Live (TTL) so that it doesn't stay stale forever.

A cache stampede can happen when:

1. High Concurrency

Thousands of users request the same piece of data.

For example:

GET homepage
GET homepage
GET homepage
GET homepage
...

2. Simultaneous Expiration

The cached value reaches its TTL and expires.

Cache
product:123 → EXPIRED

3. Race Condition

The first request goes to the database to retrieve the data.

But before that request finishes and updates the cache, thousands of other requests also see a cache miss.

Request 1 ──→ Cache MISS ──→ Database
Request 2 ──→ Cache MISS ──→ Database
Request 3 ──→ Cache MISS ──→ Database
Request 4 ──→ Cache MISS ──→ Database
...
Request 10000 ─→ Cache MISS ──→ Database

Now thousands of requests are doing the same database query.

This can overload the database and potentially cause it to slow down or crash.

How Do We Solve a Cache Stampede?

The main goal is simple:

Don't allow thousands of requests to perform the same database work simultaneously.

There are several approaches.

1. Locking

When the cache expires, the first request acquires a lock.

Other requests wait briefly instead of querying the database.

Request 1 → Cache MISS → Acquire Lock → Database
Request 2 → Cache MISS → Wait
Request 3 → Cache MISS → Wait
Request 4 → Cache MISS → Wait

The first request retrieves the data and updates the cache:

Database
   ↓
Update Cache
   ↓
Release Lock

The waiting requests can then read the newly populated cache.

2. Stale-While-Revalidate

Instead of making users wait for fresh data, we temporarily serve the existing stale value.

At the same time, a background process refreshes the cache.

User Request
     ↓
Stale Cache
     ↓
Return Immediately

Background Process
     ↓
Database
     ↓
Update Cache

This works well when serving slightly outdated data is acceptable.

3. External Pre-Warming

Another approach is to refresh the cache before it expires.

For example, a scheduled job can periodically load popular data:

Cron Job
   ↓
Database
   ↓
Update Cache

This means user traffic doesn't have to be the thing that triggers the cache refresh.

2. Hot Keys

A hot key happens when a single cache key receives a very large number of requests.

For example:

product:123

Imagine we have:

10 application servers
100,000 user requests
One Redis server

The load balancer distributes requests across the application servers:

                Load Balancer
                 /    |    \
                /     |     \
            App 1   App 2   App 3 ... App 10
               \       |       /
                \      |      /
                    Redis

The application servers can scale horizontally.

But if all requests need the same Redis key:

product:123

then all application servers send requests to Redis.

App 1 ──┐
App 2 ──┤
App 3 ──┤
App 4 ──┤
...     ├──→ Redis
App 10 ─┘

The problem is that one Redis node is receiving a huge amount of traffic for the same key.

The Redis CPU or network capacity can become the bottleneck.

Solution 1: L1 / L2 Cache

One of the simplest solutions is to introduce a local cache.

We can think of the caches as:

L1 → Application Server Local Memory
L2 → Redis

The application checks L1 first.

If the data isn't there, it checks Redis.

Request
   ↓
L1 Cache
   ↓
 Cache Hit? ── Yes → Return
   │
   No
   ↓
Redis
   ↓
Store in L1
   ↓
Return

Without L1 Cache

Suppose we have:

10 application servers
100,000 requests

Without L1:

100,000 requests
       ↓
100,000 Redis calls
       ↓
Redis becomes the bottleneck

With L1 Cache

Each application server can keep the hot data in local memory for a short period, for example 5 seconds.

Initially:

App 1 → Redis
App 2 → Redis
App 3 → Redis
...
App 10 → Redis

After that, each server has the value locally.

The remaining requests can be served directly from memory:

100,000 requests
       ↓
   L1 Cache
       ↓
Most requests served locally

So instead of sending every request to Redis, each application server can serve the majority of requests from its own memory.

Why Is L1 So Effective?

1. No Network Overhead

Reading from local memory is much faster than making a network call to Redis.

Local Memory → Very fast
Redis        → Network round trip

2. Application Servers Scale Horizontally

You can add more application servers behind a load balancer.

This helps distribute user traffic across multiple machines.

A single hot Redis key, however, can still concentrate traffic on one Redis node.

Solution 2: Key Splitting

Another approach is to split a hot key into multiple keys.

In Redis Cluster, a single key is mapped to a single shard.

So instead of:

product:123

we can create multiple copies:

product:123:1
product:123:2
product:123:3
...
product:123:100

When a request arrives, the application chooses one of the keys.

For example:

Request 1 → product:123:17
Request 2 → product:123:42
Request 3 → product:123:8
Request 4 → product:123:91

Now the traffic can be distributed across different Redis cluster nodes.

The Idea

Instead of:

100,000 requests
       ↓
product:123
       ↓
One Redis shard

we can have:

100,000 requests
       ↓
 ┌─────┼─────┐
 ↓     ↓     ↓
Key 1 Key 2 Key 3 ... Key 100

This spreads the read traffic instead of hammering a single Redis key.

Solution 3: Read Replicas

If you're using a Redis primary-replica architecture, another option is to add read replicas.

The basic idea is:

              Primary
             /   |   \
            /    |    \
       Replica Replica Replica

Writes go to the primary:

Application
     ↓
 Primary

Reads can be distributed across replicas:

Application
     ↓
 ┌───┼────────┐
 ↓   ↓        ↓
R1   R2       R3

This allows the system to distribute a large number of read requests across multiple Redis nodes.

3. Cache Penetration

Cache penetration happens when requests repeatedly ask for data that doesn't exist.

For example:

GET /users/999999

If the user doesn't exist:

Request
   ↓
Redis
   ↓
Cache MISS
   ↓
PostgreSQL
   ↓
User NOT FOUND

The problem becomes serious when thousands of requests are made for invalid IDs:

/user/999999
/user/837462
/user/123456
/user/555555
...

Every request misses the cache and reaches the database.

Request 1 → Redis MISS → PostgreSQL → NOT FOUND
Request 2 → Redis MISS → PostgreSQL → NOT FOUND
Request 3 → Redis MISS → PostgreSQL → NOT FOUND
...
Request 10000 → Redis MISS → PostgreSQL → NOT FOUND

Unlike a cache stampede, the problem here isn't that valid cached data expired.

The problem is that the requested data doesn't exist in the first place.

Solution: Bloom Filter

A Bloom Filter is a memory-efficient data structure that can quickly tell us whether an item might exist in a dataset.

The application can check the Bloom Filter before querying Redis or PostgreSQL.

Request
   ↓
Bloom Filter
   ↓
Could this ID exist?
   /          \
 No            Yes
 ↓              ↓
Return        Redis
Not Found       ↓
             PostgreSQL

For example, suppose we have these users:

The Bloom Filter knows about these IDs.

Now a request comes for:

The Bloom Filter can say:

999999 → Definitely does not exist

So we can stop the request immediately without hitting Redis or PostgreSQL.

Important Bloom Filter Property

A Bloom Filter can have false positives, but it does not have false negatives.

In simple terms:

Bloom Filter → "Definitely doesn't exist"
        ↓
     Stop here

Bloom Filter → "Might exist"
        ↓
   Check Redis
        ↓
   Check Database

So if the Bloom Filter says an ID doesn't exist, we can safely reject the request.

If it says the ID might exist, we continue normally.

Why Is This Useful?

Imagine:

1,000,000 invalid requests

Without a Bloom Filter:

1,000,000 requests
        ↓
      Redis
        ↓
  PostgreSQL

With a Bloom Filter:

1,000,000 requests
        ↓
   Bloom Filter
        ↓
Most invalid requests stopped here
        ↓
Only possible valid requests
        ↓
      Redis
        ↓
   PostgreSQL

This can significantly reduce unnecessary traffic to Redis and PostgreSQL.

Bloom Filter is especially useful when the database contains a large number of IDs and the system receives many requests for IDs that don't exist.

Cache Stampede vs Hot Key vs Cache Penetration

These three problems are related, but they are not the same.

Problem	Main Issue	Typical Solutions
Cache Stampede	Many requests hit the database after cache expiration	Locking, stale-while-revalidate, pre-warming
Hot Key	One cache key receives too many requests	L1 cache, key splitting, read replicas
Cache Penetration	Requests repeatedly ask for data that doesn't exist	Bloom Filter, negative caching, validation

Simple Way to Remember

Cache Stampede:

The cache expires and everyone goes to the database.

Hot Key:

Everyone keeps asking Redis for the same key.

Cache Penetration:

Everyone keeps asking for data that doesn't exist.

Understanding the difference helps you choose the right solution when designing a high-scale system.

Why You Shouldn’t Store Large Files in Your Database: Database vs. Amazon S3

Ammar Eyad — Mon, 07 Sep 2026 05:05:15 +0000

Storing files in a database instead of S3 introduces two massive architectural bottlenecks as your application scales:

1. CDN becomes much harder to use

One of the biggest advantages of S3 is how easily it works with a CDN such as Amazon CloudFront or Cloudflare.

Imagine you have millions of users requesting the same images.

With S3 and a CDN, the CDN can cache those images at locations around the world. Users can get the file from a nearby edge location instead of requesting it from your application every time.

This is exactly what we want for things like:

Images
Videos
PDFs
JavaScript
CSS
Other static content

Now imagine the image is stored as a BLOB inside your database.

If your images are locked in a database, your CDN is effectively blind to them. Every single image request must hit your web server, which must then execute a heavy database query to fetch the binary data, stream it back to the web server, and then serve it to the client. You are forcing your most expensive piece of infrastructure (the database) to act as a dumb file server, completely bypassing the speed and cost benefits of edge caching.

2. Database replication becomes heavier

Production databases usually have replicas for high availability.

If your database contains mostly structured data, replication is relatively straightforward.

But imagine your database contains hundreds of gigabytes or even terabytes of images and videos.

Now those large binary objects also become part of your database replication and backup workload.

For example, instead of replicating:

Users
Orders
Products
Payments

you might also be replicating:

500 GB of images
1 TB of videos
200 GB of documents

That can make replication, backups, and restores significantly heavier.

The same applies when you need to restore your database.

A database backup containing huge amounts of binary data can take much longer to restore than a database containing only structured application data.

3. Your database becomes unnecessarily large

Databases are extremely good at what they are designed for.

They are great for:

Relationships
Transactions
Queries
Indexes
Structured data
Consistency

But storing large files is a different problem.

Object storage is designed specifically for storing large objects.

So instead of making the database responsible for everything, we can separate the responsibilities.

The database stores:

User
Order
Payment
File metadata
File reference

S3 stores:

Images
Videos
PDFs
Documents

Each system does what it is good at.

What about the physical storage behind S3?

Another interesting question is:

Where does S3 actually store my file?

At the end of the day, the data has to live on physical storage hardware.

But as an S3 user, you don't need to know whether your particular object is sitting on an HDD, SSD, or which physical disk contains it.

AWS manages that infrastructure for you.

You interact with S3 as object storage:

my-bucket/users/123/profile.jpg

You don't interact with:

Disk 123
Sector 456
SSD 789

That abstraction is one of the main benefits of object storage.

What does the typical architecture look like?

A common approach is:

Application
    |
    +--- Database
    |      File metadata
    |
    +--- S3
           Actual file

For uploads, we can even use a presigned URL so the client uploads the file directly to S3 instead of sending the entire file through our application server.

The database then only needs to know things like the file name, size, content type, and S3 key.

Is storing BLOBs in a database always wrong?

No.

There are cases where it can be perfectly reasonable.

For example, if the files are very small and the application is simple, storing them in the database might be easier and completely acceptable.

The important thing is to understand the trade-off.

For a large-scale system, I generally prefer this simple rule:

Database for structured data and metadata.

S3/object storage for large files.

It keeps the architecture cleaner, allows the file storage to scale independently, makes CDN integration easier, and avoids turning your primary database into a file server.