Caching from Zero to Production

Mubashir Taj — Tue, 29 Sep 2026 13:19:41 +0000

Every backend eventually hits the same wall: the database cannot keep up, but the data barely changes. That is the moment caching stops being an optimization and becomes a requirement. This article walks through the full path, from the basic pattern to the failure modes that take down production systems, with Go examples throughout.

1. Why cache at all

It comes down to orders of magnitude. Reading from memory takes nanoseconds. Reading from disk takes milliseconds. A network round trip to the database takes longer still. When the same data is requested thousands of times per second and changes rarely, serving it from the database every time is pure waste.

The two numbers that justify a cache: request volume and data stability. High volume plus stable data means caching will pay for itself. Low volume or constantly changing data means you are adding operational complexity for little gain. Cache deliberately, not by default.

2. The cache-aside pattern

Cache-aside, also called lazy loading, is the most common pattern and the right starting point. The application owns the logic:

Check the cache for the key.
On a hit, return the value.
On a miss, read from the database, write the result into the cache with a TTL, then return it.

Here is a compact Go example using Redis:

func GetUser(ctx context.Context, rdb *redis.Client, db *sql.DB, id string) (*User, error) {
    key := "user:" + id

    // 1. Try the cache first
    if cached, err := rdb.Get(ctx, key).Bytes(); err == nil {
        var u User
        if json.Unmarshal(cached, &u) == nil {
            return &u, nil
        }
    }

    // 2. Miss: fall through to the database
    u, err := fetchUserFromDB(ctx, db, id)
    if err != nil {
        return nil, err
    }

    // 3. Populate the cache for next time (TTL: 5 minutes)
    if data, err := json.Marshal(u); err == nil {
        rdb.Set(ctx, key, data, 5*time.Minute)
    }
    return u, nil
}

The cache stays simple because it never talks to the database directly. The trade-off is the first request after every expiry pays the full database cost, and stale data is possible between the write to the database and the next cache refresh. For most read-heavy workloads, that trade-off is correct.

3. TTLs, and why fixed TTLs are dangerous

The TTL is the most consequential number in your caching layer, and teams often set it once and forget it. A fixed TTL on every key creates a subtle trap: keys written at the same time expire at the same time.

Consider a hot key absorbing 5,000 requests per second. Its TTL expires. In the next instant, all 5,000 requests fall through to the database simultaneously. CPU climbs from 40% to 100% in seconds, the connection pool saturates, and latencies spike. This is the thundering herd, also called cache stampede. The cache did not fail. The expiry pattern synchronized thousands of requests into a single attack on your own database.

The cheapest fix is jitter. Instead of a fixed 300-second TTL, use 300 seconds plus a random offset of up to 60 seconds. Expirations spread across time, and the synchronized cliff disappears:

// Jittered TTL: spreads expirations so hot keys never die together
ttl := 5*time.Minute + time.Duration(rand.Intn(60))*time.Second
rdb.Set(ctx, key, data, ttl)

One line of randomness removes an entire class of incident. This belongs in every caching layer from day one.

4. Stampede protections

Jitter handles the synchronized case, but a genuinely cold key under heavy load still funnels every request to the database. Three protections cover the remaining scenarios.

Request coalescing. When 5,000 requests need the same missing key, only one should query the database. In Go, golang.org/x/sync/singleflight does exactly this: concurrent callers for the same key share a single in-flight execution.

var group singleflight.Group

func GetProduct(ctx context.Context, id string) (*Product, error) {
    v, err, _ := group.Do("product:"+id, func() (interface{}, error) {
        return GetProductCached(ctx, id) // cache-aside logic from section 2
    })
    if err != nil {
        return nil, err
    }
    return v.(*Product), nil
}

5,000 concurrent requests become one database query. The rest wait microseconds for the shared result.

Stale-while-revalidate. Serve the slightly old value instantly and refresh it in the background. Implementation: on a miss, check for a stale copy kept under a separate key with a longer TTL. If present, return it immediately and trigger an async refresh. Users get a fast response, the database gets one query instead of thousands. This is the right choice when slightly stale data is acceptable, which covers far more endpoints than teams assume.

Probabilistic early refresh. For the highest-traffic keys, refresh the cache before the TTL expires, with probability increasing as expiry approaches. This keeps hot keys warm permanently and removes the miss path entirely for your most critical data.

Use all three together and stampedes become a non-issue rather than an incident category.

5. Eviction policies

Memory is finite, so Redis must decide what to remove when it fills up. The maxmemory-policy setting controls this, and the default (no eviction, writes start failing) surprises people.

The policies worth knowing:

allkeys-lru: evict the least recently used keys across everything. The best default for a general-purpose cache, because it automatically keeps hot data.
volatile-lru: evict least recently used, but only among keys that have a TTL. Use this when some keys must never be evicted.
allkeys-lfu: evict the least frequently used keys. Better than LRU when access patterns have long-term hot keys mixed with one-off scans that would pollute an LRU.
volatile-ttl: evict keys with the shortest remaining TTL first. A reasonable choice when TTLs genuinely reflect data priority.

Set it explicitly in your Redis configuration:

maxmemory 2gb
maxmemory-policy allkeys-lru

An unset eviction policy is a latent outage. Decide this before production, not during it.

6. Write strategies

Reads are only half the story. When data changes, the cache and the database must agree, or at least disagree briefly and predictably.

Cache-aside with invalidation (the common path): on write, update the database, then delete the cache key. The next read repopulates it. Simple and correct for most cases. The small risk is a read repopulating stale data between the database write and the delete, which the delete-then-verify patterns address if you need them.

Write-through: on write, update the cache and the database together. Reads never miss, but every write pays both costs, and you cache data nobody may read.

Write-behind: on write, update the cache immediately and the database asynchronously. Lowest write latency, but a crash before the async write means data loss. Only for data you can afford to lose or reconstruct.

For most systems: cache-aside with invalidation on writes. Reach for the others when a measured requirement demands it.

7. Observability: the metrics that matter

A cache you cannot observe is a liability. Four metrics cover nearly everything:

Hit ratio (hits / total requests). Below 80% on a read-heavy endpoint means the cache is not earning its keep. Investigate key design, TTLs, or whether the data is even cacheable.
p99 latency with and without cache. The gap between cache hits and database fallbacks tells you the real value of the layer, and a shrinking gap means the database path needs attention.
Eviction rate. A sudden rise means working set exceeded memory. Either grow the instance or tighten what you cache.
Stampede signals: watch for correlated spikes in database connections and cache misses on the same key pattern. That correlation is the fingerprint of a herd forming.

Alert on eviction rate and on miss-rate spikes for your top keys. Those two catch most caching incidents before users notice.

8. Production checklist

Before calling your caching layer production-ready:

TTLs are jittered, not fixed.
Request coalescing (singleflight or equivalent) guards every hot path.
An eviction policy is set explicitly and matches the workload.
Write invalidation is tested, including the failure case where the delete fails.
Hit ratio, eviction rate, and p99 latency are dashboarded and alerted.
The cache failing does not take down the application. Every cache call has a timeout and a fallback to the database. A cache outage should degrade latency, never cause an outage.

That last point deserves emphasis. The cache is an optimization layer. If its absence breaks you, the architecture has the dependency backwards.

Closing

Caching is simple to start and easy to get wrong at scale. The pattern fits in thirty lines of code, but production demands jittered TTLs, coalesced requests, explicit eviction, and real observability. Get those right and the cache becomes the quietest, most reliable part of your stack.

What is your go-to stampede protection in production?

hello developerws

Mubashir Taj — Thu, 24 Sep 2026 19:33:20 +0000

hello developers

DEV Community: Mubashir Taj