DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Caching That Survives Real Traffic: TTL Jitter and Single-Flight in Spring Boot

#springboot #java #redis #backend

Day 11 of building OrderHub added Redis caching with @Cacheable/@CacheEvict — the same read served ~60× faster from memory. But a naïve cache has two failure modes that only show up under load. Day 12 is about making caching safe.

Failure 1: the thundering herd

If a burst of keys all get the same TTL (say 60s), they all expire at the same instant one TTL later — and every request misses at once and stampedes the database. The fix is expiry jitter: add a small random offset (±10%) to every TTL so expiries spread into a smooth trickle instead of one cliff.

Duration ttlWithJitter(Duration base) {
  long ms = base.toMillis(), span = (long)(ms * 0.10);
  long delta = ThreadLocalRandom.current().nextLong(-span, span + 1);
  return Duration.ofMillis(ms + delta);
}
Enter fullscreen mode Exit fullscreen mode

Failure 2: the dogpile

When one hot key expires, dozens of concurrent requests all miss and all recompute the same value together. Single-flight lets exactly one request recompute while the rest wait for its result. In Spring, the simplest per-node form is one flag:

@Cacheable(cacheNames="order", key="#id", sync=true)
public Order getOrder(Long id) { return repo.findById(id).orElseThrow(); }
Enter fullscreen mode Exit fullscreen mode

sync=true makes one thread load while the others block on the cache. Across nodes you'd use a short Redis SETNX lock before recomputing.

Configurable TTLs per cache

Different data goes stale at different rates. Drive per-cache TTLs from configuration so you can tune them per environment without a redeploy:

app:
  cache:
    default-ttl: 10m
    ttls:
      order:  5m
      orders: 2m
      product: 1h
Enter fullscreen mode Exit fullscreen mode

Evict precisely, never flush

Caching is only safe if writes invalidate the right keys. Evict the specific entry that changed and any list/aggregate that includes it — but avoid flushing the whole cache, which just re-triggers the herd you fixed.

Choose an eviction policy

Redis has finite memory; maxmemory-policy decides what to drop when it fills. allkeys-lru (evict least-recently-used) is a solid default for a general cache; volatile-ttl targets keys nearest expiry. Set it deliberately so memory pressure degrades gracefully instead of erroring.

Then prove it: a Testcontainers Redis test that does a cached read and asserts the key exists with a TTL in the expected jittered band.

Live cache-strategy playground (herd vs jitter, dogpile vs single-flight) + the full Spring Boot walkthrough:
https://dev48v.infy.uk/orderhub.php

Repo: https://github.com/dev48v/order-hub-from-zero

Top comments (0)