Stop the Cache Stampede: A Small JavaScript Drill for Backend Interviews

#career #interview #javascript #backend

When twelve requests miss the same cache key at once, a normal cache-aside implementation can send twelve expensive reads to the origin. The small fix is request coalescing: keep one in-flight promise per key, let the first request own the fetch, and have the rest await it. That is a compact, explainable system-design answer—and a useful production primitive.

Why a TTL alone does not prevent a stampede

A cache is a local store that reuses a response for later requests; it reduces work only while the entry is usable. See MDN’s HTTP caching guide for the protocol-level model.

The trouble begins at an expiry boundary. Suppose a product page has a 60-second TTL:

Time	What a basic cache-aside read does
09:00:59.999	Returns the cached value
09:01:00.000	The entry expires
09:01:00.001	Every concurrent request sees a miss and calls the origin

That burst can be worse than an uncached steady state. It increases database work, stretches tail latency, and makes the origin slower exactly when traffic is high. A longer TTL changes the frequency; it does not make simultaneous misses disappear.

The invariant we want is simple: for one cache key, at most one origin fetch is active in this process.

Build the smallest useful coalescer

Here is a dependency-free Node.js drill. It deliberately keeps storage in memory so the concurrency behavior is easy to inspect. The cache entry has a fresh period and a bounded stale-on-error period; the inFlight map is the important part.

import assert from "node:assert/strict";

const cache = new Map();
const inFlight = new Map();

async function getWithCoalescing(
  key,
  fetchFresh,
  { ttlMs = 1_000, staleMs = 5_000 } = {},
) {
  const hit = cache.get(key);
  const ageMs = hit && Date.now() - hit.storedAt;

  if (hit && ageMs < ttlMs) {
    return { value: hit.value, source: "fresh-cache" };
  }

  if (inFlight.has(key)) {
    return inFlight.get(key);
  }

  const flight = (async () => {
    try {
      const value = await fetchFresh();
      cache.set(key, { value, storedAt: Date.now() });
      return { value, source: "origin" };
    } catch (err) {
      if (hit && ageMs < ttlMs + staleMs) {
        return { value: hit.value, source: "stale-on-error" };
      }
      throw err;
    } finally {
      inFlight.delete(key);
    }
  })();

  inFlight.set(key, flight);
  return flight;
}

let originCalls = 0;
const slowOrigin = async () => {
  originCalls += 1;
  await new Promise((resolve) => setTimeout(resolve, 20));
  return "profile-v1";
};

const firstWave = await Promise.all(
  Array.from({ length: 12 }, () =>
    getWithCoalescing("profile:42", slowOrigin),
  ),
);

assert.equal(originCalls, 1);
assert.deepEqual(new Set(firstWave.map(({ value }) => value)), new Set(["profile-v1"]));
assert.equal(new Set(firstWave.map(({ source }) => source)).size, 1);

const cacheHit = await getWithCoalescing("profile:42", slowOrigin);
assert.equal(cacheHit.source, "fresh-cache");
assert.equal(originCalls, 1);

cache.set("profile:42", {
  value: "last-known-good",
  storedAt: Date.now() - 1_100,
});

const staleResult = await getWithCoalescing(
  "profile:42",
  async () => {
    throw new Error("origin unavailable");
  },
  { ttlMs: 1_000, staleMs: 5_000 },
);

assert.deepEqual(staleResult, {
  value: "last-known-good",
  source: "stale-on-error",
});

console.log({ originCalls, firstWave: firstWave.length, cacheHit, staleResult });

Run it with Node 18+:

node cache-stampede-drill.js

The important assertion is not that the value is right. It is that twelve callers produced one origin call. The fresh-cache assertion proves we do not pay the origin again after the flight fills the entry. The last assertion makes the degradation policy explicit: a recently expired value may be more useful than an outage, but only inside a deliberately bounded window.

What this code does—and does not—solve

This is singleflight for one Node.js process. It is a good interview-sized primitive and it is often enough for a small service, but saying where it stops is part of a senior answer.

Key design matters. Include every dimension that changes the response: tenant, locale, authorization scope, and feature version. Coalescing an under-specified key is a data-isolation bug, not an optimization.
Always remove the flight. The finally block prevents a rejected promise from permanently poisoning the key. Waiters receive the same failure, and the next request gets a new chance to load.
Bound stale data intentionally. Stale-on-error can protect an origin, but it can be wrong for balances, permissions, or inventory. Name the acceptable staleness in the requirement instead of treating it as a universal safety switch.
Add jitter before production. If every key expires on the minute, many unrelated keys can still refresh together. A little randomized TTL variance spreads those refreshes out.
Multiple instances need a different coordination boundary. Each process has its own inFlight map. If origin protection must hold across replicas, use a shared coordination design and reason carefully about lock expiry, owner failure, and duplicate work. Do not present a distributed lock as a magic one-line upgrade.

HTTP caching has a richer vocabulary for freshness and stale responses than this toy cache. RFC 9111 is a useful primary reference when the data is actually travelling through HTTP intermediaries.

A 60-second interview answer

If an interviewer asks, “How would you avoid a cache stampede?”, I would answer in this order:

Clarify the key, read/write ratio, acceptable staleness, and whether a bad value is worse than a slow value.
Use cache-aside for normal hits.
On a miss or expired entry, coalesce requests by key: the first request creates a promise; concurrent callers await that promise instead of hitting the origin.
Delete the in-flight record in finally, record origin-call count and waiter count, and decide whether a short stale-on-error window is safe.
For many replicas, explain how the coordination changes and what happens when the owner dies.

That sequence shows more judgment than jumping straight to “put Redis in front of it.” It describes the failure mode, the local control, the observability, and the point where the design must evolve.

Practice the follow-ups, not just the headline

The useful follow-ups are predictable:

“What prevents a memory leak?” — expiry plus eviction for cache entries, and finally for flights.
“What if the origin times out?” — share the same timeout/failure with waiters; optionally serve only policy-approved stale data.
“How do you measure success?” — origin QPS per key, coalesced-waiter count, cache-hit rate, p95 latency, and stale-response rate.
“What changes for personalized data?” — put identity and authorization inputs in the key, or do not cache the response.
“Why not lock everything?” — lock scope, expiry, and failure recovery can become more expensive and less reliable than duplicate work for low-value keys.

One practical rehearsal method is to paste the code into a scratch file, change one invariant at a time, and explain the resulting failure aloud. If you want a structured technical mock after that, aceround.app — AI interview assistant is one option for practicing the follow-up loop rather than only memorizing this pattern.