DEV Community

강해수
강해수

Posted on • Originally published at dailymanuallab.com

800 simultaneous Workers, one cache miss, $40/mo surprise — the Cloudflare coalescing fix

#cloudflare #serverless #webdev #tutorial

A Korean flash sale at 9PM cost me $40 in a single month — not from KV reads, but from 800 concurrent Workers all racing to fetch the same product JSON from my origin in a 120ms window.

The naive KV pattern looks fine on paper: check KV, miss, fetch origin, write KV, return. At low concurrency it works. Under burst traffic, though, KV write latency was running ~80ms and my origin fetch took ~120ms. That gap was wide enough for roughly 400 Workers to independently conclude the cache was cold and hammer the origin simultaneously. It started returning 429s by 9:03PM.

The fix isn't faster KV writes — it's ensuring only one Worker ever starts the fetch. Everything else waits on the same in-flight promise. That's request coalescing.

The trick is that Workers are stateless across isolates — you can't share a Promise between them directly. But a Durable Object runs in a single-threaded JS environment, which makes a Map of in-flight promises inside a DO completely race-condition-free. The structure is straightforward:

if (!this.inflight.has(key)) {
  const promise = this.fetchAndCache(key).finally(() => {
    this.inflight.delete(key);
  });
  this.inflight.set(key, promise);
}
const body = await this.inflight.get(key)!;
Enter fullscreen mode Exit fullscreen mode

The finally delete is non-negotiable. A catch-only cleanup means a failed fetch permanently poisons that map entry — every future request for that key gets a rejected promise with no recovery. One other gotcha I hit early: I initially keyed all requests to a single DO instance with idFromName("global-coalescer"). A Durable Object processes requests serially, so one slow origin fetch for product A would block product B entirely. The right move is idFromName(key) — one DO instance per resource, not one global bottleneck.

The Worker entry point stays simple: KV hit returns immediately (that hot path costs nothing up to 10M reads/day), and only on a miss does the request route to the Durable Object coalescer.

I wrote up the full breakdown — including the wrangler.toml config, the KV write timing problem inside a DO, and the exact production numbers before and after — over on dailymanuallab.com.

Full post →

Top comments (0)