Exponential Backoff Isn't Enough: The Retry Storm Hiding in Your Recovery Path

The problem

A dependency blips for 400ms — a deploy, a GC pause, a brief network partition, doesn't matter which. Every client with an in-flight request fails at roughly the same instant. They all do the responsible thing: back off exponentially and retry. And yet the service doesn't recover cleanly. It gets hit by a second, sharper outage a few seconds later, caused entirely by the retries that were supposed to be polite about coming back.

I've watched this pattern take down a freshly-recovered service harder than the original blip did — not once, but as a repeating pattern, every few seconds, for the better part of a minute, while everyone stared at dashboards wondering why a "fixed" service kept re-breaking.

Why it happens

Exponential backoff fixes one problem and leaves another completely untouched. The problem it fixes: a client that fails shouldn't hammer you immediately and repeatedly — waiting longer between attempts reduces total load. The problem it doesn't fix: synchronization. If 2,000 clients all fail at t=0, and every one of them runs the identical formula delay = min(cap, base * 2^attempt), every one of them computes the identical delay. They don't spread out. They retry in lockstep.

Walk through it with base=1s, cap=32s, 2,000 clients that all fail at t=0:

• Attempt 1: all 2,000 retry at t=1s, simultaneously. The service, mid-recovery, can't absorb a 2,000-request spike. It fails again.
• Attempt 2: all 2,000 back off to 2s and retry at t=3s, simultaneously. Same result.
• Attempt 3: 4s delay, retry at t=7s. Attempt 4: 8s delay, retry at t=15s. Attempt 5: 16s delay, retry at t=31s.

Five attempts later, you've had five synchronized full-fleet spikes, each one arriving exactly when the service was most fragile — right after the previous spike knocked it back down. The growing delay buys you less frequent storms, not smaller ones. Every single storm is still the full 2,000 requests, because nothing in the formula ever desynchronizes the clients from each other. Pure exponential backoff makes the disease chronic instead of acute.

What to do about it

The fix is jitter — randomizing the delay instead of computing it deterministically — but which kind of jitter matters more than people assume:

• No jitter (what's above): everyone collides, forever, on a slowly growing clock.
• Equal jitter (delay = cap/2 + random(0, cap/2)): spreads clients across half the window. Better, but still guarantees a minimum synchronized floor every round — half the fleet is still packed into the back half of the window.
• Full jitter (delay = random(0, cap)): each client's retry lands uniformly anywhere in the window, including near zero. With 2,000 clients spread across attempt 3's 4-second window instead of landing on one instant, you get roughly 500 requests/second of smooth, absorbable load instead of 2,000 at once — and the spread widens geometrically with every attempt, so the tail trickles rather than detonates.
• Decorrelated jitter (delay = min(cap, random(base, previous_delay * 3))): approximates full jitter's spread without needing to track attempt count, which is convenient in stateless retry wrappers.

Full jitter is the one to default to unless you have a specific reason not to — it minimizes both total requests sent and the time until the last straggler succeeds, because it never re-synchronizes clients that failed together. The implementation cost is one extra random() call; there's no excuse for skipping it once you know the failure mode exists.

One more lever worth pairing with jitter: cap the maximum number of retry attempts and fail fast past it, so a client that still can't get through after, say, 5–6 attempts gives up instead of circling forever at the cap value, quietly contributing to a permanent low-grade storm.

Key takeaways

• Exponential backoff alone does not break synchronization between clients that failed at the same time — it just lowers the frequency of full-strength collisions, not their size.
• "Jitter" is not one thing. No jitter collides every time; equal jitter still guarantees a synchronized floor; full jitter is the one that actually desynchronizes the fleet.
• Default to full jitter (random(0, min(cap, base * 2^attempt))) unless you've measured a reason to deviate.
• If your service has ever had a second, sharper outage right after recovering from the first one, check your clients' retry logic before you look anywhere else — it's a textbook symptom.