The Circuit Breaker Pattern: Stop Calling a Dead Service

#architecture #backend #microservices #systemdesign

The circuit breaker pattern: stop calling a dead service

You've got a checkout service that calls a payment provider. The provider goes down. Every incoming request now sits there, holding a thread, waiting on a 30-second timeout. Your thread pool fills up. New requests queue. Response times spike from 200ms to 30 seconds across the board. Users hammer refresh. More threads blocked. More queues. Your entire service is now unresponsive because of one dependency that isn't even yours.

The service was down. You kept calling it anyway. That's the actual problem.

Retries make this worse (I wrote about that here) but they're a sibling topic. What we need is a way to stop sending traffic to something that's clearly broken.

🧠 The state machine

A circuit breaker borrows the metaphor from electrical engineering. Three states, simple transitions.

Closed is normal operation. Requests flow through to the dependency. The breaker watches failures in a sliding window and counts.

Open means the breaker tripped. Every call gets rejected immediately without touching the downstream service. No threads held. No connections opened. The caller gets an error (or a fallback) in microseconds instead of waiting on a timeout.

Half-open is the probe phase. After a configured wait duration, the breaker lets a small number of test requests through. If they succeed, back to closed. If they fail, back to open for another wait cycle.

That's it. Three states, two transition triggers: failure rate crossed a threshold (closed → open), and probe calls passed or failed (half-open → closed or half-open → open).

So what actually trips the breaker? Usually it's a percentage. If 50% or more of the last N calls failed, the circuit opens. But you need a minimum volume before that percentage means anything. Five requests with two failures is 40%, and that's noise, not a signal. Most implementations require at least 10-20 calls in the window before the breaker can evaluate.

🛠️ Configuration and code

The parameters worth tuning:

Failure rate threshold: the error percentage that trips the breaker (50% is common)
Minimum number of calls: how many requests before the breaker starts evaluating (10-100)
Sliding window: count-based (last N calls) or time-based (last N seconds)
Wait duration in open state: how long before allowing probes (30-60 seconds)
Permitted calls in half-open: how many probes to send (3-10)

Here's what this looks like with opossum, the main circuit breaker library for Node.js (v10, ~1.2M weekly downloads):

import CircuitBreaker from "opossum";

async function fetchUser(id: string) {
  const res = await fetch(`https://api.example.com/users/${id}`);
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  return res.json();
}

const breaker = new CircuitBreaker(fetchUser, {
  timeout: 3000,                  // calls slower than 3s count as failures
  errorThresholdPercentage: 50,   // trip at 50% error rate
  resetTimeout: 30000,            // wait 30s before probing
  volumeThreshold: 5,             // need at least 5 calls to evaluate
});

breaker.fallback(() => ({ name: "Guest", cached: true }));

const user = await breaker.fire("user-123");

When the breaker opens, breaker.fire() returns the fallback immediately. No network call. No timeout. The caller gets a degraded response in microseconds.

And if you want to understand the mechanics without a library, here's a minimal state machine:

type State = "CLOSED" | "OPEN" | "HALF_OPEN";

class Breaker {
  private state: State = "CLOSED";
  private failures = 0;
  private lastFail = 0;

  constructor(private threshold = 5, private resetMs = 30_000) {}

  async call<T>(fn: () => Promise<T>): Promise<T> {
    if (this.state === "OPEN") {
      if (Date.now() - this.lastFail > this.resetMs) {
        this.state = "HALF_OPEN";
      } else {
        throw new Error("Circuit OPEN");
      }
    }
    try {
      const result = await fn();
      this.failures = 0;
      this.state = "CLOSED";
      return result;
    } catch (e) {
      this.failures++;
      this.lastFail = Date.now();
      if (this.failures >= this.threshold) this.state = "OPEN";
      throw e;
    }
  }
}

Not production-ready. No sliding window, no half-open probe limit. But it shows the bones of the pattern clearly enough.

⚡ Scoping, fallbacks, and how this isn't a retry

One breaker per dependency. Actually, one breaker per endpoint if your dependencies expose multiple routes with different reliability characteristics. A single global breaker is wrong — one flaky recommendation endpoint would trip it for your perfectly healthy user-auth endpoint. Keep them isolated. If you're running a service mesh, Envoy's outlier detection does this at the proxy layer without any application code.

What do you serve while the breaker is open? Depends on the dependency.

Cached data: stale user profile, yesterday's recommendations. Better than nothing.
Degraded feature: skip personalization, show generic content, disable non-critical functionality.
Honest error: sometimes the right answer is just telling the caller "this feature is temporarily unavailable."

But here's the thing people confuse. A retry repeats a failed call hoping it'll work next time. A breaker stops making calls entirely once the failure rate proves the dependency is broken. They solve different problems. Retries help with transient blips. Breakers protect you from sustained outages where retrying just adds fuel to the fire.

If your API gateway already has circuit breaker support or you're using load balancing with health-aware routing, you might get some of this for free at the infrastructure layer. Worth checking before rolling your own.

📌 Key takeaways

A circuit breaker has three states: closed (flowing), open (rejecting), half-open (probing)
It trips when the failure rate in a sliding window exceeds your threshold
One breaker per dependency or endpoint. Never a single global breaker
While open, serve a fallback or fail fast. Don't hold threads waiting on something that's down
Breakers stop calling broken services. Retries repeat calls hoping they'll work. Different tools for different problems
Tune the minimum volume so random noise doesn't trip your breaker on low-traffic endpoints