One pipeline for retry, circuit breaker, and timeout in Dart

A slow dependency can take your whole service down. It stops answering, and your own request handlers pile up waiting on it. Now the healthy part of your service is stuck behind one upstream.

The fixes are old. Retry the failed call. Put a timeout on it. When a dependency keeps failing, stop calling it for a while so you stop adding load to something that is already in trouble.

The patterns are well known. Wiring them together in Dart without reaching for a framework is the annoying part. So I wrote resilience. It is 0.1.1 on pub.dev, MIT, and has no dependencies outside the Dart SDK.

Here is the whole idea in one screenful.

The pipeline first

final breaker = CircuitBreaker(
  failureThreshold: 5,
  resetTimeout: Duration(seconds: 10),
  onStateChange: (s) => print('circuit breaker is now $s'),
);

final limiter = RateLimiter(
  maxPermits: 5,
  per: Duration(seconds: 1),
  maxQueueLength: 32,
);

final pipeline = ResiliencePipeline([
  Retry(
    maxAttempts: 4,
    backoff: Backoff.exponential(
      initial: Duration(milliseconds: 100),
      jitter: 0.5,
    ),
    retryIf: (e) => e is ServiceUnavailableException || e is TimeoutException,
    onRetry: (event) => print(
      'attempt ${event.attempt} failed (${event.error}), '
      'next in ${event.nextDelay.inMilliseconds} ms',
    ),
  ),
  breaker,                        // one per upstream, created once
  Timeout(Duration(seconds: 2)),
  limiter,                        // one per upstream, created once
]);

final body = await pipeline.execute(() => client.getJson('/users/42'));

That wraps a flaky HTTP call in four policies. Retry is on the outside, then the CircuitBreaker, then the Timeout, and the RateLimiter gates every attempt.

Order is outermost-first, and it is a choice, not a default. More on that below.

Install is one line:

dart pub add resilience

One interface behind all of it

Every policy is the same shape. It wraps an async action and hands you back a Future.

abstract interface class Policy {
  Future<T> execute<T>(Future<T> Function() action);
}

ResiliencePipeline is itself a Policy. It just runs the ones you give it, in order. So a pipeline of policies is a policy, and you can nest one pipeline inside another with no special case.

The policies

Policy	What it does	Reach for it when
Retry	Runs the action again after a failure, with backoff and jitter	The failure is transient and the call is safe to repeat
CircuitBreaker	Opens after failureThreshold failures, fails fast for resetTimeout, then half-opens to test recovery	A dependency is down and calling it more makes things worse
Timeout	Fails the call if it runs longer than a Duration	A slow answer is worse than a fast failure
RateLimiter	A token bucket: maxPermits per per, queues up to maxQueueLength	You must not start actions faster than some rate
Bulkhead	Caps how many actions run at once	You want to bound concurrency to one dependency
ResiliencePipeline	Composes any of the above into one Policy	You want more than one of these behaviors

Retry has the knobs you would expect: maxAttempts, a backoff strategy, a retryIf predicate on the error, and an onRetry callback with the attempt number, the error, and the next delay.

That callback turns each failed attempt into a line you can read. Against a service that fails twice then answers, it prints something like:

attempt 1 failed (ServiceUnavailableException: 503 from upstream), next in 128 ms
attempt 2 failed (ServiceUnavailableException: 503 from upstream), next in 241 ms

The delays grow, and they move around from run to run, because jitter is 0.5. That is deliberate, and one section below is why.

Now the honest part. A library like this is only worth using if you get the sharp edges right, so here they are.

What the circuit breaker actually does

A circuit breaker is a switch on the caller's side. It has three states, and the value is in how it moves between them.

In closed, the normal state, calls pass straight through to the action and the breaker counts failures. Nothing is blocked.

After failureThreshold failures it flips to open. Now it stops calling the dependency at all and fails immediately for the length of resetTimeout. That is the whole point: stop calling something that is already down.

When resetTimeout elapses it goes half-open and lets a single call through. If that call succeeds the breaker returns to closed and traffic resumes. If it fails the breaker opens again and waits another resetTimeout.

You can watch every one of these transitions through onStateChange. Here it is against a dependency that is down for a moment and then recovers:

Retry needs backoff and jitter

A bare retry loop is how a small outage becomes a big one. A dependency wobbles, every client retries at the same instant, and the retries themselves become the load that keeps it down.

Backoff spreads the retries out over time. Jitter spreads them across each other, so two clients that failed together do not retry together.

resilience defaults to exponential backoff with jitter for that reason. Do not set jitter to 0 without a reason.

Share the stateful policies

CircuitBreaker and RateLimiter carry state. The breaker only opens because it remembers recent failures. The limiter only throttles because it remembers how many permits are left in the window.

Create one per request and it remembers nothing.

Future<Map<String, dynamic>> getUser(int id) {
  // WRONG: a fresh breaker every call, so it never counts a second failure
  final breaker = CircuitBreaker(
    failureThreshold: 5,
    resetTimeout: Duration(seconds: 10),
  );
  return breaker.execute(() => client.getJson('/users/$id'));
}

Make one per upstream and reuse it:

// created once, shared across calls
final _usersBreaker = CircuitBreaker(
  failureThreshold: 5,
  resetTimeout: Duration(seconds: 10),
);

Future<Map<String, dynamic>> getUser(int id) =>
    _usersBreaker.execute(() => client.getJson('/users/$id'));

One breaker per upstream, one limiter per upstream, held somewhere that outlives a single call. Retry and Timeout hold no state, so those are fine to build inline.

Only retry idempotent work

Retrying a read is free. Retrying an idempotent write is fine. Retrying a POST that charges a card can charge it twice.

The trap is that a TimeoutException does not tell you whether the request landed. The write may have succeeded on the server while the response was still in flight. Retry it and you have applied it twice.

Use retryIf to scope retries to errors that mean the call never ran, and leave the risky writes alone. The package cannot know which of your calls are safe, so that judgment stays with you.

A breaker protects the caller, not the dependency

A circuit breaker stops you from wasting calls on something that is already down. It fails fast, and it gives the dependency room instead of a fresh wall of traffic.

It does not fix the dependency. If the thing behind the breaker is broken, the breaker just makes your side degrade cleanly. Someone still has to go fix what is down.

Composition order is a decision

Order changes behavior, not just style.

Put Retry on the outside and each attempt re-runs the whole inner stack, including the Timeout. So a 2-second timeout with 4 attempts is up to 8 seconds of timeout budget across the attempts, and more than that in wall-clock once the backoff waits are added.

Put the CircuitBreaker inside the Retry and it counts every failed attempt toward its threshold. Put it outside and it sees one exhausted retry sequence as a single failure, so it takes longer to open. A retry outside the breaker will also retry the breaker's own fast-fail rejection, if your retryIf allows it.

Neither arrangement is wrong. Pick the one you meant.

One isolate, one limiter

These policies coordinate async work on a single isolate. That is the whole scope.

A RateLimiter limits one isolate's outbound rate. Run four isolates and you have four independent limiters, each allowing maxPermits per window, so your real rate is four times what one limiter says. For a single server process that is usually what you want. For a whole cluster it is not, and a cluster-wide limit needs a shared store, which this package does not try to be.

Try it

dart pub add resilience

Point it at a client that fails a few times then recovers, and watch the onRetry log and the breaker state changes.

• pub.dev: https://pub.dev/packages/resilience
• source: https://github.com/Yusufihsangorgel/resilience
• run the example: dart run example/resilience_example.dart

A slow dependency will always be someone else's bug. Whether it becomes your outage comes down to a few small policies and the order you wrap them in.