AI Agent Error Recovery: Retry, Timeout, Backoff, and Circuit-Breaker Patterns

Your agent calls an API. The API returns 429 Too Many Requests. Your agent retries immediately, gets another 429, retries again, and creates a loop that hammers an already throttled service until the run fails or costs spike. This is the naive version of “handle the error,” and it is one of the most common issues discussed on the Anthropic SDK discussion board.

Try Apidog today

Reliable error recovery is what turns an agent demo into a production service. The model is rarely the issue; the issue is how your code handles slow, throttled, or broken tool dependencies. This guide implements four core patterns:

1. Retries with exponential backoff and jitter
2. Timeouts for every outbound request
3. Circuit breakers for failed dependencies
4. Idempotency keys for safe mutations

You will also test each path against a mock API before users find the gaps. For broader context, see why AI agents break in production.

You can’t test recovery against a healthy API

A dependency that works perfectly in development never exercises your recovery code. The first real 429, 500, timeout, or malformed response then happens in production—when users are waiting and you have the least visibility.

Test recovery by creating failures deliberately:

1. Mock the API your agent calls.
2. Configure the mock to return failures such as 429, 500, slow responses, or invalid JSON.
3. Point the agent tool at the mock URL.
4. Assert on retry timing, attempt counts, request headers, and final behavior.

Apidog can stand up a mock API and script these responses so failures become repeatable test cases instead of 3am incidents.

Retry with exponential backoff and jitter

Never retry immediately in a loop. If every client retries at the same moment, a temporary outage becomes a retry storm.

Use:

• Exponential backoff: increase delays between attempts.
• Jitter: randomize each delay to prevent synchronized retry waves.
• Attempt limits: stop retrying permanent failures.
• Delay limits: prevent excessively long waits.

A typical schedule is 1s, 2s, 4s, 8s, capped at a maximum delay.

import random
import time

def backoff_delay(attempt: int, base: float = 1.0, cap: float = 8.0) -> float:
    exponential_delay = min(cap, base * (2 ** attempt))
    jitter = random.uniform(0, exponential_delay)
    return jitter

def call_with_retries(request_fn, max_attempts: int = 4):
    for attempt in range(max_attempts):
        try:
            return request_fn()
        except RetryableError:
            if attempt == max_attempts - 1:
                raise

            delay = backoff_delay(attempt)
            time.sleep(delay)

In practice, retry only transient failures:

• Connection failures
• Timeouts
• 408 Request Timeout
• 429 Too Many Requests
• Selected 5xx responses

Do not blindly retry client-side validation failures, authentication failures, or malformed requests.

The Anthropic SDK retries connection errors and selected status codes with exponential backoff for its own API calls, with a configurable maximum retry count. Your agent tools may call other APIs, though, so apply the same policy around those calls yourself. For high-risk cases, see retry logic for high-stakes APIs.

Set a timeout on every call

Retries only work when a request returns an error. A request that hangs forever never reaches your retry logic.

Every outbound call should have:

• A connect timeout for establishing a connection
• A read timeout for receiving a response
• A total agent-run budget so multiple slow calls cannot outlive the user’s patience

For example, with Python httpx:

import httpx

timeout = httpx.Timeout(
    connect=2.0,
    read=10.0,
    write=10.0,
    pool=2.0,
)

with httpx.Client(timeout=timeout) as client:
    response = client.get("https://api.example.com/search")
    response.raise_for_status()

Treat timeout exceptions as retryable when the operation is safe to retry:

try:
    response = client.post(url, json=payload)
    response.raise_for_status()
except (httpx.ConnectTimeout, httpx.ReadTimeout):
    # Apply capped backoff before retrying.
    ...

Choose timeout values from real latency data, not guesses:

• Set timeouts above the dependency’s p99 latency, with headroom.
• Avoid overly short timeouts that cancel valid requests.
• Avoid overly long timeouts that keep the agent blocked on dead dependencies.
• Give streaming responses a separate budget; legitimate streams can take longer than standard JSON responses.

Trip a circuit breaker when a dependency is down

Backoff helps when a dependency is briefly busy. It is not enough when a service is consistently failing.

A circuit breaker prevents your agent from repeatedly calling a known-bad dependency. It has three states:

• Closed: requests flow normally; failures are counted.
• Open: requests fail fast for a cooldown period.
• Half-open: one probe request is allowed through after the cooldown.

If that probe succeeds, close the breaker. If it fails, reopen it.

from time import monotonic

class CircuitBreaker:
    def __init__(self, failure_threshold=5, cooldown_seconds=30):
        self.failure_threshold = failure_threshold
        self.cooldown_seconds = cooldown_seconds
        self.failures = 0
        self.opened_at = None

    def allow_request(self):
        if self.opened_at is None:
            return True

        if monotonic() - self.opened_at >= self.cooldown_seconds:
            return True  # Half-open probe

        return False

    def record_success(self):
        self.failures = 0
        self.opened_at = None

    def record_failure(self):
        self.failures += 1
        if self.failures >= self.failure_threshold:
            self.opened_at = monotonic()

Use one breaker per dependency, not one global breaker. A broken search API should not prevent the agent from calling a healthy billing API.

For an agent, this changes “the payment API is down” from many slow timeouts into one fast, explicit failure that the agent can handle or surface cleanly.

Make retries safe with idempotency keys

Retries are dangerous when a request changes state.

Consider this flow:

1. Your agent sends POST /charge.
2. The server successfully creates the charge.
3. The response times out before the agent receives it.
4. The agent retries.
5. The customer is charged twice.

The retry behaved correctly. The request design was unsafe.

Use an idempotency key for mutations. Generate one unique value for the logical action, then reuse that exact value for every retry.

import uuid
import httpx

idempotency_key = str(uuid.uuid4())

headers = {
    "Idempotency-Key": idempotency_key,
}

response = httpx.post(
    "https://payments.example.com/charge",
    json={"amount": 5000, "currency": "usd"},
    headers=headers,
)

The server records the first request for that key. If it receives the same key again, it returns the original result instead of repeating the operation.

The critical rule: generate the key outside the retry loop.

idempotency_key = str(uuid.uuid4())

for attempt in range(max_attempts):
    response = send_charge(
        idempotency_key=idempotency_key,
        payload=payload,
    )

Do not do this:

for attempt in range(max_attempts):
    idempotency_key = str(uuid.uuid4())  # Wrong: new logical operation each retry
    response = send_charge(
        idempotency_key=idempotency_key,
        payload=payload,
    )

Use idempotency keys for any state-changing action:

• Charges
• Orders
• Emails or messages
• Record creation
• Updates with externally visible side effects

For implementation details, see idempotency keys.

Survive rate limits and the RateLimitError loop

Rate limits need special handling because the server often tells you when to retry.

A rate-limit-exceeded response usually returns 429 and may include a Retry-After header. Respect that value.

If the server says:

Retry-After: 30

wait at least 30 seconds before retrying. Retrying after two seconds guarantees another 429 and creates the loop developers describe in this SDK thread.

from datetime import datetime, timezone
from email.utils import parsedate_to_datetime

def retry_after_seconds(value: str | None) -> float | None:
    if not value:
        return None

    if value.isdigit():
        return float(value)

    retry_at = parsedate_to_datetime(value)
    now = datetime.now(timezone.utc)
    return max(0, (retry_at - now).total_seconds())

Use the header when it is available, then fall back to exponential backoff with jitter when it is not:

if response.status_code == 429:
    wait_seconds = retry_after_seconds(response.headers.get("Retry-After"))

    if wait_seconds is None:
        wait_seconds = backoff_delay(attempt)

    time.sleep(wait_seconds)

Also cap attempts. A sustained rate limit should end in a clean error, not an indefinite wait.

The Anthropic SDK honors Retry-After for its own calls. Apply the same behavior to every other rate-limited API your agent uses.

For proactive protection, rate-limit your own outbound calls with a token bucket or queue. Recovery handles limits after you hit them; pacing helps you avoid hitting them.

How to test the recovery path

A healthy API cannot prove your error handling works. Use a mock API to force scenarios and assert on observable behavior.

Scenario 1: Recover from a rate limit and a server error

Script one endpoint to return responses in order:

1. 429 with Retry-After: 2
2. 500 Internal Server Error
3. 200 OK with a valid response body

Then point your agent tool at the mock and assert that it:

• Waited at least two seconds after the 429
• Retried after the 500
• Succeeded on the third call
• Never exceeded the configured retry cap

This validates Retry-After, backoff, and successful recovery in one test.

Scenario 2: Stop after the retry cap

Configure the mock to fail every request:

500 → 500 → 500 → 500

Assert that the agent:

• Makes only the configured number of attempts
• Returns a clean failure
• Does not loop forever
• Includes enough context for logging or user-facing handling

Scenario 3: Open the circuit breaker

Configure repeated failures until the circuit breaker threshold is reached.

Assert that:

• Initial requests attempt the dependency
• The breaker opens after the threshold
• Subsequent requests fail fast during the cooldown
• A probe is attempted after the cooldown window

Scenario 4: Verify idempotency on a dropped response

This is the test that prevents duplicate charges and duplicate writes:

1. Configure the mock to accept a mutating request.
2. Simulate a dropped or timed-out response.
3. Let the agent retry.
4. Inspect both received requests.

Assert that:

• Both requests include Idempotency-Key
• Both requests use the same key
• The mock records one logical action rather than two

A new key on retry means deduplication cannot work. A duplicate logical action means you found a double-send before a customer did.

For a full harness approach, see testing agents that call your APIs.

The error-recovery checklist

Before deploying an agent, verify all of the following:

• Every outbound call has connect, read, and total-run time budgets.
• Retries use exponential backoff with jitter.
• Retry delay and attempt count are capped.
• 429 responses respect Retry-After.
• A circuit breaker exists for each dependency.
• State-changing calls use stable idempotency keys across retries.
• The give-up path returns a clean error instead of waiting forever.
• Tests force each failure mode against a mock API.

If all of these are true, your agent recovers by design instead of by luck.

Where Apidog fits (and where it doesn’t)

Apidog is not an agent framework, model host, or runtime. It does not build, run, or orchestrate your agent, and it does not evaluate model output.

Its role is the API layer your agent calls.

Use Apidog to:

• Mock the dependencies your agent calls
• Script failure sequences such as 429 with Retry-After, 500, timeouts, and malformed bodies
• Validate incoming request shape
• Check that idempotency keys are present and stable
• Verify expected call counts so duplicate sends fail tests

That is the practical fit: mock the failures your agent must survive, then verify what it sends.

Frequently asked questions

Doesn’t the Anthropic SDK handle retries for me?

For its own API calls, the SDK retries selected errors with exponential backoff, respects Retry-After, and provides a max-retries setting.

It does not cover the other APIs your agent tools call. Apply the same retry, timeout, circuit-breaker, and idempotency patterns to those dependencies yourself.

When do I need an idempotency key?

Use one for every request that creates or changes state:

• Charges
• Orders
• Messages
• New records
• Other externally visible actions

Read-only requests are generally safe to retry without one. Generate the key once per logical action so it remains stable across retries.

Rehearse one failure this week

You do not need to implement every pattern at once. Start with the failure that would hurt most—usually a rate-limit loop or a non-idempotent mutation.

Mock a 429, drop a response after accepting a write, and inspect the agent’s behavior. When you see controlled backoff and one stable idempotency key instead of a duplicate charge, you have evidence that your agent can handle the failure path—not just the happy path.

Use Apidog to mock the failure sequence and assert on what your agent does when an API pushes back.