Anthropic SDK max_retries: 1 Request Became 63 API Calls

One Tuesday night my queue processed 22 jobs. My API logs showed 1,043 requests.

Nobody deployed anything. There was no traffic spike. The model was having a rough hour and returning 529 overloaded_error, and my stack responded by retrying that error in three different places at once. Anthropic SDK max_retries was one of those places, and it was the one I had completely forgotten existed, because I never set it.

Here is the autopsy.

TL;DR

• Retry layers multiply, they don't add. SDK 3 attempts x wrapper 3 attempts x queue 7 attempts = 63 API calls for one user request.
• The Anthropic SDK retries by default (maxRetries: 2, so 3 total attempts) on 408, 409, 429, 5xx and connection errors, with its own exponential backoff. If you wrapped it in your own retry, you already have two layers.
• Nested backoff also multiplies time: my failing job took 11 minutes to reach the dead letter queue instead of failing fast.
• Fix: retry at exactly one layer, give it an absolute deadline, add jitter, and make the job idempotent with a stable job id.
• After the fix, same 22 jobs during the next overload window: 31 API calls, p99 calls-per-job dropped from 63 to 3.

What was actually running?

The system is Preterview (full disclosure: I built it and I run it), which runs voice mock interviews with a few interviewer styles and then produces a written report from the transcript. The report job is the expensive end of the pipeline: one long model call per session, a few thousand output tokens, and a user sitting there watching a spinner. That spinner is exactly why I had bolted a retry onto every layer over the previous months. Every time a report failed, I added "just one more retry" somewhere new.

Three separate incidents, three separate patches, none of them aware of each other. Here is what the code looked like by the time it bit me:

// layer 1: the SDK. I never touched this. Default is maxRetries: 2.
const anthropic = new Anthropic();

// layer 2: added the week a DNS blip killed 4 reports
await pRetry(() => generateReport(session), { retries: 2 });

// layer 3: added the week a worker got OOM-killed mid-job
reportQueue.add("report", { sessionId }, { attempts: 7 });

Each line is defensible on its own. Multiply them and you get 3 x 3 x 7 = 63 requests to the API for a single interview report.

Why did 1 request become 63 Anthropic API calls?

Because every layer treated 529 as retryable, and retry layers compose multiplicatively. The innermost layer burns all its attempts before it raises, and the layer above it sees that as one failure, so it starts a fresh full round of the inner layer.

Layer	Attempts	Calls so far
Anthropic SDK (maxRetries: 2)	3	3
p-retry wrapper	3	9
BullMQ job attempts: 7	7	63

The time math is worse than the call math and nobody talks about it. The SDK backs off between its own attempts. p-retry backs off between rounds of the SDK. BullMQ backs off between rounds of everything. Those delays nest, so a job that was never going to succeed took 11 minutes 40 seconds from enqueue to dead letter. My alerting is wired to the dead letter queue. I found out about the incident when it was nearly over.

And because all 22 jobs failed at roughly the same moment, their backoff windows lined up. No jitter anywhere in the stack meant every retry round arrived as a synchronized burst against an API that was already telling me it was overloaded. I was, very politely, making my own outage worse.

What does Anthropic SDK max_retries actually do by default?

It retries twice, so three total attempts, without you configuring anything. In both the TypeScript and Python SDKs the client-level default is max_retries=2, applied to connection errors, timeouts, 408, 409, 429 and 5xx responses, with exponential backoff and jitter, and it honors a retry-after header when the API sends one.

That is good behavior. It is the layer I should have kept. The problem was never that the SDK retries, it is that I wrote two more retry layers on top of a client that was already doing the right thing, and I did it without ever reading that default.

You can pin it per-client or per-request:

const anthropic = new Anthropic({ maxRetries: 0 });

// or leave the client alone and disable it for one call
await anthropic.messages.create({ ... }, { maxRetries: 0 });

client = Anthropic(max_retries=0)
# per request
client.messages.with_options(max_retries=0).create(...)

How do you find hidden retry layers in your stack?

Grep for them, because at least one will be in a config file you did not write. This one line found two layers I had forgotten about in my repo:

rg -n "maxRetries|max_retries|retries:|attempts:|retry|backoff|proxy_next_upstream"

The usual suspects, roughly in the order I found mine:

1. The SDK client. Anthropic, OpenAI, Stripe, AWS, most modern SDKs retry by default.
2. Your own wrapper. p-retry, tenacity, retry, or a hand-rolled for (let i = 0; i < 3; i++).
3. The job queue. BullMQ attempts, Celery max_retries, Sidekiq's 25 (!) default retries.
4. The proxy. nginx proxy_next_upstream will re-send a request to another upstream on error or timeout.
5. The client app. React Query defaults to 3 retries on failed queries. Your frontend is a retry layer.
6. The human. The user hits the "generate report" button again. Design for it.

Multiply all of them. That is your true worst-case request count per user action, and it is probably a number you would not say out loud in a design review.

How do you fix retry amplification?

Pick one layer to own retries, disable the rest, then bound the whole thing with a deadline. I chose the queue, because it is the only layer that survives a process crash and the only one that can tell a user "this is still cooking."

// SDK: off. The queue owns retries now.
const anthropic = new Anthropic({ maxRetries: 0 });

// wrapper: deleted entirely

reportQueue.add(
  "report",
  { sessionId },
  {
    attempts: 4,
    backoff: { type: "exponential", delay: 2_000 },
    jobId: `report:${sessionId}`, // dedupe: same session, same job
  },
);

Four rules that came out of this, in order of how much pain they saved:

One retry layer per failure domain. Transient network errors get retried in exactly one place. Everything else fails fast upward. If you genuinely need two layers, budget the total attempts, not the layers: outer x inner must equal the number you actually meant.

Give it an absolute deadline, not an attempt count. Attempt counts lie about wall clock. A deadline does not.

const deadline = Date.now() + 90_000;
if (Date.now() > deadline) {
  throw new UnrecoverableError("report budget exhausted");
}

Make the job idempotent before you make it retryable. Nine of my duplicated calls actually succeeded on a later attempt and wrote a second report row for the same session. Two users got two emails. A stable jobId plus an upsert on (session_id, report_version) fixed that permanently, and it is the change I should have made first.

Classify errors instead of retrying everything. A 400 invalid_request_error will fail identically 63 times. Retry 429, 500, 529 and connection errors. Never retry a 4xx that is not 408 or 429. That one filter cut my retry volume more than the layer removal did.

One more trap specific to LLM calls: if you are streaming and a proxy times out at 60 seconds while the model is at token 3,000, the retry regenerates the whole response and you pay for both. Retrying a long generation is not free the way retrying a GET /health is. Price your retries in output tokens, not in requests.

What changed after the fix?

The next overload window hit 22 jobs again. Numbers from my own logs, same workload, same failure class:

Metric	Before	After
Total API calls	1,043	31
p99 calls per job	63	3
Enqueue to dead letter	11m 40s	38s
Duplicate reports delivered	9	0

My token spend for that hour was 18x a normal hour before the fix and boringly normal after it. I also added one metric that I now consider mandatory for anything calling an LLM in a background worker: api_calls_per_job, tagged by job type. If that number is ever higher than your configured attempt count, you have a retry layer you do not know about.

So why does one request become 63 Anthropic API calls? Because retries multiply instead of adding, and because the Anthropic SDK already retries by default with max_retries=2 before your own wrapper and your job queue each add their own rounds on top. Three innocuous layers of 3, 3 and 7 attempts produce 63 calls and eleven minutes of stacked backoff for a single user action. Count the layers in your stack, pick exactly one to own retries, set the others to zero, bound the whole path with an absolute deadline, and make the work idempotent so the retries you do keep cannot duplicate anything.

---

Written by the developer behind Preterview, an interview prep platform.