A retry budget defines how many retries a client may afford in total, no matter how many attempts a single request would still have left. That sounds like bookkeeping, but it is the difference between “the partner had a brief hiccup” and “we knocked the partner over for good with our own retries”. In my article on exponential backoff with jitter, the question was when to retry. Today the question is whether you should retry at all.
The topic keeps me busy because retries feel so harmless. Three attempts, a bit of waiting, done. But once the target really falls over, every client turns into a small load generator. That is exactly when a budget helps, because it makes retries scarce as soon as too many requests fail.
In short
A retry budget limits the sum of all retries of a client, not just the attempts per request. Retries stay allowed for occasional glitches but become scarce quickly during a real outage. The biggest pitfall in Lambda: every execution environment has its own budget, and Lambda scales out exactly when things get tight.
What you will learn in this article
- What a retry budget is and why backoff alone is not enough
- Token bucket like the AWS SDK versus a 10% ratio like Google SRE
- A tested retry budget in TypeScript with your own retry loop
- How to use the budget in Lambda and where it hits its limits there
What is a retry budget?
A retry budget is an allowance for retries that all requests of a client share. The well-known limit of “at most three attempts” applies per request. The budget, on the other hand, applies to the whole process: as long as almost everything works, there is plenty left, and occasional outliers may try again. If many requests suddenly fail, the budget is used up fast. From then on, the client passes errors on immediately instead of piling on.
The idea behind it is simple. A single error is usually chance, for example a lost packet or a container that is just restarting. Many errors at once are almost never chance. In that case the target is overloaded or broken, and every further attempt makes the problem worse. That is why Marc Brooker calls retries “selfish” in the Amazon Builders’ Library: the retrying client buys better odds with the server’s time.
Why backoff alone is not enough
Exponential backoff with jitter spreads retries over time. That prevents all clients from knocking again at the same moment. However, backoff does not change the number of attempts. If every request has three attempts and all of them fail, almost three times as many calls still reach the target, just stretched out a little.
The Google SRE book works through exactly this case: with a limit of three attempts per request, the load ends up just below three times the original in the worst case. Add a per-client ratio that caps retries at 10 percent of requests, and the increase drops to about 1.1. On top of that comes the multiplication across layers, which I already calculated in the backoff article. A budget slows down each of those layers.
Two ways to build a retry budget
In practice you will meet two variants. Both work, yet they feel different.
Token bucket like in the AWS SDK
In standard mode, the AWS SDK uses a bucket of tokens. According to the current SDK reference, it holds 500 tokens. A retry after a transient error such as a timeout or a 5xx response costs 14 tokens, and a retry after throttling (HTTP 429) costs 5. Every success on the first attempt puts one token back, and a successful retry gets its own cost refunded. Once the bucket is empty, the SDK stops retrying and passes the error through.
Important context: these values belong to the new retry behavior AWS announced for 2026. Until it becomes the default, you switch it on with the environment variable AWS_NEW_RETRIES_2026=true. For the JavaScript SDK v3, the tracking issue names November 2026 as the earliest switch-over. The idea of the bucket, though, has been in the SDK since 2016.
Ratio like in the Google SRE book
The second variant keeps count: how many requests has this client made in the last time window, and how many of them were retries? A retry is allowed only as long as that share stays below a threshold, which is 10 percent at Google. The Envoy proxy knows the same principle as retry_budget, with 20 percent as the default and a floor of at least three concurrent retries.
This floor matters. Without it, a client with little traffic would hardly ever get a retry, because 10 percent of five requests is half a retry.
| Token bucket | Ratio | |
|---|---|---|
| Keeps track of | Fill level of a bucket | Requests and retries in the time window |
| Reacts to | Errors that pile up | Share of retries in the traffic |
| Recovers through | Successes that refill it | Time, as the window moves on |
| Strength | Allows a buffer for short error spikes | Scales with traffic automatically |
| Weakness | A fixed buffer does not fit every volume | Needs a floor for low traffic |
Result of the simulation from step 5: without a budget, the target receives roughly twice the load during the outage.
Build a retry budget in TypeScript
Now it gets practical. The example is a fictional bike rental whose Lambda function asks a partner how many bikes are available at a station. The partner is sometimes slow and sometimes overloaded, which is exactly the case where you want a retry budget.
Prerequisites
- Node.js 22 or newer (the example uses the built-in
fetchand thenode:testtest runner) - Basic TypeScript knowledge
- For the deployment: an AWS account and the Serverless Framework v4
Step 1: Create the project
mkdir retry-budget && cd retry-budget
npm init -y
npm install --save-dev typescript @types/node
mkdir src test
Next, adjust the package.json. The important part is "type": "module", so that Node loads the compiled files as ES modules. The tsconfig.json contains "types": ["node"], because TypeScript 7 no longer includes the Node types automatically.
{
"name": "retry-budget",
"version": "1.0.0",
"private": true,
"type": "module",
"scripts": {
"build": "tsc",
"test": "tsc && node --test dist/test/*.test.js",
"simulate": "tsc && node dist/src/simulate.js"
},
"devDependencies": {
"@types/node": "^22.20.5",
"typescript": "^7.0.2"
}
}
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"outDir": "dist",
"rootDir": ".",
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src", "test"]
}
Step 2: The two budgets
Both variants get the same interface. That way you can swap them later without touching the retry loop. The method tryAcquireRetry() is the core: before every retry it asks “may I?” and books the attempt if the answer is yes.
// src/retry-budget.ts
export type RetryKind = "transient" | "throttling";
/** Shared interface: may I try once more right now? */
export interface RetryBudget {
/** Call before every first attempt. */
recordRequest(): void;
/** Call before every retry. false = budget empty, pass the error on. */
tryAcquireRetry(kind?: RetryKind): boolean;
/** Call after every success; lastRetry = kind of the last retry, if there was one. */
recordSuccess(lastRetry?: RetryKind): void;
}
export interface TokenBudgetOptions {
capacity?: number; // size of the bucket
transientCost?: number; // cost of a retry after a timeout or 5xx
throttlingCost?: number; // cost of a retry after 429
successRefund?: number; // refund for a success on the first attempt
}
/**
* Token bucket like in the AWS SDK: retries cost tokens, successes refill.
* Once the bucket is empty, nothing is retried any more.
*/
export class TokenRetryBudget implements RetryBudget {
private tokens: number;
private readonly capacity: number;
private readonly transientCost: number;
private readonly throttlingCost: number;
private readonly successRefund: number;
constructor(options: TokenBudgetOptions = {}) {
this.capacity = options.capacity ?? 500;
this.transientCost = options.transientCost ?? 14;
this.throttlingCost = options.throttlingCost ?? 5;
this.successRefund = options.successRefund ?? 1;
this.tokens = this.capacity;
}
recordRequest(): void {
// The first attempt is free.
}
private cost(kind: RetryKind): number {
return kind === "throttling" ? this.throttlingCost : this.transientCost;
}
tryAcquireRetry(kind: RetryKind = "transient"): boolean {
const cost = this.cost(kind);
if (this.tokens < cost) return false;
this.tokens -= cost;
return true;
}
recordSuccess(lastRetry?: RetryKind): void {
// Successful retry: refund only its own cost, otherwise the small refund.
const refund = lastRetry ? this.cost(lastRetry) : this.successRefund;
this.tokens = Math.min(this.capacity, this.tokens + refund);
}
get available(): number {
return this.tokens;
}
}
export interface RatioBudgetOptions {
ratio?: number; // share of retries relative to requests, e.g. 0.1 = 10%
windowMs?: number; // observation window
minRetries?: number; // floor so that low traffic still gets retries
now?: () => number; // replaceable for tests
}
/**
* Ratio budget like in the Google SRE book: retries may make up at most
* a fixed share of the requests in the sliding window.
*/
export class RatioRetryBudget implements RetryBudget {
private readonly ratio: number;
private readonly windowMs: number;
private readonly minRetries: number;
private readonly now: () => number;
private readonly slotMs: number;
private readonly slots: { start: number; requests: number; retries: number }[] = [];
constructor(options: RatioBudgetOptions = {}) {
this.ratio = options.ratio ?? 0.1;
this.windowMs = options.windowMs ?? 10_000;
this.minRetries = options.minRetries ?? 3;
this.now = options.now ?? Date.now;
this.slotMs = Math.max(1, Math.floor(this.windowMs / 10));
}
private current() {
const t = this.now();
const start = t - (t % this.slotMs);
// Drop old slots that have fallen out of the window.
while (this.slots.length > 0 && this.slots[0].start <= t - this.windowMs) {
this.slots.shift();
}
let slot = this.slots[this.slots.length - 1];
if (!slot || slot.start !== start) {
slot = { start, requests: 0, retries: 0 };
this.slots.push(slot);
}
return slot;
}
recordRequest(): void {
this.current().requests++;
}
tryAcquireRetry(): boolean {
const slot = this.current();
const requests = this.slots.reduce((sum, s) => sum + s.requests, 0);
const retries = this.slots.reduce((sum, s) => sum + s.retries, 0);
const allowed = Math.max(this.minRetries, requests * this.ratio);
if (retries + 1 > allowed) return false;
slot.retries++;
return true;
}
recordSuccess(): void {
// The ratio budget needs no refund.
}
}
For the ratio budget, I split the window into ten slots. That gives you a sliding window with coarse resolution: old slots drop out at the back without you having to store every single request with a timestamp.
Step 3: The retry loop
The loop decides in this order: is the error retryable at all? Does this request have attempts left? Does the budget grant a retry? Only then does it wait with full jitter and try again. If the budget is empty, it throws its own error type. You can catch that further up on purpose.
// src/with-retries.ts
import { setTimeout as sleep } from "node:timers/promises";
import type { RetryBudget, RetryKind } from "./retry-budget.js";
/** Error with an HTTP status so we can tell retryable from final failures. */
export class HttpError extends Error {
constructor(public readonly status: number, message: string) {
super(message);
this.name = "HttpError";
}
}
export class RetryBudgetExhaustedError extends Error {
constructor(public readonly cause: unknown, public readonly attempts: number) {
super(`Retry budget exhausted after ${attempts} attempt(s)`);
this.name = "RetryBudgetExhaustedError";
}
}
export interface RetryOptions {
budget: RetryBudget;
maxAttempts?: number; // including the first attempt
baseDelayMs?: number;
maxDelayMs?: number;
signal?: AbortSignal;
random?: () => number; // replaceable for tests
}
/** Classifies an error: null = do not retry. */
export function classify(error: unknown): RetryKind | null {
if (error instanceof HttpError) {
if (error.status === 429) return "throttling";
if ([500, 502, 503, 504].includes(error.status)) return "transient";
return null; // 4xx except 429: a second attempt changes nothing
}
if (error instanceof Error && (error.name === "TimeoutError" || error.name === "TypeError")) {
return "transient"; // timeout or network error from fetch
}
return null;
}
export async function withRetries<T>(
operation: (attempt: number) => Promise<T>,
options: RetryOptions,
): Promise<T> {
const { budget, maxAttempts = 3, baseDelayMs = 50, maxDelayMs = 2_000, signal } = options;
const random = options.random ?? Math.random;
let lastRetry: RetryKind | undefined;
budget.recordRequest();
for (let attempt = 1; ; attempt++) {
try {
const result = await operation(attempt);
budget.recordSuccess(lastRetry);
return result;
} catch (error) {
const kind = classify(error);
if (kind === null || attempt >= maxAttempts) throw error;
if (!budget.tryAcquireRetry(kind)) throw new RetryBudgetExhaustedError(error, attempt);
lastRetry = kind;
// Full jitter: random wait between 0 and the exponential ceiling.
const ceiling = Math.min(maxDelayMs, baseDelayMs * 2 ** (attempt - 1));
await sleep(Math.floor(random() * ceiling), undefined, { signal });
}
}
}
The loop never retries errors with status 400, 401 or 404. A second attempt with the same broken request achieves nothing, it only costs budget.
🎬 Request flow with a retry budget – this animated diagram only runs on my site: watch it on sascha-ms.de
Step 4: Test
The tests check the behavior that matters: the bucket runs dry, the ratio sticks to 10 percent, a 503 is retried and a 400 is not. The time source is replaceable, so no test has to wait for real seconds.
// test/retry-budget.test.ts
import { test } from "node:test";
import assert from "node:assert/strict";
import { RatioRetryBudget, TokenRetryBudget } from "../src/retry-budget.js";
import { HttpError, RetryBudgetExhaustedError, withRetries } from "../src/with-retries.js";
test("token bucket: empty bucket blocks further retries", () => {
const budget = new TokenRetryBudget({ capacity: 30, transientCost: 14 });
assert.equal(budget.tryAcquireRetry("transient"), true);
assert.equal(budget.tryAcquireRetry("transient"), true);
assert.equal(budget.tryAcquireRetry("transient"), false); // only 2 tokens left
budget.recordSuccess("transient"); // successful retry refunds 14
assert.equal(budget.available, 16);
});
test("ratio: at most 10% retries, at least minRetries", () => {
let now = 0;
const budget = new RatioRetryBudget({ ratio: 0.1, windowMs: 10_000, minRetries: 3, now: () => now });
for (let i = 0; i < 50; i++) budget.recordRequest();
const granted = Array.from({ length: 10 }, () => budget.tryAcquireRetry()).filter(Boolean).length;
assert.equal(granted, 5); // 10% of 50
now = 11_000; // window has moved on
assert.equal(budget.tryAcquireRetry(), true); // floor applies
});
test("withRetries retries 503 and gives up when the budget is empty", async () => {
const budget = new TokenRetryBudget({ capacity: 14 });
let calls = 0;
await assert.rejects(
withRetries(async () => {
calls++;
throw new HttpError(503, "overloaded");
}, { budget, maxAttempts: 5, baseDelayMs: 1, random: () => 0 }),
RetryBudgetExhaustedError,
);
assert.equal(calls, 2); // first attempt + one retry, then the bucket is empty
});
test("withRetries does not retry 400", async () => {
const budget = new TokenRetryBudget();
let calls = 0;
await assert.rejects(
withRetries(async () => {
calls++;
throw new HttpError(400, "broken request");
}, { budget, baseDelayMs: 1 }),
HttpError,
);
assert.equal(calls, 1);
});
npm test
At the end of the output you should see pass 4 and fail 0. If you see Cannot find module instead, an import is usually missing the .js extension. With "module": "NodeNext" it is mandatory, even though the file is called .ts in your editor. If the compiler reports Cannot find name 'process', the "types": ["node"] entry is missing.
Step 5: Make the effect visible
Numbers are more convincing than claims. That is why the project includes a small simulation script. For 60 seconds it sends 100 new requests per second to a target that normally handles 150. From second 20 to 39, capacity drops to 60. It measures how many attempts arrive in total during this outage, relative to the new requests.
// src/simulate.ts
// A small thought experiment in code: what happens to the load when the target is overloaded?
import { RatioRetryBudget, TokenRetryBudget, type RetryBudget } from "./retry-budget.js";
const TICKS = 60; // one second per tick
const NEW_PER_TICK = 100; // new requests per second
const MAX_ATTEMPTS = 3;
const capacity = (t: number) => (t >= 20 && t < 40 ? 60 : 150); // outage from second 20 to 39
// Deterministic randomness so every run prints the same numbers.
function seeded(seed: number) {
return () => ((seed = (seed * 1_103_515_245 + 12_345) % 2_147_483_648) / 2_147_483_648);
}
function run(name: string, makeBudget: (clock: () => number) => RetryBudget | null) {
const random = seeded(42);
let tick = 0;
const budget = makeBudget(() => tick * 1000);
let pending: number[] = []; // attempt counters of the retries for the next tick
let attemptsInOutage = 0;
let newInOutage = 0;
let failedForGood = 0;
for (tick = 0; tick < TICKS; tick++) {
const fresh = Array.from({ length: NEW_PER_TICK }, () => 1);
fresh.forEach(() => budget?.recordRequest());
const offered = [...fresh, ...pending];
pending = [];
// The server serves randomly chosen requests up to its capacity.
for (let i = offered.length - 1; i > 0; i--) {
const j = Math.floor(random() * (i + 1));
[offered[i], offered[j]] = [offered[j], offered[i]];
}
const cap = capacity(tick);
if (tick >= 20 && tick < 40) {
attemptsInOutage += offered.length;
newInOutage += NEW_PER_TICK;
}
offered.forEach((attempt, i) => {
if (i < cap) {
budget?.recordSuccess(attempt > 1 ? "throttling" : undefined);
return;
}
const mayRetry = attempt < MAX_ATTEMPTS && (budget ? budget.tryAcquireRetry("throttling") : true);
if (mayRetry) pending.push(attempt + 1);
else failedForGood++;
});
}
const factor = (attemptsInOutage / newInOutage).toFixed(2);
console.log(`${name.padEnd(22)} load during outage: ${factor}x failed for good: ${failedForGood}`);
}
run("no budget", () => null);
run("token bucket (500/5)", () => new TokenRetryBudget({ capacity: 500 }));
run("10% ratio", (now) => new RatioRetryBudget({ ratio: 0.1, windowMs: 10_000, now }));
npm run simulate
no budget load during outage: 2.12x failed for good: 687
token bucket (500/5) load during outage: 1.25x failed for good: 780
10% ratio load during outage: 1.10x failed for good: 800
Without a budget, a good twice the load arrives during the outage. With the ratio, it is only 10 percent more, which is almost exactly the value from the SRE book. The right column deserves honesty as well: without a budget, slightly fewer requests fail for good in the end. That is due to the simplified model, in which the server stubbornly handles 60 requests per second despite the overload. Real systems often get slower under extra load and then handle less, not the same amount. That effect is exactly what the budget prevents.
Use the retry budget in Lambda
In Lambda, the budget belongs outside the handler. Code at module level runs once per execution environment and stays in place for all following invocations in that environment. That way the budget counts across many invocations, not just within a single one.
// src/handler.ts
import { RatioRetryBudget } from "./retry-budget.js";
import { HttpError, RetryBudgetExhaustedError, withRetries } from "./with-retries.js";
// Outside the handler: the budget lives as long as the execution environment.
const budget = new RatioRetryBudget({ ratio: 0.1, windowMs: 10_000, minRetries: 1 }); // keep it small, see below
const PARTNER_URL = process.env.PARTNER_URL ?? "https://partner.example.com";
// Minimal slice of the HTTP API event (payload 2.0)
interface StationEvent {
pathParameters?: { stationId?: string };
}
export async function handler(event: StationEvent) {
const stationId = event.pathParameters?.stationId;
if (!stationId) return { statusCode: 400, body: "stationId missing" };
try {
const bikes = await withRetries(
async () => {
const res = await fetch(`${PARTNER_URL}/stations/${encodeURIComponent(stationId)}/bikes`, {
signal: AbortSignal.timeout(800), // every attempt has its own timeout
});
if (!res.ok) throw new HttpError(res.status, `Partner API responded ${res.status}`);
return (await res.json()) as { available: number };
},
{ budget, maxAttempts: 3, baseDelayMs: 100, maxDelayMs: 1_000 },
);
return { statusCode: 200, body: JSON.stringify(bikes) };
} catch (error) {
if (error instanceof RetryBudgetExhaustedError) {
// Partner is overloaded: answer fast and honestly instead of pushing harder.
console.warn(JSON.stringify({ msg: "retry budget exhausted", attempts: error.attempts }));
return { statusCode: 503, headers: { "Retry-After": "5" }, body: "Partner currently unavailable" };
}
throw error;
}
}
Every attempt has its own timeout of 800 milliseconds. Three attempts plus waiting time therefore stay well below the Lambda timeout of six seconds. If the budget is empty, the function answers immediately with 503 and Retry-After. That is more honest than a timeout after 30 seconds.
# serverless.yml
service: radwerk-bikes
frameworkVersion: "4"
provider:
name: aws
runtime: nodejs22.x
region: eu-central-1
memorySize: 256
timeout: 6 # 3 attempts of 800 ms plus waits fit easily
environment:
PARTNER_URL: https://partner.example.com
functions:
stationBikes:
handler: src/handler.handler
events:
- httpApi:
path: /stations/{stationId}/bikes
method: get
npx serverless deploy
curl -i https://<your-api-id>.execute-api.eu-central-1.amazonaws.com/stations/nord-3/bikes
The Serverless Framework v4 builds TypeScript itself with esbuild, so you need no further configuration. Because the partner in the example is just partner.example.com, you will get a 503 or an error back after the deployment. For a real test, put your own test API into PARTNER_URL. You only pay per invocation, so a few test requests stay in the range of fractions of a cent.
One budget per execution environment
Here is the catch I do not want to hide in any example. A classic Lambda execution environment only handles one invocation at a time. If you have 50 concurrent invocations, you have 50 environments and therefore 50 independent budgets. And because requests take longer when the partner is slow, Lambda needs more environments during an outage, not fewer. Each new one brings a fresh, full budget.
Three things help against this:
-
Small floor: Set
minRetrieslow in Lambda, for example to 1. A single environment rarely sees enough requests for the 10 percent rule to kick in by itself. - Reserved concurrency: A fixed limit for the function also caps the number of budgets.
- Shared state: If the limit really has to apply globally, you need a shared counter, for example in DynamoDB. That costs latency on every invocation. In that case a circuit breaker with state in DynamoDB is often the better fit.
Lambda Managed Instances are a different story: there, one environment handles several invocations at once, and the module-level budget really is shared.
Common mistakes with a retry budget
Budgeting twice. For calls through the AWS SDK you do not need your own retry budget, the SDK already has one. If you still wrap your own loop around an SDK call, the attempts multiply again. Your budget belongs in front of your own HTTP calls to third-party systems.
One budget for all targets. When one partner fails, it should not use up the retries for another, healthy partner. Create a separate budget per target.
Hiding the empty bucket. An exhausted budget is a signal. Log it in a structured way and put a metric filter on it. Then you will see the partner’s outage before your users report it.
Retrying again below the budget. If your function answers with 503, the caller may retry on its own. So check whether a layer above you retries as well. The SRE book recommends retrying only in the layer directly above the overloaded one.
My take
I consider a retry budget one of the most underrated tools in distributed systems. The two core files have fewer than 200 lines including comments, the budget costs practically nothing at runtime, and it changes behavior in exactly the situation that counts. Still, most examples and tutorials only show maxAttempts: 3 and backoff, and a budget rarely shows up.
Where I disagree: a budget is no replacement for a circuit breaker. The budget makes retries scarce, while the breaker also stops first attempts. In Lambda, the breaker with shared state is often the stronger tool, because it works across all environments. My suggestion is a combination: a local budget for the small glitches, a global breaker for the big outages. And if you only build one of them, start with the budget. You will be done faster.
FAQ
What is a retry budget?
A retry budget limits the total number of retries of a client, for example to 10 percent of requests or through a token bucket. Single errors may be retried, but with many errors the client passes them on immediately.
What is the difference between a retry budget and a circuit breaker?
A retry budget only limits retries, while first attempts keep going. A circuit breaker blocks all calls to a target for a while after too many errors. The two complement each other.
Does the AWS SDK for JavaScript have a retry budget?
Yes. In standard mode, the SDK uses a token bucket. With the new 2026 retry behavior it holds 500 tokens, and a retry costs 14 tokens for transient errors and 5 for throttling.
How big should a retry budget be?
A good start is 10 percent of requests in a ten-second window plus a small floor. Then watch how often the budget runs dry and adjust it.
Your next step
Find a place in your code where you call a third-party system over HTTP and have your own retry loop. Put a budget in front of it and log when it runs dry. That is all you need to get started. And now my question to you: does only one layer retry in your system, or have you ever counted how many there really are?
Sources
- AWS SDKs and Tools Reference Guide: Retry behavior
- aws-sdk-js-v3, issue 8037: new retry behavior 2026
- Google SRE Book: Handling Overload
- Marc Brooker: Timeouts, retries, and backoff with jitter (Amazon Builders’ Library)
- Envoy: RetryBudget in the circuit breaker configuration
- AWS Lambda: Execution environment lifecycle
- Tested with Node.js 22.23 and TypeScript 7.0.2 (4 tests with
node:test, the simulation, and the handler against a local HTTP server). Theserverless.ymlwas validated as YAML, not deployed.
This post was created with AI assistance.
👉 Next step: Curious what your own Lambda functions cost? Try my Lambda cost calculator – it runs right in your browser.
📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).
Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en

Top comments (0)