Stop paying for LLM calls you already know will fail: Budget Guard + circuit breakers in TypeScript

Nitin Kaushal — Thu, 22 Oct 2026 13:08:07 +0000

Every Node.js app I've worked on ends up with the same hand-rolled code around its dependencies: a for loop with await sleep(), a Promise.race for timeouts, and a comment that says // TODO: circuit breaker. Then LLM providers showed up and added two new failure modes: 429s and 60-second hangs at peak hours, and a bill you only notice when the invoice arrives.

So I built Resili, a TypeScript-first resilience toolkit for Node.js. It wraps an async function (not a URL, not a request object) in typed policies, and it has separate packages for fetch/axios/undici and for OpenAI, Anthropic and Gemini.

Status, upfront: Resili is a release candidate (@resili/core@0.2.0-rc.2 on the npm next tag). It is not 1.0. Pin exact versions.

The core idea: one pipeline around one function

import { resili } from "@resili/core";

const users = resili((id: string) => fetch(`https://api.example.com/users/${id}`))
  .timeout({ perAttemptMs: 1_000 })
  .retry({ maxAttempts: 3, backoff: "exponential", jitter: "none" })
  .circuitBreaker({ minimumThroughput: 10, failureRateThreshold: 50 })
  .build();

const response = await users.call("42");

A few deliberate choices:

You opt in to each policy. A policy is installed only when you configure it.
Invalid options throw when the client is built, not at 3 a.m. on the first failure.
Deterministic order. When enabled, policies run as Fallback â†’ Cache â†’ Retry â†’ Circuit Breaker â†’ Timeout â†’ Dedupe â†’ Hedge â†’ Rate Limiter â†’ Bulkhead â†’ Operation. Retry outside timeout means each attempt gets its own time budget.
No surprise retries. A plain Error is not retried unless retryOn returns true, and HTTP status codes aren't failures unless you opt in (a returned 503 is still a successful function result).
failureRateThreshold is a percentage. 50 means 50%, not 0.5.
Core has zero runtime dependencies.

Circuit breaker: stop hammering a dead dependency

Retries help with blips. When a dependency is actually down, retries make it worse. Here is the circuit-breaker half of the demo in the GIF (from examples/stackblitz/demo.mjs):

import { CircuitOpenError, resili } from "@resili/core";

const guarded = resili(async () => {
  throw new Error("upstream is down");
})
  .circuitBreaker({
    minimumThroughput: 1,
    failureRateThreshold: 50,
    resetTimeoutMs: 60_000,
  })
  .build();

try { await guarded.call(); } catch {}          // real failure, circuit opens

try {
  await guarded.call();                          // never reaches the function
} catch (error) {
  if (error instanceof CircuitOpenError) console.log(error.code); // ERR_CIRCUIT_OPEN
}

The second call fails fast with ERR_CIRCUIT_OPEN without touching the dependency. You can watch retries happen too: recovered.on("RetryStarted", (event) => console.log(event.delayMs, event.attemptNumber)).

Try it in the browser, no API key: Open in StackBlitz, then run node index.mjs.

Same policies, but for LLM calls

@resili/llm is not an AI SDK. It doesn't read API keys or construct vendor clients. You pass in your own client, and the adapter wraps it:

import OpenAI from "openai";
import { createLlmClient } from "@resili/llm";
import { createOpenAiProvider } from "@resili/llm-openai";

const llm = createLlmClient({
  provider: createOpenAiProvider({ client: new OpenAI(), model: "gpt-4.1-mini" }),
  model: "gpt-4.1-mini",
  timeout: { perAttemptMs: 30_000 },
  retry: { maxAttempts: 3, baseDelayMs: 500, jitter: "none" },
});

const result = await llm.generate({ input: "Explain backpressure in one sentence." });
console.log(result.response.content);

Two details I care about:

Adapters disable the SDK's own retries, so you don't get retries stacked on top of retries (3 Ã— 3 = 9 calls).
Streaming is pull-through. The provider call starts when you iterate, and the first non-empty text delta commits the stream, so a later retry can't append a second answer to the one your user is already reading.

Anthropic (@resili/llm-anthropic) and Gemini (@resili/llm-gemini) work the same way.

Budget Guard: reject expensive calls before they happen

This is the feature I wanted most. You give createLlmClient a price table and a budget. Resili ships no model prices, so you own the numbers. From examples/budget-guard/example.mjs:

import {
  LlmBudgetExceededError,
  createLlmClient,
  createPricingResolver,
} from "@resili/llm";

const llm = createLlmClient({
  provider, // any provider, e.g. createOpenAiProvider(...) or defineProvider(...)
  model: "example-small",
  pricing: createPricingResolver([
    {
      provider: "example",
      model: "example-small",
      inputPerMillionTokensUsd: 1,
      outputPerMillionTokensUsd: 1,
    },
  ]),
  budget: { maxCostPerRequestUsd: 0.01 },
});

try {
  await llm.generate({
    input: "This text is not sent to a vendor.",
    estimatedInputTokens: 20_000,
    estimatedOutputTokens: 20_000,
  });
} catch (error) {
  if (error instanceof LlmBudgetExceededError) {
    console.log(error.limitKind, error.attemptedMicroUsd, error.limitMicroUsd);
  }
}

The provider is never called. Run it offline with node examples/budget-guard/example.mjs.

To be precise about what it guarantees:

maxCostPerRequestUsd is checked before the call, using estimated tokens. It's not a hard ceiling: real output can exceed the estimate, and the actual cost is recorded afterward.
maxAccumulatedCostUsd limits recorded spend plus in-flight reservations for a scope, which is how you catch the runaway agent loop.
The default ledger is process-local memory. Multi-instance deployments need their own shared store.

How it compares (honestly)

	Resili	cockatiel	opossum	p-retry
Retry	âœ…	âœ…	â€”	âœ…
Timeout	policy	policy	breaker option	`p-timeout`
Circuit breaker	âœ…	âœ…	âœ…	â€”
Bulkhead	âœ…	âœ…	â€”	â€”
Rate limiting	âœ…	â€”	â€”	â€”
HTTP adapters	fetch/axios/undici	â€”	â€”	â€”
LLM adapters + budget	OpenAI/Anthropic/Gemini	â€”	â€”	â€”

(From each project's own docs as of 2026-10-21. Cockatiel and opossum also document fallback.)

When to pick something else:

You only need retries? p-retry is tiny and battle-tested.
You only need a circuit breaker with a mature ecosystem? opossum.
You want a proven policy toolkit and don't care about HTTP/LLM adapters? cockatiel is excellent and was a big inspiration.

Resili makes sense when you want one typed pipeline across HTTP and LLM calls, plus cost controls that run before the request.

Try it and tell me what's wrong

npm install @resili/core@0.2.0-rc.2 @resili/llm@0.1.0-rc.2 @resili/llm-openai@0.1.0-rc.2 openai

â Repo: https://github.com/nkcodedev/resili
ðŸ“š Docs: https://resili.nkcode.dev/index.html
â–¶ï¸ Playground: StackBlitz

I'd especially like feedback on the policy-ordering defaults, the Budget Guard API, and which provider or adapter should come next. Issues and PRs are welcome.

â€” Nitin Kaushal, NK CODE DEV Â· Instagram @nk.cloud.dev

Stop writing retry logic everywhere.

Nitin Kaushal — Thu, 02 Jul 2026 05:48:04 +0000

Every service ends up with:

retry
timeout
circuit breaker
fallback

…implemented differently every time.

I built Resili — a TypeScript-first resilience toolkit 👇

import { resili } from "@resili/core";

const client = resili(fetchUser)
  .timeout({ milliseconds: 3000 })
  .retry({ maxAttempts: 3 })
  .circuitBreaker({ minimumThroughput: 10 })
  .build();

TypeScript-first • Fluent API • Production-ready

📦 https://github.com/nkcodedev/resili

Would love feedback from backend/dev folks 👇

DEV Community: Nitin Kaushal