7 things I learned trying to stop LLM API bills from silently exploding

kimbeomgyu — Sun, 12 Jul 2026 15:35:45 +0000

My first real LLM bill surprise wasn't dramatic. No infinite loop, no viral spike. A retry policy I'd written months earlier met a flaky endpoint, and a background job quietly re-sent the same long prompt all night. The bill was just... 40x normal. Nothing "failed", so nothing alerted.

I've spent the last few weeks building a hard spending cap for LLM calls, and most of what I learned wasn't about caps at all. It was about how many small boring ways money leaks when the meter only exists on the provider's side. Here's the list I wish someone had handed me.

1. Billing dashboards are rear-view mirrors

Every provider dashboard answers "how much did you spend?" hours later, after aggregation. Nobody answers "how much are you spending right now, from which feature?" By the time the dashboard shows the leak, the leak already happened. If you take one thing from this post: put the meter inside your process, at the call site, where it can act before the request instead of reporting after it.

2. Total spend is a useless number

"$120 today" tells you nothing actionable. The question that matters is which feature is leaking. The fix is one tag per call (feature: "chat", feature: "summarizer") and a per-feature breakdown. The first time I ran a breakdown on a real app, a background enrichment job I'd forgotten about turned out to be 60% of spend. For me, the per-feature breakdown was the entire diagnosis.

3. Every provider reports usage differently, and they all lie a little

OpenAI gives you prompt_tokens and completion_tokens. Anthropic gives input_tokens/output_tokens. Gemini nests everything in usageMetadata. Bedrock and Cohere have their own shapes. Then it gets worse: cached input tokens bill at a different rate, and reasoning tokens are billed as output even though you never see them. Two providers even disagree on whether reasoning tokens are counted inside the output number at all. xAI and Gemini report them outside it, so if you naively bill the output count you're undercounting exactly when reasoning models get expensive. Budget a week of reading API docs that contradict the actual responses.

4. Streaming hides the receipt until the very end

With stream: true, usage arrives (if at all) in the final chunk. OpenAI won't even send it unless you pass stream_options: { include_usage: true }. Anthropic splits it across message_start and message_delta events you have to assemble, and the delta is cumulative, so adding instead of replacing double-counts. Gemini sends a running usageMetadata where only the last value is real. Any cost tracking that ignores streaming is blind to the calls most modern apps actually make.

5. A cap that checks after the call is a receipt, not a guard

The naive cap goes: call, add to a counter, compare. It always overshoots by one call. Concurrency makes it much worse. Ten parallel requests all read the same counter, all pass the check, all land. I only really fixed this with an atomic reserve-then-settle: estimate the cost, reserve it against the cap before the call (one atomic operation, Lua script if the counter lives in Redis), settle the difference after. If you run multiple workers the counter has to live in shared storage anyway, otherwise each worker politely enforces its own private budget while the fleet blows through ten of them.

6. Retry storms are the leak nobody instruments

The single most expensive bug class I found wasn't prompts. It was retries: exponential backoff on a 429 while the request is actually succeeding on the provider's side, queue re-drives, at-least-once delivery re-running completed jobs. Track consecutive failures per feature and alert when the streak gets weird. A hard cap turns this failure mode from "unbounded bill" into "capped error you notice in the morning", and that trade is almost always correct for background workloads.

7. Caps and traces are two sides of the same thing

A reader of my first post put it better than I had: cost caps and execution traces are both answers to "what is my agent doing when I'm not looking?" The cap is the brake. A per-call spend event (feature, model, usd, running total) piped into the logs you already have is the gauge. You want both, because a brake without a gauge just means you get stopped without knowing why.

I ended up packaging all of this into a small MIT-licensed library called budget-guard: zero-dependency hard caps that hold under concurrent workers, per-feature attribution, streaming usage, and adapters for Vercel AI SDK, LangChain.js, LlamaIndex.TS and Mastra. But honestly, even if you never install anything: tag your calls, meter before the request, and go look at what your retries did last night.

I built a free circuit breaker for your LLM API bill

kimbeomgyu — Sun, 28 Jun 2026 10:39:29 +0000

Update: a few things landed since this was published. v0.2 added a Redis shared store so multiple workers can share one cap. After that came an onSpend hook (per-call cost events), streaming usage for OpenAI/Anthropic/Gemini, monthly caps with timezones, and adapters for Vercel AI SDK, LangChain.js, LlamaIndex.TS and Mastra. As of v0.6 the caps are also concurrency-safe (atomic reservation) and there's a built-in pre-call estimator, so over-budget calls get blocked before they're sent.

We've all seen the screenshot: someone leaves an agent running overnight, a retry loop goes sideways, and they wake up to a bill that's 10x what they expected — "$40 from a $5 task." The scary part isn't the money, it's that nothing stopped it. You find out when the invoice lands.

I wanted the dumbest possible fix: a hard cap that just blocks the next call when you're over budget. Not a dashboard I have to remember to check, not a gateway I have to deploy. So I built budget-guard — a tiny drop-in wrapper for the OpenAI/Anthropic clients.

The whole idea in 3 lines

import { guard } from 'budget-guard';

const ai = guard(openai.chat.completions, { project: 'my-app', dailyCapUSD: 50 });

await ai.create({ model: 'gpt-4o', messages }, { feature: 'summarize' });

Once today's spend for my-app crosses $50, the next create() throws BudgetExceededError before it bills you. A runaway loop dies at the breaker instead of draining your account. It stays out of your critical path — your calls still go straight to the provider; budget-guard just meters the usage and trips when you're over.

Bonus: where did the money go?

It tags spend per feature, so you can finally answer "what cost that?":

import { spendReport } from 'budget-guard';
spendReport('my-app'); // → { summarize: 2.41, chat: 0.88 }  (today, USD)

It auto-detects OpenAI (prompt_tokens/completion_tokens) and Anthropic (input_tokens/output_tokens) usage shapes; for anything else you pass a one-line extractor.

Being honest about v0.1

I'd rather tell you the limits than have you find them on an invoice:

In-memory, per-process. Great for a single script, agent, or worker. Run multiple instances and the cap is per-instance and resets on restart. A shared (Redis) store is the obvious next step.
Enforced on the next call. No pre-call token estimation yet, so one call can overshoot before the breaker trips.
Prices are a hand-maintained table. PRs welcome to keep them current.

So today it's best for solo devs, side projects, and single-process agents — exactly the people most likely to get surprised by a bill.

Try it / break it

npm i budget-guard

MIT, no runtime deps, TypeScript. Repo: https://github.com/kimbeomgyu/budget-guard

I'd love feedback on two things: should it block by default, or warn + callback? And if you run a fleet, would a Redis-backed shared cap actually get used, or do you already handle this at a gateway layer? Issues and PRs very welcome.

DEV Community: kimbeomgyu