LLM API Spending Limits: Why 20 Agents Under Budget Still Blow the Cap (with Python Fix)

Twenty agents start at the same moment. Each one checks the budget, sees $4.10 spent against a $5.00 cap, and sends a request that costs about $0.15. All twenty checks pass. Together they add $3.00, so you end at $7.10 on a $5.00 cap.

Nothing crashed and nothing threw an error. The guard read a number that was already out of date by the time the request went out, and every agent read that same number.

Most homegrown LLM budget code has this bug. The controls teams normally rely on don't catch it either. Below is how each control fails, followed by code that does stop the spend.

The best-measured teams overrun the most

In February 2026, Sapio Research surveyed 500 finance leaders at large US and UK enterprises for DoiT. 79% said they'd had AI cost overruns in the previous 12 months. The odd result is at the top. Among organizations that rated their FinOps "very mature or leading edge", 89% overran, and their mean overspend was 30.9%. That's the worst of any segment in the study.

The survey doesn't say why. My guess is that good dashboards find overruns faster, and by the time you've found one, the money is already spent. These are large enterprises, not solo builders. But they use the same controls an indie dev sets up first: a dashboard and an alert.

Source: DoiT, "AI spending survey".

Four controls and what each one misses

Control	What it does	What it fails to stop
Budget alert	Sends a notification at a threshold	All of it. Spend keeps running until someone reads the message
Provider hard limit	Rejects requests once the org or project cap is reached	One user or tenant draining a shared account, and overshoot while enforcement catches up
Rate limit (RPM/TPM)	Limits throughput	Total spend. A loop running just under its RPM limit can still run all night
Pre-flight reservation	Reserves the worst-case cost before the call and refuses calls that don't fit	Only its own estimation error, which max_tokens bounds

The provider docs say the first row plainly. Google's billing docs say an alerts-only budget "doesn't automatically cap" usage or spending. Google's real spend caps are configured per project, cover only services that support them, and keep those services paused until you lift the cap by hand. OpenAI's spend-limit guide says "Spend alerts do not enforce a cap" and treats alerts and hard limits as two separate controls.

The second row is a real cap. OpenAI fails requests with organization_spend_limit_exceeded once you hit the limit. But provider limits work at the account or workspace level and take time to enforce. They don't know your customers exist. If you run a SaaS, one user's runaway loop uses up the org cap and blocks everyone else.

Sources: Google Cloud budgets, OpenAI spend limits.

The naive guard

spent = 0.0
CAP = 5.00

def call(client, **kw):
    global spent
    if spent >= CAP:
        raise BudgetExceeded()
    resp = client.chat.completions.create(**kw)  # 1-30 seconds in flight
    spent += cost_of(resp.usage)
    return resp

This is check-then-act with a network round trip in between. Every thread that arrives during that round trip sees the old spent value. Under concurrency the cap has no effect, and the more parallel agents you run, the more you overshoot.

Fix: reserve first, settle after

Before the call, work out the worst-case cost: estimated input tokens plus max_tokens of output. Reserve that amount atomically. If it doesn't fit, raise an error, and the provider never sees the request. After the call, release the reservation and record the actual cost.

The code counts integer micro-dollars instead of floats. With floats, 4.10 + 0.15 * 6 can land just above 5.0 and block a call that fits.

import math
import threading

# USD per 1M tokens. Copy these from your provider's pricing page.
# Handy unit fact: $X per 1M tokens == X micro-dollars per token.
PRICES = {
    "your-model": {"in": 0.0, "out": 0.0},
}

class BudgetExceeded(Exception):
    pass

def cost_micros(model, tokens_in, tokens_out):
    p = PRICES[model]
    return math.ceil(tokens_in * p["in"] + tokens_out * p["out"])

class Budget:
    def __init__(self, cap_usd):
        self.cap = round(cap_usd * 1_000_000)
        self.committed = 0
        self.reserved = 0
        self._lock = threading.Lock()

    def reserve(self, amount):
        with self._lock:
            if self.committed + self.reserved + amount > self.cap:
                raise BudgetExceeded(f"need {amount}µ$, cap {self.cap}µ$")
            self.reserved += amount

    def settle(self, reserved, actual):
        with self._lock:
            self.reserved -= reserved
            self.committed += actual

class CappedOpenAI:
    def __init__(self, client, budget):
        self.client = client
        self.budget = budget

    def chat(self, *, model, messages, max_tokens, **kw):
        text = "".join(m["content"] for m in messages
                       if isinstance(m.get("content"), str))
        est_in = len(text) // 3 + 50 * len(messages)  # pessimistic on purpose
        worst = cost_micros(model, est_in, max_tokens)

        self.budget.reserve(worst)  # raises before any network I/O
        actual = worst              # failed mid-flight? assume the worst
        try:
            resp = self.client.chat.completions.create(
                model=model, messages=messages, max_tokens=max_tokens, **kw
            )
            u = resp.usage
            actual = cost_micros(model, u.prompt_tokens, u.completion_tokens)
            return resp
        finally:
            self.budget.settle(worst, actual)

Two details matter here.

First, max_tokens is a required argument. Without an output limit there's no upper bound on cost, so there's nothing to reserve. On OpenAI models that take max_completion_tokens, pass that instead. Reasoning tokens count toward it, so the bound still holds.

Second, a call that fails is charged at the worst case. That's deliberately conservative: a request that times out may still have been billed. If you'd rather be generous, settle known 4xx errors at zero.

The Anthropic SDK already requires max_tokens, so only the inner call and the usage field names change:

resp = self.client.messages.create(
    model=model, messages=messages, max_tokens=max_tokens, **kw
)
actual = cost_micros(model, resp.usage.input_tokens, resp.usage.output_tokens)

Here's the twenty-agent scenario again, with a fake $0.15 call:

from concurrent.futures import ThreadPoolExecutor

budget = Budget(cap_usd=5.00)
budget.committed = 4_100_000  # $4.10 already spent

def attempt(_):
    try:
        budget.reserve(150_000)  # $0.15
        return "ok"
    except BudgetExceeded:
        return "blocked"

with ThreadPoolExecutor(20) as ex:
    results = list(ex.map(attempt, range(20)))

print(results.count("ok"), results.count("blocked"))  # 6 14

Six calls fit into the $0.90 that's left. The other fourteen are blocked before they reach the network. The run ends at $5.00 instead of $7.10.

More than one process: move the lock to Redis

A threading.Lock only protects one process. With several workers, pods, or a queue consumer fleet, do the check and the increment in one Lua script so they happen together:

-- KEYS[1] = budget key, ARGV[1] = cap, ARGV[2] = amount (micro-dollars)
local used = tonumber(redis.call('GET', KEYS[1]) or '0')
if used + tonumber(ARGV[2]) > tonumber(ARGV[1]) then
  return 0
end
redis.call('INCRBY', KEYS[1], ARGV[2])
return 1

Call it with r.eval(LUA, 1, key, cap, worst). To settle, run INCRBY key (actual - worst), which is usually a negative number. Use one key per tenant and period, such as budget:{user_id}:2026-10. That gives you the per-customer cap a provider can't provide.

If you'd rather not maintain this

I packaged this pattern as baar-core, an open-source Python library. It does the atomic reservation and returns a 402 before the provider is contacted. It's pip install baar-core. It's free because cost-control code should be something you can read before you trust it with your bill.

For teams that need per-user budgets across a product, noburn.dev is built on top of it. It does the same pre-flight enforcement and blocks a user's API calls before they fire once that user goes over budget.

The library matters less than where the check sits. An alert reports the overrun after it's happened. A reservation made before the request goes out is what stops it.

Does anything in your stack refuse an LLM call before it's sent, or does all of it report after?

---

Originally published at https://www.robatdasorvi.com/chapters/building/why-i-built-hard-spending-caps-for-llm-apis