Why Budget Alerts Never Stop Runaway LLM Spend

An "84% of monthly budget used" email landed on a Tuesday. I read it. I forwarded it to myself with a note that said watch this. Then I let the batch job run overnight anyway, and by Thursday morning the account had burned $1,900 against a $600 cap.

Nothing malfunctioned. The alerting pipeline worked exactly as specified, delivered on time, to the right person, who understood it. That's the uncomfortable part, and it's not a story about discipline.

A budget warning is a scarcity signal, and scarcity signals don't produce caution

Telling someone a resource is running low is one of the most-studied moves in behavioral psychology. The studied outcome is not restraint. It's an increase in perceived value and a pull toward consuming the remainder.

Worchel, Lee, and Adewole ran the cleanest version of this in 1975, published in the Journal of Personality and Social Psychology, with 200 participants. Same cookie, same room, same everything, handed over from either a jar holding ten or a jar holding two. The two-cookie jar produced higher ratings on value and attractiveness. Then they added a disclosure condition: some participants were explicitly told the supply had dropped because other people wanted them. Being told the mechanism didn't cancel the effect. That group rated the cookie highest of all.

Now read your dashboard banner again. "You have 16% of your budget remaining" is structurally identical to two cookies in a jar. It's a depletion cue with a full explanation attached, and the explanation is not a brake. You know the number is a threshold someone configured. You know the cap is arbitrary. Knowing changes what you can say about the banner. It does not change what you do next, which is usually to run the job.

What a soft limit looks like in code

Here's the version almost everyone writes first, in some dialect:

spent = usage_store.month_to_date(user_id)

if spent > BUDGET * 0.9:
    log.warning("user %s at %.0f%% of budget", user_id, spent / BUDGET * 100)
    metrics.incr("budget.near_limit")

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,
    messages=messages,
)

usage_store.record(user_id, response.usage)  # ledger updates *after*

Every line of that is defensible in review. It still fails, in three specific ways.

The ledger is always behind by exactly the calls that matter. record() runs after the response returns. During a burst, the spend you most need to see is the spend that hasn't landed yet.

Concurrency reads the same stale number. Twelve workers pull month_to_date inside the same second, all twelve see 92%, all twelve pass, all twelve fire. No single request overshot the cap. Collectively they cleared it by 4x.

Agent loops move faster than any aggregator. A tool-calling agent that hits a malformed schema and retries will happily make forty calls in ninety seconds. If your usage rollup runs on a one-minute cron, the loop is finished before the graph even flinches. My $1,900 was mostly this: a retry loop with no ceiling, feeding a context window that grew on every pass.

The common thread: that if statement doesn't gate anything. It observes and then gets out of the way. It's a very expensive log line.

Reserve before you spend

The fix is boring and it's the same one databases have used forever. Don't check the balance, hold against it.

class BudgetExceeded(Exception):
    status_code = 402


def reserve(user_id: str, estimated_cost: float) -> Reservation:
    with db.transaction():
        row = db.query(
            "SELECT spent, reserved, cap FROM budgets "
            "WHERE user_id = %s FOR UPDATE",
            user_id,
        ).one()

        if row.spent + row.reserved + estimated_cost > row.cap:
            raise BudgetExceeded(
                f"{user_id}: {row.spent + row.reserved:.2f} committed, "
                f"needs {estimated_cost:.2f}, cap {row.cap:.2f}"
            )

        db.execute(
            "UPDATE budgets SET reserved = reserved + %s WHERE user_id = %s",
            estimated_cost, user_id,
        )
    return Reservation(user_id, estimated_cost)

The call site becomes a hold, then a settle:

estimate = price_of(model, count_tokens(messages), max_tokens)
hold = reserve(user_id, estimate)   # raises 402 before any network I/O

try:
    response = client.messages.create(...)
    hold.commit(actual_cost(response.usage))
except Exception:
    hold.release()
    raise

Two details carry the weight. FOR UPDATE means those twelve concurrent workers serialize instead of all reading the same stale total, which kills the race that soft checks can't see. And the estimate is an upper bound computed from input tokens plus max_tokens, so you reserve the worst case and refund the difference on commit. Over-reserving is a rounding error. Under-reserving is how you get a Thursday like mine.

Most of the annoying parts here are already solved. baar-core is an open-source Python library that wraps this into a decorator around your LLM client: pip install baar-core, set a cap, and it raises a 402 before the provider is ever contacted rather than after the tokens are billed. Atomic reservation is the piece I'd have gotten wrong on my own, and it's the piece that stops concurrent calls from jointly clearing a limit that neither one individually exceeded. noburn.dev is what we built on top of it for teams: the same pre-flight enforcement blocking the API call before it fires when a user is over budget, plus per-user caps and the ledger view.

Three rules that survive contact with a real incident

Enforce at one boundary. A single wrapper around your client, not if statements scattered across seven call sites where the eighth one, added last month by someone in a hurry, has none.

Reserve, don't observe. If your limit reads a number that a different process is responsible for updating, it isn't a limit. It's a lagging indicator wearing a limit's clothes.

Fail typed and loud. Return a real 402 with a machine-readable reason so callers, retry middleware, and agent loops can all distinguish "out of budget" from "transient 500" and stop instead of backing off and trying again. A log line is advice. An exception is a wall.

The reason this matters more than it should is the cookie jar. Alerts are built on the assumption that information changes behavior, and the research on depletion cues says information mostly changes your narration of the behavior. I didn't ignore my 84% email. I read it, understood it, and then did the thing anyway while explaining to myself why this run was different. Every person on your team will do the same, including the one who wrote the alert.

So stop shipping warnings where you meant to ship a wall.

What's the biggest gap you've measured between a spend alert firing and anything actually stopping?

---

Originally published at https://robatdasorvi.com/stories/why-scarcity-works-even-when-people-know-it-is-fake