Your phone buzzes at 7:12 a.m. Stripe: "$47.18 charged to OpenAI."
You didn't ship anything. You didn't run a demo. You left an agent "watching" a queue overnight — and it watched the queue straight into your credit card.
If that sentence made your stomach drop, welcome. You're not bad at prompting. You're missing a budget guardrail.
Agents don't have a "stop spending" instinct
A normal API call is a vending machine: one coin, one snack, done.
An agent is a toddler with your wallet in a candy store. It can:
- retry a flaky tool five times
- re-read the same 12k-token context on every loop
- call a "just in case" research tool that itself calls three more tools
- keep going because nothing told it the night was expensive
That's not malice. That's missing constraints. Agents optimize for "finish the task," not "finish the task under $2."
The two meters that actually matter
Think of agent cost like a taxi meter with two dials:
- Token meter — every thought, every tool result stuffed back into context, every retry.
- Tool meter — every external call (search, browser, code exec, email, your own APIs).
Most teams only watch the first dial. The second dial is where the $47 hides.
A single "helpful" web-research step can cascade into plan → search → open 4 pages → summarize → search again. Each hop looks cheap. The cascade does not.
Stop doing this: unlimited loops with premium models
The default anti-pattern looks innocent:
while not done:
thought = llm.think(state) # frontier model every turn
result = tools.run(thought.action) # maybe expensive
state = state + thought + result # context balloons
No max steps. No max dollars. No model downgrade. No "if we're spinning, bail." That's how you wake up to a receipt.
Do this instead: a hard spend cap + cheap-first routing
Drop this scaffold around any agent loop. It tracks estimated USD, enforces a hard stop, and routes easy steps to a cheaper model.
from dataclasses import dataclass, field
@dataclass
class Budget:
max_usd: float = 2.00
spent_usd: float = 0.0
max_steps: int = 20
steps: int = 0
# rough public-ballpark rates — replace with your real pricing table
rates: dict = field(default_factory=lambda: {
"cheap": {"in": 0.15 / 1e6, "out": 0.60 / 1e6}, # small/fast
"smart": {"in": 2.50 / 1e6, "out": 10.00 / 1e6}, # frontier
"tool": 0.002, # flat estimate per external tool call
})
def charge_llm(self, model: str, in_toks: int, out_toks: int):
r = self.rates[model]
self.spent_usd += in_toks * r["in"] + out_toks * r["out"]
self._check()
def charge_tool(self, n: int = 1):
self.spent_usd += self.rates["tool"] * n
self._check()
def _check(self):
self.steps += 1
if self.spent_usd >= self.max_usd:
raise RuntimeError(
f"Budget exhausted: ${self.spent_usd:.4f} >= ${self.max_usd}"
)
if self.steps > self.max_steps:
raise RuntimeError(f"Step cap hit: {self.steps} > {self.max_steps}")
def pick_model(task_hardness: str) -> str:
return "smart" if task_hardness == "hard" else "cheap"
def run_agent(goal: str, tools, budget: Budget | None = None):
budget = budget or Budget(max_usd=2.00, max_steps=20)
state = {"goal": goal, "scratch": []}
while True:
hardness = "hard" if needs_deep_reason(state) else "easy"
model = pick_model(hardness)
thought, usage = llm_call(model, state) # returns token usage
budget.charge_llm(model, usage["in"], usage["out"])
if thought.done:
return thought.answer
result = tools.run(thought.action)
budget.charge_tool()
state["scratch"].append((thought, result))
Three ideas in one snippet:
- Hard USD ceiling — the agent cannot outspend the night.
- Step ceiling — infinite loops die of old age, not bankruptcy.
- Cheap-first routing — most steps don't need the Ferrari model.
Is the pricing table exact? No. Is "some meter beats no meter" true? Extremely.
Make the bill visible during the run
A budget you only inspect in the morning is a postmortem, not a control. Log a one-liner every step:
step=07 model=cheap in=1204 out=318 tools=1 spent=$0.41/$2.00
When spent crosses 50% and 80%, get loud. Agents fail silently on quality; they should fail noisily on money.
Analogy time (because receipts are boring)
- No budget = giving an intern a corporate card and saying "figure it out."
- Token-only budget = watching the gas gauge while ignoring toll roads.
- USD + steps + model routing = a trip with a prepaid transit card, a max number of transfers, and a rule that says "walk the short blocks."
You still get there. You just don't fund a mystery tour.
A 15-minute checklist before you "let it run overnight"
- Set
max_usdto something you'd shrug at (start at $1–$2). - Set
max_steps(start at 15–25). - Default the loop to a cheap model; promote to frontier only on hard branches.
- Cap tool fan-out (e.g. max 3 searches per plan, not 30).
- Truncate or summarize scratchpad so context doesn't compound every turn.
- Log
spent/maxevery step. Alert at 50% and 80%. - Kill switch: if the same action fails twice, stop — retries are where bills go to party.
Do those seven and the overnight $47 becomes an overnight $1.80 with a clean "budget exhausted" error — annoying in a good way.
The punchline
Agentic AI isn't expensive because models are evil. It's expensive because autonomy without a meter is just unsupervised spending.
Prompts make agents clever. Tools make agents useful. Budgets make agents shippable.
If you've already been burned, drop your scariest overnight receipt (and the one guardrail you added after) in the comments — max_usd, model routing, or tool fan-out caps. I'll go first: after the $47 surprise, every long-running agent here boots with Budget(max_usd=2.00, max_steps=20) and cheap-first routing. Sleep got cheaper.
Top comments (0)