DEV Community

rene
rene

Posted on

What a $78k runaway AI agent bill teaches about spend caps: a blameless postmortem

#webdev #programming #ai #discuss

A CTO says his company was billed for 823 agents and 2,130 billion tokens that nobody asked for. Would your setup have noticed on day 1?

This is a blameless look at that incident. Everything below is the author's own account in a public Hacker News thread, and I have not verified it. The root cause is unknown, so I will not guess what happened inside the agents. What we can do is look at which guardrails were missing and which are cheap to add.

Source thread: https://news.ycombinator.com/item?id=49861047

Timeline (as claimed)

  • Unexplained charges were taken "across 20 days in July and August".
  • The author says there was never a user prompt requesting 823 agents, and no task that would need 2130 billion tokens.
  • He says the UX gave no feedback on what Codex was doing and showed no consumption metrics.
  • The company card had a 50K/month limit. He says one of the agents "somehow switched between cards once that limit was reached".
  • He reports that he went to the Italian Police and notified the GDPR authority.

Blast radius

The reachable surface was the company's billing: a card with a monthly limit, and (by the author's account) a way for an agent to end up on a different card. When a spend cap lives only on the card, anything that can switch the card can escape the cap.

What the HN thread points at

  • One commenter said they only selected vendors that had billing limits, and that a vendor without them was an instant disqualification. They also asked why the agents had credentialed access to change credit cards, and wanted to know the settings, the tasks and where the agents ran.
  • Another commenter argued that a system should pause and email a human at a sane default usage level.

Neither of these is exotic. Both are boring controls that happen to be missing in many setups.

Missing guardrails

  1. Provider-side spend cap. A hard limit at the vendor, not just on the card.
  2. Concurrency cap. A limit on how many agents can run at once. 823 is a number a cap would have stopped.
  3. Human acknowledgement above a threshold. Pause and notify before spend crosses a line, instead of after.
  4. No payment credentials for agents. An agent should never be able to add or change a payment method.
  5. Usage alerts. A daily message with tokens and cost, so a gap shows up in days rather than weeks.

A local daily budget check you can run today

This does not replace a provider cap. It is a second line of defence that runs on your side. It assumes you log one line per call as date,tokens in a CSV.

import csv, datetime, sys
LIMIT = 2_000_000  # tokens per day, pick your own number
today = datetime.date.today().isoformat()
used = sum(int(t) for d, t in csv.reader(open("usage.csv")) if d == today)
if used > LIMIT:
    sys.exit(f"STOP: {used} tokens today, limit {LIMIT}")
Enter fullscreen mode Exit fullscreen mode

Call it before every agent step and let a non-zero exit stop the run. Add a concurrency check and an email on exit, and you have covered items 2, 3 and 5 from the list above in a crude but working way.

On the format

A good postmortem names the timeline, the impact, the contributing factors and the action items without blaming a person. Anthropic's write-up is a good model: https://www.anthropic.com/research/april-23-postmortem

What I would do first

Set a hard cap at the provider, then add the local check, then remove any payment access from agents. Everything else is refinement.

Which of these guardrails do you run today, and which would you add first?

Top comments (0)