The DN42 agent that ran up a bill while scanning a network — everyone's blaming the operator. And sure, the operator should have set a budget. But I keep coming back to a different question: why did the budget have to be set at all?
Every agent framework I've touched ships with the same defaults. Tools on. No spend limit. No per-call cost. No approval gate. The default state of an agent is "unlimited," and you have to actively work to make it otherwise. That's backwards, and it's the real story here.
Think about how we build everything else. Your database has connection limits by default. Your web server has timeouts by default. Your cloud account has billing alerts by default. Nobody ships software where the safe configuration is the one you have to discover and the dangerous one is the default. Except agents. Agents ship wide open, and we act surprised when someone runs one without reading the manual.
I've been guilty of this myself. First agent I put in front of a paid API, I spent a week on the prompt, the tools, the memory — and zero minutes on what it would cost if it went off the rails. The budget was an afterthought, something I'd add "later." Later is where unbounded agents live.
The fix isn't a better operator. It's better defaults. Budgets should be on by default. Expensive tools should be opt-in, not pre-enabled. An agent that can't estimate an action's cost should refuse and ask, instead of guessing and billing. That's a framework problem, and it's solvable — it just requires the people building these tools to treat cost the way they treat security: a property of the system, not a feature the user remembers to turn on.
I'm not letting the operator off the hook. But the reason this keeps happening isn't that operators are careless. It's that the tools make careless the path of least resistance. When the default is unlimited, the failure isn't the operator's mistake. It's the design's.
Fix the defaults and this story stops being a cautionary tale and starts being a bug report.
Top comments (0)