Three incidents made the rounds this season, and every AI-agent operator should read them as one story:
- An agent loop ran silently for eleven days and burned $47,000 before anyone noticed (write-up).
- An agent hit an AWS error and "fixed" it by spinning up duplicate CloudFormation stacks until the bill reached $6,531 (HN thread, 1,278 points).
- A coding agent found an over-privileged token in an unrelated file and deleted a production database — with backups — in 9 seconds (PocketOS post-mortem).
The common thread isn't model quality. In all three cases the model did something locally reasonable given bad context. The failure was the outer loop: nobody was watching, nothing bounded the blast radius, and stopping the agent required a human physically at the machine it ran on.
What an outer loop actually needs
If you run long agent sessions (Claude Code, Cursor, Codex, self-hosted), you can build most of this yourself:
1. A pre-execution gate, not a post-hoc diff. Reviewing the diff after rm -rf or a force-push is archaeology, not control. The gate has to sit between "agent decides" and "command executes." Most harnesses expose a hook for exactly this (Claude Code's PreToolUse hooks, for example).
2. Deny rules that persist. An agent blocked once will happily retry the same class of action next session. A denial should compile into a durable rule — "never git push --force on this repo" — that outlives the conversation.
3. Budget and iteration caps. The $47k loop was an eleven-day incident that a daily spend cap would have turned into a $50 one. Wall-clock watchdogs, token budgets per session, and loop-iteration limits are boring and they work.
4. Reachability when you're not at the desk. This is the piece almost everyone skips. Agents run for hours precisely so you can leave — which means the approval surface has to be on the device that's actually in your pocket, not in a terminal at home.
5. Least-privilege credentials. The 9-second database deletion happened because a blanket-scope token sat readable in a repo. No approval UX saves you from that; scope your tokens.
Disclosure and a soft plug
I ship with coding agents daily, and after my own close calls I built the missing piece for myself: Hermes Mobile (iOS, source and gateway on GitHub). Agents route through a gateway on my own machine; risky tool calls pause and my phone shows the exact command to approve or deny before it executes, and a thumbs-down becomes a standing block rule.
Honest state: it shipped this month, it has essentially zero users, and I'm the first one. Remote access is your own network/Tailscale/tunnel — no vendor cloud in the middle. If you try it, I want the bug reports more than the installs.
Whatever you use — mine, hooks you wrote yourself, or a cron job that kills anything running past midnight — give your agents an outer loop before they teach you the lesson the expensive way.
Top comments (0)