There's a story going around about an AI agent that was given network access to scan DN42 — the hobbyist BGP network people run to practice internet-scale routing — and it ran up a bill big enough to bankrupt the operator. I don't know the exact numbers and I'm not going to repeat them, because the numbers aren't the point. The point is that it happened at all, and that it was entirely predictable.
We keep talking about agent safety as if it's a sandboxing problem. Can the agent escape its container? Can it exfiltrate data? Can it touch the host? Those are real questions, but they're the wrong ones for most of the failures I actually see. The agent doesn't need to escape to hurt you. It just needs to keep going.
The DN42 scan is the perfect example. Scanning a network is the classic "cheap per action, expensive in aggregate" workload. One ping costs nothing. A million pings cost something. A hundred million pings cost a lot. And an agent with a goal — "map this network" — will happily do a hundred million pings, because each one is a step toward the goal. It's not malicious. It's not even stupid. It's an agent doing exactly what you asked, with no concept of what the doing costs.
That's the failure mode we're not designing for. We sandbox the agent to protect the environment from the agent. We rarely budget the agent to protect ourselves from the agent. And the second one is the one that actually bites.
I run agents with a budget object. Not a sandbox — a budget. Every tool call checks it. Every API call decrements it. When it hits zero, the agent stops, logs what it was doing, and asks. Not "asks nicely" — asks, and waits. There's a hard ceiling on spend per run, a hard ceiling on tool calls, a hard ceiling on tokens. The agent can't go over because the runtime won't let it. It's not a suggestion. It's a circuit breaker.
The DN42 operator's mistake — and I say this with sympathy, because I've made the same mistake — was giving the agent network access without a budget. The sandbox was probably fine. The agent probably couldn't touch the host. But it could touch the network, and touching the network costs money, and nobody told it to stop. So it didn't.
Here's the thing about agents: they're great at doing a million cheap things. That's their whole value proposition. A human would get bored after a hundred pings and go make coffee. An agent will ping until the bill is due. So if you're going to give an agent a network, you need to answer one question before you give it the goal: what's the most this is allowed to cost? If you can't answer that, you're not ready to give it the network.
Blast radius is the other half. Cost ceilings stop the bleeding, but they don't stop the damage. The agent should have the least privilege that still lets it do the job. If it's scanning a network, it should have a scanning account with a scanning budget, not the operator's credentials. If it's calling an API, it should have a key that's rate-limited and capped. The agent should never be able to spend more than the task is worth, and it should never be able to touch anything the task doesn't require.
I've seen the alternative. I've watched an agent retry a failed tool call fourteen times because nobody told it to stop. Fourteen calls, each one billed, each one a step toward a goal that was already dead. A circuit breaker would have stopped it at three. A budget would have stopped it at one. The agent wasn't broken. The design was.
There's a deeper point here, and it's the one I keep coming back to. Autonomy without accountability is a bug. If you give an agent a goal and no budget, it will pursue the goal to the end of your money. That's not a bug in the agent. That's a bug in your design. The agent is doing exactly what you asked. The problem is you didn't ask it to stop.
So when I hear about the DN42 operator, I don't think "AI is dangerous." I think "someone gave an agent a network and forgot to give it a wallet." The sandbox protects the world from the agent. The budget protects you from the agent. You need both, and you need the budget first, because the sandbox can't pay your bill.
I'll be honest: I haven't run an agent against DN42, and I'm not going to. But I've run agents against real networks, and I've paid for their mistakes. The fix is always the same. Hard ceilings. Blast-radius limits. A budget object that the runtime enforces, not the agent. And a kill switch that a human can hit, because the agent won't hit it for you.
If you're building an agent today, ask yourself one question before you give it any tool: what's the most this is allowed to cost, and what's the most it's allowed to touch? If you can't answer both, you're not building an agent. You're building a bill.
Top comments (0)