DEV Community: MD Faiz

I replayed two months of my Claude Code sessions. $1,675 at API prices, and 117 rm -rf.

MD Faiz — Wed, 30 Sep 2026 16:04:35 +0000

I build Paveo, a checkpoint that sits between an AI agent and everything it calls and refuses the calls it shouldn't make, before they run. Before I asked anyone else to trust that, I ran it against the agent I use most: my own Claude Code, over the two months I spent building Paveo.

I'm on a subscription, so none of this was billed. That matters for everything below: these are the numbers my usage would have cost on the API, and the commands my agent would have been stopped from running.

Where the data comes from

Claude Code keeps every session on your disk, one .jsonl file per session under ~/.claude/projects. Each assistant message records its token usage, and each tool call records its name and arguments. paveo replay reads those files, prices the model calls from a dated price table, and puts every tool call through a policy. It stores nothing and prints only counts and rule names.

59 days. 106 sessions. 8,113 model calls. 8,176 tool calls.

What it would have cost

What	Amount
Total at API list prices	$1,674.75
Costliest day	$136.93
Costliest single session	$452.92
Opus 5	$1,300.47 over 5,248 calls
Opus 5.5	$374.28 over 2,861 calls

Almost every call in the files omits which region served it, and US-only inference costs more on some models, so those calls are priced at the US rate. The total can read up to 10% high. It never reads low, which is the direction a budget should err in.

The number that stayed with me is the single session: $452.92. On a subscription it was an ordinary long afternoon. On the API, as an unattended agent, it's the incident report.

What it would have stopped

The starter policy for Claude Code is short: a handful of patterns for commands that destroy work, and the paths to the agent's own settings and the guard's own files. Replayed over 8,176 tool calls, it would have refused 670:

Refused	Count
`rm -rf` (recursive force delete)	117
`git clean -f`	3
`git reset --hard`	2
`git push --force`	1
edits to `.claude` settings or the guard's own files	547

Two honest notes on those numbers.

Not every rm -rf was a mistake. Plenty were the agent cleaning up a build folder it had just made. The guard would have made it find another way, or ask. I'm fine with that trade: a false refusal costs a retry, a missed one costs the work.

Most of the 547 are false alarms, and I kept the rule anyway. I was building a tool whose files live in .paveo, so the agent touched them constantly. But "the agent edits its own settings or its own guard" is the one thing I least want to allow, because it's how every other limit gets switched off. A noisy rule that guards the guard stays.

Three things I'd tell anyone running agents

Price your sessions before you move them to the API. The files are already on your disk. If you add it up yourself, count each message id once: a streamed message can appear on several lines.
Set a ceiling per run, before it starts. A monthly alert tells you about the $450 session after it happened. A ceiling checked before each call refuses the call that would breach it.
Keep the never-list short and mechanical. The acts worth a hard stop are the ones cheap to catch: recursive deletes, force-pushes, anything that spends money. Write the refusal as an instruction ("don't retry it another way, tell me what you wanted"), not a verdict, or the model goes looking for another route.

What's next

Paveo's repository goes public this week. It runs in your own process, opens no network sockets, and the same policy file guards Claude Code, Codex, Cursor or an agent you build yourself.

If you've priced your own sessions, I'd like to know: what was your costliest one?

Four quiet ways an AI agent's budget guard gets the bill wrong

MD Faiz — Thu, 24 Sep 2026 12:19:04 +0000

An agent's budget guard has one job: refuse the call that would take it past its ceiling. To do that it must hold the worst case before each call and settle the real cost after. The settling half is where guards quietly break, and they rarely throw an error when they do.

This week I found four of these in my own code. One was live, and undercharged. Two more would have undercharged, and one would have overcharged the user for something they had chosen not to buy.

1. The placeholder that looks like a count

When Claude streams, the first event, message_start, carries a usage block. Its output count is tiny: 1, 2 or 3 in Anthropic's own documentation examples. The real output count arrives later, in message_delta, and the docs are explicit: "The token counts shown in the usage field of the message_delta event are cumulative."

A tracker that folds usage as events arrive works perfectly, until the stream ends before message_delta. A proxy on a custom base_url that drops or reshapes it, a dropped connection, a regression after an upgrade. Then the call settles on the placeholder, and a 20,000-token reply is billed as one token. Nothing errors.

Fix: a stream is final only when a message_delta carrying an output count has arrived. Anything else is charged the worst case that was held for it.

2. Missing counts read as zero

record({}) settled a call at $0. An empty usage dict, a response with no usage, a key the parser skipped: all of them looked like a call that cost nothing.

Fix: usage without both an input and an output count is unreadable, and unreadable is charged the worst case, then raised. One check at the point where every provider's usage is settled, not a copy per provider, because the copies had already started to disagree.

3. Assuming a refused request is free

The first draft gave the hold back on any 4xx: the API refused it, so surely nothing was billed. Then I went looking for the provider page that says so: the errors page, pricing, rate limits, streaming. None of them does. The only line I found was that a failed web search is not billed.

An assumption nobody can cite is not a ceiling. Fix: a call that raises is charged its worst case. If that ever overcharges someone, it is visible and recoverable. The other direction is how a limit gets breached.

4. Reading the stream to find the bill

To get final usage on a messages.stream() call, the obvious move is get_final_message(). In the Anthropic SDK that calls until_done(), which reads the rest of the stream. So a user who stopped a generation early, to save money, would be made to download and pay for the part they walked away from.

Fix: read usage from what has already arrived, never read further than the caller did, and charge the worst case if the final count never came.

The rule underneath all four

A guard may be wrong, but only in the direction that protects the ceiling. Every one of these bugs was a guard being optimistic: trusting a placeholder, treating silence as zero, assuming an error was free. Optimism in a guard is a limit that stops holding without telling anyone.

I'm building Paveo, a checkpoint that runs inside your own process before every model and tool call. A budget is one rule it enforces; which models and tools an agent may use are others. Nothing leaves your machine. It works with Anthropic, OpenAI and Gemini today, and it's in a private alpha: if you run agents in Python and want to try to break it, say so in the comments.

Earlier: I went through 14 reports of AI agents burning money. Every budget that failed, failed one of two ways.

A question for anyone running agents: besides cost, what would you want to block outright, before the call leaves?

I went through 14 reports of AI agents burning money. Every budget that failed, failed one of two ways.

MD Faiz — Tue, 22 Sep 2026 13:37:12 +0000

Everyone measures AI agent spend. Almost nobody stops it. I spent two weeks in GitHub issues, bug trackers and Show HN threads reading what actually happened to people, and the failures are far more repetitive than I expected.

The scale first, so this isn't abstract. One report: 416 API calls and 112,057,985 input tokens in a single session. Nine of that user's ten sessions ended in failed, and several kept firing calls for 20 to 23 hours after the point of failure. The dashboards worked perfectly the whole time.

Failure one: the gate passes, then the call changes.

The budget check runs against the model you asked for. Then a fallback or a router sends the request to a different deployment, and that one bills. From litellm #41344: "Nothing re-checks budget on the deployment that actually bills." The check was correct and irrelevant.

Failure two: everyone checks at once.

Five sub-agents each read the remaining budget. All five see headroom. All five proceed. Every individual check passed and the total is over the ceiling. A Cisco security spec puts it exactly: "multiple agents can observe remaining budget and start model calls before the governor's next poll."

This one is nastier than it looks, because it only appears under concurrency, which is precisely when you're not watching.

What connects them: both are a gap between deciding and spending. Anything that reads a number and then acts on it loses that race. The fix is the same shape banks have used for decades on card authorisations: claim the worst case before the call, settle to the real number after it returns. You over-reserve slightly and you cannot overspend.

That's what I'm building, as a library that sits in the process and refuses the call rather than reporting on it.

The question I keep asking everyone, because the answers keep surprising me: besides cost, is there anything you'd want an agent blocked from doing outright? Tools that write, external actions, specific models? Or is cost the whole of it for you?