A few months ago I became convinced my AI assistant was bleeding money through incompetence. It felt true. Sessions ran long, things failed, and the meter was always spinning. So I did what I do with any suspected leak: I measured it. I streamed 45 days of local session transcripts through a script that classifies every failed tool call and prices the recovery turn that follows it. It covered 660 session files, roughly June through early August. The parse ran entirely on my machine and cost zero tokens, which turns out to matter more than I expected, because once something is free to measure you actually measure it weekly, and I had been treating measurement itself as a one-time project.
The number I went looking for was big. Here is what the audit actually found: 729 failed tool calls across 45 days, about sixteen a day, and 718 recovery turns where the model stopped to clean up after a failure. Those recovery turns cost roughly 15.85 million of my local "effective unit" weighting. Total spend in the same window was about 1.05 billion units, which puts the cleanup bill at one and a half percent.
The mistakes were real, and fixing them is worth doing, but they were not where the money went.
About forty percent of the failures were genuine agent mistakes: edits against stale files, paths that stopped existing two sessions ago, a PowerShell script using an operator the installed version doesn't have, tool-call JSON the schema rejected. The other sixty percent were the environment being the environment. There were pages that never reach DOM-idle, so the browser tool waits 45 seconds and gives up. Screenshot injection timed out on busy screens. Windows locked files underneath edits at the worst moments. The worst offender is a professional networking site you have heard of. It never idles, on purpose, and the tool waits faithfully every time. None of that is fixable by prompting harder, and I had been mentally invoicing all of it to the model.
While I had the transcripts open, I priced the thing nobody warns you about. A long session re-reads its own history on every reply, and the meter spins for every one of those re-reads. This is the quiet failure mode of AI-assisted work: nothing errors, nothing alerts, the session just grows and every turn gets a little more expensive until a routine question is quietly hauling a few megabytes of dead conversation uphill with it. The audit put that re-read tax at roughly 80% of total spend. The single worst session had burned 240 million cache-read tokens on its own, across 665 messages and a 7.2MB transcript, and I couldn't have told you what most of those messages were about.
Three sessions like that accounted for about 493 million tokens in one week. That was 20% of the week's entire burn, sitting in three tabs I kept open because closing them felt like losing something.
I made three changes.
The first one was the least glamorous. The hygiene rule stopped being aspirational. The top burner sessions get archived the moment the audit names them, and a fresh session opens with a one-line handoff so nothing of value depends on the old context surviving. It sounds too small to matter, and that is exactly why it took an audit to make me actually do it.
Then I checked the boring tiering instead of assuming it. Every scheduled routine in the estate is pinned to the cheapest model that can do the job, and the model-fit check confirmed that held, with one catch worth naming. The same check flags workflow fan-out calls that inherit the premium model by omission, which is exactly the kind of leak you only see when you go looking. When it flags one, pinning a cheaper tier is a small config change, so there is no reason to leave it inherited.
The last change was making the measurement itself a routine. The scanner is about 200 lines of PowerShell, it runs Sunday, it parses the week's transcripts locally and writes a snapshot file, and because it never calls the API it never shows up in its own report, which keeps the measurement honest in the cheapest possible way.
Here are the snapshots, for the record. Rolling seven-day cache-read volume across all models: 2.45 billion tokens on August 9. 447 million on August 27. 151 million on August 30. Weekly message count fell from 17,385 to 2,899 to 1,014 over the same snapshots. The two later windows overlap and my estate migration landed in the same stretch, so I won't pretend I can cleanly attribute the drop to any single cause. The message count is the honest signal, because closing sessions is a behavior, and behavior shows up in the count no matter what the platform charges per token.
One caveat before anyone quotes these numbers at a vendor: "effective units" are my local weighting for comparing spend across token types, and the vendor's invoice uses different math. The ratios are the finding, not the absolutes, and the ratio says my own habits were the line item, which was not the conclusion I wanted from a report I built to blame the model.
The agent was never the expensive part. I was the expensive part, one open tab at a time.
Top comments (4)
The 1.5% number for recovery turns is the result I wish more people got to before deciding what to optimize. I spent a month convinced my agents were burning money on retries and bad tool calls before I actually measured it. Same thing. The failures were real but they weren't the bill.
The re-read tax hit me differently because I run long-lived agent sessions that accumulate context across dozens of tool calls. I had one session that stayed open for three days and by the end every routine turn was dragging 6MB of conversation it didn't need. Archiving that session and handing off a one-line summary dropped my weekly spend by something like 15%. No prompt changes, no model swaps, just closing a tab.
Your 40/60 split between agent mistakes and environment failures is the part I'd push people to replicate. When I broke mine down the environment side was almost entirely timeouts from web pages that never reach idle and file locks from other processes. You can't prompt your way past a site that deliberately never finishes loading. Separating those two buckets made the agent-side fixes a lot simpler because I stopped trying to solve problems that weren't the model's fault.
Your 80% number — the "re-read tax" from long sessions — is the same failure mode I hit building a retrieval pipeline: long context silently accumulates, every turn re-reads it, and nothing alerts because nothing errors. I catalogued this in my own system as the "absent answer" problem — a cost that never shows up on any dashboard until you build the tool that measures it.
brainbootdev's question about the ratio drifting is the right one, and I'd push it further: the ratio itself is a third state. If the 40/60 split drifts toward environment-heavy, that means the infrastructure is degrading — more external tools failing, more pages not rendering. If it drifts toward agent-heavy, the agent is degrading — more stale file references, more hallucinated paths. The ratio is a health metric for both the agent AND the environment, not just a cost breakdown.
One thing I'd add: your scanner runs locally and costs zero tokens, which means it can measure every week forever without itself becoming a cost item. That's the discipline: measurement infrastructure should never be a line item in its own report. The moment your auditor costs more than the audit, you stop auditing.
The 40/60 split is the part I would want other people to copy. Most cost postmortems stop at "the model made mistakes" because that is the only bucket the vendor dashboard can show you. Separating genuine agent error from environment latency changes what you do next: the first is a scaffolding problem, the second is a timeout policy problem, and they have almost nothing to do with each other.
The thing I trust most here is that you went looking for a big number and published the small one. 1.5% is an unrewarding headline, and that is exactly why the rest of the piece reads as credible.
Question, since you say measuring is now cheap enough to do weekly: has the 40/60 split held steady across the 45 days, or does the environment share move around week to week? If it drifts, that ratio is arguably a better health metric for the setup than the absolute spend is.
The 1.5 percent number is the useful part of this post, and it is the kind of result that only shows up when measuring is free. When an audit costs a day of work you run it once, get a scary anecdote, and optimize the thing you happened to look at. When it costs zero you run it weekly and discover the scary anecdote was rounding error. That reframes where to spend effort: 729 failures over 45 days is a quality problem worth fixing on its own terms, but it is not a cost problem, and treating it as one would have bought you a 1.5 percent ceiling. Curious whether the remaining 98.5 percent broke down mostly into context re-reading, or into genuinely long reasoning turns - those two have completely different fixes.