Why Your AI Agent Costs 6x More Than You Calculated

pop3zxcv — Mon, 03 Aug 2026 18:27:06 +0000

Pricing an LLM call is simple arithmetic. Input tokens times the input rate, plus output tokens times the output rate. Every pricing page shows you this, every calculator computes it, and for a chatbot it is correct.

For an agent it is wrong by a factor of six, and the factor gets worse the longer the agent runs.

The reason is not hidden or subtle. It follows from one property of language models that everybody knows and almost nobody puts into their cost estimate.

Models have no memory

A language model does not remember your last request. Each call is independent. If you want the model to know what happened three steps ago, you send it again.

So an agent loop does this:

step 1  send: system prompt + task
        get:  a tool call

step 2  send: system prompt + task + step 1 output + tool result
        get:  another tool call

step 3  send: system prompt + task + step 1 + step 2 + both tool results
        get:  another tool call

Take a concrete shape. A 2,000-token system prompt with tool schemas, a 500-token task, 400 tokens of model output per step, and 1,200 tokens of tool results per step. Here is what each step actually sends:

Step	Context sent
1	2,500 tokens
2	4,100 tokens
3	5,700 tokens

Each step costs more than the one before it, forever. Step 3 is more than twice the price of step 1 and it is doing the same amount of new work.

The formula

Across N steps, every step pays for the system prompt and the task. That part is linear:

N × (system + user)

The accumulated history is the part that hurts. Step i carries the output and tool results of all i − 1 steps before it. Summing that over the whole run gives the triangular number:

(output + tool_result) × N × (N − 1) / 2

Put together:

total input tokens = N × (system + user) + (output + tool_result) × N × (N−1) / 2

The second term is quadratic. Double the step count and that part roughly quadruples.

What that does at different step counts

Same workload shape as above, no retries, no caching:

Steps	Raw conversation	Billed input	Multiplier
3	7,300	12,300	1.7×
5	10,500	28,500	2.7×
10	18,500	97,000	5.2×
12	21,700	135,600	6.2×
20	34,500	354,000	10.3×
30	50,500	771,000	15.3×
50	82,500	2,085,000	25.3×
100	162,500	8,170,000	50.3×

"Raw conversation" is the number you get if you add up everything the agent said and everything it read. It is the number your intuition reaches for. "Billed input" is what appears on the invoice.

At 3 steps the gap is small enough to ignore. At 12 steps you are paying 6.2 times your estimate. At 50 steps, 25 times. A deep research agent that runs a hundred steps bills fifty times the tokens the conversation contains.

Nothing is broken when this happens. It is what the pricing model does when you loop it.

Retries make it worse

A failed tool call, malformed JSON, a guardrail rejection: whatever the cause, a retry re-sends the context too, at whatever depth the failure happened.

At a 10% retry rate the 12-step example moves from 6.2× to 6.9×. Not dramatic on its own, but it stacks on top of a number that is already six times your estimate, and retry rates in production are rarely zero.

Three things that move the bill

Take the 12-step agent at a 10% retry rate on Claude Sonnet 5. Baseline is $0.351 per run. At 1,000 runs a day that is $10,534 a month.

Prompt caching. Most of what you are paying for is context re-sent verbatim, and cache reads cost roughly 90% less than fresh input. A 90% hit rate takes the run to $0.109. That is 69% off, with the same model and the same agent.

Step count. Halving the loop from 12 steps to 6 takes it to $0.112, or 68% off. Almost identical to what caching bought you, which is worth sitting with for a second: removing half the reasoning steps and caching everything are about equally valuable.

Tool result size. Trimming tool results from 1,200 tokens to 400 takes it to $0.235, or 33% off. Smaller than the other two but easier than either. Every tool result is re-sent by every step that follows it, so truncating a verbose search result compounds down the whole run.

The counterintuitive one

The instinct when a bill is too high is to switch to a cheaper model. Compare that against cutting steps, on a 20-step agent:

Change	Cost per run	Saving
Baseline: Sonnet 5, 20 steps	$0.867
Swap to Haiku 4.5, still 20 steps	$0.433	50%
Stay on Sonnet 5, cut to 10 steps	$0.257	70%

Ten steps on the frontier model is cheaper than twenty steps on the small one, and you keep the better model.

This is not a quirk of these two models. Price is linear: a model at half the rate costs half as much, and that is the ceiling on what switching can buy you. Step count is superlinear, so halving it saves more than half. The gap widens as the agent gets longer.

Which means the first question about an expensive agent is not "what cheaper model could do this", it is "why does this take twenty steps".

Why the usual calculators miss it

Nearly every LLM pricing calculator prices one request. Input times rate, plus output times rate. That is the right model for a chat completion and it is what the pricing pages describe, so it is a reasonable thing to build.

The closest prior treatment I found is Softcery's voice agent calculator, which applies a flat 1.8× "reality factor" to LLM cost and notes in a footnote that conversation history "compounds O(n²) with turns". They diagnosed the mechanism correctly. For voice at roughly four turns a minute with short turns, a constant is probably a fair approximation, and theirs also absorbs function-calling round trips and barge-in handling.

Tool-using agents sit somewhere else on the curve. Turns are fewer but each one drags a large tool result behind it, so the quadratic term takes over much earlier. A constant that fits a voice call will not fit a research loop, which is the case for computing the curve rather than picking a number.

What this arithmetic does not tell you

Worth being clear about the edges.

Retries here are a flat multiplier on the total. Real retries happen at a specific depth, so a failure at step 18 costs far more than one at step 2, and a flat rate under-counts late failures.

Cache hit rate is one number. In reality your system prompt might cache at 99% while your tool results never cache at all.

Providers tokenize differently, so comparing token counts across vendors is approximate. And cache writes are not free everywhere: OpenAI's GPT-5.6 charges 1.25× uncached input to write to cache, which the numbers above do not include.

None of this changes the shape of the curve. It does mean you should treat any figure here as a planning estimate rather than a billing forecast.

Run it on your own numbers

I built a calculator that models the loop instead of a single request: step count, tool result size, retry rate, cache hit rate, across 17 models, with the per-step accumulation and cost attribution broken out. Free, no signup, runs entirely in the browser.

costperrun.com

The pricing data is on GitHub with a source URL and verification date against every rate. If you find a stale price, open an issue.

The number worth checking first is your step count. It is almost always higher than you think, and it is the term that squares.

DEV Community: pop3zxcv