If you price a coding agent the way you price a chatbot, you will misread the bill every time.
A normal code chat turns one prompt into one completion. A coding agent runs a loop: it reads files, calls tools, reads the output, edits, re-runs the tests, and self-corrects — often a dozen turns. 2026 research (Stanford, MIT, and others) puts a single agentic-coding task at roughly 1–3.5 million tokens, with ~76% of them reads. One study found agentic coding burns up to 1000× more tokens than a normal code chat.
The takeaway is not "agents are expensive." It is that the unit is wrong. You do not pay per million tokens. You pay per task — and two levers move that number 5–10× more than the model name does.
Lever 1: caching (the one that beats model-switching)
A coding agent re-sends the same context — system prompt, project files, tool definitions — on nearly every turn. Cache reads bill a fraction of base input. On a gateway with transparent cache pricing, the gap is stark:
| Model | Input (¤/1M) | Cache read (¤/1M) | Cache advantage |
|---|---|---|---|
| deepseek-v4-Pro | 560 | 10 | ~56× |
| qwen3.5-plus | 130 | 20 | ~6.5× |
| hy3-preview | 180 | 60 | 3× |
| minimax-m3 | 390 | 80 | ~4.9× |
Structure prompts so the stable context is cacheable and new content appends at the end, and that discount compounds across a long session. One change, bigger impact than most model swaps.
Lever 2: route by difficulty, not by habit
The same studies show a ~40× per-task cost spread across models for the same task. The pattern that holds: run the cheapest model that passes your evals by default, and escalate only the hard tasks. A unified gateway with 25+ models across 8 providers makes that a one-line switch instead of a refactor:
REQUEST -> AUTH -> ROUTE(model:"auto") -> RESPONSE -> METER
^ picks by cost / quality / speed
The catch most teams miss: you cannot route by difficulty if you cannot see the cost. That is why per-request traceability matters — every call should show model choice, latency, tokens, and cost on one line, not buried in a dashboard.
The move
Stop benchmarking models on $/M. Benchmark them on $/task for 15–30 real tasks from your own repo. Cache the stable context. Route the routine work to cheaper models. And make the meter visible, because an agent that hides its own cost is the most expensive agent you will run.
Building agentic systems on a unified, OpenAI-compatible gateway (SG-hosted, PDPA-aligned, up to 90%+ cheaper on China models): tokenlat.com
Top comments (4)
A task is kind of ephemeral IMO. I don't have clear definition of it when I code. And I don't believe in accurate estimation upfront. A good metric to consider would be the rate of picking a wrong model for a task (not matching its complexity).
Agreed — the fuzzy "task" boundary is exactly why per-token pricing hides the real number. Your wrong-model-for-complexity rate is the metric that matters: it captures the rework tax. A cheap model that needs three retries on a hard subtask usually costs more than routing that subtask to a frontier model once. The win isn't one model — it's matching each subtask to the right tier and counting the retries as cost.
Per-task pricing changes the eval math more than it looks, because a retry loop stops being just slower and becomes a line item. Once I started scoring agents on cost per successful task instead of raw tokens, a few cheap models got expensive fast on the reruns needed to land a correct answer. The billing model basically forces you to measure success rate, not throughput.
This is the insight that flips the whole eval: once you score cost-per-successful-task, "cheap" models quietly fail. The rerun needed to land a correct answer is the line item nobody budgets. We've watched "cheap" routing eat its own savings the moment the metric is tokens instead of outcomes. Measuring success rate instead of throughput is what makes routing defensible.