Model A costs half as much per token as Model B. Model A also needs almost twice the tokens to actually solve the task correctly. Guess which one was cheaper by the time the invoice came.
This is the gap nobody accounts for when they search "cheapest LLM API" and sort a comparison table by the input/output price columns. The table isn't wrong. It's just answering a narrower question than the one people think they're asking.
The Metric Everyone Optimizes, and What It Leaves Out
Price-per-token is the number every pricing page leads with, and it's the number every comparison article sorts by. It's also, on its own, an incomplete predictor of what a task will actually cost — because price-per-token tells you nothing about how many tokens a model needs to get you a usable answer.
Two models can have identical per-token pricing and produce wildly different real-world costs, because token count isn't fixed by the task — it's a function of the model's behavior on that task. A model that's verbose, that pads its answers with unnecessary preamble and repetition, that needs a longer, more carefully engineered prompt to produce reliable output, or that has a higher rate of producing an answer you have to retry — all of that shows up as more tokens consumed per successfully completed task, regardless of how cheap each individual token is.
"Cheapest LLM API" answered by price-per-token is answering "which model has the lowest sticker price." What most people actually want answered is closer to "which model gets me a correct, usable result for the least money" — and those are only the same question if every model needs exactly the same number of tokens to get there, which is almost never true.
A Concrete Way to See the Gap
Picture two models, both priced at output tokens, one at $1 per million and one at $2 per million — the first one looks twice as cheap on the pricing page. Now run the same task through both: a structured data extraction job, say, pulling five fields out of unstructured text.
The cheaper-per-token model produces a verbose response — some preamble explaining what it's about to do, the extracted data, then a summary restating what it just extracted — averaging 400 output tokens per task. The pricier-per-token model returns exactly the structured output requested, no preamble, no restatement — averaging 90 output tokens per task.
Run the actual math: the "cheap" model costs $0.0004 per task. The "expensive" model costs $0.00018 per task. The model with double the sticker price ends up costing under half as much per completed task, purely because of how differently the two models behave on the same prompt.
This isn't a hypothetical edge case — verbosity, formatting habits, and instruction-following precision vary meaningfully across models, and none of that variance shows up in a price-per-token comparison table.
Why This Gets Worse With Retries and Prompt Length
Two more factors compound the gap, and both get ignored by a pure per-token comparison:
Retry rate. If a model produces an unusable or malformed response some percentage of the time — invalid JSON, a hallucinated field, an instruction it didn't follow — every retry is additional tokens spent on a task that hasn't actually been solved yet. A model with a meaningfully higher retry rate can lose a per-token price advantage entirely once you account for the wasted attempts.
Prompt engineering overhead. Some models need a longer, more explicit system prompt to reliably follow formatting instructions — extra input tokens paid on every single call, forever, to compensate for something a different model does correctly with a shorter prompt. This cost is easy to miss because it gets baked into your prompt template once and then becomes invisible, but it's still being paid on every request.
Both of these push the real cost further from the number on the pricing page, and both push in the same direction: toward the per-token price being a weaker predictor of actual cost than it appears.
How to Actually Measure the Metric That Matters
The fix isn't complicated, it's just a different thing to measure: price per successfully completed task, calculated as (input tokens + output tokens + retry overhead) × price per token, averaged across a representative sample of your actual use case — not a generic benchmark, your specific task.
A rough way to approximate this without heavy tooling:
Pick 15-20 representative examples of your actual task.
Run them against each model candidate, recording total tokens consumed (including any retries needed to get a usable result).
Multiply by each model's per-token pricing.
Compare the resulting per-task cost, not the per-token rate.
This takes an afternoon, not a research project, and it's the only version of "cheapest" that actually predicts your bill. It also tends to produce genuinely surprising results — the model with the higher sticker price wins this comparison more often than pure price-per-token rankings would suggest, precisely because verbosity and retry rate are real, common differentiators that the pricing page doesn't capture.
Where This Connects to a Bigger Pattern
This is also a decent argument for why testing multiple models against your actual task matters more than picking one off a price comparison chart and committing — since the real cost differences here only show up empirically, per task, not on a spec sheet. Standardized, OpenAI-compatible access across multiple providers makes that kind of task-level comparison meaningfully easier to run, since testing model B doesn't require a separate integration from testing model A. RouteAI is one example of infrastructure built around that specific need — one interface across DeepSeek, Qwen, Kimi, GLM, and other models, which makes running the kind of per-task cost comparison described above a matter of switching a model parameter rather than rebuilding an integration for each candidate.
The Actual Takeaway
"Cheapest LLM API" measured by price-per-token is a real, useful number — it's just not the number that determines your actual bill. The number that determines your actual bill is price-per-completed-task, and the only way to know it is to actually run your specific task against your specific candidates and count.
The pricing page will tell you the price. It won't tell you the cost. Those turn out to be different things more often than the sorted comparison table would have you believe.
TL;DR: Comparing LLM APIs by price-per-token misses how many tokens each model actually needs to complete a task — verbosity, retry rates, and prompt-engineering overhead can make a "cheaper" model cost more per solved task than a pricier one. The metric that actually predicts your bill is price-per-completed-task: run 15-20 representative examples through your candidate models, count total tokens including retries, and multiply by each model's rate. Standardized access across multiple providers (like a gateway such as RouteAI) makes running this comparison easier, since testing a new model becomes a config change instead of a new integration.
Here's the tool I referenced in this post: www.fastrouteai.com
Top comments (0)