The Token Price on the Landing Page Isn't Your Real LLM Cost

Comparing LLM providers by the token price on their landing page is a bit like comparing phone plans by the advertised monthly rate before taxes, fees, and overage charges. Technically a number, not the number that ends up on your bill.

What the headline price usually hides

• Output vs. input pricing — a lot of providers charge meaningfully more for output tokens than input. If your workload generates long responses (summaries, drafts, chat), output pricing dominates the actual bill, not the input rate that got top billing on the pricing page.
• Gateway/platform fees — on top of the underlying model rate, some routing layers add their own fee, and it's not always obvious it's stacked on top rather than included.
• Failed calls — a failed request doesn't refund itself in time. Retries, monitoring, and support tickets are real costs that never show up in a per-token comparison.
• Search/context fees — anything doing retrieval or search-grounding (like Perplexity's Sonar) often bills extra for the search context on top of generation tokens.
• Engineering time — every additional provider you integrate directly is SDK maintenance, another auth flow, another set of edge cases. That's a real cost even if it never shows up on an invoice.

The unit that actually matters: cost per completed task, not cost per token

If an agent workflow makes five model calls to complete one user-facing task, the number worth tracking is total cost for that task end-to-end, not the sticker price of whichever model handled step three. A "cheap" model that needs more retries or produces worse output that requires a second pass can end up costing more per completed task than a pricier model that gets it right the first time.

Practical version of this: route by task complexity, not by defaulting everything to one model.

classification / simple extraction  -> cheapest capable model
summarization / drafting            -> mid-tier model
final reasoning / code review       -> your strongest model

Escalate only the steps that actually need it. Sending every request through your best (and priciest) model because it's the one you're most confident in is the single fastest way to overspend.

Where a low-cost unified endpoint actually helps

If your workload is genuinely high-volume and mostly simple (classification, short-form generation, agent substeps), the pricing model of your routing layer matters more than usual. Something like GonkaRouter, which lists per-token pricing as low as $0.0004/1M tokens on an OpenAI-compatible endpoint, is worth knowing about specifically for that high-volume-simple-task tier, not necessarily as a wholesale replacement for whatever handles your hardest reasoning steps.

Same caveat applies to every pricing figure in this space, including that one: token prices move, sometimes tied to underlying compute cost. Check the live pricing page before you build a cost model around any specific number, including ones in this post.

The actual checklist before switching providers for cost reasons

• [ ] Did you compare output pricing, not just input pricing?
• [ ] Did you account for gateway/platform fees stacked on top of the base model rate?
• [ ] Did you measure cost per completed task, not cost per raw API call?
• [ ] Did you test real prompts, not just the benchmark examples on a comparison page?
• [ ] Are you routing simple steps to cheap models and reserving your best model for the steps that actually need it?

Curious what routing strategies people here have found actually move the needle, task-complexity-based routing, or is most of the savings coming from prompt compression and caching instead?