Every guide to cutting LLM costs is about the same thing: send fewer tokens. Trim the system prompt, cache the prefix, retrieve less, route easy requests to a small model. All of it works. And all of it is one half of the equation.
Your bill is:
bill = tokens × price_per_token
Two factors. Multiplication. Every cost-cutting article I have read attacks the left one and treats the right one as a law of physics. It is not — it is a number on a pricing page, and pricing pages are not the only place those models are sold.
The two levers, in the same units
A quick vocabulary check, because the whole argument lives in these words. A token is roughly ¾ of a word — models are billed per million of them. Input tokens are what you send (prompt, files, chat history), output tokens are what the model writes back. Output costs 3–10× more per token, which is why it usually dominates the bill. Prompt caching means the provider stores your unchanged prefix and re-reads it at a large discount, often around 10% of the input price — you pay a small premium the first time to write the cache, then read it cheaply.
Now, one workload. A team runs a coding agent, and last month it burned 200M input and 40M output tokens on Claude Sonnet 5. At list price ($2.00 in / $10.00 out per 1M):
200 × $2.00 + 40 × $10.00 = $400 + $400 = $800
Lever one — fewer tokens. Say 70% of that input is a stable prefix (the same repo files, the same instructions) and you make it cacheable. That 140M drops to roughly $0.20 per 1M instead of $2.00. Input becomes 140 × $0.20 + 60 × $2.00 = $148. Output does not cache — it is generated fresh every time. New total: $548, a 31% cut. That is a good month of engineering work.
Lever two — cheaper tokens. Same tokens, same model, price reduced 15%. $800 becomes $680. Zero engineering work.
Neither is the interesting part. The interesting part is that they multiply:
| Configuration | Monthly bill |
|---|---|
| List price, no caching | $800.00 |
| Caching only | $548.00 |
| Cheaper price only | $680.00 |
| Both | $465.80 |
Pulling both levers is a 42% cut. Pulling the one everybody writes about is 31%.
Why the second lever gets ignored
Because it does not feel like engineering. Caching is a diff you can review, benchmark, and put in a changelog. "We pay less per token" is a procurement decision, so it falls between the team that owns the code and the team that owns the invoice, and nobody optimizes it.
It is also the lever with no quality tradeoff. Every token-side optimization trades something: caching adds cache-invalidation bugs, retrieval trims context the model might have needed, routing to a smaller model risks a worse answer that costs more in retries. Changing the price per token changes exactly one thing — the price per token. The weights, the context window, and the output are identical.
Where the price is actually negotiable
Disclosure before the numbers: I work on altrouter.ai, a gateway that resells the same vendor models below their list price — Claude Sonnet 5 at $1.69 / $8.50 per 1M against the official $2.00 / $10.00, Opus 4.5 at $3.74 / $18.75 against $5.00 / $25.00, GPT-5.6 at $2.13 / $12.75 against $2.50 / $15.00. Discounts run 10–25% depending on the model, the API is OpenAI-compatible, so switching is a base_url change. Prices as of August 2026.
The honest gaps, so you hear them from me and not from the comments: no embedding models, no data residency options, no SLA of our own beyond what the upstream vendor delivers, and no invoicing for legal entities yet. If any of those is a requirement, this lever is closed for you and lever one is all you have.
What to do this week
- Pull last month's spend and split it into input, output, and cached input. If you cannot produce that split, you cannot price anything — in/out prices differ by 3–10×, and cached input by another 10×.
- Multiply your actual token counts by two or three candidate prices. Not benchmark tokens — yours. The ratio is workload-specific.
- Do the caching work anyway. The levers multiply; skipping one to take the other is leaving the smaller half on the table.
The point is not that one lever beats the other. It is that you are running an optimization with two variables and holding one constant for no reason other than habit.
Top comments (0)