LLM API bills grow quietly. A code-review bot here, a nightly summarizer there, and suddenly the invoice has a comma in it. This post walks through four cost levers in order of effort vs. payoff, with a worked example: a representative workload that starts at $412/month on GPT-5.5 for everything, and ends around $54/month with the same models and the same output quality.
Numbers use OpenAI and Anthropic list prices as of mid-2026.
1. Stop defaulting to the flagship (saves 40-70% on the right calls)
The laziest win. Most pipelines send everything to the biggest model because that is what was in the quickstart.
A rule of thumb that holds up well:
- Classification, extraction, routing, short summaries: Claude Haiku 4.5 ($5/Mtok output list) or GPT-5 ($10/Mtok output list)
- Code generation, multi-step reasoning, long-context analysis: GPT-5.5 ($30/Mtok list) or Claude Opus 4.7 ($25/Mtok list)
Route by task type with a 10-line dispatch function. In most agent-style pipelines, half or more of the calls turn out to be Haiku-shaped: short inputs, structured outputs, no deep reasoning. In the worked example, moving 60% of calls down a tier takes the bill from $412 to $246.
2. Actually use prompt caching (saves up to 90% on input)
Both vendors ship it, both are underused.
- OpenAI: cached input is 90% off. GPT-5.5 input drops from $5.00 to $0.50 per Mtok.
- Anthropic: cache reads are 0.10x base input. Cache writes cost 1.25x (5-min TTL) or 2.0x (1-hour TTL), so caching pays off from the second hit onward.
The catch: caching only works if your prompt prefix is byte-stable. The classic failure mode is interpolating a timestamp or request ID into the system prompt "for context". That one line disables caching on every call. Move anything dynamic to the end of the prompt, keep the system prompt and few-shot examples byte-identical, and the discount shows up on its own.
A review bot with a 6K-token system prompt and rubric resends that prefix on every diff. Cached, it becomes a rounding error. Worked example: $246 down to $187.
3. Trim the transcript you resend
Chat-style loops resend the whole conversation every turn, so token cost grows quadratically with turn count. A common fix: cap history at the last N turns plus a running summary, refreshed every ~10 turns by a cheap model. Cutting input volume roughly in half on long sessions takes the example from $187 to $148.
As a bonus, steps 1-3 also reduce latency. The cheap tier answers easy calls in a fraction of the flagship's time.
4. Buy the same tokens at outlet prices (saves 80% on whatever is left)
After 1-3 you still pay list price for every remaining token. The last lever is not paying list.
corouter is a token outlet that bills the same GPT and Claude models at 20% of the official list price. Same model IDs, same JSON shapes, same SDK. The migration is one line:
from openai import OpenAI
client = OpenAI(
base_url="https://api.corouter.co/v1", # was api.openai.com/v1
api_key="cr-...",
)
resp = client.chat.completions.create(
model="anthropic/claude-sonnet-4-6", # Claude via the same endpoint
messages=[{"role": "user", "content": "ship it"}],
)
The details that matter for the math:
- The cache discounts from step 2 pass through 1:1 and stack on top of the 80% off. GPT-5.5 cache reads land at $0.10/Mtok.
- Pay-as-you-go credits, $5 minimum. No subscription to babysit.
- Failed requests auto-refund to the balance on 5xx.
- Streaming, tool calls, JSON mode, and structured outputs pass through unchanged.
Full rate card is on the pricing page. corouter is not affiliated with OpenAI or Anthropic; it routes, meters, and bills. Check your own compliance requirements before moving production traffic, same as with any gateway.
Applied to the worked example, the remaining $148 becomes about $54. (Not exactly $148 x 0.2, because the cache-read line was already deeply discounted.)
The math in one table
| Lever | Monthly cost after |
|---|---|
| Baseline, GPT-5.5 for everything | $412 |
| + tiered routing (step 1) | $246 |
| + prompt caching (step 2) | $187 |
| + history trimming (step 3) | $148 |
| + outlet pricing (step 4) | $54 |
Where to start
Start with step 2. Caching is the highest payoff per minute of work if your prompts share any prefix at all, and restructuring prompts for cacheability forces you to look at what you are actually sending. Teams routinely find kilobytes of dead weight in system prompts that have been shipping to production for months.
If you have a lever this list missed, drop it in the comments.
Top comments (0)