Originally published on AI Tech Connect.
Why routing is the biggest lever you have Every cost-reduction technique for LLM workloads sits somewhere on a spectrum of effort against return. Prompt caching is cheap to enable and saves on repeated prefixes. Compression trims tokens at the margin — we covered the mechanics in our guide to prompt compression in production. Batch processing halves the price of anything that can wait. All worth doing. But none of them touches the single largest line in most inference budgets, which is this: the price gap between model tiers is a multiple, not a percentage, and most teams send everything to the top tier. As of August 2026, Claude Haiku 4.5 costs $1 per million input tokens and Claude Opus 5 costs $5 — with output at $5 against $25. That is a five-times spread inside one vendor's line-up,…
Top comments (0)