Originally published on the TierUp blog. Per-token prices fell ~280x in two years and enterprise AI budgets still tripled — here's the math behind the paradox.
Here's the paradox defining AI budgets in 2026: per-token prices have been in freefall, and total spend keeps going up anyway. Henon's analysis leads with the headline version — token prices fell 98% while enterprise AI costs tripled. Oplexa's inference-cost report, citing Epoch AI and AnalyticsWeek data, frames it even more starkly: effective per-token costs down roughly 280x over two years (from ~$30/M in 2023 to ~$0.10/M for comparable capability in 2026), while average enterprise AI budgets grew from about $1.2M in 2024 to $7M in 2026 — and inference now eats ~85% of the AI budget, up from 40% in 2023.
Falling prices didn't fail. Volume won.
Where the volume comes from
Agents multiply calls. A chatbot answers a question with one model call. An agent plans, calls tools, reads results, retries, and self-checks. Gartner's March 2026 analysis, as cited by Oplexa, found agentic workflows make 10–20 LLM calls per user-initiated task and consume 5–30x more tokens than a standard chatbot interaction. Every product that quietly upgraded from "chat" to "agent" this year multiplied its token volume by an order of magnitude without changing its pricing page — or yours.
RAG inflates every call. Retrieval-augmented requests carry 3–5x more tokens than the bare question, per the same Gartner-cited analysis. That's the point of RAG — but it means your input volume scales with your document chunking strategy, not your user count. And as we covered in the tokenizer tax post, fat contexts can also push you across long-context pricing thresholds.
Always-on beats per-request. Monitoring agents, background summarizers, and scheduled pipelines consume tokens around the clock whether or not a human is watching. Usage stops tracking headcount.
Humans, given leverage, use more of it. TechCrunch's June 2026 report on the industry's cost scramble has the receipts: Jellyfish's research head measured per-developer token consumption rising ~18.6x in nine months. Their study found the heaviest token users were about twice as productive — but spent 10x more tokens getting there. Uber reportedly blew through its entire 2026 AI coding budget by April. Priceline saw a Cursor renewal come back 4–5x more expensive, with one engineer spending $40,000 on tokens in a single month. One company reportedly discovered a $500 million Claude bill after failing to set usage limits.
The pattern across all four: cost per token fell, tokens per outcome exploded, and outcomes per user grew. Multiply three curves and the product points up.
The mitigation checklist
The response emerging across the industry — TechCrunch describes the conversation shifting wholesale from capability to "guardrails," and a Tokenomics Foundation standards body launching this month — amounts to FinOps for AI. The practical version:
- Route by task difficulty. Most calls in an agent loop are glue — classification, extraction, formatting — and don't need a frontier model. Oplexa reports model routing cutting spend 60–80%, the single largest lever on their list.
- Set hard budgets and per-group limits. Priceline's approach per TechCrunch: token limits on employee groups. Alerts are not limits; limits are limits. (See also: the reported $500M bill.)
- Cache aggressively. Prompt caching (up to 90% off cached input) and semantic caching (30–50% savings per Oplexa) attack the RAG-inflation problem directly.
- Batch what isn't interactive. Batch APIs run 50% off at major providers. Background summarizers and nightly pipelines rarely need real-time pricing.
- Cap agent loops. Set maximum iterations and maximum tool calls per task. An agent that retries itself into a 20-call loop is a cost incident, not a feature.
- Trim retrieval. Measure whether your 3–5x context inflation actually improves answers. Rerank harder, stuff less.
- Meter tokens per outcome. Track tokens-per-resolved-task, not spend-per-month. It's the only metric that separates "we're doing more" from "we're wasting more."
The honest takeaway
Rising AI spend isn't automatically a problem — Jellyfish's data shows the heavy spenders really were more productive. The problem is unexamined spend: frontier models doing glue work, uncapped loops, and nobody owning the tokens-per-outcome number. Prices will keep falling. Your bill will keep rising. The only variable you control is how much of that bill buys something.
Routing every call to the cheapest tier that clears your quality bar is item one on the checklist — and it's the entire premise of TierUp.
Top comments (0)