This week in the AI price war: OpenAI cut GPT-5.6 Luna pricing by 80% and GPT-5.6 Terra by 20%. Claude Opus 5 landed on Amazon Bedrock holding at $5 in / $25 out per million tokens with a 1M context window. Every headline says the same thing: intelligence is getting cheaper, fast.
So why does every engineering team I talk to report the same thing — the AI line on the cloud bill went up again this quarter?
Because tokens are the only part of the stack getting cheaper, and tokens are becoming the smallest part of the bill. Let's do the math.
Jevons paradox, but for tokens
In 1865, economist William Jevons noticed that more efficient steam engines didn't reduce coal consumption — they increased it, because efficiency made steam viable for things it was previously too expensive for.
Swap coal for tokens. When Luna gets 80% cheaper, teams don't pocket the savings — they take workloads that were marginal at the old price and turn them on:
- The code-review bot that was "too expensive to run on every PR" now runs on every PR.
- The log-summarization job that ran daily now runs hourly.
- The agent that answered questions now acts — and an acting agent burns 10–50× the tokens of an answering one, because plan/execute/verify loops are token furnaces.
An 80% price cut followed by a 10× usage increase is a 2× bill increase. That's not a failure of discipline; it's the price cut working exactly as intended — for the vendor.
The bill is quietly changing shape
The more important shift: token spend is becoming the minority of AI infrastructure cost. Here's the stack that came online around it this year:
1. Agent runtime. AWS Bedrock AgentCore, Azure Foundry Agent Service, Vertex AI's agent stack — every major cloud shipped managed agent infrastructure this year. Runtime, gateway, identity, managed memory: each is a new metered line item that didn't exist on your 2024 bill. You're not paying for intelligence; you're paying for the scaffolding around intelligence, and scaffolding doesn't get 80% cheaper on a Tuesday.
2. Agent memory and state. Long-term memory stores, vector databases, session persistence. Memory is storage + retrieval compute, priced like storage + compute — on the classic cloud cost curve (slow decline), not the model cost curve (cliff dives).
3. Observability. Tracing what an agent did, evaluating outputs, storing full conversation traces for audit. Teams routinely discover their LLM observability spend rivals their token spend — you're storing and querying every token twice.
4. The GPU floor. If you run any inference yourself, you know the dirty secret: self-hosted model economics are dominated by utilization, and bursty agent workloads are utilization poison. A GPU node pool sized for peak agent activity idles most of the day at full price.
Rough shape of what I see in real accounts: what was ~80% tokens / 20% everything-else in 2024 is heading toward ~30% tokens / 70% runtime + memory + observability + GPU — while total AI spend grows quarter over quarter.
Why this is a FinOps problem now
For two years, AI cost had a comforting story: "wait six months, the price drops." True for tokens. Irrelevant for the rest of the stack — the rest of the stack is ordinary cloud infrastructure, and it responds to ordinary FinOps levers, not to model-vendor price wars:
- Idle agent runtimes and GPU pools obey the same physics as idle EC2 — schedule them. An eval environment's GPU node group has no business running at 3am (we schedule ours to sleep the same way we schedule staging — same tooling, ZopNight treats a GPU node group like any other schedulable resource).
- Observability retention is a knob. Ninety days of full traces for a chatbot is a choice, and it's usually the default, and the default is expensive.
- "Cheaper model" is a rightsizing decision. Luna at −80% makes model selection a cost lever on par with instance selection. Routing the easy 70% of requests to the cheap tier is this year's version of moving dev to spot instances.
- Anomaly detection needs to cover AI resources. An agent stuck in a retry loop is the new "forgot to turn off the p4d instance" — invisible on daily billing granularity until it's a very visible number.
The takeaway
The price war headlines are real, and they will keep coming — Luna won't be the last 80% cut. But "tokens got cheaper" and "AI got cheaper to run" stopped being the same sentence sometime this year. The bill's center of gravity moved into the infrastructure around the model, and that part doesn't do price-war cliff dives. It does what cloud bills have always done: grow quietly until someone looks.
Is anyone actually seeing their total AI spend fall after a price cut? I keep asking and I have not found one yet — if you're the exception, I'd genuinely like to know what you're doing differently.
Top comments (0)