TL;DR: OpenAI and Anthropic have cut LLM prices by up to 80% since July 2026. Before you celebrate, check which line of your bill actually moved. For agent-style workloads, cached input dominates spend, small models now handle most "AI feature" work for pennies, and the safest architecture is one that lets you switch models with a config change. Here's how I'd re-plan an app's AI costs this week.
What changed
Max Woolf published a clear breakdown of the recent price cuts on 25 October. The short version:
- OpenAI's small "Luna" tier dropped about 80% in July, then halved again with GPT-6 Luna in September.
- Anthropic made a Sonnet 5 launch discount permanent, cut cache-read pricing hard on Fable 5.1, and cut Opus 5.5 by 40–60% overall compared to Opus 5.
- OpenAI's flagship "Sol" models got cheaper in steps through August and September, with GPT-6.1 Sol halving cache reads again.
His explanation is competition: open-weight models and dozens of inference providers sell near-identical quality, so price is the lever left. Whatever the cause, it changes the maths for a lot of features that were "too expensive" in the spring.
But the headline number hides the part that matters for your app.
1. Find out which tokens you're actually paying for
Most of us look at the input and output price and stop there. The same write-up points out that in agentic coding, cache reads are 96–99% of token spend. Agents re-send the same system prompt, tool definitions and file context on every turn, and almost all of it is a cache hit.
So for an agent, a 75% cut on cache reads matters more than a 20% cut on output. For a one-shot summariser with no shared prefix, the reverse is true.
Before deciding anything, split your usage into those buckets. Both major APIs return usage per request. A tiny logger is enough to start:
// usage-log.ts
type Usage = {
feature: string;
model: string;
inputTokens: number;
outputTokens: number;
cacheReadTokens: number;
cacheWriteTokens: number;
};
export function logUsage(u: Usage) {
// Swap console for your DB or analytics later.
console.log(JSON.stringify({ ...u, at: new Date().toISOString() }));
}
Log it per feature, not just per model. "The AI bill" is useless. "The ticket-triage feature costs X per 1,000 tickets" is something you can make decisions about.
2. Structure prompts so the cache actually hits
If cache reads are now very cheap, the win is making sure your requests are cacheable in the first place. The rule is simple: stable content first, changing content last.
With Anthropic's Messages API, you mark the end of the stable prefix with cache_control:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
export async function answerWithDocs(docs: string, question: string) {
return client.messages.create({
model: process.env.LLM_MODEL!, // keep model IDs in config
max_tokens: 1024,
system: [
{ type: "text", text: "You answer questions using only the provided docs." },
{
type: "text",
text: docs, // large, rarely changes -> cache it
cache_control: { type: "ephemeral" },
},
],
messages: [{ role: "user", content: question }], // changes every call
});
}
OpenAI applies prompt caching automatically on long prompts, but the same ordering rule applies: anything that changes per request (timestamps, user names, the question) belongs at the end. A single Today is ${new Date()} at the top of your system prompt quietly breaks the cache for everything after it.
Things that commonly bust the cache:
- Dynamic values in the system prompt
- Tool definitions generated in a different order each time
- Re-serialising JSON context with unstable key order
3. Route by task, not by habit
Lots of products send everything to the flagship model because that's what the prototype used. Now the small tiers cost cents per million tokens, so it's worth being deliberate.
A simple router covers most apps:
// llm-router.ts
type Task = "classify" | "extract" | "summarize" | "reason" | "code";
const MODEL_FOR: Record<Task, string> = {
classify: process.env.LLM_SMALL!,
extract: process.env.LLM_SMALL!,
summarize: process.env.LLM_SMALL!,
reason: process.env.LLM_LARGE!,
code: process.env.LLM_LARGE!,
};
export function pickModel(task: Task): string {
return MODEL_FOR[task];
}
Then measure. Run a few hundred real inputs through the small model and compare with the large one on the cases you care about. If the small model is right often enough, ship it. If not, you've lost an afternoon, not a quarter.
I'd keep one honest caveat here: cheaper models are not automatically good enough. A specialised, boring approach (a regex, a classic classifier, a database query) still beats an LLM for plenty of tasks. Price cuts don't change that.
4. Put the provider behind one function
The other side of commoditisation: if quality is close and prices keep moving, switching providers becomes a normal operation. Make it cheap.
# llm.py
import os
from dataclasses import dataclass
@dataclass
class LLMResult:
text: str
input_tokens: int
output_tokens: int
def complete(prompt: str, *, task: str) -> LLMResult:
provider = os.environ["LLM_PROVIDER"] # "anthropic" | "openai" | ...
if provider == "anthropic":
from .providers import anthropic_complete
return anthropic_complete(prompt, task=task)
if provider == "openai":
from .providers import openai_complete
return openai_complete(prompt, task=task)
raise ValueError(f"Unknown provider: {provider}")
Nothing clever here, and that's the point. Your app code calls complete(). Model IDs, providers and fallbacks live in config. When the next price change lands, you change an environment variable and re-run your evals.
5. Re-check the features you shelved
This is where the price cuts can make actual product money. Go back through the AI ideas you dropped for cost and price each one at today's rates:
| Question | Why it matters |
|---|---|
| Which tier does it really need? | Small-model pricing changes most features from "no" to "yes" |
| How much of the prompt is a stable prefix? | Cheap cache reads reward long, shared context |
| What does it cost per user per month? | Compare against what that user pays you |
| Does it still work at 2x today's price? | Prices can move in either direction |
That last row is my rule of thumb. If a feature only makes sense at this quarter's prices, build it so you can swap models or switch it off without a migration. If it still makes sense at double the price, it's a real feature, so build it properly.
What I would not do
- Don't jump to the most expensive model just because it's cheaper than last month. Measure first. Bigger isn't always better for narrow tasks.
- Don't hard-code a model ID in twenty places. You will want to change it, probably soon.
- Don't assume subscription plans follow API prices. Woolf notes he cut his own subscription quota, and he expects plan allowances may shrink before API prices rise.
Wrapping up
The price cuts are real, and they make a lot of AI features newly viable. The developers who benefit most will be the ones who know which tokens they pay for, keep their prompts cache-friendly, and can switch models without a rewrite.
Which line of your LLM bill surprised you most when you finally broke it down: input, output or cache?
I build web and AI products for clients; more about my work at papansarkar.com.
Top comments (6)
Answering your closing question from a session I actually broke down: the line that surprised me was cache-write, not cache-read. Reads were most of the token count, exactly as you say, but they're the cheap direction, and now that providers are cutting cache-read pricing hardest, the gap between a read and a rewrite gets wider, not narrower. The cheaper reads get, the more a single cache miss stings relative to a clean turn.
That changes what the usage logger is for. Splitting the buckets (you've got it right, cacheReadTokens and cacheWriteTokens as separate fields) matters less for the steady state than for catching the handful of turns where cached input collapses and the whole prefix gets re-sent at write price. In the session I measured, a small number of turns carried almost all the cache-write cost, and they lined up with the prefix breaking (a tool list reordering, an MCP server reconnecting) rather than with the hard work. A workload that's 99% cache-read in aggregate can still have its bill decided by the 1% of turns where the cache busts.
So one line I'd add to your "things that bust the cache" section: watch the per-turn cache-write series, not just the totals. The total looks flat and cheap; the spikes are where the money is, and they're roughly invariant to the price cuts, because on those turns you're paying the write rate, which moved far less than the read rate.
Great point, and I think you're right. Reads tell you where the tokens go, but the cache-busting turns decide the bill, and that gap only widens as reads get cheaper. Tool reordering and MCP reconnects are sneaky because nothing in the code looks wrong. I'll add "watch per-turn cache writes, not just totals" to the post. Thanks for measuring it.
Task-based routing to cheap models usually looks great until you measure the error distribution. If a small tier cuts sticker price by 80% but misses a schema or hallucinates a parameter 8% of the time, the expected cost flips. In multi-step pipelines, a cheaper model rarely fails cleanly on turn one. It drifts for two or three exploratory tool calls, expands the prompt history, and then forces an automated fallback to the frontier model anyway. By that point you pay for the dead-end turns, the expanded transcript, and the large model run. Nominal token deflation creates an illusion of free margin, but the actual clearing cost is determined per completed task outcome rather than per million tokens on the rate card.
One important point here is that cheaper tokens don't always translate into cheaper AI workflows. Cache misses, retries, fallback calls, and model inaccuracies can quickly increase the actual cost per completed task. At IT Path Solutions, we recognize the importance of balancing AI cost optimization with performance, reliability, and output quality when developing AI-powered applications. Tracking cost alongside task success rate gives a much more realistic picture of AI economics. The idea of keeping model selection configurable is also a smart way to stay flexible as pricing continues to change.
Logging per feature instead of per model is the part I would keep. The AI bill on its own never tells you which feature is worth it.
The list price only moves the tokens you send, and caching is the line that changes most for an agent that resends its history. Did your invoice drop as much as the headline?