DEV Community

MT_Notes
MT_Notes

Posted on

Two Price Sheets in One Day, the Same $0.20 for Cache Reads: Why Multi-Model Routing Math Just Changed

#ai #apigateway

1. Background

On the afternoon of Tuesday, September 22, Anthropic and OpenAI shipped new models within about an hour of each other: Claude Opus 5.5 from Anthropic, and GPT-6 Sol plus GPT-6 Luna from OpenAI. Both labs put the price cut in the headline:

  • OpenAI priced Sol at $2 input / $10 output per million tokens and Luna at $0.10 / $0.50, a 50% cut against GPT-5.6 promotional pricing. The flagship Astra is unchanged at $10 / $50. OpenAI confirmed to the press that these are permanent prices, not promotional ones.
  • Anthropic priced Opus 5.5 at $4 / $20 (Opus 5 was $5 / $25), cut cache reads from $0.50 to $0.20, and says typical workloads cost 40% less than Opus 5 at default settings. Most coverage concluded that "Opus 5.5 is still twice the price of Sol." For coding agents and long conversations, that is only half right. One number is identical on both sheets: cache reads cost $0.20 per million tokens. And both labs say plainly that cache reads are where most agentic and coding spend goes.

2. The Technical Picture

2.1 Three price columns side by side, and where cache prices come from

(Per million tokens.)
On input and output, Opus 5.5 is exactly 2x Sol. On cache reads, they are equal. On cache writes, both charge 125% of the input rate: $2.50 for Sol, $5.00 for Opus 5.5.
The number worth remembering is the ratio between cache reads and the input price. The whole GPT-6 family sets it at 10% of input, a 90% discount. Opus 5.5 sets it at 5% of input, a 95% discount. So the $0.20 collision isn't coordination between the two labs. It's two independent policies -- "input costs twice as much" and "the read discount cuts twice as deep" -- landing on the same number by coincidence. That ratio outlives the absolute prices: when the next generation ships, you don't need to memorize a new table, just the multiplier.
2.2 Caching is already the default state, not an optional discount
This shows up most clearly in real traffic. OpenRouter's model page publishes the first-day distribution for GPT-6 Luna: on OpenAI's own endpoint, roughly 85% of input tokens were served from cache, with a hit rate near 86%. The result is a weighted-average input price of $0.0318 -- less than a third of the $0.10 list price.
Put another way, the price sheet describes a usage pattern that barely exists. A production agent reuses the same system prompt, tool definitions, and repository context on nearly every call, and that is precisely the part caching covers. Any cost estimate that bills every input token at the list price will be systematically too high.
2.3 Re-running the math on an agent session
Take a coding-agent session of 50 steps. Each step reuses a 60K cached prefix, appends 2K of new content (billed as a cache write), and produces 1K of output. For the whole session: 3M cache reads, 100K cache writes, 50K output.

On the same workload, the Opus 5.5 vs. Sol gap narrows from 2x on the sticker to about 1.56x, and the entire gap sits in the cache-write and output columns. The cache-read column is identical.
Then add tokens per task. Anthropic says Opus 5.5 at its default medium effort matches GPT-6 Astra on Terminal-Bench 4.0 at about 40% of the cost per task, and beats Astra on FrontierCode at roughly one fifth of the cost. OpenAI says Sol beats Opus 5 on AutomationBench at about 9% of the cost per task. The two "cheaper" claims were measured on different benchmarks against different rivals, and nobody has run Sol against Opus 5.5 head-to-head. The takeaway: list prices are good for ruling models out, not for making routing decisions.
2.4 Caching has two boundaries, and both are harder than price
The first is time. OpenAI is unusually explicit about the cost here: the cache discount applies only to shared prefixes reused inside a 30-minute window. Step away from a session for a meeting and come back, and that 60K prefix is no longer a cache hit -- it is billed at the input rate again. The longer the agent task and the more human checkpoints it has, the easier this boundary is to trip.
The second is ownership. Caches are stored per vendor-and-model pair. Switch vendor or switch model mid-session and the prefix is written fresh: the same 60K prefix costs 60K x $5 / 1M = $0.30 on Anthropic's side, while the identical step on Sol would have been $0.012 as a cache read -- roughly 25x less. Switching once is fine. Switching per step hands back everything you saved.
Together the two boundaries say one thing: a cache is a local asset, not a global one.
OpenAI pushed that second boundary outward inside its own family. GPT-6 caches now survive changes to reasoning effort and to tool definitions, and developers get explicit cache breakpoints, a Prompt Caching Dashboard, a diagnostics tool for misses, and a prewarming interface. GitHub reports that over the past several months these changes cut the share of prompt tokens requiring fresh processing by more than half. In engineering terms: turning effort up and down inside one vendor is now close to free, while swapping models to change difficulty still costs you a cache rebuild.
2.5 So routing needs two layers

  1. Pick the vendor per session. Decide once when the task starts. Try Opus 5.5 for long migrations and code audits, Luna for high-volume extraction and summarization, Sol for everyday coding -- then avoid switching for the rest of the session.
  2. Tune the tier per step. Inside a session, handle changes in difficulty with reasoning effort rather than a model swap, so the cache stays warm.
  3. Calibrate on cache hits. Watch the cached_tokens field in each response, not the price sheet.

3. In Practice: Session-Level Vendor Choice as One Parameter

This strategy only holds if switching vendors doesn't mean a new SDK, a new key, and another bill to reconcile. Accels is a Singapore-based company, and router.accels.tech exists to solve exactly those three problems:

  • Stable. One OpenAI-compatible endpoint serves models from multiple vendors, so you don't maintain separate integration and retry logic for each vendor's peaks and hiccups.
  • Complete catalog. The GPT-6 family and the Claude family sit behind the same interface. When a new model ships, you change one model string -- use the names listed in the console.
  • Unified billing. Usage across vendors lands in a single bill, so answering "what did this agent session actually cost?" doesn't mean stitching two dashboards together. Here is a minimal example that picks a model per session, keeps the vendor fixed inside it, and logs cache hits on every step: 
from openai import OpenAI

client = OpenAI(base_url="https://router.accels.tech/v1", api_key="your-accels-key")

# Session-level routing: choose once at the start, keep it for the whole session to preserve the cache
ROUTES = {
    "migration": "claude-opus-5-5",  # long migrations / code audits
    "coding":    "gpt-6-sol",        # everyday coding agents
    "extract":   "gpt-6-luna",       # high-volume extraction / summarization
}

def run_session(task_type: str, system_prompt: str, steps: list[str]):
    model = ROUTES[task_type]
    # Put the static parts first: system prompt, tool definitions, repo context
    messages = [{"role": "system", "content": system_prompt}]
    for step in steps:
        messages.append({"role": "user", "content": step})
        resp = client.chat.completions.create(model=model, messages=messages)
        messages.append({"role": "assistant", "content": resp.choices[0].message.content})
        details = getattr(resp.usage, "prompt_tokens_details", None)
        cached = getattr(details, "cached_tokens", 0) if details else 0
        print(f"{model} prompt={resp.usage.prompt_tokens} cached={cached}")
    return messages
Enter fullscreen mode Exit fullscreen mode

Two practical tips. Put the parts that never change at the very front so the prefix stays long and stable. And log cache hits on every step, then use those numbers rather than list prices to judge which route actually saves money.

4. Closing

The two price sheets of September 22 look like a price war. For engineers they raise a more specific problem: caching is now the biggest line item in agent costs, and caches are tied to a vendor, tied to a model, and expire in 30 minutes. So good multi-model orchestration is no longer "pick the cheapest model at every step." It is "pick the right vendor for each session, change effort rather than models inside it, and keep the cache warm." Anthropic has said Sonnet 5.5 and Haiku 5.5 follow in the coming weeks, so these sheets will change again. Make the model name a config parameter and route through router.accels.tech for one endpoint and one bill. Then when the next price cut lands, you change one line of config and rerun the comparison.

Sources

Top comments (0)