TL;DR: Claude Opus 5.5 came out yesterday with cache reads at $0.20 per million tokens, 60% cheaper than Opus 5. I redid the cost math for my text-to-SQL CRM assistant. For one design choice, the cost gap between "retrieve only the relevant schema" and "send the whole schema every time, cached" shrank from 2.1x to 1.2x. At that point the choice stops being about cost and becomes about accuracy. The migration guide also has two breaking changes that hit text-to-SQL setups directly.
Some context: how Aria picks the schema to send
A few months ago I wrote about Aria, an AI assistant that lets CRM agents ask questions in plain English and get answers from live SQL.
The core trick is RAG over the schema, not over the data. A Python pipeline writes a plain-English description of every table, column and enum value in the CRM (~90 docs in total). When an agent asks a question, I embed the question, run a pgvector search over those docs, and send only the top 5 to the model along with the question.
Here's why I built it that way:
- Cost. I didn't want to pay for the whole semantic layer on every request.
- Focus. I reasoned that a model shown 5 relevant tables writes better SQL than one shown all 15.
Opus 5.5's pricing mostly takes away reason #1. That leaves reason #2, which I never actually measured.
The pricing change that matters here
| Opus 5 | Opus 5.5 | |
|---|---|---|
| Input | $5 / MTok | $4 / MTok |
| Output | $25 / MTok | $20 / MTok |
| 5-min cache write | $6.25 / MTok | $5 / MTok |
| Cache read | $0.50 / MTok | $0.20 / MTok |
Most of the headlines are about the 20% cut on input and output. For a schema-heavy workload, though, cache reads matter most. They used to cost 0.1x the input price and now cost 0.05x.
The napkin math
These are my assumptions. Plug in your own numbers:
- 800 questions/day (40 agents × ~20 questions, which is roughly the load I described in my last post)
- Full semantic layer: ~20k tokens (~90 docs × ~220 tokens)
- Top-5 retrieval: ~1.1k tokens per question
- ~10 cache writes/day. The default cache lives 5 minutes, and every hit refreshes it for free. 800 questions over a 9-hour workday is about 1.5 per minute, so the cache stays warm most of the day and only goes cold at the start of the day or after lunch.
I'm only counting the schema part of the prompt, because that's the only part that differs between the two designs.
Option A: retrieve top 5, no caching (the retrieved docs change on every question, so there's no stable prefix to cache)
Option B: send the full semantic layer on every request as a cached prefix
| Per day | Option A: retrieve | Option B: full + cache | B vs A |
|---|---|---|---|
| Opus 5 | $4.40 | $9.25 | 2.1x |
| Opus 5.5 | $3.52 | $4.20 | 1.2x |
Per question on Opus 5.5, that's $0.0044 vs $0.0053. Over a 22-working-day month, it's about $77 vs $92.
On Opus 5, "just give the model everything" cost twice as much, which made the pgvector step easy to justify. On Opus 5.5, it costs about $15 a month more across the whole team.
At that price, cost no longer decides the question. Accuracy does.
What Option B would get me
- No embedding call or vector search before each question. That's one less network hop and one less thing that can fail.
-
No retrieval misses. With retrieval, if the question says "stale leads" and the embedding doesn't pull up the doc that explains
last_activity_at, the SQL is wrong before the model even starts. With the full layer, the model can always see every table. - A static prompt prefix, which matters for the next section.
What it wouldn't replace: my intent examples, the question→SQL pairs that get promoted from thumbs-up feedback. That set grows over time, so it still belongs in a retrieval step. The likely end state is a hybrid: a static cached schema plus retrieved examples.
Two breaking changes that hit text-to-SQL setups
I read the Opus 5.5 migration guide with Aria in mind. Two changes stood out.
1. Forced tool use now returns a 400
tool_choice of type tool or any is rejected. Many text-to-SQL setups force the model to call their run_sql tool so it can't answer from memory. On Opus 5.5, the fix is tool_choice: auto, marking the tool as strict, and saying clearly in the prompt when the tool must be used.
My SQL validator doesn't change. That step never trusted the model anyway: SELECT-only checks, agent-ID injection from the JWT, and a read-only Postgres role. Moving to a better model shouldn't move your trust boundary.
2. Conversations should be append-only
Thinking is always on in Opus 5.5 (you control it with effort instead of turning it off). The guide says to keep conversations append-only, with no edits to system, tools or earlier messages mid-conversation. For newer accounts, replaying a thinking block after such an edit returns a 400 by default.
That's a problem if I move my current design over to Opus 5.5 unchanged. I rebuild the system prompt on every turn with a new set of retrieved schema docs. That's an edit to system in the middle of a conversation.
So there are two ways to comply:
- move the retrieved docs out of
systemand into each new user turn, or - make
systemstatic, which is Option B again.
Both the pricing and the API now push in the same direction: put the large, rarely changing context in a static prefix, cache it, and add new context only by appending.
Here's roughly what the request would look like:
{
"model": "claude-opus-5-5",
"max_tokens": 1024,
"output_config": { "effort": "low" },
"system": [
{ "type": "text", "text": "You write read-only PostgreSQL for a student-housing CRM. Always answer by calling query_crm_database." },
{ "type": "text", "text": "<full semantic layer: every table, column, enum>",
"cache_control": { "type": "ephemeral" } }
],
"tools": [{ "name": "query_crm_database", "strict": true, "...": "..." }],
"tool_choice": { "type": "auto" },
"messages": [
{ "role": "user", "content": "<retrieved intent examples>\n\nWhich of my leads haven't been contacted in 3 days?" }
]
}
(I'd use effort: low for the formatting pass and try medium, the new default, for SQL generation.)
What I'm going to measure
I haven't switched Aria over yet. This is napkin math and a close read of the docs, not a benchmark. The plan:
- Take the 30 question→SQL examples my schema pipeline already generates and use them as an eval set.
- Run both options: top-5 retrieval vs full cached layer.
- Compare how often the generated SQL returns the same rows as the reference SQL, plus latency and actual cost from the
usagefields (cache_read_input_tokenstells you whether caching actually happened).
If the full-context version is at least as accurate, I'm removing the pgvector step from the question path. I'll post the numbers when I have them.
Question for you: if you're doing text-to-SQL, do you retrieve schema per question or send all of it? And has cheaper caching made you rethink the pieces around RAG? I'd love to hear in the comments.
Top comments (1)
The join-table retrieval miss is what killed dynamic schema retrieval for us long before cache pricing dropped. A vector search on 'stale leads' pulls the leads table and maybe activities, but it almost always drops the join table or audit enum that actually links the two. The model gets a hallucinated join condition because the glue schema was ranked 7th instead of top 5.
At 20k tokens, putting the full schema into a static cached prefix completely sidesteps that failure mode. The only thing I would watch in production with the 5-minute ephemeral cache is latency jitter after brief idle gaps. When an agent pauses for six minutes between calls, that first turn takes the full cache-write recomputation latency before it warms up again. Keeping the examples in the append-only user turn and the DDL pinned in the system block is the cleanest separation for that.