If your agent's token spend keeps climbing, the model isn't the problem.
You are paying, per query, to re-explain your schema to something that should already know it.
Where the money actually goes
| Cost | Cause | Fix |
|---|---|---|
| Schema in every prompt | No resolved context layer | Resolve against the graph, not the prompt |
| Retry loops | Generated SQL fails or returns empty | Prove the join before execution |
| Multi-turn clarification | Ambiguity handled in conversation | Typed intent resolves it in one pass |
| Bigger model tier | Bought to compensate for missing context | Fix the context instead |
| Warehouse compute | Failed and re-run queries | Fewer, correct queries |
Notice that four of the five are the same root cause wearing different clothes: the model is being asked to do work that belongs upstream.
The arithmetic
Stuff a schema dump into the context window and every question re-pays for it. Add retries and you pay several times per question. Then someone proposes a larger model to improve accuracy, and the per-token price goes up on a volume that was already inflated.
Teams read that curve as "AI is expensive." It isn't. Re-deriving meaning at inference time is expensive.
Compiled context inverts the economics
When resolution happens against a semantic graph instead of inside the prompt, the model receives a small typed intent rather than your entire warehouse. Fewer tokens in, fewer retries, smaller model sufficient.
The bill drops and accuracy rises for the same reason — the work moved to the layer that can do it deterministically.
That is the part worth internalising: token cost and accuracy are not a trade-off you have to manage. They are both downstream of the same architectural decision.
The full breakdown — the cost model, worked examples of prompt-side vs compile-side resolution, and where the savings actually land — is here:
👉 Token Cost: Why Brittle Semantic Layers Bleed Capital
Originally published at colrows.com/blogs/token-cost-hidden-tax-semantic-layer
Top comments (0)