We rebuilt our agent stack around model routing about two months ago. This is what we learned, including the part where our first version was worse than doing nothing.
The setup that broke
Our research agent reads vendor documents, extracts claims, checks them against each other, and drafts a recommendation for a human to sign. Forty or so memos a month.
We built it on one frontier model because that's what was in the tutorial we started from. Every step, same model. It worked well enough that nobody looked at it for a quarter, which is exactly how these things go.
Then we looked at the bill.
The distribution was the problem. About 4% of our calls were doing something a cheap model would get wrong. The other 96% were classification, extraction, formatting, and retries, where a cheap model gets the same answer. We had been paying a premium on all of it.
Attempt one: a global downgrade (don't do this)
Our first fix was to swap the whole pipeline to a cheaper tier and see what broke.
Plenty broke, but not where we expected. Classification and extraction were fine. Formatting was fine. What fell apart was the step where the agent had to look at two documents that disagreed and decide which one to trust. The cheap model didn't fail loudly there. It picked one, wrote a confident sentence about it, and moved on. The error flowed straight into the memo.
That's the lesson that reorganized how we think about this. Cheap models don't fail by producing garbage. They fail by producing something plausible that survives review. On a classification task you notice immediately. On a judgment task you notice when a client does.
Attempt two: route by blast radius
The question we now ask at every step is not "is this hard." It's:
If this step returns a mediocre answer, how far does it travel before someone catches it?
That reframing did more for us than any model comparison.
Blast radius of one row goes to the cheapest tier that passes. A misclassified document gets caught by the next step or by a human scanning a list. Wrong is cheap here. This is most of the pipeline.
Blast radius of a section goes to a balanced tier. Structuring evidence, drafting a paragraph, ordinary summarization. Errors are visible on read-through.
Blast radius of the whole deliverable goes to the premium tier. For us that's exactly two steps: reconciling sources that contradict each other, and the final pass before a human signs. Under 5% of calls.
Blast radius beyond the deliverable goes to a human. Anything touching a customer, money, or production.
Our spend dropped by a little over half. Memo quality went up slightly, which I attribute mostly to the fact that writing this table forced us to articulate what each step was for. Half the win had nothing to do with models.
Why the tiers actually differ
Worth understanding why this works rather than cargo-culting the table.
When Anthropic launched Claude Fable 5 it made a claim that reads like marketing but functions as engineering guidance: the model's lead grows as tasks become longer and more complex. Anthropic says it's a Mythos-class model made safe for general use whose capabilities exceed any model the company has made generally available, and that it stays focused across millions of tokens on long-running tasks while improving its output using its own notes. Anthropic says Stripe used it to complete a codebase-wide migration in a 50-million-line Ruby codebase in a day, against more than two months of manual work.
The advantage is conditional on difficulty. On a short easy call the tiers converge and you're paying for a margin you can't measure. On a long tangled one the gap is your product.
That's the whole justification for routing. If the advantage were flat across task types, routing would be pointless and you'd just buy the best model. It isn't flat.
The pricing math
Anthropic lists Fable 5 at $10 per million input tokens and $50 per million output tokens.
The trap in agent workflows is that spend doesn't scale with tasks, it scales with turns. One "check these twelve documents" task is not twelve calls. It's twelve reads, a pile of pairwise comparisons, a revision, a validation pass, and however many retries. The loop is the product and the loop is the cost.
We run on the Buda AI Agent Workspace, where Fable 5 is a subscription-only premium tier rather than the default, and the credit multipliers are stated where you pick: Sonnet 4.6 at 1.0x, Opus 4.8 at 1.7x, Fable 5 at 3.3x. The specific stack matters less than the property. If the cost of a routing decision isn't visible when you make it, you'll make it once in week one and never revisit it.
The persistent workspace turned out to matter more than the multipliers. The reason we resisted routing for so long was a worry that switching models mid-workflow would lose the thread. When every step reads the same context, that stops being a concern and routing costs you nothing to adopt.
Gotchas
The safety fallback. Anthropic says Fable 5 includes safety classifiers and that some cybersecurity, biology, chemistry, and distillation-related requests may automatically get a response from Claude Opus 4.8 instead, at under 5% of sessions on average. We hit this on a security-review workflow and spent a day chasing a nonexistent bug. Log which model actually answered. It's a one-line change that will save someone an afternoon.
Retries are invisible. Each retry looks cheap, so nobody instruments them. Ours were about 2.5x my estimate. Track retries per task per tier before you trust any cost model you've built.
Don't route by vibes. "This step feels important" is not a criterion. Blast radius is, and two people looking at the same step will usually agree on it.
What I'd do differently
Start with the routing table, not the model choice. Walk your pipeline, mark each step by how far a bad answer travels, and you'll usually find the expensive question is a small identifiable minority you've been funding by inertia.
The best model available is the right choice for the steps where being wrong is expensive. It's a bad default for everything else, and everything else is most of your bill.
Top comments (0)