In a previous post I built a harness for probing where an AI coding agent's boundaries actually are. This follow-up is about a problem that showed up immediately after: once you know what an agent can do, you still have to decide which model should do it — and routing everything to the most expensive model is a quiet budget leak.
Strong cheap models keep shipping, and frontier models keep getting better at the hard stuff. The practical question isn't "which model is best" — it's "which tasks in my day actually need the best model." This post describes the routing setup I use, with a small reproducible artifact you can adapt.
The observation that started this
I logged two weeks of my own agent-assisted coding tasks and labeled each one after the fact: did the task actually require deep reasoning, or would any competent model have done? The split was roughly:
- Mechanical transforms (rename + fix imports, generate boilerplate tests, convert config formats): a mid-tier model succeeded nearly every time.
- Local reasoning (explain this function, find why this test flakes): mid-tier was fine most of the time.
- Cross-cutting design or subtle bugs (race conditions, API contract decisions, refactors touching many files): this is where weaker models produced plausible-but-wrong output that cost me more time to review than it saved.
So the optimization target isn't model quality. It's review cost. A wrong answer on a mechanical task is cheap to catch (the test suite catches it). A wrong answer on a design question is expensive to catch (you are the test suite).
A three-tier routing rule
| Tier | Task type | Failure is caught by | Route to |
|---|---|---|---|
| 1 | Mechanical, verifiable by tests/linters | CI | Free / cheapest model |
| 2 | Local reasoning, single-file scope | Quick skim | Free / cheap model |
| 3 | Multi-file design, concurrency, security, irreversible changes | Only human review | Strongest model you have |
The rule I follow: a task starts at the cheapest tier, and only gets escalated when the failure detector is a human. If a test suite, type checker, or linter can catch a bad answer, there is no reason to pay frontier prices for it.
The reproducible part: a pre-flight classifier
Instead of deciding vibes-based at prompt time, I run a tiny classifier first. It's deliberately dumb — heuristics, not ML — because I want it auditable:
# route.py — decide tier before sending a task to an agent
import re
TIER3_SIGNALS = [
r"race|deadlock|concurren|async.*order",
r"auth|token|secret|permission|inject",
r"migrat|schema change|breaking",
r"refactor.*(across|all|whole)",
]
def tier(task: str, files_touched: int, has_tests: bool) -> int:
t = task.lower()
if any(re.search(p, t) for p in TIER3_SIGNALS):
return 3
if files_touched > 3 and not has_tests:
return 3 # no safety net + wide blast radius
if files_touched <= 1:
return 2
return 1 if has_tests else 2
if __name__ == "__main__":
print(tier("rename getUser to fetchUser and fix imports", 6, True)) # 1
print(tier("explain why this parser drops trailing commas", 1, True)) # 2
print(tier("possible race in the retry logic", 2, True)) # 3
The inputs (files_touched, has_tests) come from whatever scaffolds the task — in my case the agent harness reports them before execution. The point isn't the exact thresholds; it's that the routing decision is explicit, logged, and changeable. After a week you can count how often each tier's output had to be redone, and tighten or loosen the signals with actual data.
Where the free tier lives in this setup
Tiers 1 and 2 need a model that's always available and costs nothing, because that's where most of my volume goes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I route tier-1/2 tasks through MonkeyCode since it offers free model access and a free server option, which maps neatly onto the high-volume, low-stakes tiers — I don't have to think about per-call cost when the router sends a boilerplate-generation task down. Tier-3 tasks, where review cost dominates, still go to whichever frontier model I'm currently trusting for design work; the free tier doesn't change that part of the decision.
One honest caveat on free tiers in general: expect variability. Latency, availability, and which models are offered can change, so the router treats the free endpoint as the default, not a guarantee — if a tier-1 task fails or times out, it retries once, then escalates one tier rather than looping.
Limitations, and who shouldn't do this
- The classifier is only as good as its signals. Anything the regex list doesn't anticipate defaults to a lower tier. The mitigation is the escalation-on-failure behavior plus logging, but if you skip the logging, you're flying blind.
- Tier 3 misclassification is the expensive failure mode. A concurrency bug routed to a weak model can produce confident nonsense. When in doubt, route up — the table's last column is the one to be conservative about.
- If your tasks are almost all tier 3 (security-critical systems, novel algorithm work), routing saves you nothing; just use the best model.
- If you can't verify tier-1 output mechanically (no tests, no linters, no type checker), the whole premise breaks. Write the harness first.
If you want to try this, start with the logging before the routing: label a week of tasks by hand, see your actual tier distribution, and only then automate the split. The router is the easy part — knowing your own workload is the artifact that matters.
Top comments (0)