The Router Is the Harness
Every production agent harness already runs a classifier in its control loop. Claude Code gates tool calls through a background safety classifier on Sonnet 4.6 before a single file is touched. Gemini CLI stacks seven routing layers deep, each one a different classification strategy, and the first to produce a decision wins. Codex and Cursor enforce their own permission classifiers before execution. None of them call it a "router model." None of them sell it as a product. But every one of them proves the same thesis: the decision layer in an agent system is a classification problem, not a generation problem, and the industry has been paying generation prices for it.
π Read the full version with charts and embedded sources on AgentConn β
TypeSafe's Jev, launched September 15, 2026, did not invent this pattern. It gave it a name, a price tag, and a typed API. The timing was not accidental. Within days, LangChain published a full walkthrough on building an agent harness with Jev as the decision layer, including a Request Router that picks the right model tier and a Tool Gate that blocks risky calls before execution. The walkthrough is demand proof: builders already want this separation, and now there is a purpose-built model to slot into the gap.
This article is not about Jev the product. We covered that already, including the calibration question TypeSafe still owes the community. This is about the pattern β the architectural convergence toward separating classification from generation in the agent control loop β and how to implement it today, whether you use Jev, a fine-tuned classifier, or a prompted frontier model with structured outputs.
The Pattern Was Already There
Rewind to before Jev existed. What does an agent's control loop actually do between receiving a user request and producing a result?
A recent source-code analysis of 13 production coding agents β Claude Code, Codex, Cursor, Gemini CLI, Aider, and others β found that the dominant architectural pattern is not a single model call. It is a loop of decisions punctuated by model calls. The agent decides what to do next, whether to use a tool, which model to invoke, whether the output is safe, and whether the task is complete. Each of those decision points is a classification: route this request (Choice), assess this risk level (Score), determine if this is done (boolean).
The paper's most striking finding: Gemini CLI implements a 7-layer routing strategy β the most complex routing mechanism observed in any production agent. Each layer applies a different classification strategy, from static rules to a local Gemma model running client-side. Simple queries resolve cheaply at the first layer. Ambiguous ones fall through to progressively more sophisticated classifiers. No other agent performs client-side model selection, but every agent performs some version of this routing logic.
βΉοΈ The decision stack is already separated from the generation stack. Claude Code runs its safety classifier on Sonnet 4.6, not on the main model handling the conversation. Gemini CLI runs Gemma locally for routing. The pattern exists β it just was not named or productized until now.
Claude Code's architecture makes this explicit. Before any tool executes, the harness evaluates the action through four permission modes. In auto mode, a background safety classifier running on Sonnet 4.6 sees only the user request and the proposed tool call β deliberately isolated from the main model's reasoning to prevent the model from arguing the harness into approving risky operations. That classifier decides: allow, deny, or ask the user. It is a pure classification task. It does not generate text. It does not explain its reasoning. It returns a decision.
The Cost Math That Forces the Split
Why separate the classifier from the generator? The arithmetic is brutal.
An analysis from Interesting Engineering's "Twelve Decisions in an Agent System" lays out the economics. The author identifies twelve distinct decision points in a typical agent loop β intake routing, budget gating, completion verification, tool-result checking, action admissibility, progress monitoring, condition gating, evidence checking, escalation tiering, fan-out extraction, retrieval filtering, and rubric scoring. Each decision point has three implementation options: code (free but limited to deterministic logic), typed classifier (~$0.000006 per decision), or frontier LLM (~$0.02 per decision).
That is a 3,333x cost ratio between a typed decision and a frontier model call. An agent that makes 12 decisions per loop iteration, running 30 iterations per task, burns 360 LLM calls just on control flow. At frontier pricing, that is $7.20 per task in decisions alone. With a typed classifier, it is $0.002.
View original post on Substack β
β οΈ The counter-argument matters. A frontier model answering "is this task done?" while it is already loaded costs marginal tokens, not a fresh $0.02 call. The 3,333x ratio assumes standalone calls. In practice, piggybacking decisions on existing context windows shrinks the gap. The typed classifier wins on latency (single forward pass vs. autoregressive generation) and isolation (the classifier cannot be prompt-injected through the conversation context), but the cost advantage is smaller than the headline number suggests when the LLM is already warm.
The pattern that emerges from production systems is not "replace all LLM decisions with classifiers." It is confidence-gated routing: the classifier handles the fast, clear cases; the LLM handles the ambiguous ones. TypeSafe documents this as a first-class pattern. The classifier returns a decision and a confidence score. Above a threshold, act on the classifier's decision. Below it, escalate to a frontier model. Below a second threshold, escalate to a human.
How LangChain Implements It
The LangChain walkthrough is the clearest available implementation guide for this pattern. It builds two decision nodes into a standard LangGraph agent loop:
1. Request Router (model selection). Before the agent calls its LLM, the router classifies the incoming request by complexity. A lookup query ("what is the capital of France?") routes to a cheap, fast model (gpt-4o-mini). An architecture question ("design a microservices migration plan") routes to a capable, expensive model (gpt-4o). The routing policy is defined in plain English as the classifier's state, and the classifier returns a Choice with probabilities.
2. Tool Gate (permission check). Before any tool call executes, the gate classifies the proposed action by risk level: low, medium, or high. Low-risk actions (reading a file, running a search) proceed automatically. Medium-risk actions (editing a file) proceed with logging. High-risk actions (deleting files, running shell commands) are blocked and escalated to the user for approval.
This mirrors exactly what Claude Code, Codex, and Cursor already do with their built-in permission systems β but extracted into an explicit, configurable middleware layer. The key architectural insight: the classification is middleware, not part of the model. It wraps the agent loop, inspects proposed actions, and gates execution. The LLM inside the loop never sees the gate's logic or knows it is being filtered.
# Simplified pattern from LangChain's Jev walkthrough
from typesafe import TypeSafeClassifier
classifier = TypeSafeClassifier()
# Route by complexity
routing = classifier.invoke(
state={"user_query": query, "context": context},
questions={"complexity": Choice(["simple", "moderate", "complex"])}
)
if routing["complexity"].answer == "simple":
model = cheap_model
else:
model = capable_model
# Gate by risk before tool execution
risk = classifier.invoke(
state={"tool_name": tool, "args": args, "policy": risk_policy},
questions={"risk_level": Choice(["low", "medium", "high"])}
)
if risk["risk_level"].answer == "high":
await escalate_to_user(tool, args)
Three Decision Points Where Classifiers Win
Not every decision in the agent loop benefits from a standalone classifier. The Twelve Decisions analysis finds that four of the twelve operate better as pure code (budget gating, progress monitoring, fan-out extraction, retrieval filtering), because they involve deterministic thresholds, not semantic judgment. But three decision points are clear wins for purpose-built classifiers:
1. Model Routing
The highest-ROI application. A router classifier reads the incoming request and picks the cheapest model capable of handling it. LangChain's walkthrough implements exactly this pattern. The economics are straightforward: if 60% of your agent's queries are simple lookups, routing them to a model that costs 10x less than your default saves 54% of your LLM spend with no quality loss on those queries.
The Gemini CLI takes this further with its 7-layer approach. The first layer is static rules (command patterns that always map to the same model). The second is an embedding-based classifier. Deeper layers use progressively more sophisticated methods. The design principle: resolve cheaply when you can, escalate when you must.
2. Tool Gating (Permission Checks)
This is where Claude Code's architecture is most instructive. The safety classifier runs on a separate model instance, sees a deliberately limited context (only the user request and the proposed tool call), and returns a ternary decision (allow/deny/ask). The isolation is the point β it prevents the conversational model from manipulating the safety check through its reasoning chain.
The LangChain walkthrough calls this the AutoModeMiddleware: "uses Jev to check tool calls for risky decisions it may take, and block calls before the tool executes." Every major coding agent ships some version of this. The question is not whether you need a tool gate β it is whether your gate should be a prompted LLM, a fine-tuned classifier, or a purpose-built decision model.
3. Completion Verification
The agent needs to decide: "Am I done?" This is a boolean classification β the simplest form of the pattern. But it is also where LLM-based decisions fail most visibly. A generation model asked "are you done?" has incentive structures (built through RLHF) to be helpful, which means it tends to claim completion prematurely rather than ask for clarification. A classifier trained specifically on completion detection, with calibrated confidence scores, can set a threshold: below 0.8 confidence, continue the loop.
For more on verification patterns in agent loops, see our piece on why verification beats review for coding agents.
The "Classification Is Feature Engineering" Insight
A parallel conversation on Hacker News β "LLM Classification Is Feature Engineering" (116 points) β arrives at the same conclusion from a different angle. The argument: using an LLM as a standalone classifier is fundamentally flawed because LLM outputs lack reliable confidence scores, making threshold adjustment impossible. The solution is to treat LLM classification as a feature engineering step β extract the LLM's reasoning and verdicts as features, then feed them to a logistic regression or decision tree that provides calibrated probabilities.
View discussion on Hacker News β
This is exactly what TypeSafe claims Jev does internally. Strip out the generation capability. Train the model to produce typed decisions with calibrated probabilities. Let the downstream system set thresholds. The innovation, if it holds, is not that classifiers are better than LLMs for classification β that is obvious. It is that you can train a transformer-based model that has all the contextual understanding of an LLM but outputs only typed decisions with calibrated confidence, giving you the semantic depth of an LLM and the reliability properties of a traditional classifier.
βΉοΈ The convergence is real. The HN discussion, the academic routing papers, and the production agent architectures are all arriving at the same place: classification and generation are different jobs, and running them on the same model is an architectural shortcut that costs you money, latency, and reliability. The question is no longer whether to separate them, but how.
What the Community Is Saying
The developer response to the router-as-harness pattern has been immediate and practical. Within days of LangChain's walkthrough, builders started integrating Jev into existing harnesses beyond what TypeSafe or LangChain documented.
Alvaro Cintas shared a Claude Code integration called jev-model-router, an early-access mod built on Claude Code's function hooks. Before every turn, the mod calls Jev to classify three things: how mechanical the task is, how much reasoning it needs, and whether it is risky. The result determines which model and effort level Claude Code uses for that turn.
Daniel San noted that "Jev's classifier is incredibly well designed and flexible enough to be integrated directly into other harnesses like Claude Code. Jev can decide which model and effort level to use based on the user's prompt. For me, this is where Jev becomes really useful."
The ComplianceGate paper from June 2026 provides the academic validation for this pattern in regulated industries: a classifier-gated multi-tier routing system achieved 99.2% accuracy at just 7ms inference overhead, with 39% latency reduction and 33-52% cost savings compared to routing everything through a single frontier model.
How to Implement This Today
You do not need Jev to implement the router-as-harness pattern. You need three things:
1. A classification layer that returns confidence scores. Options, from cheapest to most capable:
- Prompted frontier model with structured outputs (OpenAI function calling, Anthropic tool use): $0.01-0.03 per decision, highest accuracy on ambiguous cases, highest latency
- Fine-tuned small model (distilled from frontier model decisions): $0.001-0.005 per decision, good accuracy on your specific distribution, moderate latency
- Jev or similar typed decision model: ~$0.00004 per decision, unknown accuracy on your specific distribution (calibration unproven), lowest latency
- Traditional ML classifier (logistic regression on LLM-extracted features): ~$0.00001 per decision, requires labeled training data, sub-millisecond latency
2. A confidence threshold and escalation path. The classifier's confidence score determines what happens next:
- High confidence (>0.9): act on the classifier's decision
- Medium confidence (0.7-0.9): act but log for review
- Low confidence (<0.7): escalate to a frontier model
- Very low confidence (<0.5): escalate to a human
3. Isolation between the classifier and the generator. This is the lesson from Claude Code's architecture. The classifier must not share context with the generation model. If it does, the generation model can influence the classification through its reasoning chain β exactly the failure mode that Claude Code's auto-mode classifier was designed to prevent.
π‘ Start with the tool gate. Of the three decision points above, tool gating is the easiest to implement, the hardest to get wrong (false positives just ask for confirmation), and the most immediately valuable for safety. Model routing is higher ROI but requires careful measurement of your query distribution. Completion verification requires labeled examples of "done" vs. "not done" states. The tool gate needs only a risk policy and a list of tools.
What This Means for Builders
The router-as-harness pattern is not a future architecture β it is the current architecture of every major agent harness, and the only question is whether you make it explicit or leave it implicit.
Here is the honest assessment:
Do this now: Audit your agent loops for "generation-shaped classification." Every place you prompt an LLM with "choose one of these options" or "rate this on a scale of 1-5" or "is this safe to execute" is a candidate for a standalone classifier. Calculate the cost: multiply the number of such decisions per task by your LLM's per-call cost, then compare to a classifier alternative.
Do this carefully: Confidence-gated routing. The pattern works, but the thresholds are empirical, not theoretical. You need to measure classifier accuracy on your specific distribution before setting thresholds. Start conservative (low threshold, frequent LLM fallback) and tighten as you collect data.
Do not do this yet: Ripping out your LLM-based decision layer entirely and replacing it with Jev or any other single classifier. TypeSafe has not published calibration data. No independent benchmark has validated Jev's confidence scores on a diverse task distribution. The pattern is proven; the specific product is not. Use it as one option in your escalation chain, not as a load-bearing replacement.
The deeper lesson is structural. As the Twelve Decisions analysis puts it: a frontier model costs ~$0.02 per decision. A typed classifier costs ~$0.000006. At that ratio, you can afford 350 classifier decisions for the cost of one LLM call. The agent that wins is not the one with the best model β it is the one that knows which decisions need a model and which do not.
The harness is not the thing around the model. The harness is the router. And the router just got its own model.
For our deep-dive on Jev itself β including the calibration question, RLCD training, and benchmark analysis β see Jev by TypeSafe: A New Agent Layer, If Calibration Holds. For more on why the harness matters more than the model, see Your Agent Harness Is the Product.
Originally published at AgentConn
Top comments (2)
this closes your arc cleanly: sep 3 you argued the harness decides who wins,
yesterday you argued the money moves on price not capability, and today the
router is the harness that does both at once β it picks which model ever sees
the request, so it sets the bill and the ceiling in one decision. the layer
everyone treated as plumbing turns out to be the system.
the part that lit up for me is what this does to eval, and it is my own
failure coming back in a new coat. in my february incident i tagged every
retrieval verdict with the embedder it was earned under, because a swap makes
old green results describe a retrieval that no longer exists. a router does
the same thing continuously and silently: every answer is served by some
model on some route, so a golden-set pass without a route tag is a green lie
about cost β you don't know which model earned the point, and the router may
have quietly degraded to the cheap branch while the suite stayed green because
the answer happened to match anyway.
and this is your own precondition line turned on you, which i think is the
highest compliment i can pay: "any time your test's precondition is itself
probabilistic, a green result is ambiguous." the router's precondition is
probabilistic by construction β it selects a branch by policy or by chance β
so a cheap accepted answer means either "the router correctly picked a cheap
model that handled it" or "the router picked wrong and the answer coincided."
only a route tag plus negative tests run per branch separates those two. your
principle, applied to your thesis, says the router needs the same stamping my
embedder needed.
which connects straight back to the metric i floated at you yesterday: cost
per verified answer, tracked by route. routing on price without a route-tagged
eval is optimizing the bill without measuring the quality at that bill. with
the tag, "we cut 20%" becomes "we cut 20% and kept N verified answers on the
cheap branch" β the number your money-moved piece actually wants.
curious how you verify the router hasn't drifted between runs today: do you
stamp the route on the verdict at all, or do you lean on the routing policy
being deterministic enough that you trust it without checking? asking because
my embedder was deterministic too, and it still lied to me for a week.
We run the confidence-gated escalation shape you describe, in production, and the thing that has cost us most sits upstream of the classifier. A router routes by a tier ordering. We could not produce a stable one.
Same 160 scored HumanEval+ problems, same prompts, same scorer, temperature 0, two identical runs an hour apart:
First and last swapped. On run 1 the candidate lost to our cheap tier by 1.3 points, on run 2 it won by 1.4, and McNemar exact two sided on the paired set gives p = 0.815. The run moved 2.5 points while the gaps we were reading were 1.3 to 1.4, so the ordering was noise both times. A router configured off either run alone encodes a ranking the next run reverses.
What survives re-running is the partition, and it speaks to your point about which decisions need a model. Take the same problems and split by task instead of by column total: all three models correct on 130 and all three wrong on 2, in both runs, with the three way disagreement at 17.5% then 14.8%. Best single model 92.5%, any-of-three correct 98.8%, so about 6 points sit outside the reach of picking the right model however good the picker is.
The defensible version of that is narrower than the headline. Difficulty alone makes all three models wobble on the same hard items, so the evidence worth keeping is the 10 problems contested in both runs with zero flips by any model. Those are the ones where A is reliably right and B is reliably wrong twice, which is a model by item interaction and is the thing a router could in principle exploit.
Caveats: 160 problems, code only, our own scorer, one pair of runs, tool calling path not included, and floors do not transfer between benchmarks. On BFCL our total moved 0.375 points while 4.8% of items flipped, because flips cancel.
So the cheapest line I would add to your implementation section, ahead of setting any threshold: run the tier comparison twice at temperature 0 and check the ordering survives. And re-run the one that agrees with you first. Run 1 had our cheap tier on top and I nearly wrote it up that way.