A Cost-Aware Model Router for Claude in About 80 Lines of Python

The fastest way to cut an LLM bill without hurting quality is to stop sending every request to your most capable model. Most traffic in a typical agent workflow is classification, extraction, or short replies. A smaller, faster model handles those fine. The bigger model should be reserved for the steps where it measurably changes the outcome.

This post walks through the simple router I use with Claude models. It's not a framework. It's a pattern you can adapt in an afternoon.

The idea

Every request gets a cheap first pass that decides how hard it is. Easy requests go to a fast model. Hard ones go to a stronger model. If the fast model's answer fails a check, the request is retried on the stronger one.

Three ingredients:

1. A difficulty signal: rules, a small classifier, or a short call to the fast model itself.
2. Routing rules kept in config, not hard-coded.
3. An escalation path when the cheaper answer isn't good enough.

The router

import anthropic
from dataclasses import dataclass

client = anthropic.Anthropic()

ROUTES = {
    "fast": {"model": "claude-sonnet-5", "max_tokens": 1024},
    "strong": {"model": "claude-opus-5-5", "max_tokens": 4096},
}

HARD_TASKS = {"multi_step_plan", "contract_review", "root_cause_analysis"}

@dataclass
class Request:
    task_type: str
    text: str
    needs_tools: bool = False

def pick_route(req: Request) -> str:
    if req.task_type in HARD_TASKS:
        return "strong"
    if len(req.text) > 20_000:
        return "strong"
    if req.needs_tools and req.task_type == "multi_step_plan":
        return "strong"
    return "fast"

def call(route: str, system: str, user: str):
    cfg = ROUTES[route]
    resp = client.messages.create(
        model=cfg["model"],
        max_tokens=cfg["max_tokens"],
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    return resp, cfg["model"]

The rules here are deliberately boring. Task type and input size go a long way before you need anything clever. Check the current model names and pricing in Anthropic's documentation before copying this, since both change over time.

Escalation when the cheap answer isn't good enough

Routing up front catches the obvious cases. Escalation catches the rest:

import json

def validate(output_text: str, schema_keys: set) -> bool:
    try:
        data = json.loads(output_text)
    except json.JSONDecodeError:
        return False
    return schema_keys.issubset(data.keys())

def route_and_run(req: Request, system: str, schema_keys: set):
    route = pick_route(req)
    resp, model = call(route, system, req.text)
    text = resp.content[0].text
    if route == "fast" and not validate(text, schema_keys):
        resp, model = call("strong", system, req.text)
        text = resp.content[0].text
        log_escalation(req.task_type, reason="validation_failed")
    log_usage(req.task_type, model, resp.usage)
    return text

Validation can be anything cheap and deterministic: valid JSON, required fields present, a confidence field above a threshold, or a citation check for retrieval answers. Avoid using another LLM call as the validator for every request, or you lose most of the savings.

Measuring whether it works

A router without measurement is a guess. I log four things per request:

• Task type
• Model used
• Input and output tokens
• Whether it escalated, and why

From that, two numbers matter most:

Escalation rate per task type. If a task escalates more than about a third of the time, it probably belongs on the strong route from the start. Paying for two calls is worse than paying for one larger one.

Quality by route. Sample outputs from both routes each week and score them against the same rubric. The goal is equal quality at lower cost, not lower cost at any quality.

Where routing goes wrong

Routing on the wrong signal. Text length is a weak proxy for difficulty. A short question about a complex contract can be hard, and a long but simple form can be easy. Task type is usually the better signal.

Ignoring latency. The fast model is often fast as well as cheap. For user-facing steps, that matters as much as cost.

Forgetting tool use. Multi-step tool use with dependent calls is where larger models tend to be more reliable. I keep planning steps on the strong route even when the text is short.

Hard-coding models. Models change often. Keep routes in config so switching a model is a deploy-free change, and rerun your eval set when you do.

Adding a learned difficulty signal later

Rules get you surprisingly far, but eventually you'll want a better signal than task type. The cheapest upgrade I've found is to let your own logs train it.

After a few weeks of routing, you have labelled data for free: requests that the fast model handled and passed validation, and requests that escalated. A small classifier on simple features can learn which new requests are likely to escalate and send them straight to the strong route.

from sklearn.linear_model import LogisticRegression

def features(req: Request):
    return [
        len(req.text),
        req.text.count("?"),
        int(req.needs_tools),
        int(req.task_type in HARD_TASKS),
        count_entities(req.text),  # IDs, amounts, dates mentioned
    ]

X = [features(r) for r in logged_requests]
y = [r.escalated for r in logged_requests]
clf = LogisticRegression().fit(X, y)

def pick_route_learned(req: Request) -> str:
    p = clf.predict_proba([features(req)])[0][1]
    return "strong" if p > 0.4 else "fast"

Keep the threshold in config and tune it against your eval set. A lower threshold sends more traffic to the strong model, raising cost and lowering escalations. The right point depends on how much you value latency and how expensive a wasted fast call is for you.

Cost guardrails beyond routing

Routing reduces average cost. It doesn't stop a single runaway task. I add two simple guards:

• A token budget per task, enforced in the agent loop. When a task crosses it, the agent summarizes its progress and hands off instead of continuing.
• A daily budget per tenant or team, with an alert at 80 percent. This catches a new integration that suddenly sends ten times the expected traffic.

How much does it save?

It depends entirely on your traffic mix. If most of your requests are simple, routing makes a large difference. If most are complex reasoning, it helps less. The only reliable way to know is to log a week of real traffic by task type, then simulate the routing rules against it before turning them on.

For a deeper comparison of the two models, including where the larger one tends to justify its price, I wrote up when Sonnet is enough and when Opus earns its cost.

I work on enterprise agent deployments at AI Hive, where model routing is one of the first things we set up for cost control. How do you decide which requests get your most expensive model?