Jarvis Osorio on Model Routing: Why Startups Should Not Bet on a Single AI Model

Canonical: Jarvis Osorio on Model Routing: Why Startups Should Not Bet on a Single AI Model — also at jarvisosorio.com. Originally published by Jarvis Nelson Osorio (Jarvis Osorio).

The comforting story in AI engineering is that one "god model" will eventually do everything well enough. I do not believe that story, and I do not build as if it were true. I am Jarvis Osorio (Jarvis Nelson Osorio), a technology entrepreneur and innovator based in Chile, working on AI systems, DAO institutions, books and art. The more useful pattern is model routing: send each job to the model — or the small ensemble — that fits the job, then keep a record of which path you took.

This is not a vendor review. Routing layers such as OpenRouter and similar gateways are one way to practice the idea. The architecture matters more than any single dashboard.

Beyond a single default

A startup usually begins with one API key and one favorite model. That is fine for a prototype. It becomes a hidden concentration risk once the product has users:

• A price change or rate limit can stall the whole roadmap.
• A policy change can silently alter tone, refusal behavior, or tool use.
• A model that writes beautiful prose may be a weak classifier, a slow retrieval helper, or an expensive way to extract a date from a form.
• Teams in Latin America also feel latency and currency swings that a single distant endpoint does not advertise in a launch blog post.

Routing is how you treat models as replaceable components instead of as a personality you have to stay loyal to.

A simple routing map

I start with a map, not a prompt. For each product action, write four lines:

1. Job. Classify a ticket, draft an email, extract a field, review a pull request, answer a child-facing question.
2. Constraints. Latency budget, cost ceiling, language, data residency, whether the output may be shown verbatim.
3. Failure cost. What happens if the answer is wrong, delayed, or too agreeable?
4. Preferred path and fallback. A primary model, a cheaper or local backup, and a human gate when the failure cost is high.

Once that map exists, routing is almost boring — which is the point. Boring paths can be tested. A single mysterious genius cannot.

Fusion is not the same as a committee

There is a fashion for sending the same prompt to several models and picking the majority. I have written about the trap: three agents can agree and all three can be wrong if they share assumptions. Routing is different. It assigns different jobs to different specialists, and it uses a judge only when the jobs require a second look.

A healthy pattern looks like this:

• A small, fast model handles classification, routing, and "do I have enough context?"
• A stronger model handles the hard reasoning step, with sources attached.
• A separate checker — preferably a different family of model, or a rules engine, or a person — reviews irreversible actions.
• The product logs the path: which model, which version, which tools, which fallback.

That last line is governance. It is the same instinct I defend in Breaking the Centralized Order: if you cannot reconstruct a decision, you do not really have an institution.

Practical cautions

Routing can fail in ordinary ways. Always sending work to the cheapest model is not architecture; it is a discount addiction. Always sending work to the largest model is a luxury tax. Switching models mid-conversation without telling the user can produce a jarring change in voice. And a routing layer is itself a dependency — it needs monitoring, just like a payment provider.

For child-facing or high-trust features, I would rather route away from systems that never refuse. The Compliant Mind is about that bias toward agreement. A router that only optimizes for "the user seemed happy" will happily pick the most compliant voice.

A week-one experiment

If you already have one model in production, try this without rebuilding the company:

• Pick three real tasks from last week.
• Run each task through two models you can actually afford.
• Score them on correctness, latency, and tone — not on eloquence alone.
• Write the routing rule you would have wanted, including a fallback.
• Add a log field so the next incident review can see the path.

You will learn more from twelve scored examples than from another month of opinions about which laboratory is "winning."

More from me at jarvisosorio.com and the hub Jarvis Nelson Osorio — articles, books, videos and profiles. The institutional companion to this essay is Breaking the Centralized Order.

The future of AI engineering is not a race to one singular mind. It is a craft of sending the right work to the right mind, and of staying honest about the path.