Most agent stacks pick one model and use it for everything.
Tagging an email gets the same model as reviewing a contract.
That works.
It is also like hiring a senior lawyer to sort your mail.
On October 8, 2026, at Gemini at Work 2026, Google Cloud announced the Gemini agent, which it calls "a universal agent for work." It plans the work, uses skills and tools, and connects to business systems.
One line in the launch stood out to me:
It chooses the best model for the job, has built-in cost controls, and most importantly, has the security, administration, and governance required by enterprise customers.
In his keynote remarks, Thomas Kurian added that the agent orchestrates across Google's Gemini models and Anthropic's Claude models today. He also said "the best model for the task is not always the largest one." Google also says the agent has built-in cost controls.
That distinction matters.
The agent and the model are now separate choices.
Something has to make that choice on every task.
Routing is a policy, not a vibe. If you can't print why a model was picked, you don't have a router.
So let's build a tiny version.
By the end, you'll run one command:
npx tsx router.ts
And watch seven example tasks get routed, paused, rejected, and resumed.
This is my small model of the idea. It is not how Google's Gemini agent works inside. Every model, price, and task below is made up.
Table of Contents
- The Problem
- Step 1: Describe the Models
- Step 2: Decide What Fits
- Step 3: Pick the Cheapest Fit
- Step 4: Enforce the Spend Cap
- Step 5: Run It
- Where It Breaks Down
- The Bigger Idea
1. The Problem
A router has to answer three questions per task.
Which models can do this at all?
Which one of those is cheapest?
Is there still budget to run it?
Most code answers only the first one, by hardcoding a model name.
Then the bill arrives, and it answers the third one for you.
Capability first. Price second. Budget last. The order is the design.
2. Step 1: Describe the Models
A model is just a few facts the router can compare.
type Tier = "small" | "medium" | "large";
type Model = { name: string; tier: Tier; usdPer1kTokens: number; maxContext: number; tools: boolean };
type Task = { id: string; project: string; need: Tier; tokens: number; tools?: boolean; override?: string };
type Route =
| { kind: "run"; task: string; model: string; cost: number; why: string }
| { kind: "paused"; task: string; model: string; cost: number; why: string }
| { kind: "rejected"; task: string; why: string };
Each model has a tier, a price, a context limit, and a tools flag.
Each task says what tier it needs and how big it is.
override lets a human take the wheel.
Google said users can pick a model themselves. That felt worth keeping.
3. Step 2: Decide What Fits
Fit is a hard filter. No scoring yet.
const RANK: Record<Tier, number> = { small: 0, medium: 1, large: 2 };
const MODELS: Model[] = [
{ name: "mini", tier: "small", usdPer1kTokens: 0.0004, maxContext: 32_000, tools: false },
{ name: "mid", tier: "medium", usdPer1kTokens: 0.003, maxContext: 128_000, tools: true },
{ name: "big", tier: "large", usdPer1kTokens: 0.015, maxContext: 1_000_000, tools: true },
];
function fits(m: Model, t: Task): boolean {
if (RANK[m.tier] < RANK[t.need]) return false;
if (t.tokens > m.maxContext) return false;
if (t.tools && !m.tools) return false;
return true;
}
A model that is too small does not fit.
A model with too little context does not fit.
A task that needs tools needs a model with tools.
A cheap model that can't do the job isn't cheap. It's a retry with extra steps.
4. Step 3: Pick the Cheapest Fit
Now we sort what's left by price.
function cost(m: Model, t: Task): number {
return Math.round((t.tokens / 1000) * m.usdPer1kTokens * 10_000) / 10_000;
}
function pick(t: Task): { model?: Model; why: string } {
if (t.override) {
const m = MODELS.find((x) => x.name === t.override);
if (!m) return { why: `override "${t.override}" is not a known model` };
if (!fits(m, t)) return { why: `override "${m.name}" does not fit this task` };
return { model: m, why: "user override" };
}
const candidates = MODELS.filter((m) => fits(m, t)).sort((a, b) => cost(a, t) - cost(b, t));
if (candidates.length === 0) return { why: "no model fits" };
return { model: candidates[0], why: `cheapest model that fits (${candidates.length} fit)` };
}
An override still has to fit.
If someone forces a small model onto a huge task, the router says no.
An unknown model name is rejected, not guessed.
Otherwise the router takes the cheapest model that passed the filter.
Every result carries a why string.
That string is the whole point.
5. Step 4: Enforce the Spend Cap
Each project gets a cap. The ledger tracks spend.
class Ledger {
spent = new Map<string, number>();
constructor(private caps: Record<string, number>) {}
route(t: Task): Route {
const { model, why } = pick(t);
if (!model) return { kind: "rejected", task: t.id, why };
const c = cost(model, t);
const spent = this.spent.get(t.project) ?? 0;
const cap = this.caps[t.project] ?? 0;
if (spent + c > cap) {
return { kind: "paused", task: t.id, model: model.name, cost: c, why: `cap $${cap} would be hit (spent $${spent.toFixed(4)})` };
}
this.spent.set(t.project, Math.round((spent + c) * 10_000) / 10_000);
return { kind: "run", task: t.id, model: model.name, cost: c, why };
}
raise(project: string, cap: number) {
this.caps[project] = cap;
}
}
The check happens before the task runs, not after.
If the next task would cross the cap, it pauses.
Paused is not failed.
The work waits for a human to raise the cap, and then it resumes.
Google says its agent ships built-in cost controls. This is my tiny version of one.
6. Step 5: Run It
// ===== EXAMPLE INPUTS (made-up tasks, prices, and caps) =====
const tasks: Task[] = [
{ id: "tag-email", project: "support", need: "small", tokens: 2_000 },
{ id: "draft-reply", project: "support", need: "medium", tokens: 6_000, tools: true },
{ id: "contract-review", project: "legal", need: "large", tokens: 180_000 },
{ id: "summarize-thread", project: "support", need: "small", tokens: 40_000 },
{ id: "quick-fix", project: "support", need: "small", tokens: 1_000, override: "big" },
{ id: "translate", project: "support", need: "small", tokens: 3_000, override: "gpt-x" },
{ id: "second-review", project: "legal", need: "large", tokens: 180_000 },
];
const TAG = { run: "[RUN] ", paused: "[PAUSED]", rejected: "[REJECT]" } as const;
console.log("tiny-model-router demo");
const ledger = new Ledger({ support: 0.1, legal: 3 });
const paused: Task[] = [];
for (const t of tasks) {
const r = ledger.route(t);
const detail = r.kind === "rejected" ? r.why : `${r.model} $${r.cost.toFixed(4)} ${r.why}`;
console.log(`${TAG[r.kind]} ${t.id.padEnd(17)} ${detail}`);
if (r.kind === "paused") paused.push(t);
}
console.log("\nraise legal cap to $6, resume paused work");
ledger.raise("legal", 6);
for (const t of paused) {
const r = ledger.route(t);
const detail = r.kind === "rejected" ? r.why : `${r.model} $${r.cost.toFixed(4)} ${r.why}`;
console.log(`${TAG[r.kind]} ${t.id.padEnd(17)} ${detail}`);
}
console.log("\nspent:", Object.fromEntries(ledger.spent));
Output:
tiny-model-router demo
[RUN] tag-email mini $0.0008 cheapest model that fits (3 fit)
[RUN] draft-reply mid $0.0180 cheapest model that fits (2 fit)
[RUN] contract-review big $2.7000 cheapest model that fits (1 fit)
[PAUSED] summarize-thread mid $0.1200 cap $0.1 would be hit (spent $0.0188)
[RUN] quick-fix big $0.0150 user override
[REJECT] translate override "gpt-x" is not a known model
[PAUSED] second-review big $2.7000 cap $3 would be hit (spent $2.7000)
raise legal cap to $6, resume paused work
[PAUSED] summarize-thread mid $0.1200 cap $0.1 would be hit (spent $0.0338)
[RUN] second-review big $2.7000 cheapest model that fits (1 fit)
spent: { support: 0.0338, legal: 5.4 }
Here is what happened.
tag-email fits all three models, so it gets mini.
draft-reply needs tools, so mini is out and mid wins.
contract-review needs the large tier. Only big fits.
summarize-thread is "small" work, but 40,000 tokens is past mini's context. So it routes to mid, and that would cross the support cap. Paused.
quick-fix is overridden to big. Allowed, and logged as an override.
translate asks for a model that does not exist. Rejected.
second-review would push legal past $3. Paused.
Then a human raises the legal cap, and second-review resumes.
summarize-thread stays paused, because nobody raised the support cap.
Correct. The router does not get to give itself a raise.
7. Where It Breaks Down
This is a teaching model. Real routing is harder.
- Tiers are a guess. Real tasks don't label themselves "small." You need a classifier, or a cheap first pass, to decide.
- Prices are per token here. Real costs include tools, sandboxes, retries, and cached tokens.
- No quality check. A real router should watch whether the cheap model's answers held up, and route up when they don't.
- Estimates before the run. Token counts are known after the run. The cap check uses an estimate before it.
- One ledger, one process. Real spend tracking is shared, and two tasks can race past a cap together.
None of those change the shape.
Filter by capability. Sort by price. Check the budget. Print why.
8. The Bigger Idea
task
└→ filter: tier, context, tools
└→ sort: cheapest fit (or override)
└→ ledger: under the cap?
├→ run, with a reason
├→ pause, until a human raises the cap
└→ reject, with a reason
The task provides the requirements.
The models provide the options.
The filter provides the floor.
The price provides the order.
The cap provides the ceiling.
The human provides the raise.
This part is prediction, not history.
I think "which model are you on?" is becoming the wrong question for agents. The model will change every few months. Google said almost exactly that in the launch.
The better question is: what is your routing policy, and can you read it?
Picking the model is now part of the job. So it should be part of the log.
Try Roster
I'm building Roster around this idea: AI employees with real responsibilities, tools, memory, schedules and computer access. Each one has a lane, and a budget that lane can't quietly outgrow.
If the same follow-ups, handoffs, and waiting loops keep eating your week, give them to an AI employee.
Top comments (0)