DEV Community

Riley Zhang
Riley Zhang

Posted on

Stop Picking One Coding Model: A Free Router That Sends Each Task to the Cheapest Model That Passes

#tooling #ai #tutorial #productivity

Every week another open-weight or cheap API coding model drops, and every week somebody asks me which one they should switch to. My honest answer after running a small eval harness for months: that's the wrong question. No single model wins across task types. The model that nails your regex-heavy refactors may mangle your SQL migrations, and the expensive one you keep as a default is probably overkill for half your prompts.

So instead of picking a winner, I built a tiny router: classify the task, look up the cheapest model that has proven it passes that task class on my own repo, and only escalate when the cheap one fails. This post is the whole setup. It costs nothing to run if you use free tiers, and it takes about an hour.

Why routing beats choosing

A single-model default has two failure modes:

  1. Overpaying for easy tasks. Docstring generation, simple test scaffolding, and boilerplate edits pass on almost any current model. Paying frontier prices for these is waste.
  2. Under-trusting cheap models on hard tasks. Some budget models are genuinely good at one narrow thing (in my harness, one free model beat a paid one on TypeScript type-error fixes specifically). You only learn this by measuring per task class, not overall.

A router fixes both. It is also future-proof: when next week's shiny model drops, you don't re-argue the switch — you run it through the same harness and let the routing table update itself.

Step 1: A task taxonomy you actually have

Don't copy a benchmark's categories. Grep your own history. I pulled my last 300 prompts from editor logs and clustered them into five classes that covered ~90% of volume:

Class Example prompt Share of my usage
fix-typo-lint "fix these eslint errors" 22%
write-test "add pytest cases for this function" 19%
refactor-small "extract this into a helper" 18%
explain-debug "why does this throw X" 16%
sql-migration "write the Alembic migration for this schema change" 8%

Your table will differ. That's the point — the router is only as honest as the taxonomy.

Step 2: Score each candidate model per class

You need a handful of verifiable tasks per class — things with tests or diffs you can check mechanically, not vibes. I keep 6 tasks per class in tasks/<class>/, each a directory with a prompt.md, starter files, and a check.sh that exits 0 on success.

The scoring harness (deliberately boring bash):

#!/usr/bin/env bash
# score.sh <model-name> — runs every task against one model, writes results.tsv
MODEL="$1"
: "${RESULTS:=results.tsv}"

for task_dir in tasks/*/; do
  class=$(basename "$task_dir")
  for task in "$task_dir"*/; do
    work=$(mktemp -d)
    cp -r "$task"/* "$work"/
    prompt=$(cat "$work/prompt.md")

    # llm_call is your adapter: sends prompt + starter files to $MODEL,
    # writes the model's file edits back into $work. ~20 lines of curl/python.
    if llm_call "$MODEL" "$prompt" "$work" && (cd "$work" && bash check.sh >/dev/null 2>&1); then
      echo -e "$MODEL\t$class\t$(basename "$task")\tpass" >> "$RESULTS"
    else
      echo -e "$MODEL\t$class\t$(basename "$task")\tfail" >> "$RESULTS"
    fi
    rm -rf "$work"
  done
done
Enter fullscreen mode Exit fullscreen mode

Then build the routing table — cheapest passing model per class:

#!/usr/bin/env bash
# route.sh — emits routing-table.tsv: class -> cheapest model with pass rate >= threshold
awk -F'\t' '
  { key=$1 FS $2; total[key]++; if ($4=="pass") passed[key]++ }
  END {
    for (k in passed) {
      rate = passed[k]/total[k]
      if (rate >= 0.8) print k, rate   # threshold: 80% per class
    }
  }' results.tsv | sort -t$'\t' -k2,2 -k4,4nr | \
awk -F'\t' '!seen[$2]++ { print $2 "\t" $1 "\t" $3 }' > routing-table.tsv
Enter fullscreen mode Exit fullscreen mode

sort here stands in for "sort by your cost per class"; I keep a static costs.tsv mapping model → relative cost tier (free = 0) and join on it. Free models automatically win every tie, which is the behavior I want.

Step 3: Classify and dispatch

Classification doesn't need an ML model. A 15-line classifier using keyword heuristics on the prompt plus file globs from your editor covers most cases (.sql or migration in path → sql-migration; --fix or eslint in prompt → fix-typo-lint, etc.). Anything unclassifiable goes to a default class mapped to your strongest passing model. Escalation rule: if the routed model's output fails check.sh (or you reject it in review), retry once with the next model up the cost ladder for that class and log the escalation. The escalation log is signal — a class with rising escalations means the routed model is drifting and needs rescoring.

Where the free capacity comes from

The catch with "just measure everything" is that scoring 5 models × 30 tasks, then re-scoring whenever a new model drops, burns API budget fast. I run the harness on MonkeyCode, which offers free access to a set of coding models and a free server option to run the harness itself — so the whole eval loop (runner + candidate models) costs me nothing, and paid API calls only happen in production routing, where they're justified. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use it for the eval side specifically; the harness above is plain bash and works against any provider with an API, so nothing here locks you in.

If you want to try this, the fastest path is: pick your two most frequent task classes, write 6 verifiable tasks each, and score one free model against your current default. That alone usually surfaces one surprise. If you're evaluating options, MonkeyCode's free model tier is a low-friction place to pull candidates from.

Limitations, and who should skip this

  • Small samples lie. 6 tasks per class gives you a rough filter, not a confidence interval. Treat pass rates as "good enough to route," revisit monthly, and never quote them as benchmarks.
  • Verifiable tasks bias the taxonomy. "Explain this architecture decision" has no check.sh. My router handles subjective classes by always sending them to the strongest model — which means the savings are concentrated in mechanical work. Fine, but know that's what you're optimizing.
  • The taxonomy rots. New project, new stack, new prompt habits — recluster every few months or the router quietly routes garbage.
  • Heuristic classification misroutes. My explain-debug prompts containing the word "migration" got sent to the SQL model for a week. Log every routing decision; you'll want the audit trail.
  • Skip this if your volume is low (a few prompts a day — just use the model you like), if your work is mostly novel design (no repeatable task classes), or if you can't write mechanical checks for anything you delegate. Routing without verification is just faster guessing.

The takeaway

The weekly model-release cycle isn't going to slow down, and "which model should I use" will never have a stable answer. "Which model should handle this class of task, according to my own tests" does — and it updates itself every time you rerun the harness. Build the table once, let the routing table absorb the churn, and spend your attention on the tasks that actually need it.

Top comments (0)