Free LLM endpoints appear and vanish constantly — a provider opens a generous
tier, a new open model shows up on someone's gateway, a promo runs for a month.
Chasing them by hand is a losing game. So I built a small radar
(model-radar) that watches for
free models and, when one is genuinely good enough, adopts it automatically into
my agent stack — but only after it clears a gate strict enough that it has never
once adopted something that embarrassed me.
The auto-adoption is the flashy bit. The gate is the bit worth your time, because
"just use the free model" is advice that will quietly wreck an agent pipeline.
Why free models are a trap you still want to fall into
The appeal is obvious: a lot of agent work — summarise, triage, draft, classify —
doesn't need a frontier model, and paying frontier prices for it is silly. A free
tier that's "good enough" for the boring 90% is real money saved.
The catch is that "free model" and "works in an agent" are almost unrelated
properties. A model can score beautifully on a leaderboard and be useless for
agentic work, because agent work needs things benchmarks don't measure:
- Reliable tool-calling. The model must emit correct, structured tool calls — and, more importantly, actually use the results it gets back. Plenty of models emit a tidy call and then ignore the response, which is worse than not calling the tool, because it looks like it's working.
- Honest context handling. Some endpoints silently truncate a long prompt and answer from the half they kept, with no error. Confident, wrong, and invisible.
- Stability. A free tier that rate-limits mid-task or disappears next week can't be in your critical path.
So a radar that adopted models on quality scores would be a machine for
introducing silent failures. The whole design problem is: how do you let a
process adopt something automatically without letting it adopt something bad?
The answer: adoption is a gate, not a vote
The radar can find candidates freely. It can promote one only by passing every
stage of a fixed, deterministic gate — and if any stage fails, it's rejected, full
stop. No score, no averaging, no "it's mostly fine." The gate checks, in order,
things a leaderboard never will:
- Does it actually respond, at the endpoint, right now, within a sane timeout?
- Can it tool-call for real — emit a correct call and incorporate the result into its next turn — on a concrete test task, not a synthetic one?
- Does it respect context — feed it something long and confirm it didn't silently drop the tail?
- Is it stable enough across repeated calls, not a one-lucky-response fluke?
- Does it beat what I'm already running on the specific jobs I'd actually route to it?
Only a model that clears all five gets adopted. Everything is measured on real
tasks and written down, so adoption is a reproducible decision I can audit later,
not a vibe. And there's a blocklist: models measured to fail agentic use in a way
that matters are banned by name, so the radar can't rediscover and re-adopt a known
dud.
The principle underneath is the same one my whole estate runs on: let the
automated part gather and propose freely; let it act only through a gate that a
bad candidate cannot argue its way past. A guardrail written as a suggestion in a
prompt is decoration. A guardrail written as a pass/fail function is a wall.
What it actually buys me
- Most of my agent work runs on free or cheap models, adopted automatically as better free options appear, with the expensive frontier models reserved for the genuinely hard slice.
- I'm not manually chasing which provider opened a tier this week.
- And nothing junk gets in, because adoption is gated on the properties that actually matter for agents — tool-calling, context honesty, stability — not on a benchmark number.
The failure I designed the whole thing to avoid is the one that gets everyone else:
a cheaper model quietly swapped in that looks like it's working — emits calls,
returns fluent text — while ignoring tool results or truncating context, producing
confident garbage that no error ever flags. The gate exists specifically to catch
the model that fails silently, because that's the only kind of failure that
survives long enough to hurt you.
Take a look
model-radar is on GitHub. Even if
you never auto-adopt anything, the transferable idea is the gate: before you route
real work to a free model, test it on the thing agents actually need — can it
tool-call and use the result, does it keep your whole prompt, is it stable — and
make that test a hard pass/fail, not a score. Cheap intelligence is worth
chasing. Cheap intelligence that fails silently is worth fearing, and the only
difference between the two is whether you gated it.
Notes from someone who wanted a coffee-a-month agent bill and refused to pay for
it in silent failures, so built a radar that adopts free models automatically and a
gate strict enough to trust it.
🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.
Top comments (0)