If you've tried to point a local model — Qwen, a quantized Llama, whatever fits on your GPU — at a real task in a real repo, you already know the feeling. It starts confidently. It edits three files. It announces it's done. And then you run the tests and half of them are red, one of the files it "edited" is byte-for-byte unchanged, and the function it swore it added isn't there.
The usual conclusion is: the model is too weak. Get a bigger model, or rent a frontier API, and the problem goes away.
That conclusion is mostly wrong, and it's expensive. A large share of what looks like model weakness is actually the absence of a system around the model — the scaffolding that a frontier model partially compensates for on its own and a weak model does not. If you build that system, a much weaker model becomes usable for work you'd have assumed it couldn't touch. The industry has a name for that system: the harness.
I've spent the last several months building one — a coding harness called Atlarix that's designed to run any model, including small local ones, and get real work out of them. Along the way it's produced code that got merged into projects like Remix, Caddy, Traefik, and Valkey (more on that, honestly, at the end). This post isn't a pitch for it. It's the set of principles I learned building it, written so you can apply them to your own agent, whatever it's built on. Atlarix is just the reference implementation I'll point at to prove I actually did each thing rather than just theorizing.
Here's the core claim, stated plainly:
A weak model doesn't need a bigger prompt. It needs a system that catches its mistakes instead of trusting them.
Four mechanisms do most of the work. None of them require a better model.
1. Verify edits against reality, not against the model's word
The single most common failure mode of a weak model in a coding loop is the confident false completion: it reports success for work it didn't actually do. The edit didn't apply. The file didn't change. The function it described isn't on disk.
A frontier model does this too — it just does it less often, so you get away with trusting it. With a weak model you cannot trust the report at all. So don't. The fix is a principle I'd now build into any agent from day one:
Never let the model be the judge of whether its own change succeeded. Check the world.
Concretely, after every edit, the harness re-reads the file from disk and confirms the change is actually present. If the edit didn't land, that fact goes back to the model as a tool result — "the file is unchanged" — instead of forward to the user as "done." The model gets a chance to notice and retry, in the same turn, before anything reaches you.
The same principle extends to commands. When the agent runs the project's own checks — tsc, eslint, ruff, mypy, pytest, whatever the repo uses — a non-zero exit code is not something the agent gets to narrate its way past. In Atlarix the turn is held open: a command that exits non-zero, or an edit that didn't verify on disk, is routed back to the agent rather than surfaced as a finished result. The agent literally cannot declare a task done while the project's own tests are failing.
That one rule — the agent can't mark work complete while the repo's checks are red — eliminates the most damaging class of weak-model errors, because the most damaging errors aren't wrong code. They're wrong code reported as correct. Wrong-but-flagged is recoverable. Wrong-but-confident is what ships bugs.
You can implement this in any loop. Run the checks. Read the exit code. If it's non-zero, feed the failure back as the next observation instead of returning to the caller. It's not clever. It's just refusing to take the model's word for anything you can verify yourself.
2. Enforce control in the harness, not in the prompt
There's a strong temptation to solve agent misbehavior with more instructions. "Always wait for the command to finish before continuing. Never claim completion prematurely. Always ask before running destructive commands." You write a longer and longer system prompt, and a frontier model mostly follows it, and a weak model mostly doesn't — because following a paragraph of procedural instructions is itself a capability that weak models lack.
So stop asking the model to behave and make the behavior structural.
If a rule matters, the harness should enforce it mechanically, so a model that "forgets" the rule physically can't break it.
Tool approvals, background command handling, wait-states, sub-agent orchestration — in a prompt-driven agent these are all things you ask the model to handle correctly. In a harness-driven agent they're enforced by the system around the model. A weak model can't emit a premature "task complete" if completion is gated on verification it doesn't control (see #1). It can't skip an approval if the approval is a queue the execution path must pass through, not a politeness the model chooses to observe.
The practical test: for every "always/never" line in your system prompt, ask "what happens if the model ignores this?" If the answer is "something bad happens," that rule doesn't belong in the prompt — it belongs in the harness, as a gate the model routes through whether it wants to or not. The prompt is for guidance. The harness is for guarantees. Weak models need guarantees.
3. Give it a sandbox, so a mistake is contained instead of catastrophic
A weak model will try to run something it shouldn't — a command scoped too broadly, a write outside the project, a destructive operation it didn't reason through. If your safety story is "the model is careful," you don't have a safety story.
Contain execution at the OS level, so the blast radius of a bad call is bounded by the system, not by the model's judgment.
Atlarix confines command execution per-OS — Landlock on Linux, AppContainer on Windows, Seatbelt on macOS — so the agent physically cannot write outside the project you opened. There's an approval queue with hunk-level accept/reject on every diff, and a danger gate on destructive operations. The point isn't the specific primitives; it's the principle: the model's mistakes are caught by the system, not trusted on faith. When containment is structural, you can let a weaker, less-trustworthy model act, because the cost of it being wrong is bounded.
This is also what makes local-first viable at all. If the model runs on your machine and the execution is sandboxed to your project, "the AI touched my system" stops being a leap of faith. The containment is what earns the autonomy.
4. Feed it structure on demand, not the whole repo
Weak models have smaller effective context and degrade faster as you fill it. The instinct is to stuff the repo into the prompt so the model "has everything." This is exactly backwards — you drown the small model in tokens and it performs worse.
Retrieve narrowly and on demand. Let the model pull what it needs when it needs it, instead of pre-loading everything it might need.
Atlarix searches the repo on demand — bundled ripgrep for grep/glob, no index to build, no background watcher — and keeps durable notes on what it learns about the codebase so it doesn't re-derive the same structure every turn. The model asks for what it needs; it isn't forced to hold the whole tree in its head.
The general principle for any agent: on-demand, tool-driven retrieval beats context-stuffing, and it beats it more the weaker your model is. A frontier model can afford to waste context. A local model can't. Give it a way to look things up, and a way to remember what it found, and you've freed its limited context for actual reasoning.
Why this matters beyond saving on API bills
The obvious payoff is cost — run a local model, pay nothing per token, keep your code on your own machine. That's real. But the deeper payoff is trust. Every one of these mechanisms replaces a place where you were trusting the model with a place where the system checks. Verified edits replace trusting the completion report. Harness-enforced control replaces trusting the model to follow rules. The sandbox replaces trusting it not to do damage. On-demand retrieval replaces trusting it to juggle the whole repo.
Stack them and something surprising happens: the model's reliability stops being the ceiling. The system's guarantees become the floor, and a weaker model operating above a solid floor beats a stronger model operating on trust. That's the whole thesis. The harness, not the model.
The honest part
I said I'd be straight about the merges, so here it is.
Atlarix, running on open-weight and local models, has produced code that maintainers merged into real projects — Remix (merged by its co-creator), Caddy, Traefik, Valkey, and others. Every one is a public, verifiable pull request; the links are on atlarix.dev if you want to check them, and you should.
But I want to be precise about what that does and doesn't mean, because a technical audience deserves it and will figure it out anyway. These were human-directed. I drove the agent — chose the target, steered the work, reviewed the diff before it went out — and the agent appears as a co-author on the commits, not the sole author. A maintainer of Remix or Caddy reviewed the change and merged it after dealing with me as the human behind it. What the harness did was get an open-weight model's output to the point where it could clear that bar: correct, verified against the project's own checks, and clean enough to survive a real maintainer's review.
That's the claim, and it's a narrower and more honest one than "AI merged code into Remix." It's: a well-built harness plus a modest model, driven by a human who knows what they want, can produce work that passes the same gate a human contributor's work passes. That's not magic. It's the four mechanisms above, doing their job.
If you're building your own agent, take the principles and leave the tool. If you want to see the reference implementation, it's Atlarix — a private AI workstation that runs any model, local or hosted, with your code staying on your machine. Either way: stop blaming the model. Build the harness.
I'm Amariah Abishai, a self-taught engineer in Nairobi building AI developer tooling at NorahLabs. If you want the deeper version of the retrieval design, there's a published paper on an earlier structural-retrieval approach I tried, measured, and eventually moved away from — which is its own lesson for another post.
Top comments (0)