The hardest problem in AI-assisted coding isn't picking a model. It's engineering the loop around it. It will soon be quaint to ask, which model is best? As reasoning and tool-use models converge in capabilities, the model matters less than the harness you build for it. This month alone: Anthropic released Fable, yanked it because guardrails 'could not be trusted', then re-released it. OpenAI dropped three frontier models just this week. And Meta shipped its first agentic model. Yet agentic coding issues continue.
Loop Engineering: Complete The Circle
Yes, the model matters. But the system around it increasingly matters more. A strong model inside a weak system can be unruly and expensive or even downright destructive. A weaker model in a disciplined system ships more useful work, at lower cost and lower risk.
That's why two IDEs running the same model feel nothing alike. One reads the right files, makes a focused change, runs the relevant tests, and reports uncertainty. The other burns context, edits unrelated code, retries failed approaches, and declares victory.
The difference isn't the model. It's the harness.
Benchmarks measure what a model can do under controlled conditions. The same model posts different scores on the same benchmark depending on the platform. But developers don't care about marketing numbers. They care what a system does inside a messy repository.
So the competitive edge isn't access to the smartest model. It's the best execution environment around whatever model you have.
Prompt engineering vs. loop engineering
Prompting is a single request.
Loop engineering is the operating system around the request.
Developers need to stop treating agents like chatbots that need elaborate instructions. A giant spec is not a workflow. It's an undocumented program with weak control flow and no failure handling.
Agents have their own ADHD. They keep working after the useful work is done: refactoring unrelated code, rewriting tests, "cleaning up." You know the moment. You set the task up perfectly, stepped away from the terminal, and came back to a WTF?! diff.
More prompting doesn't fix this. Boundaries do.
Stop prompting a model to behave. An agent without a harness is a driver on a road with no lanes, no signs, no speed limit, just a post-it called AGENTS.md. A harness paints road lanes, puts up stoplights, adds a highway patrol. Imagine driving in a foreign country where roads didn't have signs or markers. Do you drive on the left or the right or is it all one road?
A harness loop defines the task, gathers context, plans, makes a bounded change, runs checks, inspects failures, retries selectively, records decisions, and STOPS at a success condition or budget limit. Instructions should not just tell agents, “Do not do.” A harness should detect and stop changes. It should restrict paths, inspect diffs, enforce scope, and reject work that violates a contract.
My Python harness and the Typescript port are wrappers around a bash loop with a hard timeout. None of it is clever. It enforces boring coding standards humans try to follow and agents do when guardrails are set. Like most software it's a work in progress.
Verification Debt
Agents generate code faster than humans can verify it. The bottleneck has moved from generation to diff review, and we're accruing verification debt. Like technical debt, except it piles up before you've proven anything works. A 1,000-line agent patch compiles. It passes the tests the agent wrote. Neither proves intended behavior. Maybe it misunderstood the requirement. Mocked away the hard part. Quietly changed a public interface.
The cost is measurable. In METR's trial, experienced open-source developers took 19% longer with AI tools while thinking they'd been 20% faster. Review sucks time. But going fast on a merry-go-round isn't speed.
And the backlash against "AI slop" is earned. Five agents don't produce five times the output. They produce conflicting abstractions and five times the review burden. Engineers were never graded on lines of code. There is elegance in less. Yet "tasks completed" is now a boast, and a dangerous metric.
Verification belongs inside the loop: executable acceptance criteria, independent tests, static analysis, type checks, security scans, and human review proportional to risk.
Repo damage: how AI agents degrade a codebase
The failure that should make you nervous isn't one bad patch. It's cumulative degradation. An agent solves today's narrow task while making the codebase harder to maintain. Every change is individually defensible. The sum? Entropy.
You see it when agents operate without context. They recreate functions that already exist because they never looked. They route around architectural constraints they don't understand. They add dependencies for problems an installed library already solves.
Fixing this takes more than updating AGENTS.md. The workflow is controlled delegation, a harness with local quality gates:
- Before the edit: system understands the repository.
- During: constrains the scope.
- After: evaluates the diff as a change to the whole, not an answer to a prompt.
Humans build the harness. Agents work inside it. The harness manages context, tools, permissions, budgets, checkpoints, and verification. That makes engineering the loop central work.
Audit your AI coding loop
The best developers won't write the most code. They'll build the loops that write the best code. The model is rented. Your harness is an asset.
No harness? Big problem.
- [ ] Every run has a hard budget: time, tokens, iterations
- [ ] Agents can only touch scoped paths
- [ ] Acceptance criteria are executable and written before agents start
- [ ] Tests do not come from the agent that wrote the code
- [ ] Agents *stop* on success
Top comments (1)
My own harness is a boring bash loop: hard timeouts, scoped paths, a stop condition. Nothing clever.
What's in yours?