DEV Community: Nikita Kitaychik

Stop Prompting. Write Loops

Nikita Kitaychik — Tue, 21 Jul 2026 17:14:38 +0000

"You shouldn't be prompting coding agents anymore. You should be designing the loops that prompt them for you." — Peter Steinberger

Back in November, Boris Cherny — the creator of Claude Code — deleted his IDE. Stopped using it. In a recent interview he says it plainly: "I don't prompt Claude anymore. I have loops running that prompt it and decide for themselves what to do. My job is to write the loops." And he adds: this is a transition the market will spend the rest of the year living through.

The shift is simple to say and enormous in its consequences. Before, you're in a chat: prompt → answer → edit → prompt again. The human is the bottleneck on every iteration. Now you design a system once, and it finds the work itself, hands it out to agents, checks the result, writes down what got done, and decides what comes next. It pokes the agents instead of you.

The ladder of abstraction

Daniel Demmel laid this out as a ladder, with prompting on the bottom rung:

Prompt engineering — how to phrase the request.
Context engineering — what to put in the context.
Feedback-loop engineering — letting the agent see the consequences of its own work: tests, logs, real APIs.
Harness engineering — running the whole apparatus: what launches, what verifies, when to call a human.

The trend is a move up the ladder. You stop machining the part and start designing the assembly line. Why it all lined up right now: models got reliable enough to trust with a step you don't check on every iteration; parallelism got cheap; and the infrastructure drove straight into the tools.

Anatomy of a loop

A loop closes without a human for one reason: the code has a cheap, objective oracle — compilation, tests, lint, execution. The agent sees for itself where it lied. That's how you build a closed-loop: agents compile, run, screenshot, hit real keys — they verify themselves.

The loop itself boils down to a formula: find work → bounded action → verification → report → escalation. You formalize this as a one-page contract for each loop:

trigger:  when it fires
scope:    what it touches / what it doesn't
budget:   max attempts / time / files
stop:     conditions for escalating to a human
report:   where to write the outcome

Six orchestration primitives

Anthropic described six patterns that an agent assembles into a harness for a specific task — like snapping together building blocks:

Classify-and-act — routing by type. Whatever comes in (an issue, an email, a ticket, an error) gets classified, and each type goes to its own handler. This is the "dispatcher" at the front of the loop: don't treat everything the same, first figure out what actually arrived.
Fan-out-and-synthesize — the task is split into independent pieces, each solved in parallel in a clean context, then the results are pulled back into one. This is how an agent analyzes a large codebase with a dozen sub-agents at once, while a synthesizer assembles the whole picture. Parallelism where the pieces don't depend on each other.
Adversarial-verify — a separate skeptic agent checks the result against a rubric, and its job is to disprove, not to confirm. This removes self-preferential bias: the author shouldn't be the one grading their own work. The core trust pattern is that whoever checks isn't whoever did.
Generate-and-filter — produce many candidates, filter by quality, throw out the duplicates. Useful when it's cheaper to produce ten options and pick than to try to guess the one right answer on the first try.
Tournament — several agents solve the same task with different approaches, and judges compare them pairwise until one winner is left. This is for cases where the solution space is wide and it isn't clear in advance which path is best.
Loop-until-done — spawn agents until a stop condition holds: nothing new is found, zero errors, the goal is reached. This is how you do an exhaustive sweep — for example, hunting bugs until two passes in a row come back empty.

Notice the key thing: almost every primitive rests on the verification step — adversarial-verify checks, filter selects, until-done watches for "zero errors." Which means they all bottom out in the oracle again. Without a way to say objectively "this is correct," the primitives turn into a pretty imitation of work.

Order matters: tests before code

And here's the nuance that decides accuracy. If the agent writes the code and then the tests, it's checking itself: the tests inherit the same misunderstanding as the code, they converge, and a green build means nothing. If tests come before code, the oracle becomes independent of the implementation: the test encodes the intent before there's a solution. Then "tests passed" is a real signal. Test-first turns the test from a check into a spec-as-oracle, and the loop's accuracy climbs sharply.

The trap: the agent can write weak tests or bend them to the code just to get green. So the test has to be derived from the contract and the acceptance criteria, not invented by the agent, and it has to be protected from rewrites. With that discipline the loop converges on the correct thing, not the convenient one.

And where there's no oracle at all — design, a product decision, a specification — the loop doesn't converge on its own, and a human has to stand at the gate.

The sober risk

Addy Osmani puts it honestly: the loop makes mistakes unsupervised — and they stay unnoticed. Comprehension debt grows: the code moves faster than your understanding of it. And there's "cognitive surrender" — when you stop forming an opinion and accept whatever the machine produced. His formula: "Build the loop. But build it as someone who intends to stay an engineer, not just hit 'run.'"

What it looks like in practice: from contract to dev loop

If code agents work in loops, the bottleneck moves: it's not in writing the code, but in the quality of the contract the loop checks itself against. The loop is exactly as good as your definition of "done" is precise.

Step 0. The requester hands over a contract, not a task. It has three things that make it loop-ready:

claims about the code are grounded — a method/field actually exists, verified, not "should be there";
acceptance criteria are observable — you can run them, not eyeball them;
the why is written down — so the decision transfers instead of being re-invented every time.

Step 1. The dev loop starts from acceptance, not from code. The agent reads the criteria → writes tests that encode them (test-first, from the contract) → implements → runs tests and lint → fixes red itself within budget → repeats until green.

Step 2. Escalation only on real forks. The loop calls a human when "the contract says X but the code does Y" or "the criterion is ambiguous." It doesn't call over a missing bracket.

Step 3. The output is a PR and a test run. Review happens over the PR and the contract, not step-by-step behind the agent.

What changes for the developer, concretely:

routine verifiable pieces stop being hand-written — the loop drives them;
the role shifts from implementer to loop owner and PR reviewer;
the engineer's judgment moves to where there is no oracle — an architectural fork, a non-obvious tradeoff — instead of "implement the obvious thing."

The honest boundary. This works where acceptance is objective and the contract is grounded. Where there's no oracle — design, the product choice of "are we building the right thing" — a human is at the gate, the loop doesn't converge on its own. And test-first is mandatory: without it, the agent grades itself.

So why keep a senior engineer?

Let's follow the logic all the way, without flinching. If the requester hands over a verifiable contract, and the loop implements and verifies it — why keep a senior engineer?

There's an error hidden in the question. It assumes that the architecture that satisfies the contract is itself unambiguous. For the simple case — yes. For the complex one — no. The contract specifies the WHAT and the acceptance. It doesn't specify the HOW: the data model, the transaction boundaries, where the lock goes, the routing, the behavior on a network partition, the migration strategy. In that gap between the "what" and a sound "how" there's no test-first oracle, because the solution space is open and the tradeoffs run deep.

So the contract doesn't transfer in a single act — it's an enrichment chain:

Three things make this work.

The technical contract is also verifiable, and its criteria are the oracle. The senior doesn't enrich with prose: the technical acceptance criteria are observable and become the very tests-before-code. The senior finishes writing the oracle — after which the loop can run.

Validation runs upward. If the contract is ambiguous or unbuildable, the senior sends it back to the requester rather than quietly patching it. The contract isn't "handed down," it's negotiated between two gates.

The amount of enrichment scales with architectural novelty. A trivial feature — nothing to enrich, the loop runs almost straight from the contract, the senior barely touches it. A complex one — the enrichment is the senior's core value. Involvement scales with complexity, not with the number of features.

From this comes the symmetry that holds the whole conclusion. The requester owns the intent gate — "is this the right feature" has no oracle. The senior owns the architecture gate — "is this the right design, is it correct under load, under a race, under attack" has no oracle either. The loop fills the verifiable middle. Remove either gate and you get confident-wrong at scale: either the wrong feature, or a beautifully described feature on a naive, insecure architecture that passed shallow tests.

The honest, uncomfortable part: the headcount drops. The "implementer of clear specs" engineer is in the danger zone. The "architect and loop owner" engineer is needed more than ever. For the business, that's exactly the point: the same number of people delivers several times more. Leverage.

The one-line takeaway: the loop pushes everyone up — to where there is no oracle — and erases the middle that the oracle covers. It doesn't remove the role, it removes the oracle-coverable part of the role. The requester on intent. The senior on architecture. The loop in between, running exactly as far as the contract is verifiable from both ends.

Building on Boris Cherny, Peter Steinberger, Addy Osmani, and Anthropic Engineering.

The Oracle Lens

Nikita Kitaychik — Tue, 21 Jul 2026 17:09:16 +0000

In the last piece I said a specification deserves a compiler: a gate that fails when the input is wrong. That sounds clean until you try to build one, because you immediately hit a question the word "wrong" was hiding: wrong how?

Verification is not one thing. When you check a spec, or a plan, or a chunk of generated code, you are actually asking up to three different questions that feel the same and are not. Conflating them is the most common mistake I see in AI-for-products tooling, including in tools that are otherwise very good. So before any gate, the lens. There are three axes, and they are orthogonal.

Truth, Coherence, Intent

Truth is groundedness in reality. Is this claim actually so? "The API returns this field." "This table has a status column." "The consumer can tolerate the extra parameter." Each is true or false against a world that exists independently of the document: the codebase, the running system, the data. You check Truth by going and looking: grep the source, call the endpoint, run the query. Truth has an external referent, which is exactly why it's the most mechanizable of the three.

Coherence is internal consistency. Does the document contradict itself? Are there gaps, behavior specified for some inputs and silent for others? Is the same term used two ways? Coherence needs no world outside the document; you can check it by reading the document against itself. This is the axis most spec tooling has actually reached. When Kiro flags an ambiguous sentence or two requirements that can't both hold, or when Spec Kit's checklist asks whether every case is covered, that's a coherence oracle. It's genuinely useful and I don't want to undersell it.

Intent is the match to what was actually wanted. Not "is this consistent" and not "is this true" — "is this the right thing." A spec can be flawlessly coherent and perfectly grounded in reality and still describe a feature nobody needed, or the wrong feature, or the right feature with a subtly wrong default that only the person who asked would recognize as wrong. Intent lives in a human's head. It has no external referent you can grep and no internal test the document can pass.

The reason to name all three is that they come apart. That's the whole utility of the lens.

They pull apart, and that's the point

A spec can be coherent and false. It hangs together beautifully, every requirement consistent with every other, and it's built on a claim about the system that isn't true. Nothing internal will catch it; the document is at peace with itself. Only a truth check, reaching outside the document, catches it.

A spec can be true and incoherent. Every individual claim checks out against the codebase, but two requirements quietly contradict each other, or half the input space is unspecified. Grounding passes claim by claim; coherence fails at the seams.

And a spec can be true, coherent, and wrong: the hardest case. It lies about nothing, contradicts nothing, and describes something other than what was wanted. No mechanical oracle catches this, because there's nothing mechanical to check it against.

Here's the tidy real-world proof that these axes are separable rather than a philosopher's distinction: look at where the industry stopped. Everyone who climbed to the spec layer (Kiro, Spec Kit) built a coherence oracle and planted a flag. They can tell you your spec contradicts itself. Not one of them can tell you your spec is lying about the API, because that's a different axis entirely and needs different machinery: something that leaves the document and goes to look. The coherence gate and the truth gate are not the same gate at different strengths. They're different gates. The market accidentally ran the experiment and confirmed the lens.

A live example: coherence without truth

I'll give you one from the making of this very series, because it's the cleanest illustration I have and it happened to me last week.

I was researching whether anyone had already published a Truth/Coherence/Intent-style taxonomy. One of my agents came back with a perfect hit: a 2026 paper, it reported, that split verification oracles along exactly those lines and declared intent-matching to be fundamentally non-mechanizable, complete with a quotation. It was on-topic, well-phrased, and it fit my thesis so snugly I almost used it.

It was fabricated. A second pass that actually opened the PDF found the word "intent" appears twice in the paper, in unrelated sentences, and the quoted line does not exist. The summary was internally flawless (coherent, fluent, plausible) and false at the root. It passed every coherence check a reader could apply to the summary itself. It failed the only check that mattered: go read the source. Coherence green, truth red. If I had a coherence gate and mistook it for a truth gate, that fabrication would be a citation in this article right now.

That is the entire argument for the lens in one anecdote. The dangerous errors are not the incoherent ones — those look wrong. The dangerous errors are the coherent-but-false ones, because they look right, and only an oracle pointed at the world rather than at the document will stop them.

The strength gradient — and the axis you can't mechanize

The three axes don't just differ in kind; they differ in how mechanizable they are, and that ordering should drive where you spend effort.

Truth is the most mechanizable. It has an external referent, so you can automate the check: grep, compile, execute, query. Build the hardest, most automatic gate here. This is where a spec compiler earns its keep, and, per the market, where the least has been built.

Coherence is partly mechanizable. Consistency and completeness checks can be automated to a real degree, and LLMs are decent at flagging contradictions. It has diminishing returns but it's tractable. The field is already here.

Intent is not mechanically oracle-able, and pretending otherwise is the trap. There is no external referent and no internal test; the ground truth is a state of mind that is often underspecified even to the person holding it, and that changes when they see the result. You can approximate it — acceptance tests encode a slice of it, examples pin corners of it, accumulated human rulings (a later piece: case law) give you a weak, drifting proxy. But approximate is the ceiling. The honest engineering move is not to fake an Intent oracle with a confident LLM judge; it's to name Intent as the axis that requires a human, put the person exactly there, and mechanize Truth and Coherence hard enough that the human's scarce attention is spent only on the question no machine can answer.

That is what the lens buys you. Not a new gate — a way to know, for every gate you build, which of the three questions it actually answers, so you never again let a coherence check stand guard over a truth-shaped hole, or let a machine pretend to hold the one gate that was always yours to hold.

Coming later in the series: the truth gate itself — grounding a specification against reality, and why the interesting failures are about layers, not lies.

Specification Is a Compile Target

Nikita Kitaychik — Tue, 21 Jul 2026 17:09:09 +0000

For most of my career, the specification was the cheap part. You wrote a paragraph, maybe a ticket, and the expensive, skilled, irreplaceable work happened downstream: turning that paragraph into correct code. The spec was a suggestion. The implementation was the truth.

That ratio has inverted, and almost nobody has repriced their work to match.

When an agent can take a precise description of a change and produce a working, tested implementation in minutes, the implementation stops being the bottleneck. It stops being where the skill lives. The skill moves up, to the only place left that the machine cannot do for you: deciding what to build, and stating it precisely enough that the rest is mechanical. The senior engineer doesn't disappear. They get pushed one rung up the ladder, onto the specification.

And here's the part the tooling hasn't caught up to: if implementation is now a compile step, then the specification is a compile target. It's no longer prose you hand to a human who will fill the gaps with judgment. It's a source artifact that gets compiled into a system. And like any source artifact fed to a compiler, it deserves a type checker, a linter, and a build that fails when the input is wrong.

That's the whole thesis. The rest of this series is what it looks like when you take it literally.

What "compile" actually demands

A compiler is unforgiving in a specific, useful way. It doesn't ask whether your program is a nice idea. It asks whether your program is grounded: do these symbols exist, do these types line up, does this call match a real signature? It refuses to proceed when the answer is no. The refusal is the feature. A compiler that warned politely and emitted a binary anyway would be useless.

Apply that standard to a product specification and most of them fail instantly. A typical spec is full of claims about reality: "the API already returns this field," "this is just a UI change," "the consumer can handle the extra parameter." Each of those is a claim that is either true or false against an actual codebase, and almost none of them are checked. They're asserted in a planning doc, they survive review because they're plausible, and they detonate three weeks later in implementation when someone discovers the field was never there.

When a human wrote the implementation, this was survivable: the human caught the lie while coding, cursed, and patched the spec in their head. When an agent writes the implementation, the lie goes straight through. The agent trusts the spec. The spec was wrong. Now you have a confidently wrong system, built fast.

Take the shape of a failure I see constantly. A draft spec claims the system already contains a dormant mechanism for the thing you're about to build (there's a flag, or a commented-out block, or a function whose name matches exactly), and the proposal is, essentially, it's already there, just switch it on. Plausible. The symbol exists; a lazy check (does the name appear in the code?) passes. But a gate that refuses to accept name-matching as evidence opens the body instead, and finds that the code with the right name does something unrelated: it guards a different resource, at a different layer, than the claim assumes. Right name, wrong thing, and an entire workstream would have been built on it. When you force every claim to a citation and re-read it in context, the ones that fail are almost never absurd inventions. They're plausible claims that happen to be true about a different part of the system.

So the first thing a spec-as-compile-target needs is the compiler's core move: a blocking gate that refuses to admit an unverified claim. Not a reviewer who might notice. A gate that demands evidence: grep it, cite the file and line, or the claim doesn't enter the spec. I'll spend a whole piece on this one, including what a leak rate of unverified claims looks like when you actually measure it, and why the interesting failures aren't lies but layer confusion — claims that are true, but true about the wrong layer of the system.

The cell nobody is standing in

Here's the uncomfortable, and I think defensible, claim: I looked, and almost everyone is building the compiler one floor too low.

The spec-driven-development movement is real now. GitHub's Spec Kit, AWS's Kiro, Tessl — there's a credible map of who occupies what, and it's worth reading. Some of them even have a gate at the spec layer, which is more than I expected when I went looking. Kiro walks you from requirements to design to tasks and checks the requirements document as it goes. Spec Kit ships a Review & Acceptance Checklist you validate before generating.

So the room isn't empty. But look at what those gates check, because that's the whole point. Kiro's spec gate flags ambiguity, internal inconsistency, incompleteness, wrong level of detail. Spec Kit's checklist asks whether the spec is complete and well-formed. These are real, useful checks, and every one of them is a coherence check. Is the document self-consistent? Is it complete? Does it contradict itself?

None of them asks the other question: is it true? Does "the API already returns this field" survive contact with the actual API? None of these tools grounds the spec's claims about reality against the actual codebase before it generates. Spec Kit's reality check runs after implementation: it compares the built code back to the spec, which is a fine thing to do and the opposite of what I'm describing. The grounding research that does verify against source (2026 papers with blocking gates, typed evidence, mechanical citation-checking) operates one floor down, on code and retrieved chunks, not on requirements. The design-system tooling grounds the implementation in real components. The render oracles judge the output.

That's the room that's actually empty, and it's a sharper room than "nobody has a spec gate." Everyone who reached the spec layer built a coherence oracle. Almost nobody built a truth oracle for the spec: a gate that refuses a requirement because its claim about the existing system is false. "Refuse my spec when it's lying" — not "when it's vague." That distinction between coherence and truth turns out to be the spine of this whole series, and I'll make it explicit in the next piece.

I want to be honest about why that room is empty, because the flattering explanation is also the wrong one. It's not empty because it's a clever niche I alone spotted. It's empty because it's early. The hybrid it requires, someone who decides what to build and can engineer the verification of that decision, is rare, and the whole industry's attention is pointed at automating implementation, which is the thing that just became easy. My bet is that this is temporary. My own argument predicts it: as implementation automates, every senior gets pushed onto the spec layer, and they'll arrive needing exactly these tools. This series is a playbook written from inside the room, before the crowd arrives.

That's a bet, not a certainty. "Early" is a synonym for "risk." The audience that needs a spec compiler today is small, and a fair reader will say this is just good engineering hygiene with grandiose framing. Maybe. I'm writing for the version of you that's six months from now, standing on the rung you're about to be pushed onto.

The series: a map, not the territory

What I've built is a real pipeline that does this: a compiler for specifications, in daily use on a production codebase. I'm going to give away the map: the frameworks, the vocabulary, the patterns, the reasoning, and the honest numbers. I'm keeping the territory: the actual prompts, the harness wiring, the accumulated rulings, the datasets. The map is what recruits you to the idea. The territory is the part that doesn't copy out of a blog post anyway.

Here's what's coming next:

The Oracle Lens. Before you can build a gate you need to know what kind of truth it can check. I split verification oracles into three axes: Truth (is this grounded in reality), Coherence (is this internally consistent), and Intent (is this what was actually wanted). They're orthogonal, which is why a spec can pass grounding and still be the wrong thing.
Grounding the spec. The blocking gate above, in full: evidence-or-it-doesn't-count, the leak-rate metric, and layer confusion as the real enemy.
Contracts, Not Tasks. How you hand work to an agent matters more than which agent, and the principle underneath: decomposition granularity should scale inversely with the strength of your acceptance gate.

There's more behind these: why the whole thing compounds over time, what to do about the one axis no machine can check, and the honest numbers from measuring my own pipeline. The series will get to it. But it starts with the reframe, because it reorganizes everything else: the implementation was never the hard part. Stating the truth precisely enough to compile it — that's the job now. Build the compiler for that.

Next: The Oracle Lens — the three kinds of truth a gate can check, and the one it can't.