AI can now write the feature.
Then AI can write the tests.
Then AI can open the pull request.
Then AI can review the pull request.
Then AI can ...
For further actions, you may consider blocking this person and/or reporting abuse
i think the more interesting question here is not whether ai can write correct code, but how we verify that the original requirement was understood correctly.
if the same model writes the code, creates the tests and reviews the pull request, you can end up with a lot of confidence without much independent verification.
the developer's role may be shifting from checking every line to questioning the assumptions, business rules and decisions behind the code.
good points, enjoyed the read.
The shared-assumption chain is the right frame, and there is an executable version of "verify the requirement first" that survives a fast pipeline better than a checklist does.
Take the implementation the model produced, flip one condition in it, and re-run the tests the model wrote. If they still pass, the tests encode the same misunderstanding as the code, and you learned that in about ten seconds rather than in a postmortem. A test that cannot fail is not evidence of anything, and when the same prompt produced both halves, that is the default case rather than the unlucky one.
The second thing your refund example shows is where the gap actually lives. "Refund only if the payment was captured" and "refund if a payment record exists" produce identical behaviour on every row except one: a record exists and the capture failed. The AI does not write a wrong test for that state. It writes no test for it, because the state is absent from its model of the world. So the human question is not "are these tests correct", which is slow and subjective. It is "which state is missing from this table", which is shorter, mechanical, and answerable by someone who knows the business rule and cannot read Go.
That reframes the review from judging code to enumerating cases, which is the part a model is worst at and a domain expert is best at.
Agreement Loop is a useful name for what otherwise just looks like a row of green checks. What I can't tell yet is whether the explain-before-fix step keeps the human's read independent, given the explanation comes from the same reviewer that proposed the fix.
The interesting part for me is that verification is moving one level above the code. If AI can generate the implementation, tests and review, the developer should not simply become another layer checking the output. The real question becomes: why was this decision made, and does it fit the architecture, the product and the real-world constraints? A function can be correct. A test can pass. An AI review can approve it. And the system can still be making the wrong architectural decision.
So I would separate it into three layers:
Code → Is it correct?
System → Is it designed correctly?
Reality → Are we solving the right problem?
AI can help with all three, but the developer still needs to own the decisions between them.
"AI Agreement Loop" is the perfect term for what is happening across modern CI pipelines.
When Model A writes the code, Model B (or the same model with a different prompt) generates tests, and Model C reviews the diff, we haven't created an independent quality gate—we've created a synthetic echo chamber. They share the same training priors, the same prompt misunderstandings, and the same architectural blind spots.
As @deanlee pointed out below, running correlated models over correlated code creates the illusion of verification, not external validity. It proves internal consistency, not operational correctness.
This is why I love Point #4 (Verify the Blast Radius).
In an AI-dominated pipeline, the engineer's job fundamentally shifts from "syntax reviewer" to "risk underwriter." The question is never whether the code compiles or tests pass; it's:
What is the blast radius when the AI's unstated assumption collides with the production database at 2:00 AM?
Fantastic and timely essay, Robert. The shift from checking mechanics to evaluating blast radius and reality is where true engineering begins.
The human is, where the human has always been. Accountability. Whether you use Vim, Jetbrain's suite, vs code, or visual studios, it doesnt matter. Each IDE has it's perks and quirks and each has a plethora of extensions to make it 'suck less', that's kinda where we've been living. Sometimes you get a quirk, like I did in Jetbrain Rider, where it decided to use a stale build cache after a merge and I tested, everything worked, pushed, then got chewed out for pushing broken code... (wasnt even my code that was broken, but sure... thanks) The developer exists as the accountability layer, if you let Claude write it for you, or you use Jetbrain's autocomplete, or you manually write it, or copy it from stack overflow, it really doesnt matter... Your job is to be accountable when things go wrong, so there's someone to point the finger at and someone in charge of fixing it. How you do it, has always been up to you.
That's the bit that confuses me so much, everyone is treating AI like it's heaven and hell at the same time, yet the reality is inbetween and we havent shifted an inch. A developer still produces code, a developer still uses all the tools at their disposal to boost their performance and quality, a developer still relies heavily on programs to do the heavy lifting. Nothing has changed, despite all the hype, it's just a new generation of tools and we learn them the same way we did in the past. Through trial and error. When auto-completes first came out, they were fantastic! Except when they assumed the wrong pattern... Your job as a developer is identifying that and fixing it. Now AI goofs up and your job is the same. Nothing changed. Well actually, 1 thing changed, we went from single threaded, to multi-threaded. Now you dont just code with 1 IDE, you code with 5 agents at once. Doesnt change the job in the slightest though, just how you do it.
Every comment here treats "the AI reviewer" as one thing. In our runs it isn't, and I think that's the missing piece.
We ran the same tool-calling tasks through six models. The overall scores were too close to rank. The failure types were not. One family only ever got an argument wrong. Another was the only one that skipped the call entirely. A third was the only one that picked the wrong tool.
So the question changes. Not "is AI review independent?" but "which reviewer is blind to which mistake?" If your writer's usual failure is a wrong but valid argument, a reviewer that only checks structure will pass it every time. You want the reviewer whose strength covers the writer's weakness.
Two things we learned the hard way:
Your refund example is the one no reviewer catches, because the mistake lives in the requirement. What helped us there: any rule with a threshold has to name one input that would make it fail. If you can't name one, the check can't fire, and it will pass forever.
What is left for the human is the part neither pass can check: whether this was the right thing to build at all. The review catches wrong code. Nothing in that loop catches a correct implementation of a feature nobody asked for, and that has cost me more time than any bug ever has.
Running an AI reviewer over AI-generated code treats two correlated draws as independent observations. Because both models inherit similar token priors and training distributions, the review step tests internal consistency rather than external validity. It confirms that the implementation matches the prompt, but it cannot evaluate whether the prompt solved the real economic requirement.
That leaves the engineer underwriting tail risk rather than proofreading syntax. The actual job in that loop is pricing the production downside when an unstated business rule breaks, because the financial liability never stays with the generator.
The refund example shows why reading the diff can't catch this: the misunderstanding isn't a line you can point at, it's a case that's missing, and a reviewer scanning green checks only sees what is there. The cheapest counter I know is to write one test yourself before any code exists, in the requirement's own words: a payment record that was never captured, a refund request, and the expected answer is no. It takes a couple of minutes, and it's the only artifact in the loop that didn't come from the model's reading of the prompt. If that test is still present and green at approval time, the human verified something.
One practical distinction I’ve found useful is separating the reviewer’s job into finding problems and making findings actionable.
A reviewer can identify a suspicious line, but the useful part is connecting it to the exact code location, explaining the risk, and giving the developer enough context to decide whether it actually matters.
That still leaves the human responsible for validating the finding against the real requirement.
"The developer still has to review reality" is a good way to put it. Same-model-writes-tests-and-reviews-its-own-PR is the exact setup that produces confidence without real independent verification.
Câu hỏi thực sự: khi AI viết code, viết test, mở PR, rồi còn có AI review luôn thì developer còn "verify" gì ngoài... việc đọc log và bấm approve? Tớ đang dùng workflow tương tự và thấy điểm núi lở chất lượng không phải ở từng dòng code mà ở chỗ AI tự tin đưa ra quyết định kiểu "tôi đã kiểm chứng bằng cách chạy thử test" mà thực ra test đó chạy trên môi trường mock không đại diện được thực tế. Khi AI là cả người viết và người review, trách nhiệm của developer dời hoàn toàn về kiểu "tôi phải tự mình viết lại câu hỏi đúng để test AI phản hồi", chứ không còn là kiểu "tôi đọc và đồng ý" (site: labagent .tech)
the question that gets me is the signal quality problem: AI reviewers trained on the same distribution as AI writers miss the same classes of bugs. we saw this in a code gen pipeline where the model introduced a subtle race condition that its own review pass consistently ignored because it treated the pattern as idiomatic. the developer's job shifts to adversarial testing, not code review. the useful check becomes: does this behavior match the spec under edge cases. what's your threshold for trusting the AI review versus overriding it?
The shared-assumption chain is the scariest part, and I've watched it happen: the spec gets misread once and every later step politely confirms the misreading. What helped me was splitting verification into two kinds. Structural rules (layering, where logic may live, what may call what) go into a deterministic check that no model can talk its way past. That frees human review for the one thing nothing else can do: comparing the change with the intent. My reviews got shorter, but much more focused on "is this the right behavior?" instead of "is this code tidy?".
I’ve found the useful test is giving the agent a case it didn’t see in the original task. If it can’t explain what should happen there, the green tests don’t tell you much.