AI can now write the feature.
Then AI can write the tests.
Then AI can open the pull request.
Then AI can review the pull request.
Then AI can fix the review comments.
And finally, a developer clicks:
Approve.
That workflow sounds incredibly efficient.
It also creates a very uncomfortable question:
If AI writes the code and AI reviews the code, what exactly is the human verifying?
This is no longer theoretical.
GitHub says Copilot code review now accounts for more than one in five code reviews on GitHub, and its review system can explore repository context, inspect large pull requests, review bot-authored PRs, and re-check its own findings after changes.
That can be extremely useful.
But only if we are clear about what the human reviewer is still responsible for.
Because:
AI reviewing AI-generated code is not the same thing as independent verification.
The New Development Loop
A growing number of workflows now look like this:
Requirement
↓
AI generates implementation
↓
AI generates tests
↓
AI opens pull request
↓
AI reviews pull request
↓
AI fixes findings
↓
Human approves
On paper, this looks great.
Everything has been:
- implemented
- tested
- reviewed
- fixed
So what is left for the developer?
A lot, actually.
Because every step above can share the same misunderstanding.
The Biggest Risk: Shared Assumptions
Imagine the requirement is:
A user should only receive a refund if the payment was successfully captured.
AI misunderstands that as:
A user can receive a refund if a payment record exists.
The AI then writes the implementation.
Then it writes tests.
The tests use the same interpretation.
Then an AI reviewer inspects the code.
It sees:
- clean structure
- tests passing
- correct types
- reasonable error handling
Everything looks good.
But the requirement is still wrong.
The pipeline becomes:
Wrong assumption
↓
Correct implementation of wrong assumption
↓
Correct tests for wrong assumption
↓
Correct review of wrong implementation
↓
Green CI
This is why passing tests and positive AI review are not enough.
They can verify consistency.
They cannot guarantee that the original understanding was correct.
AI Can Review Code Without Understanding Your Real Intent
This is where human judgment still matters.
An AI reviewer can often detect:
- obvious bugs
- null handling
- unsafe patterns
- missing validation
- suspicious logic
- inconsistent naming
- duplicated code
- test gaps
That is valuable.
GitHub has been expanding Copilot code review with deeper repository exploration, severity levels, and stronger agentic review capabilities.
But your actual product intent may not live inside the codebase.
It may live in:
- a conversation with a customer
- a product meeting
- a support ticket
- a legal requirement
- a business rule
- an undocumented edge case
- a decision made six months ago
The model may never see that context.
And even if you provide it, it can still misunderstand it.
That means the human reviewer has to verify something deeper than syntax.
The Developer Should Verify the Requirement First
The first question should not be:
“Is this code good?”
It should be:
“Is this solving the right problem?”
Before reviewing the implementation, ask:
- What was the original requirement?
- What behavior should the user actually see?
- Which business rules matter?
- What should never happen?
- What assumptions did the AI make?
- Are those assumptions correct?
This is requirement verification.
And it may become one of the most important developer skills in AI-assisted development.
Tests Are Not Independent If AI Wrote Them From the Same Prompt
This is another subtle problem.
Suppose you give an AI:
Build a password-reset flow.
The AI writes the feature.
Then you say:
Write tests.
Those tests are often based on the same mental model the AI used for the implementation.
So if the model forgot an important requirement, the tests may forget it too.
You get:
Implementation assumption
↓
Test assumption
↓
Both agree
↓
Tests pass
That is not independent verification.
It is agreement.
And agreement is not the same as correctness.
Ask Tests to Challenge the Implementation
A better approach is to change the role of the test-generation step.
Instead of:
Write tests for this implementation.
try:
Act as a skeptical QA engineer.
Do not assume the implementation is correct.
Based on the requirement, identify:
- failure cases
- abuse cases
- race conditions
- boundary cases
- invalid states
- unexpected user behavior
Then write tests that try to break the implementation.
That creates more separation between:
creator
and
critic
It is still AI reviewing AI.
But at least you are changing the objective.
The Human Reviewer Should Look for Decisions
AI is very good at examining code.
Humans should spend more time examining decisions.
For example:
Instead of asking:
Is this function correct?
ask:
Why does this function exist here?
Instead of:
Does this API return the right type?
ask:
Should this endpoint be allowed to perform this operation at all?
Instead of:
Is this query efficient?
ask:
Should this data be queried this way in the first place?
Instead of:
Do the tests pass?
ask:
Are we testing the behavior the business actually needs?
This is where developers add the most value.
Architecture Still Needs Human Attention
AI reviewers can catch many local issues.
But architecture is often about tradeoffs across the entire system.
An AI-generated feature may introduce:
- another service
- another queue
- another database table
- another cache
- another dependency
- another abstraction
Each one can look reasonable individually.
Together, they may make the system worse.
That is why the human reviewer should ask:
Does this change make the system easier or harder to understand?
That question is often more important than whether every line is technically correct.
Watch for “AI Agreement Loops”
One pattern I think teams should actively avoid is:
AI writes code
↓
AI reviewer suggests change
↓
AI author accepts
↓
AI reviewer approves
↓
Human sees green check
↓
Merge
Everything in that loop may come from models with similar assumptions.
The process looks highly reviewed.
But the independence between steps may be weak.
I think of this as an:
AI Agreement Loop
The system creates the appearance of verification because multiple stages agree.
But those stages may not be truly independent.
That is the key danger.
Green Checks Can Create False Confidence
Developers are trained to trust signals like:
Tests passed
Lint passed
Type check passed
AI review passed
Security scan passed
Those signals are useful.
But they can create a dangerous feeling:
“Everything is green, so this must be safe.”
Not necessarily.
A green pipeline tells you:
The system passed the checks you chose to run.
It does not tell you:
You chose the right checks.
That distinction matters more than ever.
A Practical Human Review Checklist
When AI writes the code and AI reviews the code, I think the developer should verify at least these 7 things.
1. Verify the Requirement
Ask:
Does this actually solve what the user or business asked for?
Not:
Does the code match the prompt?
Those are different questions.
2. Verify the Assumptions
Ask the AI:
List every assumption this implementation makes about:
- user behavior
- data
- permissions
- APIs
- infrastructure
- ordering
- timing
Then review them manually.
Hidden assumptions cause a huge number of production bugs.
3. Verify the Failure Modes
Ask:
- What happens if the network fails?
- What happens if the DB write partially succeeds?
- What if the request is repeated?
- What if the external API times out?
- What if two requests happen simultaneously?
- What if the input is technically valid but unexpected?
Happy-path code is easy.
Production failures live outside the happy path.
4. Verify the Blast Radius
If this change fails, what breaks?
One component?
One customer?
All customers?
Payments?
Authentication?
Data integrity?
A change touching a critical path deserves more human attention than a small UI change.
5. Verify the Architecture
Ask:
- Did the AI add a new abstraction?
- Did it introduce another dependency?
- Did it duplicate existing functionality?
- Did it bypass an existing architectural boundary?
- Did it make the system harder to reason about?
Correct code can still create bad architecture.
6. Verify the Tests
Do not only read whether tests pass.
Read what they actually test.
Ask:
If the requirement were wrong, would these tests notice?
If the answer is no, you need better tests.
7. Verify That You Can Explain the Change
This is my final rule.
If someone asks:
“Why was this implemented this way?”
you should have an answer better than:
“The agent generated it and Copilot approved it.”
If you cannot explain the change, you are not ready to own it.
Use Different Agents for Different Roles
One useful pattern is to deliberately separate roles.
For example:
Agent 1 — Implementer
Implement the requirement using the existing architecture.
Agent 2 — Adversarial Reviewer
Assume this implementation contains subtle bugs.
Try to find:
- incorrect assumptions
- security problems
- failure modes
- race conditions
- architectural problems
Agent 3 — Test Designer
Ignore the existing tests.
Design tests directly from the original requirement.
Human
Verify:
Did all three understand the actual problem correctly?
Multiple agents do not replace human review.
But role separation can reduce simple agreement loops.
Review the Requirement and the Diff Together
One practical habit helps a lot.
Do not review a pull request with only the code visible.
Keep the original requirement next to it.
For every significant change, compare:
Requirement
↕
Implementation
↕
Tests
These three should agree.
If only:
Implementation ↔ Tests
agree, you may still have a perfectly tested wrong feature.
Ask AI to Explain Before Asking It to Fix
Suppose the AI reviewer finds a problem.
Avoid immediately clicking:
Apply suggestion
Instead ask:
Explain:
1. Why this is a problem.
2. What real-world failure it could cause.
3. What assumptions the current implementation makes.
4. Why your proposed change fixes it.
5. What new risks your fix introduces.
Now you are reviewing reasoning, not just accepting another generated diff.
That helps maintain ownership.
AI Review Should Increase Human Leverage, Not Remove Human Judgment
I am not arguing against AI code review.
Quite the opposite.
AI review can be extremely useful.
It can catch issues humans miss.
It can reduce repetitive review work.
It can inspect large changes quickly.
It can help teams focus attention where it matters.
But the goal should be:
AI checks mechanics
+
Human checks meaning
Not:
AI writes
+
AI tests
+
AI reviews
+
Human clicks approve
That second workflow may be fast.
But eventually the developer becomes little more than the final button in an automated pipeline.
And that is dangerous.
The Human Role Is Moving Up a Level
The developer's value is gradually shifting.
Less time may go into:
typing syntax
More time may go into:
understanding requirements
designing systems
challenging assumptions
verifying behavior
evaluating risk
debugging failures
making tradeoffs
taking ownership
That is not the disappearance of software engineering.
It is software engineering becoming more about judgment.
The Question I Now Ask Before Approving AI Code
I used to ask:
“Does this code look correct?”
Now I think the better question is:
“What do I know that the AI systems in this loop might not know?”
Maybe it is:
- customer intent
- historical context
- production behavior
- an undocumented dependency
- an organizational constraint
- a previous incident
- a business rule
That missing context may be the most important part of the review.
Final Thought
We are quickly approaching a workflow where AI can:
write the code
write the tests
review the code
fix the review
and maybe even:
merge and deploy it
That does not make human verification less important.
It changes what humans need to verify.
The developer's job is increasingly not to check whether every semicolon is correct.
It is to ask:
Did we build the right thing?
Are the assumptions correct?
Do we understand how it can fail?
Can we explain why this design exists?
Are we willing to own the consequences?
Because if AI writes the code and AI reviews the code, the most valuable thing the developer can provide is something neither system automatically guarantees:
Independent judgment.
AI can review the implementation.
The developer still has to review reality.
Top comments (16)
i think the more interesting question here is not whether ai can write correct code, but how we verify that the original requirement was understood correctly.
if the same model writes the code, creates the tests and reviews the pull request, you can end up with a lot of confidence without much independent verification.
the developer's role may be shifting from checking every line to questioning the assumptions, business rules and decisions behind the code.
good points, enjoyed the read.
The shared-assumption chain is the right frame, and there is an executable version of "verify the requirement first" that survives a fast pipeline better than a checklist does.
Take the implementation the model produced, flip one condition in it, and re-run the tests the model wrote. If they still pass, the tests encode the same misunderstanding as the code, and you learned that in about ten seconds rather than in a postmortem. A test that cannot fail is not evidence of anything, and when the same prompt produced both halves, that is the default case rather than the unlucky one.
The second thing your refund example shows is where the gap actually lives. "Refund only if the payment was captured" and "refund if a payment record exists" produce identical behaviour on every row except one: a record exists and the capture failed. The AI does not write a wrong test for that state. It writes no test for it, because the state is absent from its model of the world. So the human question is not "are these tests correct", which is slow and subjective. It is "which state is missing from this table", which is shorter, mechanical, and answerable by someone who knows the business rule and cannot read Go.
That reframes the review from judging code to enumerating cases, which is the part a model is worst at and a domain expert is best at.
Agreement Loop is a useful name for what otherwise just looks like a row of green checks. What I can't tell yet is whether the explain-before-fix step keeps the human's read independent, given the explanation comes from the same reviewer that proposed the fix.
The interesting part for me is that verification is moving one level above the code. If AI can generate the implementation, tests and review, the developer should not simply become another layer checking the output. The real question becomes: why was this decision made, and does it fit the architecture, the product and the real-world constraints? A function can be correct. A test can pass. An AI review can approve it. And the system can still be making the wrong architectural decision.
So I would separate it into three layers:
Code → Is it correct?
System → Is it designed correctly?
Reality → Are we solving the right problem?
AI can help with all three, but the developer still needs to own the decisions between them.
"AI Agreement Loop" is the perfect term for what is happening across modern CI pipelines.
When Model A writes the code, Model B (or the same model with a different prompt) generates tests, and Model C reviews the diff, we haven't created an independent quality gate—we've created a synthetic echo chamber. They share the same training priors, the same prompt misunderstandings, and the same architectural blind spots.
As @deanlee pointed out below, running correlated models over correlated code creates the illusion of verification, not external validity. It proves internal consistency, not operational correctness.
This is why I love Point #4 (Verify the Blast Radius).
In an AI-dominated pipeline, the engineer's job fundamentally shifts from "syntax reviewer" to "risk underwriter." The question is never whether the code compiles or tests pass; it's:
What is the blast radius when the AI's unstated assumption collides with the production database at 2:00 AM?
Fantastic and timely essay, Robert. The shift from checking mechanics to evaluating blast radius and reality is where true engineering begins.
The human is, where the human has always been. Accountability. Whether you use Vim, Jetbrain's suite, vs code, or visual studios, it doesnt matter. Each IDE has it's perks and quirks and each has a plethora of extensions to make it 'suck less', that's kinda where we've been living. Sometimes you get a quirk, like I did in Jetbrain Rider, where it decided to use a stale build cache after a merge and I tested, everything worked, pushed, then got chewed out for pushing broken code... (wasnt even my code that was broken, but sure... thanks) The developer exists as the accountability layer, if you let Claude write it for you, or you use Jetbrain's autocomplete, or you manually write it, or copy it from stack overflow, it really doesnt matter... Your job is to be accountable when things go wrong, so there's someone to point the finger at and someone in charge of fixing it. How you do it, has always been up to you.
That's the bit that confuses me so much, everyone is treating AI like it's heaven and hell at the same time, yet the reality is inbetween and we havent shifted an inch. A developer still produces code, a developer still uses all the tools at their disposal to boost their performance and quality, a developer still relies heavily on programs to do the heavy lifting. Nothing has changed, despite all the hype, it's just a new generation of tools and we learn them the same way we did in the past. Through trial and error. When auto-completes first came out, they were fantastic! Except when they assumed the wrong pattern... Your job as a developer is identifying that and fixing it. Now AI goofs up and your job is the same. Nothing changed. Well actually, 1 thing changed, we went from single threaded, to multi-threaded. Now you dont just code with 1 IDE, you code with 5 agents at once. Doesnt change the job in the slightest though, just how you do it.
Every comment here treats "the AI reviewer" as one thing. In our runs it isn't, and I think that's the missing piece.
We ran the same tool-calling tasks through six models. The overall scores were too close to rank. The failure types were not. One family only ever got an argument wrong. Another was the only one that skipped the call entirely. A third was the only one that picked the wrong tool.
So the question changes. Not "is AI review independent?" but "which reviewer is blind to which mistake?" If your writer's usual failure is a wrong but valid argument, a reviewer that only checks structure will pass it every time. You want the reviewer whose strength covers the writer's weakness.
Two things we learned the hard way:
Your refund example is the one no reviewer catches, because the mistake lives in the requirement. What helped us there: any rule with a threshold has to name one input that would make it fail. If you can't name one, the check can't fire, and it will pass forever.
What is left for the human is the part neither pass can check: whether this was the right thing to build at all. The review catches wrong code. Nothing in that loop catches a correct implementation of a feature nobody asked for, and that has cost me more time than any bug ever has.
Running an AI reviewer over AI-generated code treats two correlated draws as independent observations. Because both models inherit similar token priors and training distributions, the review step tests internal consistency rather than external validity. It confirms that the implementation matches the prompt, but it cannot evaluate whether the prompt solved the real economic requirement.
That leaves the engineer underwriting tail risk rather than proofreading syntax. The actual job in that loop is pricing the production downside when an unstated business rule breaks, because the financial liability never stays with the generator.
The refund example shows why reading the diff can't catch this: the misunderstanding isn't a line you can point at, it's a case that's missing, and a reviewer scanning green checks only sees what is there. The cheapest counter I know is to write one test yourself before any code exists, in the requirement's own words: a payment record that was never captured, a refund request, and the expected answer is no. It takes a couple of minutes, and it's the only artifact in the loop that didn't come from the model's reading of the prompt. If that test is still present and green at approval time, the human verified something.