DEV Community

Cover image for When AI code generation outruns human review
Kristiyan Stoyanov
Kristiyan Stoyanov

Posted on AI-assisted

When AI code generation outruns human review

A coding agent finishes a feature, opens a merge request and starts the next task. The engineers who need to understand the first change are still reviewing yesterday's work.

Faster generation can produce bigger merge requests, more merge requests, or both. If meaningful review cannot keep pace, the delivery queue grows even while the coding work appears to accelerate.

My approach would combine stronger automated checks, approval rules that match the consequences of a change, and AI assistance for reviewers. Gradual release then limits initial exposure and gives the team a chance to detect problems before expanding it. Each part addresses a different constraint.

Smaller merge requests solve part of the problem

Thirty small-to-medium merge requests are not necessarily easier to review than ten large ones. That comparison depends on what the changes do, how they interact and how much context each reviewer must reconstruct.

Small, coherent changes still help. Google's engineering guidance describes a useful change as self-contained, including its related tests and enough context to understand it. Splitting a feature at arbitrary file boundaries can work against that goal.

Consider a change to order retries. Separate merge requests for the retry policy, the duplicate-request protection and its tests might each be short. The reviewer still needs to understand their combined behavior. A short diff is not necessarily a small decision.

I would preserve coherent changes while looking at the whole queue: how much work is waiting, what needs human judgment, and what evidence could make that judgment easier. Generating more work while review is saturated is a reason to limit work in progress too.

Automate checks that protect important behavior

Compilation and a coverage threshold cannot express every delivery requirement. I want checks tied to the ways the application could fail: business rules, real dependency interactions, service contracts, security boundaries and recovery behavior.

For an order-retry change, a useful scenario is a request that succeeds but loses its response. The client retries. Does the system create another order, or recognize the same operation? A passing test that only checks the successful response leaves that question unanswered.

An agent can help implement the test, but its expected result should come from the requirement. Review that expectation independently of the implementation. Keep the checks and the policies governing them protected from casual weakening in the feature change itself.

The pipeline should report what it checked, what failed and what remains untested. That gives a reviewer specific evidence to inspect. It also gives the coding agent a concrete failure to repair.

Set approval requirements according to consequences

A wording correction and an authorization change should receive different attention. Diff size alone cannot determine that: a one-line change can broaden access to every customer's data.

Here is a starting policy I would discuss with a team, assuming its organizational requirements permit it:

Change category Example Human approval approach
Narrow, low-risk category Non-executable documentation typo outside legal or operational instructions Zero required approvals if the defined checks pass
Bounded behavior change A reversible application change within an established component One relevant reviewer
Higher-consequence change Authorization, billing, data migration or a shared service contract Two relevant reviewers, including the responsible owner where needed
Uncertain or mixed scope A change whose consequences or dependencies are unclear Escalate classification before deciding the approval path

These counts are proposed policy choices. Extra approvals only help when the reviewers understand the relevant risks. Conversely, zero required approvals should be a deliberately narrow category with an accountable owner and an auditable history.

GitLab's approval rules illustrate the underlying mechanism: specify required counts and approvers, including code owners. The risk classification above is a team decision, not a capability that an approval counter supplies automatically.

An agent may propose a category and explain it. Enforcement should come from the agreed policy, with uncertain cases routed for human judgment. Treat changes to the approval policy or its checks as sensitive changes themselves. Reassess the classification when the diff changes.

Three complementary safeguards feed gradual release: automated checks, risk-based approvals and AI-assisted review. Canaries, feature flags and recovery controls limit and manage exposure.

Give reviewers useful AI assistance

The reviewer can have an agent too. Its job is to help the engineer find the consequential decisions and verify them efficiently.

For the order-retry example, a useful review brief would identify the retry trigger, the identity used to recognize the same operation, the place that enforces uniqueness and the tests covering a lost response. It should point to the relevant code, including unchanged dependencies that matter.

Mechanical edits can be grouped separately, with an explanation. “Generated files” and “trivial” are insufficient reasons to ignore a change. A regenerated client can introduce a meaningful contract difference; a small configuration edit can change runtime behavior.

A prompt I would use:

Read this merge request and the code it depends on. Explain the intended behavior change, identify the decisions most likely to affect correctness or operations, and link each concern to code and test evidence. Group mechanical changes separately and justify that classification. List assumptions and missing context. Do not approve the change or modify files.

The engineer then checks the brief against the implementation and investigates the important paths. The summary is an aid to review; it does not prove that omitted code is harmless.

Release gradually and decide when to stop

Review and automated checks happen before the new behavior encounters production conditions. Rollout controls provide another opportunity to observe problems and contain exposure.

With a canary, expose a small, representative cohort to the new version and compare it with the existing version. Select signals relevant to the change. For order retries, those could include duplicate-order detection, successful completion and retry volume, alongside errors and latency.

Define the conditions for pausing or reversing the rollout before starting. Give the cohort enough traffic and time to reveal the failure modes being evaluated. A quiet dashboard immediately after deployment is weak evidence if almost nobody has exercised the changed path. Google's SRE guidance explains the importance of representative traffic, attributable metrics and comparison with a control.

A small cohort does not guarantee a small impact. Both versions may share a database or queue. A destructive migration or a burst of retry traffic can affect customers outside the canary too. Those shared dependencies belong in the rollout assessment.

Feature flags add a different control: enable or disable selected behavior without deploying another application version. Keep a working fallback path and test the relevant flag states. Assign an owner to temporary flags and remove them when their rollout purpose is complete. Pete Hodgson's feature-toggle guidance covers both operational switches and the complexity of maintaining them.

Turning off a retry feature can stop future retries. It will not remove an order already duplicated or reverse an external action already completed. Keep the data-repair and recovery plan separate from the switch that stops further exposure.

Start with one category and measure the whole path

I would pilot the approach on one bounded category of change. Define its required evidence, approval policy, review brief and rollout controls before expanding the policy across a repository.

Track time waiting for review, human effort spent reviewing, time to deployment, defects that escape and the effort needed to recover. Compare reasonably similar changes and keep the sample size visible. More merged requests alone would not show that delivery improved.

Use the results to adjust the policy. If reviewers regularly find important omissions in the briefs, improve the brief and its verification. If a category produces unexpected incidents, revisit its checks, approval requirements and rollout limits. If the queue keeps growing, reduce concurrent work while addressing the constraint.

The question I would put to a team is concrete: which changes could ship with less manual approval, and what evidence and recovery controls would make that a responsible choice?

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev •

The retry example shows why review burden follows the decision surface, not line count. I would require the change description to name the idempotency key, the lost-response scenario, and the evidence for duplicate protection; that gives reviewers a stable checklist even when generated code expands the diff.