6 checks before merging AI-agent generated code

Marcin Brzozka — Fri, 26 Jun 2026 16:36:07 +0000

AI coding agents are useful because they can make large changes quickly.

That is also the reason I do not want to merge their patches just because the final answer says “done”.

The risky failure mode is not usually obvious broken code. It is a plausible patch that quietly touches a risky area.

Here is the checklist I use before merging AI-agent generated diffs.

1. Did dependencies change?

Look for package files and lockfiles:

package.json
lockfiles
requirements.txt
pyproject.toml
go.mod
Docker base images

Dependency changes should get explicit review. A tiny source diff plus a large dependency change is not tiny.

2. Did auth, payment, security, or config files change?

Slow down if the patch touches:

authentication middleware,
session/token handling,
payment or checkout code,
webhook handlers,
.env parsing,
deployment config,
CI workflows.

These are exactly the areas where “it builds” is not enough.

3. Did source change without tests changing?

Not every patch needs new tests, but source changes with zero test changes should be visible in review.

At minimum, the author/agent should provide real command output showing what was run.

4. Did generated or bundled files change?

Large generated files can bury important edits.

If a patch changes a minified file, lockfile, generated client, or build artifact, review the source of that generated output too.

5. Are there secret-like literals?

Search for suspicious strings:

api_key
token
secret
password
private keys
webhook secrets

Even test fixtures deserve a second look.

6. Is “tests passed” backed by actual output?

I want to see the command and real result, not just a summary.

Good:

npm test
18 passed

Weak:

Tests should pass.

Turning the checklist into a local gate

I packaged this workflow as a small local Python CLI that scores a unified diff before merge.

Example:

git diff > change.patch
python src/agent_change_risk_auditor.py audit --diff change.patch

It flags dependency changes, sensitive paths, source-without-tests, large/generated changes, and secret-like literals.

The point is not to replace human review. The point is to make “slow down and inspect this patch” visible before merge.

Related resources

I put the checklist and example report here:

Blog/checklist page: http://152.239.117.170/blog/ai-agent-code-review-checklist-before-merge.html
Sample report: http://152.239.117.170/sample-audit-report.html
Product page: http://152.239.117.170/

There is also a small paid Gumroad kit for teams that want the source, CI template, and Pro workflow pack:

Basic Kit: https://marcnova48.gumroad.com/l/cakkb
Pro Pack: https://marcnova48.gumroad.com/l/bdyklr

Question: what risk category would you add to this checklist for AI-generated patches?

I built a local risk gate for AI-agent code changes

Marcin Brzozka — Fri, 26 Jun 2026 15:19:20 +0000

AI coding agents are now good enough to create a lot of code quickly. That also means they are good enough to create a lot of risky changes quickly.

The failure mode I keep seeing is not “the agent writes obviously broken code.” The harder problem is quieter:

a dependency or lockfile changes without anyone noticing,
auth/payment/config files are touched in a broad refactor,
source files change but tests do not,
generated-looking rewrites bury a small important change,
a secret-like literal appears in a diff,
the final report says “tests passed” but the evidence is thin.

So I built a small local CLI workflow: AI Agent Change Risk Auditor.

It reads a unified diff/patch file and returns a risk score before merge.

git diff > change.patch
python src/agent_change_risk_auditor.py audit --diff change.patch

Example output:

AI Agent Change Risk Audit
Risk level: high
Risk score: 63/100
Files changed: 2
Lines: +3 / -1

Flags:
- DEPENDENCY_CHANGE:package.json
- SOURCE_CHANGED_WITHOUT_TEST_CHANGE
- POSSIBLE_SECRET_LITERAL_IN_DIFF

Recommendations:
- Add or update tests for changed source files before merge.
- Remove secret-like literals and rotate exposed credentials if real.
- Review dependency changes manually and run lockfile/security checks.

What it checks

dependency and lockfile changes,
auth/payment/security/config paths,
source changes without test changes,
large or generated-looking rewrites,
secret-like literals in the diff.

What it does not do

It is not a security guarantee. It is not a replacement for tests, code review, or proper SCA.

It is just a cheap local guardrail that catches “slow down and review this” signals before an AI-agent patch gets merged.

Why local-first?

Teams often do not want to upload private diffs to a third-party service just to get a basic risk score.

A local script is easy to inspect, easy to modify, and easy to run in private CI.

Who it is for

founders using AI coding tools,
agencies reviewing AI-generated client changes,
small teams that need a pre-merge safety checklist,
anyone who wants a simple local guardrail before a human review.

The minimal checklist

If you want the idea without any tool, start here:

Did dependencies change?
Did auth/payment/security/config files change?
Did source change without tests?
Did a large generated file change?
Are there secret-like literals in the diff?
Can the agent’s “tests passed” claim be tied to actual output?

That checklist alone catches a surprising amount.

Demo assets

I made two public demo pages for the workflow:

Sample audit report: http://152.239.117.170/sample-audit-report.html
ROI calculator: http://152.239.117.170/roi-calculator.html

Commercial note

I packaged this as a small paid starter kit with source, tests, docs, CI templates, and a commercial-use summary.

Basic Kit: $5 one-time — https://marcnova48.gumroad.com/l/cakkb
Pro Pack: $19 one-time — https://marcnova48.gumroad.com/l/bdyklr

Product page and demos: http://152.239.117.170/

The goal is not to promise perfect security. The goal is to save reviewer attention and reduce obvious AI-agent change risk.

Question for other teams using coding agents: what additional risk category would you add?

DEV Community: Marcin Brzozka