When a coding agent hands me a patch, I do not start by asking another model whether the diff looks sensible. That can help me find risk. It cannot prove that the requested behavior works.
Verification needs to start with the task and end with reproducible evidence.
Freeze the task and revision
I write down the requested outcome, the repository revision before the run, and the exact agent change. If the task is ambiguous, I keep that ambiguity visible instead of silently choosing the easiest interpretation.
This gives every later check an identity. Without it, a green test result can be real and still belong to the wrong revision or requirement.
Find the behavioral boundary
The useful question is what a user or caller can observe. That might be a browser transition, an API response, an authorization denial, a persisted record, or a regression that must stay fixed.
I prefer checks against that boundary. Tests that merely repeat the implementation can pass while the product behavior remains wrong.
Run the smallest authoritative checks
Repository-owned commands and tests come first. If the existing suite does not exercise the changed behavior, I add a focused browser or API check.
The evidence should retain the command, environment, exit state, and bounded output. A passing command without context is weak evidence. A pile of unrelated checks is not automatically stronger.
Separate different kinds of failure
An agent regression, an existing repository failure, an environment problem, a timeout, and an unverified requirement are different outcomes. Treating all of them as the same red mark hides what needs to happen next.
This separation also prevents flaky infrastructure from turning into a false product verdict.
Package the result
I want a reviewer to be able to answer five questions without reconstructing the whole run:
- What task was checked?
- Which revision and change were checked?
- What commands or journeys ran?
- What failed or remained unknown?
- Why did the final verdict follow from that evidence?
A machine-readable evidence bundle makes those answers portable instead of leaving them scattered across terminal output and chat history.
Re-check after the fix
A changed diff is not closure. I rerun the failing check and the smallest relevant regression set, preserving both the earlier failure and the later pass. That shows the path from problem to proof.
This task-to-evidence workflow is the method behind CodeVetter. The complete checklist and its limits are at https://codevetter.com/verify-ai-generated-code.
Top comments (0)