If the same agent that wrote the code also decides whether it is ready, the outcome
is predictable: almost always approved. Not out of bad faith — by construction.
Why self-grading does not work
Whoever wrote it holds the mental model of what they meant to do. When reviewing, they
compare the code against the intention, not the result. The question “does this work?” becomes
“is this how I imagined it?” — and the answer is yes, because they wrote it.
Worse: the same model has an incentive to finish. Saying “done” closes the task; saying “not
yet” opens more work. Without role separation, the short path wins.
What separates the two roles
The critic does not read code, it runs commandsIt runs the test, the lint, the
build, the endpoint. Opinion about the code does not count: the exit code does.The score is numeric and has a floor0-100 with a declared threshold (85, say).
“I think it is fine” does not pass; below the threshold, it fails.Failing requires the reasonSaying “it failed” is not enough: the report lists
what failed, with the command output. Without it, the agent “fixes” blind.The critic does not fixIt measures. The one who fixes is the author — and goes
back to the same ruler until it passes.
In practice
SCORE=100.0 THRESHOLD=85.0 GATE=passed
SCORE=60.0 THRESHOLD=85.0 GATE=failed
GAPS=["pytest: exit code 1, expected 0"]
The same loop, the same code, two different answers because the second round had a deliberately
broken test. The critic did not know that — it just measured.
What changes day to day
You stop reviewing promisesThe report arrives with a score and a reason, not with
“implemented successfully”.The agent improves on its ownWith the failure reason in hand, it fixes what broke
instead of rewriting everything.You see the regressionA score that drops between two rounds signals something
broke — before it reaches production.
See the AI Test Loop skill
See all skills
Want this wired into your project? Guided setup from
R$ 1.500, with the first cycle run together.
Top comments (0)