Most AI quality checks work like this: one model writes the answer, and a second model grades the answer. Ask what grades the second model. Usually nothing.
The research community measured the problem early. In June 2023, a Berkeley-led team studying language models as judges named the failure modes:
"position, verbosity, and self-enhancement biases"
Position bias: the judge prefers whichever answer comes first. Verbosity bias: the judge prefers the longer answer. Self-enhancement bias: the judge prefers answers resembling its own. The same paper found strong judges agree with humans about as often as humans agree with each other, above 80 percent. Read the number the other way round. On roughly one call in five, your grader and a human disagree, and nobody tells you which call.
Three years later, the vendors selling judge-based evaluation admit the same thing in their own documentation. LangChain, in March 2026:
"Early on, developers wrote rubrics and configured LLM judges hoping the scores would hold, but they didn't."
"An unvalidated judge might measure verbosity rather than whether the output is high-quality and genuinely helpful."
The second sentence is the 2023 verbosity bias, restated by a vendor in 2026. Their fix is to collect human corrections and tune the judge against them. Sound advice. Notice what the advice concedes: the judge is accurate only after you supply the ground truth the judge was supposed to replace.
AWS offers built-in judges for agent quality, with names like Helpfulness. You do not get to inspect or change how they decide:
"Built-in evaluator configurations, including their evaluator models and prompt templates, cannot be modified."
The review vendor Greptile states the problem with an animal. An agent reviewing its own code, Greptile writes, is
"like a fox guarding the henhouse."
And Anthropic, in a November 2025 engineering post on long-running coding agents, described the failure plainly. The model tended to mark a feature complete without proper testing. The recommended fix:
"Self-verify all features. Only mark features as “passing” after careful testing."
The fox, asked to audit the henhouse more carefully.
Google's Harness² paper, October 2026, is the most honest version I have read. The method improves agents by comparing runs with model judgment, and the authors write in their limitations section about the method's signals:
"remain sensitive to execution variability and shared model biases."
Shared model biases. When the writer and the grader come from the same family of models, their blind spots overlap. A second opinion from a twin is not a second opinion.
We learned this the expensive way. In one of our own production runs, the model asked to authorize a final answer was willing to call a run complete with 21 tasks still pending, zero operations performed and zero files written. So Merlin's model authorizer gets a vote and never the last word. Rules counting, hashing and looking up records run around the model and override the model.
Here is the test I would put to any verifier in your stack. Ask what the verifier reads. If the verifier reads the answer, the verifier shares the answer's weaknesses. If the verifier reads the record of what happened, the verifier checks something the model had no chance to argue with.
Top comments (1)
The test of asking what the verifier reads is a good one. Where a model judge is still needed, two cheap mitigations help: use a judge from a different model family than the writer, and score each pair in both orders so position bias cancels out. Then check the judge against a small human-labelled set before trusting the numbers. Deterministic checks on the record, like task counts and file hashes, should still hold the final say.