A model that grades your AI feature makes mistakes too, and they leak into the score. Measure how often it is wrong on a hundred outputs a person checked, and you can take its mistakes back out.
Engineer's view. This is a health check that returns 200 while the database is down. Your uptime number then mixes real uptime with the check's blind spot. A model grading your outputs does the same thing. It passes some bad ones, so the score runs high and every real fix looks smaller than it was.
TL;DR. The pass rate a grading model reports is part your product and part the grader's own mistakes. Measure those mistakes on a small set a person has checked, and you can subtract them back out.
Explain Like I'm 5
You run a fruit stand, and a helper sorts apples into a good basket and a bad basket. She
is quick, but she is kind. Most bruised apples land in the good basket too.
So you count the good basket and feel proud. Nine apples in ten look fine. That number is
partly your apples and partly her kindness.
To find the real number, you test her. Hand her fifty apples you know are good and fifty
you know are bruised. Count how many of each she gets right. Now you know how many bruised
apples she lets through on a normal day. Take that many out of the good basket, and what is
left is the true count.
The Problem
You have shipped this bug already, in a system with no model in it.
Your uptime dashboard said 99.9% for a month. Users kept filing tickets about errors. After
a day of digging you found the health check. It pinged an endpoint that returned 200 as long
as the web process was alive. When the database was down, the check still passed.
Nothing was wrong with the arithmetic. The dashboard faithfully counted what the check said.
But the check had a blind spot, so the number mixed two things: real uptime and the check's
habit of passing broken states. You could not fix the dashboard until you knew how often the
check was wrong.
An eval that uses an LLM judge has that bug at full size. An LLM judge is a model you
prompt to grade each output of your feature as Pass or Fail. Its pass rate goes on the
dashboard as if it were your product's pass rate. It is not. Judges make mistakes in both
directions, and they lean lenient. One 2025 study of judges grading feedback on student code
found they passed 96% of good answers but failed fewer than a quarter of bad ones.
That leniency does two kinds of damage. The score runs high. Worse, it hides progress. When
the judge passes most bad outputs anyway, fixing some of them barely moves the number.
The fix: label about a hundred outputs by hand, measure how often the judge agrees on the
good ones and on the bad ones, and use those two rates to undo the blend.
Read the full post on The AI Commit
How to Correct an LLM Judge's Pass Rate With 100 Human Labels carries the rest: the fix and why it works, the code to change for each role, how to tell it worked, when it is the wrong tool, and an interactive model you can play with.
Originally published at theaicommit.com, a daily AI engineering lab for software engineers. The original has an interactive version you can play with, and the full runnable code is on GitHub.
Top comments (1)
Small mismatch in the full write-up: the sample stops when
TPR + TNR - 1 < 0.30, which is the same asTPR + TNR < 1.30. With a sum of 1.20, the sample refuses to correct while the prose's 0.30 cutoff on the sum lets it through. I'd keep the- 1in the prose so both use the same guard.