DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

Evals and LLM-as-a-Judge: a model can grade another model, if you cancel position, verbosity, and self-preference bias

#ai #llm #machinelearning #beginners

You built a smart model. Now — is it actually good? A benchmark gives one number for a leaderboard; a task eval tells you it works for your job. And to grade at scale, you can ask a strong model to be the judge — an LLM-as-a-judge agrees with human preference about as often as two humans agree with each other, at API speed and near-zero cost. The catch: the judge inherits biases a tick-box test never had. I built an interactive harness that shows each bias flipping a verdict and each mitigation cancelling it; here's how a trustworthy eval stack fits together.

The pointwise eval loop — a reproducible score

Start with your tasks, not a public benchmark. Run each case, score the answer against a rubric, pass if it clears a bar. Because it's deterministic, the same inputs give the same number every run — so a drop is a real regression, not noise, and a private set doesn't leak into training data the way public benchmarks do.

def run_eval(model, cases, judge, pass_bar=3.0):
    results = []
    for c in cases:                                  # YOUR real inputs
        answer = model(c.input)
        score  = judge_pointwise(judge, c.input, answer, c.rubric)  # 0..5
        results.append(score >= pass_bar)            # PASS / FAIL
    return mean(results)                             # one aggregate number
# same inputs -> same number. a drop between runs = a real regression.
Enter fullscreen mode Exit fullscreen mode

Write the judge prompt — a rubric + a parseable verdict

A vague "which is better?" invites every bias. Anchor the judge on explicit criteria, tell it to ignore length and confidence, and force a structured output you can parse. A rubric plus a fixed schema turns a vibe into a number.

JUDGE_PROMPT = """
You are grading an answer. Score 0-5 on THESE criteria ONLY:
  - correctness  (is it factually right?)        # anchor on substance,
  - relevance    (does it answer the question?)  # NOT length or style
  - instructions (did it follow the format?)
Ignore how long or confident the answer sounds.
Question: {q}
Answer:   {a}
Reply as JSON: {"reasoning": "...", "score": <0-5>}
"""
Enter fullscreen mode Exit fullscreen mode

Pairwise is more reliable — but it opens the order bias

Judges (and humans) are far more consistent at comparing two answers than at giving an absolute 0–5, which is why preference data is pairwise. So for head-to-heads, show both answers and ask which wins. But this opens the one bias absolute scoring doesn't have: many judges systematically favour the answer shown in position 1.

def judge_pairwise(judge, q, ans1, ans2):
    prompt = f"Question: {q}\n[1] {ans1}\n[2] {ans2}\n" \
             "Which answer is better? Reply 1 or 2. Judge on the rubric, not length."
    return judge(prompt)          # -> "1" or "2", but sensitive to ORDER
Enter fullscreen mode Exit fullscreen mode

Swap-and-average — kill position bias

Judge both orders and only count a win if the same answer wins both ways. The order-dependent bonus lands on slot 1 in each run, so it cancels out of the comparison. This is the "swap & average" toggle in the demo.

def debiased_pairwise(judge, q, a, b):
    first  = judge_pairwise(judge, q, a, b)   # A in slot 1
    second = judge_pairwise(judge, q, b, a)   # B in slot 1 (swapped!)
    if first == "1" and second == "2":  return "A"    # A won BOTH orders
    if first == "2" and second == "1":  return "B"
    return "TIE"                                        # order flipped it -> tie
Enter fullscreen mode Exit fullscreen mode

Rubric + blind + panel — kill verbosity and self-preference

Swap-and-average fixes order, not length or family — length isn't order-dependent. For verbosity (judges reward longer answers even when padded with fluff), anchor on the rubric and forbid rewarding length. For self-preference (a judge rates its own model family higher), keep the judge blind to who wrote each answer and average a panel of different-family judges.

prompt += "Do NOT reward longer answers. Judge correctness & relevance only."   # verbosity
answers = shuffle_and_strip_model_names([a, b])   # self-pref: judge can't see who wrote what
judges  = [judge_vega, judge_nova, judge_atlas]    # different model families
verdict = majority(debiased_pairwise(J, q, a, b) for J in judges)
# never let a model be the sole judge of its own contest.
Enter fullscreen mode Exit fullscreen mode

Calibrate against humans, then gate the release

Is the judge even trustworthy? Compare it to a small human-labelled anchor set and measure agreement — and Cohen's κ, which corrects for lucky agreement. Only once the judge tracks humans do you let it block a release when quality regresses.

agree = mean(judge_label[i] == human_label[i] for i in anchor_set)
kappa = cohens_kappa(judge_labels, human_labels)   # chance-corrected
assert agree > 0.8 and kappa > 0.6, "judge not reliable enough to gate on"
if run_eval(candidate) < run_eval(current) - 0.02:
    block_release("quality regressed on the eval suite")
Enter fullscreen mode Exit fullscreen mode

Both halves matter: pairwise tells you which of two models people prefer, with order, length, and family biases carefully cancelled; pointwise gives a stable, reproducible score you can gate a release on. Neither replaces the human anchor set that keeps the judge honest. The demo lets you flip each bias on the same two answers — where Answer A is genuinely better — and watch the verdict get hijacked, then recover the moment you switch on the matching fix.

Flip a bias, watch the judge change its mind for the wrong reason, then fix it:
https://dev48v.infy.uk/ai/days/day38-evals-llm-judge.html

Top comments (0)