How LLM-as-Judge Works
LLM-as-Judge delegates the quality evaluation of one LLM's output to another LLM (or the same one).
Two basic forms:
Pointwise scoring (single answer)
JUDGE_PROMPT = """Evaluate this AI response (1-10 per dimension):
Dimensions: accuracy, relevance, clarity
Question: {question}
Response: {answer}
Return JSON: {{"accuracy": int, "relevance": int, "clarity": int}}"""
Pairwise comparison (two answers)
COMPARE_PROMPT = """Which answer is better?
Question: {question}
Answer A: {answer_a}
Answer B: {answer_b}
Reply with only "A" or "B"."""
Pairwise works well for comparing two versions (Prompt A vs Prompt B). Pointwise works better for monitoring a single system's quality trend over time.
Three Experiments: How Severe Are the Biases?
Experiment 1: Position Bias
Design: Same answer pair, evaluated twice — first with A shown first, then with B shown first. Without bias, the winner should be independent of order; first-position win rate should be ~50%.
def experiment_position_bias():
for pair in answer_pairs:
winner_order1 = judge_pairwise(question, answer_a, answer_b) # A first
winner_order2 = judge_pairwise(question, answer_b, answer_a) # B first
# Unbiased: winner_order1 == winner_order2 rate should approach 100%
Results:
First-position win rate: 67% (unbiased baseline: 50%)
Second-position win rate: 33%
Consistency rate: 67% (unbiased baseline: 100%)
Position bias magnitude: 17% above 50/50 baseline
A consistency rate of 67% means 33% of judgments flip when the same answer pair swaps order. If you use this Judge for A/B testing, one third of the results track presentation order, not answer quality.
Experiment 2: Verbosity Bias
Design: Same question, two answer versions — a concise version (2-3 sentences) and a verbose version with identical core content but padded with filler, repetition, and elaboration. Content is equivalent; only length differs.
Concise version:
Lists are mutable (add/remove/change); tuples are immutable.
Tuples are faster, use less memory, and work as dictionary keys or multi-return values.
Verbose version:
That's a great question! Let me explain in detail...
(same core information, expanded to ~300 words with intros, repetition, and summary)
Results:
Avg concise answer score: 7.5/10
Avg verbose answer score: 8.0/10
Verbosity bonus: +0.5 points
(unbiased baseline: 0.0 point difference)
Equal content, and the longer version scores 0.5 points higher. Evaluating two prompts where one produces concise answers and the other verbose ones, the Judge systematically favors verbosity — independent of which version actually helps the user more.
Experiment 3: Framing Bias
Design: Same answer, evaluated with two prompts — a standard Judge prompt vs a strict expert prompt (explicitly stating that ordinary answers don't exceed 6/10, only excellent ones reach 8+).
Results:
Standard prompt avg score: 8.0/10
Strict expert prompt avg score: 8.0/10
Framing delta: +0.00 points
What this means: glm-4-flash doesn't respond to the strictness level specified in the prompt — even with an explicit higher standard, scores stay the same. This is useful information: for this model, you don't need to worry that prompt wording will systematically inflate or deflate scores. But stronger models (GPT-4, Claude) may show more framing sensitivity. Always test before assuming.
Three Mitigation Strategies
Mitigate Position Bias: Multi-Round Randomization
Single evaluations are unreliable with 33% flip rate. Fix: evaluate each pair multiple times with randomized order.
import random
def pairwise_with_randomization(judge, question, answer_a, answer_b, rounds=5):
a_wins = 0
b_wins = 0
for _ in range(rounds):
if random.random() > 0.5:
result = judge(question, answer_a, answer_b)
if result == "A": a_wins += 1
else: b_wins += 1
else:
result = judge(question, answer_b, answer_a)
if result == "A": b_wins += 1 # "A" position holds B
else: a_wins += 1
return {"a_win_rate": a_wins / rounds, "b_win_rate": b_wins / rounds}
5 rounds of randomization averages position bias toward 50%. Cost is 5x API calls; reliability is substantially higher.
Mitigate Verbosity Bias: Per-Dimension Scoring + Anti-Verbose Instructions
Option A: Score dimensions separately, no combined score
ANTI_VERBOSE_JUDGE = """Evaluate this response. Score each dimension 1-5 independently:
1. Accuracy: Is the technical content correct? (length-independent)
2. Conciseness: Is the information density high? (more concise = higher score)
3. Relevance: Does it directly answer the question?
Note: longer answers are not better — evaluate content quality only."""
Option B: Explicit declaration in the prompt
"Note: response length does not affect scoring. A concise accurate answer and a detailed accurate answer should receive equal scores."
Mitigate Self-Bias: Use a Stronger or Different Judge Model
If you generate with one model and evaluate with the same model, self-bias leads to systematic overestimation.
# Generate with cheaper model
generator = ChatOpenAI(model="glm-4-flash", ...)
# Evaluate with stronger or different-vendor model
judge = ChatOpenAI(model="gpt-4o", ...)
# or
judge = ChatAnthropic(model="claude-sonnet-4-6", ...)
Cost increases; evaluation reliability improves. For important release decisions, use Claude to evaluate GPT output (or vice versa) rather than self-evaluation.
Production-Ready Judge Prompt Template
Incorporating all three mitigations:
PRODUCTION_JUDGE_PROMPT = """You are a strict technical content reviewer.
Scoring rules:
- Score range 1-5, NOT 1-10 (reduces variance)
- Length does not affect scores — evaluate content quality only
- Vague or verbose writing is NOT a positive — clarity and conciseness are
- 3 means "meets expectations", not "barely acceptable"
Dimensions (each 1-5):
1. Accuracy: Is the technical content correct? Any factual errors?
2. Relevance: Does it directly answer the question? Any off-topic content?
3. Usefulness: Can a user act on this answer to solve their problem?
Question: {question}
Response: {answer}
Return JSON only:
{{"accuracy": int, "relevance": int, "usefulness": int, "reasoning": "one sentence explaining the lowest-scoring dimension"}}"""
Design choices:
- 1-5 not 1-10: reduces variance; LLMs are inconsistent at fine-grained distinctions (7 vs 8)
- "3 = meets expectations": prevents LLMs from treating 3 as negative, which causes score inflation across the board
- Require reasoning: forces the Judge to explain the lowest-scoring dimension, reducing arbitrary scoring
- Anti-verbosity declaration: explicitly counters verbosity bias
Practical Recommendations
Single evaluations have limited reliability: With 33% flip probability from position bias, single results aren't enough to make decisions. For important comparisons, run at least 3 rounds and report win rates, not single outcomes.
Calibrate before scaling: Before running a new Judge Prompt at scale, test it against 20-30 human-annotated samples. Spearman correlation between Judge scores and human ratings should exceed 0.7 before using the Judge for decision-making.
Monitor score distributions: If the Judge assigns 90% of outputs a score of 7-8, the distribution is too concentrated and the Judge lacks discriminative power. Adjust the prompt or use a stronger model.
Use scenario-specific Judges: A code generation Judge should focus on correctness and runnability; a summarization Judge should focus on faithfulness; a conversational AI Judge should focus on helpfulness. A generic Judge applied to all scenarios loses evaluation precision.
Summary
Three experiments produce three specific action items:
- Position bias (67% first-position win rate): Pairwise evaluation must randomize order across multiple rounds — 33% flip probability makes single-shot results unreliable
- Verbosity bias (+0.5 points): Declare "length doesn't affect scoring" in the prompt, and add a conciseness dimension to counteract
- Framing bias (0.0 delta on glm-4-flash): This model doesn't respond to prompt strictness level — but different models behave differently, always verify before assuming
References
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Zheng et al., 2023)
- Full demo code: eval-03-llm-judge
Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.
Find more useful knowledge and interesting products on my Homepage
Top comments (0)