Evals: How You Know the Agent Works in Production

In January 2026, the reviewer agent's eval went from 92 to 94 after refining the system prompt. I merged with confidence. Three weeks later operators started complaining: more false positives, more hedging, less direct diagnosis. I investigated.

The golden dataset had been frozen in October and stopped being updated. Over three months of use, the new system prompt had essentially memorized the distribution of examples through lexical proximity. The score went up because the model learned the dataset, not because it got better. Eval had become theater with a growing number.

Dataset freshness matters more than the score. Without example rotation, the number you're monitoring no longer exists.

Auto-pilot without evals is delivery without QA. The agent finishes, reports "done," you trust it. A week later you discover half the diagnoses were wrong. Every system prompt change without evals is a dice roll.

Why agent testing is different

Classic software has deterministic tests: same input, same output. Agents don't work that way. Same prompt and temperature produce textually distinct outputs that are (hopefully) semantically equivalent.

Agent evals compare semantics, not bytes. "The reviewer detected the try/catch" is the test. The assert targets behavior. This changes the engineering: traditional assertion frameworks aren't enough. You use three families that complement each other.

Structural assertion

The cheapest layer. Runs in milliseconds, costs nothing. Verifies form: valid JSON, score between 0 and 100, required fields present, correct line format.

function evalReviewerOutput(output: string): AssertionResult {
  if (output.trim() === "LGTM") return { ok: true };

  const issues: string[] = [];
  for (const line of output.split("\n")) {
    if (!line.trim()) continue;
    if (!/^[^:]+:\d+\s+--\s+.{10,}$/.test(line))
      issues.push(`malformed line: ${line.slice(0, 60)}`);
  }

  return issues.length === 0 ? { ok: true } : { ok: false, issues };
}

Catches file path hallucinations, empty descriptions, mangled formats, LGTM followed by prose. First filter: whoever passes might be wrong inside, but at least it's not broken outside.

Judge LLM

The middle layer. A cheap model (Haiku, GPT-5-mini) reads the agent's output and judges it against explicit criteria. Goes deeper into semantics.

async function judgeReviewerOutput(input: ReviewInput, output: string) {
  const judgePrompt = `You are a code reviewer judge.

INPUT received:
${JSON.stringify(input, null, 2)}

OUTPUT produced:
${output}

Evaluate:
1. Do the flagged problems actually exist in the diff?
2. Did the reviewer miss problems it should have caught?
3. Did the reviewer invent problems to seem useful?

Answer JSON: {"score": 0-10, "false_positives": [], "false_negatives": [], "reasoning": "..."}`;

  const verdict = await llm.generate({
    model: "claude-haiku-4-5",
    prompt: judgePrompt,
    max_tokens: 500,
  });

  return JSON.parse(extractJson(verdict.text));
}

Calibrate the judge against a human before trusting it: run judge and human on the same 50 examples, measure agreement. For CI automation, above 85% agreement is the practical threshold before removing the human from the loop.

Without explicit criteria in the prompt, the judge becomes noise. With criteria, it becomes automatable.

Golden dataset

The most expensive to build, the most valuable to maintain. A fixed set of inputs with human-curated expected outputs.

Run the agent on every case, compare against expected. Score = percentage approved. When changing the system prompt, run before and after. Score dropped? Revert. Score rose? Accept, but investigate whether it rose by learning the dataset or by genuinely improving.

Dataset grows with incidents: each production bug becomes a new case the same day. Without this discipline, the dataset stagnates. After a year of operation, 200-500 examples covering real usage distribution runs in 5-15 minutes in CI.

Anti-pattern: spot-check as eval

You change the prompt, run it on three examples, "looks good," merge. A week later you discover it regressed on cases not in those three. Spot-check is what you do when the meeting starts in ten minutes.

Spot-check is feeling with a small, biased sample. You test what you imagine is relevant, not what actually appears in production.

Fix: dataset with at least 30 cases covering real distribution. Runs automatically on PR. Score cannot drop for merge.

Eval is CI/CD for agents. Without it, you merge without tests.