Originally published on vibki.com.
Have you ever changed a prompt, liked a few answers, and shipped it without knowing what else broke? LLM evals fix that: an eval is an automated test for your prompt, with fixed inputs, a success criterion, and a score. By the end you will build your first one and know if a change helped.
One tiny prompt tweak, what could possibly break?
Meet Brewly, a coffee shop with an LLM (Large Language Model) support bot made of two prompts. The classify prompt, the router, returns JSON like {"intent": ..., "needs_human": ...}. The persona prompt is Mia, a warm barista: at most 3 sentences, always the exact menu path such as "Settings > Billing", never a promised refund.
The team asks for "shorter and more playful", so you edit both prompts. Harmless, right?
# classify_v1.md (last two lines)
{"intent": "cancel_subscription | refund | shipping_status | other", "needs_human": true or false}
Set needs_human to true if the customer is angry or mentions a lawyer.
# classify_v2.md (last two lines)
Read the customer message and reply with JSON: a short snake_case "intent" and a boolean "needs_human".
Keep it brief.
# persona_v2.md (whole file)
You are Mia, Brewly's support barista. Be playful and keep replies short.
Three test chats look fine, and that is the mirror: it shows today's face, not your weight. In production the router answers cancel_plan, a label no code handles, and Mia drops the menu path.
So what are LLM evals, actually?
What would have caught it? LLM evals give you three parts: cases (fixed inputs, here five messages in cases.json), a success criterion (what counts as right) and a score (a number to compare across edits).
{
"id": "cancel",
"message": "How do I stop my coffee box?",
"intent": "cancel_subscription",
"needs_human": false,
"golden": "Sure! You can cancel any time in Settings > Billing > Cancel plan, and your next box still ships until the period ends. Anything else I can help with?"
}
Anthropic's docs say a good criterion is specific, measurable, achievable and relevant. "Brewly replies should be good" is none of those, while "the router returns one of four labels, the right one, and Mia gives the menu path for every account action" is all four. The router's half, intent.schema.json:
{
"type": "object",
"required": ["intent", "needs_human"],
"additionalProperties": false,
"properties": {
"intent": {"enum": ["cancel_subscription", "refund", "shipping_status", "other"]},
"needs_human": {"type": "boolean"}
}
}
Mia gets a golden answer, a trusted reference reply, and a set of them is a golden dataset (per the Langfuse guide: inputs with reviewed reference outputs). That covers one prompt, and Brewly ships two.
Is one check really enough for a production bot?
No, because each kind of eval answers a different question, like gauges on one scale:
- Code-based: code checks a measurable fact.
-
Classification: exact match on the label, then accuracy and recall on
needs_human(angry customers caught). - Prompt review: lint the prompt text, no model call.
- Golden answer: ROUGE-L F1 (word overlap, 0 to 1) similarity to the reference.
- LLM-as-judge: a second model grades against a rubric (a written rule).
Pairwise asks which reply wins, a human just reads them (as in Human-in-the-Loop), regression reruns old cases, and consistency asks the same question twice. Anthropic's docs say to pick the fastest, most reliable, most scalable grader, so you want the cheapest one that sees the failure.
# evals/test_support_bot.py, part 1: the router tests
import json, os, re, pathlib
HERE = pathlib.Path(__file__).parent
V = os.environ.get("BOT_VERSION", "v1") # v1 or v2
CASES = json.loads((HERE / "cases.json").read_text())
SCHEMA = json.loads((HERE / "intent.schema.json").read_text())
LOGGED = json.loads((HERE / "logged" / f"{V}.json").read_text()) # logged model outputs, one per case
PROMPT = (HERE.parent / "prompts" / f"classify_{V}.md").read_text()
def schema_errors(obj, schema): # the few JSON Schema rules we use
errs = [f"missing key {k!r}" for k in schema["required"] if k not in obj]
errs += [f"extra key {k!r}" for k in obj if k not in schema["properties"]]
for k, rule in schema["properties"].items():
if k in obj and "enum" in rule and obj[k] not in rule["enum"]:
errs.append(f"{obj[k]!r} is not one of {rule['enum']}")
if k in obj and rule.get("type") == "boolean" and not isinstance(obj[k], bool):
errs.append(f"{k!r} is not a boolean")
return errs
def test_output_schema(): # 1. code-based: is the JSON the shape we parse?
bad = []
for c in CASES:
try:
bad += [f"{c['id']}: {e}" for e in schema_errors(json.loads(LOGGED[c["id"]]["raw"]), SCHEMA)]
except json.JSONDecodeError as e:
bad.append(f"{c['id']}: not JSON ({e})")
assert not bad, bad
def test_intent_accuracy(): # 2. classification: exact match per label
got = [json.loads(LOGGED[c["id"]]["raw"]) for c in CASES]
hits = [g.get("intent") == c["intent"] for g, c in zip(got, CASES)]
assert sum(hits) / len(hits) >= 0.8, f"accuracy {sum(hits)}/{len(hits)}"
angry = [g for g, c in zip(got, CASES) if c["needs_human"]]
assert all(g["needs_human"] for g in angry), "angry customer not sent to a human"
def test_classify_prompt_review(): # 3. prompt review: lint the prompt, no model call
for label in SCHEMA["properties"]["intent"]["enum"]:
assert label in PROMPT, f"prompt never mentions label {label}"
assert "needs_human" in PROMPT
# part 2: Mia's tests and a tiny runner
def test_persona_rules(): # 4. code-based: the persona's hard rules
for c in CASES:
reply = LOGGED[c["id"]]["reply"]
assert len(re.findall(r"[.!?]", reply)) <= 3, f"{c['id']}: more than 3 sentences"
if "golden" in c:
assert "Settings > Billing" in reply, f"{c['id']}: no menu path"
def lcs(a, b): # longest common subsequence length
prev = [0] * (len(b) + 1)
for x in a:
cur = [0]
for j, y in enumerate(b):
cur.append(prev[j] + 1 if x == y else max(prev[j + 1], cur[j]))
prev = cur
return prev[-1]
def test_persona_golden(): # 5. golden answer: ROUGE-L F1 against a reference
scores = {}
for c in CASES:
if "golden" in c:
got, want = LOGGED[c["id"]]["reply"].split(), c["golden"].split()
p, r = lcs(got, want) / len(got), lcs(got, want) / len(want)
scores[c["id"]] = round(2 * p * r / (p + r), 2)
assert all(f1 >= 0.7 for f1 in scores.values()), scores
return f"F1 {scores}"
RUBRIC = ("You grade support replies written by Mia, a warm barista. Answer `correct` only if "
"the reply is friendly, in plain words, and tells the customer what to do next. "
"Otherwise answer `incorrect`. Reply with one word.")
def test_persona_judge(): # 6. LLM-as-judge: rubric, one-word verdict
raise Skip("needs a judge model") # real version: send RUBRIC + reply to a model
class Skip(Exception):
pass
if __name__ == "__main__": # tiny runner, no framework
print(f"--- bot {V}")
for name, fn in [(n, f) for n, f in list(globals().items()) if n.startswith("test_")]:
try:
print(f"PASS {name}", fn() or "")
except Skip as s:
print(f"SKIP {name}: {s}")
except AssertionError as e:
print(f"FAIL {name}: {e}")
# real output: BOT_VERSION=v1 python evals/test_support_bot.py, then v2
--- bot v1
PASS test_output_schema
PASS test_intent_accuracy
PASS test_classify_prompt_review
PASS test_persona_rules
PASS test_persona_golden F1 {'cancel': 0.91, 'double': 0.79}
SKIP test_persona_judge: needs a judge model
--- bot v2
FAIL test_output_schema: ["cancel: 'cancel_plan' is not one of ['cancel_subscription', 'refund', 'shipping_status', 'other']", "double: 'billing_issue' is not one of ['cancel_subscription', 'refund', 'shipping_status', 'other']"]
FAIL test_intent_accuracy: accuracy 3/5
FAIL test_classify_prompt_review: prompt never mentions label cancel_subscription
FAIL test_persona_rules: cancel: no menu path
PASS test_persona_golden F1 {'cancel': 0.77, 'double': 0.83}
SKIP test_persona_judge: needs a judge model
On v2 the router fails three ways: cancel_plan and billing_issue break the schema, accuracy is 3/5, and the prompt never mentions cancel_subscription. Mia fails with cancel: no menu path, and guess which test still passes? The golden answer one. That is 96 lines of plumbing.
💡 Note: Model answers are logged outputs, a saved file standing in for live calls.
Do I really have to hand-roll all of this?
Not really, so let's run the same LLM evals on Brewly in frameworks. DeepEval is pytest-style: a Golden is an input plus its expected output, ExactMatchMetric and PatternMatchMetric need no LLM, and GEval is the LLM-judged golden metric, so it needs a judge key:
import json
import pytest
from deepeval import assert_test
from deepeval.dataset import EvaluationDataset, Golden
from deepeval.metrics import ExactMatchMetric, PatternMatchMetric, GEval
from deepeval.test_case import LLMTestCase, LLMTestCaseParams
from conftest import CASES
# The golden dataset: input + the answer we agreed is right.
dataset = EvaluationDataset(goldens=[
Golden(input=c["message"], expected_output=c["intent"], name=c["id"]) for c in CASES])
@pytest.mark.parametrize("golden", dataset.goldens, ids=lambda g: g.name)
def test_intent_exact_match(golden, logged): # no-LLM metric: score is 0 or 1
got = json.loads(logged[golden.name]["raw"])["intent"]
tc = LLMTestCase(input=golden.input, actual_output=got, expected_output=golden.expected_output)
assert_test(tc, [ExactMatchMetric(threshold=1.0)])
def test_reply_has_menu_path(logged): # no-LLM metric: regex
tc = LLMTestCase(input="How do I stop my coffee box?", actual_output=logged["cancel"]["reply"])
assert_test(tc, [PatternMatchMetric(pattern=r".*Settings > Billing.*")])
def test_reply_matches_golden(logged): # LLM-judged against the golden reply (needs a judge key)
g = next(c for c in CASES if c["id"] == "cancel")
match = GEval(name="Matches golden", threshold=0.7,
criteria="The actual output gives the same instructions as the expected output, in Mia's warm tone.",
evaluation_params=[LLMTestCaseParams.ACTUAL_OUTPUT, LLMTestCaseParams.EXPECTED_OUTPUT])
assert_test(LLMTestCase(input=g["message"], actual_output=logged["cancel"]["reply"],
expected_output=g["golden"]), [match])
# real run on the v2 logged outputs (v1: 6 passed)
FAILED evals/test_deepeval.py::test_intent_exact_match[cancel] - AssertionError: Metrics: Exact Match (score: 0.0, threshold: 1.0, strict: False, error: None, reason: The actual and expected outputs are different.) failed.
FAILED evals/test_deepeval.py::test_intent_exact_match[double] - AssertionError: Metrics: Exact Match (score: 0.0, threshold: 1.0, strict: False, error: None, reason: The actual and expected outputs are different.) failed.
FAILED evals/test_deepeval.py::test_reply_has_menu_path - AssertionError: Metrics: Pattern Match (score: 0.0, threshold: 1.0, strict: False, error: None, reason: The actual output does not match the pattern.) failed.
3 failed, 3 passed, 1 deselected, 3 warnings in 0.38s
promptfoo does it in YAML, grading through its echo provider with no API call:
description: Brewly intent router
prompts: ["{{raw}}"]
providers: [echo] # echo = grade logged outputs, no API call
defaultTest:
assert:
- type: is-json # schema check, no LLM
value:
type: object
required: [intent, needs_human]
properties:
intent: {enum: [cancel_subscription, refund, shipping_status, other]}
needs_human: {type: boolean}
- type: javascript # classification: label equals the expected label
value: "JSON.parse(output).intent === context.vars.expected"
tests:
- description: cancel on v1
vars: {raw: '{"intent": "cancel_subscription", "needs_human": false}', expected: cancel_subscription}
- description: cancel on v2
vars: {raw: '{"intent": "cancel_plan", "needs_human": false}', expected: cancel_subscription}
# real run: npx promptfoo@0.120.0 eval -c promptfooconfig.yaml, plus the v2 reasons
Successes: 1
Failures: 1
Errors: 0
Pass Rate: 50.00%
is-json: JSON does not conform to the provided schema. Errors: data/intent must be equal to one of the allowed values
javascript: Custom function returned false
Inspect AI builds a task from a dataset, a solver that answers and a scorer that grades, and Ragas covers LLM apps. Frameworks give you history and CI (continuous integration) gating, but green is only as honest as your cases.
When does a green score lie to you?
Not always, because green in LLM evals only means the checks you wrote found nothing. On v2 the golden-answer test passed at F1 0.77 and 0.83 (threshold 0.7) and rated v2's double reply above v1's (0.83 against 0.79) while the menu path was gone, so similarity misses the one fact that matters. Every kind is blind somewhere: a schema accepts a valid wrong label, exact match fails a correct paraphrase, a judge needs checking against a human. A scale only measures what you put on it.
Repeat each run, because temperature varies answers, and never tune until only your own cases pass.
💡 Attention! The most important thing: one new case per production failure, from the cheapest grader that sees it.
Conclusion
So would an eval have caught the "shorter and more playful" edit on the Brewly bot? Yes: accuracy 3/5, 'cancel_plan' is not one of [...] and cancel: no menu path all show before it ships, the way a scale beats the mirror. Cases, a criterion and a score, graded per prompt: that is what LLM evals are. Tomorrow, write 5 failure cases for your JSON prompt and your user-facing prompt, add a schema, label, rule and golden check, and run them before your next edit. Next time: prompt caching, and why a cache read costs a fraction of normal input.
Sources
- Anthropic: Define success criteria and build evaluations
- Framework docs: DeepEval: Exact Match, DeepEval: getting started, promptfoo: Assertions & metrics and promptfoo: Echo provider
- Inspect AI docs: Hello, Inspect and Ragas docs: Introduction
- Langfuse: Golden dataset evaluation
Written with AI assistance and reviewed by the author.
Top comments (0)