Here is the kind of sentence that should keep a team up at night:
"At this clarity grade, you won't see a thing with the naked eye."
Our assistant produced it while explaining a diamond. The grade it cites is real — SI1, printed on
the stone's lab report. The promise attached to it is invented. A clarity grade is assigned under
10× magnification; it does not resolve what a naked eye will see. Two SI1 stones can look entirely
different — one clean at arm's length, the other with a visible speck right in the middle.
The model took a true fact and manufactured its consequence, because that is what a helpful
sentence sounds like.
Notice what the sentence is not. It is not a banned word. It is not a hallucinated stone. It
contains no wrong number. It breaks no rule you could have written down in advance. It is simply
fluent — and fluency is the problem. Fluent and true are different variables, and a language model
optimizes the first one.
We build AI assistants for exactly this kind of product. The one in this essay explains diamonds:
a short, two-voice spoken conversation about one stone — a host asks the questions a shopper
would ask, a gemologist answers — generated from that stone's evidence. A diamond is the
definition of a high-consideration purchase: thousands of dollars, bought rarely, chosen under
uncertainty. And it has a property that makes it an unforgiving test bed for generative AI —
nearly every claim about a diamond can be checked against its certificate from GIA or IGI, the
independent laboratories that grade the world's diamonds. Overselling is not just tacky here. It
is checkable.
In an earlier essay we argued that the language model should conduct the conversation while a
deterministic core makes every decision. This essay is the same worldview applied to quality. The
model writes. It does not get to grade.
At one generation you can catch a sentence like the one above by reading it. Across a catalogue of
stones you cannot. So before we made the writer better, we built an instrument: an independent LLM
judge that reads every generated claim against the evidence and returns a verdict we can count.
This essay is about how that judge is built, how we keep the judge itself honest, and what changed
once we had it — which is that the arguments stopped.
Part I — Three gates, and who owns which rule
The assistant explains one diamond at a time, from that diamond's evidence: the lab certificate,
the price, the retailer's published analysis of the stone. One rule governs everything: every
claim about this stone must trace to supplied evidence. If the evidence cannot answer — is it
eye-clean, meaning no marks visible to the naked eye? — the assistant says so, out loud: the
honest answer is a maybe, and it will not promise more. Then it points the shopper to a human gemologist.
Enforcing that rule is not one mechanism. It is three, layered, and they differ in kind.
The first gate prevents. It runs in code, inside the pipeline. When the assistant generates a
follow-up question about a stone, the question must name the evidence path it would be answered
from. A question that names no available path is rejected — by code, not by prompt hope. This is
what stops the system from asking "how does this price compare to the market?" about a stone
whose data has no comparable set. The model never gets the chance to improvise an answer, because
the question never survives.
The second gate catches. A deterministic validator runs over every finished script: a
hard-fail lexicon of the words the assistant must never say — amazing, must-have,
investment, the manufactured urgency, the "trust me." Word-boundary regexes, a few dozen tests,
no judgment involved. If a banned phrase appears, the script fails. Cheap, fast, and absolute.
The third gate judges. What remains after prevention and regex is the dangerous class: the
plausible-but-unsupported claim. "You won't see a thing with the naked eye" passes both gates. No
rule names it in advance, because it is not a phrase — it is a relationship between a sentence and
a lab report. Checking it requires reading both. That is a job for a reader, and at scale the
reader has to be a model.
One detail here taught us more than we expected: the boundary between the gates is itself a
design decision. When we extended the banned-word list from the domain experts' style guide, the
unambiguous terms went into the regex gate. But the context-dependent ones deliberately did not —
and the best example is flawless. In most products, "flawless" is empty praise. In diamonds it
is literally a grade: FL, the top of the GIA clarity scale. "It is not flawless" is a factual
statement about an SI1 stone, not a compliment withheld. Likewise "guaranteed": a real test
fixture proved that "that would need a guarantee the report doesn't give" is the assistant at its
most honest. A blunt regex would fail exactly those sentences. Those words went to the judge
instead, which can read the sense and not just the string. Which gate owns which rule is not a
default. It is a decision, and we made it with test evidence.
Part II — Anatomy of a verdict
"Use an LLM judge" usually means: ask a model to rate the output from one to ten. A score like
that is an opinion with a number attached. You cannot appeal it, you cannot audit it, and you
cannot tell whether a 7 means two small problems or one large one. It is not an instrument; it is
a mood.
Our judge is built differently, and every difference exists because a naive version failed first.
It returns a triage, not a score. For every sentence that asserts something beyond restating
the input data, the judge answers typed questions: Is this a claim about this stone, or about
diamonds in general? If it is about this stone — does the evidence establish it? If it is general
— is it accurate gemology? "Lab-grown and natural diamonds are chemically identical" is category
teaching: permitted, and wanted — the judge's instructions say teaching is never a violation.
"This stone will look white to the eye" is an item claim, and it had better trace to something. A real
violation is only the intersection: about this stone, and unsupported.
A claim also has an owner. Supported is not the only axis. GIA graded the stone; the retailer
analyzed it; the assistant speaks for the house. If it says "we grade this VS1," every word is
factually true and the sentence is still a violation — it credits the lab's work to the house.
The judge flags misattribution as its own class, separate from unsupported claims, because
provenance errors are how a trustworthy voice quietly claims authority it does not have.
It must report the claims it cleared. The judge is required to list the general statements and
the supported claims too, with the classification visible — and it is forbidden from letting them
inflate the violation count. This sounds like bureaucracy. It is the whole point. A verdict that
only lists problems is a number you have to trust. A verdict that shows every ruling — each quote
verbatim, each one naming the evidence path it was checked against — is an audit trail you can
re-check. When the judge is wrong, you can see exactly where.
Orthogonal signals stay orthogonal. The judge also measures tone: does the assistant sound
like a trusted expert across the counter or a closer working a commission? Does the conversation
sound like two people responding to a stone, or a script? But tone is a separate, subordinate
channel, and the judge's instructions are blunt: warmth never excuses an unsupported claim. A
formatting check rides in a third channel and changes nothing in the honesty count. There is no
blended score, deliberately. A blended score is where warmth hides dishonesty.
The writer and the judge read the same constitution. Every rule about what a lab report does
and does not support lives in one content-standard document. That document is spliced into the
writer's prompt and into the judge's prompt. The spec is a single artifact: the writer implements
it, the judge enforces it, and editing one sentence changes both sides of the contract at once.
There is no version of the rules that only the judge believes.
It never grades its own output. The writers run on Claude Sonnet 5. The judge's seat has been
held by two models — first GPT-5.5, a different lab entirely, and today Claude Opus 4.8, a
different model from the writers — and we verified the discipline held across the change. A model
asked to grade its own writing has a conflict of interest baked into its weights. Independence
has to be constructed.
It never touches production. The judge is a development instrument, not a runtime guardrail.
It grades output during the dev cycle to find where the writer's prompts leak; it adds zero
latency and zero cost to serving. Grading one diamond costs about thirty cents — four passes,
each re-reading the full evidence — which is trivially cheap in development and pointlessly
expensive in production, where the deterministic gates already stand. A judge is not a content
filter. It is how you find out what to fix.
Part III — Calibrate the instrument
The uncomfortable part: the judge is also a language model. It is an unreliable narrator hired to
audit an unreliable narrator. This is less absurd than it sounds — auditors are people too — but
only if the workflow plans for it. Ours does, in three ways.
The judge can be wrong, and the loop says so. Our written dev cycle is: the judge finds a
violation → we fix the writer's prompt, or we fix the judge if the judge is wrong → we log it →
we re-run. The same humility applies to the deterministic gates. We ban the word flaw — the
marks inside a stone are characteristics, never flaws. Then the assistant, explaining a modest luster
rating, said it was "not a flaw; just part of why the rating sits at four instead of five or
six" — and the regex failed the most honest sentence in the script. So the ban was narrowed:
the corrective "not a flaw" now passes, while "no flaws in this stone" — a perfection claim —
still fails. The exemption got its own tests. False positives are not noise; they are
calibration data.
The verdict is schema-checked, and malformed verdicts retry. In real runs the judge would
intermittently drop a required field, or return a field with the wrong type. We do not average
around that; the harness detects the malformed verdict and retries it, and every consumer of the
verdict is guarded against a bad shape. An instrument with unhandled failure modes doesn't stop
measuring — it starts lying quietly.
One sample is noise. The same stone, judged twice, does not score the same. We watched one
diamond score fifteen violations on one run and six to eight on reruns. If we had compared two
models using single samples, we could have concluded anything at all. The discipline is boring:
average at least two runs before believing any comparison, and never let a single verdict decide
anything important.
An instrument you never calibrate is just another opinion.
Part IV — Three arguments we never had to have
Here is what actually changed when the judge existed. Three decisions that AI teams normally
settle by taste, seniority, or vendor reputation got settled by reading the same number before and
after.
Which model. We ran two writer tiers over the same stones, with the same prompts and the same
judge. Claude Haiku 4.5 — the small, fast tier — averaged about ten real violations per diamond.
Claude Sonnet 5 averaged 0.8. A twelve-fold honesty gap, on identical inputs. And the direction of
the tradeoff surprised us: Sonnet also scored higher on specificity, so the honesty did not cost
richness. Every arithmetic blunder in the corpus came from the small model — including the run
where it declared one stone "59 carats more" than another. The real gap between those stones was
0.13 carats — and fifty-nine dollars. The model had turned a price difference into a weight.
Fifty-nine carats is not an engagement ring; it is a museum piece. A model that cannot keep
dollars and carats apart is not a bargain at any latency.
The judge also caught something no benchmark would have: the fast tier existed only to shave
seconds off time-to-first-word, yet it had quietly generated every record in our persisted
"quality" collection. The latency shortcut had poisoned the quality artifact. No unit test flagged
it, because every unit test passed. The judge flagged it, because the judge reads output.
The ending is the good part. The fast model's one advantage evaporated when we fixed a parser to
accept the larger model's output format — and measured end to end, the strong model at low
reasoning effort was fast enough to serve. Reputation said "the small model is fine for this." The
instrument said otherwise, and the instrument had receipts.
Whether RAG. Everyone's default architecture for grounding is retrieval. We shipped something
smaller: a deterministic key-join. The gemological knowledge — vetted definitions, approved
analogies, the boundaries of what a report can and cannot tell you — is keyed by characteristic,
and each stone's generation receives only the blocks matching what its report actually lists: a
cloud, a feather, a needle — the trade's names for the tiny marks inside a stone. An exact
lookup, not a search. So when the assistant explains a
cloud, it uses the vetted definition — a cluster of microscopic pinpoints — and keeps the vetted
boundary: the report cannot tell you whether it causes visible haze in this stone. The
knowledge base is content-hashed into the cache key, so editing one entry regenerates exactly the
output that depended on it.
Did we lose anything by skipping the vector database? That is normally a religious question. For
us it was an empirical one: the judge held the grounded output at zero real violations per stone.
Retrieval remained unnecessary for a bounded, one-stone-at-a-time task — not as an article of
faith, but as a measured result we can revisit the day the task stops being bounded.
There was a quiet confirmation along the way. A gemological standard written independently by the
domain's own experts — people who had never read our prompts — was later diffed against our
system. It landed on the same rules: the same evidence discipline, the same refusal to infer
eye-cleanness from a clarity grade, the same banned reassurances. Two instruments, built apart,
giving the same reading. That is what being right feels like from the inside: unremarkable.
Did the edit help. The prompts are the product, so we treat them like one. Every prompt edit
carries a changelog entry with four fields: the case (the concrete finding that forced it —
almost always a judge finding), the reason (the principle it serves), the change (what
actually moved), and the verification (a judged real run, with numbers). A typical case: the
judge caught the assistant volunteering why a stone had earned its value rating — a cause the
evidence never states — so the rule against manufactured explanations was widened, and the
verification showed the pattern gone across redraws with the honesty counts unchanged. About
sixty entries accumulated in the first week. No silent edits, ever. And because the prompt version
is folded into the cache key, no cached episode can ever have come from an unknown prompt: edit
the prompt, and everything it touched regenerates by design.
This is test-driven development for a non-deterministic system. The judge's verdict is the failing
test you drive to green. And the tests that matter run against the real API, on real output —
because in this project, every finding above came from a real run, and the mocked tests passed the
whole time. Over four hundred automated tests guard what can be asserted deterministically; not
one of them counts as evidence that the writing got better. Only the instrument says that.
Part V — What generalizes
Diamonds are the setting, not the point. This pattern transfers to any team shipping generative
output where being wrong is expensive. The short version:
- Build the instrument before polishing the product. You cannot improve what you argue about. A judge you trust converts taste into experiments.
- Layer the gates, and decide which gate owns which rule. Prevent in code what can be prevented. Hard-fail on regex what is unambiguous. Send to the judge only what requires reading. The boundary is a design decision — make it with test evidence.
- Demand a triage, not a score. Typed questions, verbatim quotes, named evidence, cleared claims reported alongside violations. If you cannot re-check a verdict, you do not have an instrument; you have a mood.
- Keep channels orthogonal. Honesty, provenance, tone, formatting — measured separately, never blended. A single number is where problems hide behind virtues.
- Construct independence. A different model, ideally a different lab, reading from the same written standard as the writer. Never let a model grade its own homework.
- Calibrate, and distrust single samples. Fix the judge when the judge is wrong, guard against malformed verdicts, average runs. Variance is real and it will lie to you.
- Keep the judge out of production. It is a development instrument. The gates that serve customers should be deterministic, cheap, and boring.
The payoff is cultural as much as technical. Model selection stopped being a debate about vendor
reputation. Architecture stopped being a debate about fashion. Prompt engineering stopped being a
vibe. We stopped arguing about which model was smarter and started reading the same number twice.
One honest caveat, because our instrument would flag us if we skipped it: the judge measures
whether the assistant is honest about a diamond. It cannot measure whether a shopper wants to
listen. That is a different experiment, with a different instrument — and no model gets to grade
it for us.
Originally published at xinfer.ai — where there is also a 2-minute audio spotlight and a full narration of this essay.
I'm Sam Wen, founder of XInfer.AI. We build AI agents that transact — chat, voice, phone, and SMS — with one deterministic core underneath all of it.
Top comments (0)