DEV Community: Sam Wen

The LLM Narrates, the Core Decides: Anatomy of an AI Checkout

Sam Wen — Mon, 27 Jul 2026 01:43:31 +0000

How we built a checkout that runs the same protocol in a chat window, a voice session, and a live phone call.

One word, many moves

On the AI storefront platform I work on, a customer can type "checkout" into a chat window — or say it to a voice agent or a phone line — and, a short conversation later, they have an order number. The transcript looks unremarkable: a question about special instructions, maybe a contact form, a quick "Is everything correct?", done.

Beneath that brief exchange lies the most carefully engineered path in the system. Checkout is where a language model, several tools, an order engine, and a human who may be walking down the street all have to agree on what just happened — and where any disagreement costs real money.

This post explains our approach: what it is, why the constraints force this particular shape, and how the pieces move together in text and in voice. Then, because a design only means something relative to its alternatives, I'll compare it with the two other ways this is usually built.

Part I: The approach

The whole design reduces to one sentence: the language model conducts the conversation, but a deterministic core makes every decision.

Concretely, checking out involves two tools the model can call:

shopping_cart — cart operations, with submit as the consequential one.
collect_user_info — a small sub-protocol for contact details, with four actions: get (read the current state, rendering nothing), show (open a form, possibly prefilled), update (fill in one field), and confirm (save and close).

When the model calls shopping_cart(action="submit"), the request lands in a single server-side function shared by every channel. That core resolves who is buying, decides whether anything still needs to be asked, and answers with a typed outcome rather than prose. Five of them cover every case:

NEED_CUSTOMER_INFO — no usable contact information; open the form.
CONFIRM_CUSTOMER_INFO — contact is on file from before; read it back and get a yes.
CART_EMPTY — there's nothing to check out.
SUBMITTED — the order exists; here is its number.
A failure — carrying a reason code the model can relay. The most common is NEED_ORDER_DETAILS: the order was submitted before the order questions were asked, so the order engine refuses and the model goes back and asks.

The customer data that drives a checkout decision never lives in the model's prompt — the contact used to place the order rides in the tool results, and the core resolves it server-side by provenance, not from anything the prompt says. The prompt otherwise holds only the protocol for reacting to each signal. And every resubmission carries accumulated flags: orderDetailsConfirmed=true once the order questions were asked, contactConfirmed=true once the customer approved their on-file details. The flags are how a multi-turn negotiation avoids going in circles — anything already settled travels forward, so no step is ever repeated, and no step can be skipped, because the core simply won't proceed without the flag.

That "never in the prompt" needs one qualifier. On text and SMS the prompt does carry a session-context block with the customer's profile, cart, and location for ambient awareness; on voice, where the supplier configs are frozen, even that is absent. Either way the core ignores it: presence of data in the prompt is never permission to spend it.

That's the approach. The model narrates; the core decides. Everything else in this post is the reasoning behind that split and the machinery that makes it work.

Part II: Why this shape

Four constraints pushed us here, and I think at least three of them apply to anyone building an agent that transacts.

1. Channels multiply everything you fork. The same store sells through web chat, an embeddable widget, a mobile app, in-app voice, real phone calls, and SMS. Early on, each channel had its own glue between the model and the business logic — and the glue drifted. An audit found the kinds of bugs drift produces: an order lookup that skipped its ownership check on one channel but not another, phone-only customers who could buy in text but were refused by voice, and one channel that reported failures as if they had succeeded. None of these were design mistakes; they were synchronization mistakes. The fix wasn't more review discipline — it was making sure a decision exists in exactly one place, so there is nothing to keep in sync.

2. The voice prompt is frozen. Our voice agents run on third-party conversational AI suppliers, and their system prompts are compiled into configurations that persist on the suppliers' systems. You cannot inject "this customer's name is Pat" into the prompt at call time. This kills the most common trick in LLM app design — personalizing the prompt — and forces a cleaner one: the prompt stays static and describes only the protocol; anything per customer rides in the tool results. A prompt written weeks ago handles today's caller because today's caller's details arrive as data, not as instructions.

3. A phone call has no screen. In text chat, "collect the customer's info" can mean rendering a form. On a phone call, there is no form, no button, nothing to tap. Every step of the protocol, therefore, needs a purely conversational equivalent — which turns out to be a forcing function for good design: if a step can't survive being spoken aloud, it was probably leaning on UI to paper over ambiguity.

4. The model cannot be trusted with money. Not as an insult — as an engineering fact. A language model will occasionally fill in a customer's email address with something it saw earlier in the conversation, announce an order as placed before anything has happened, or retry a timed-out submission, creating the order twice. These aren't edge cases you prompt away; they're the model doing what models do. The decisions that cost money — whose contact to use, whether to place the order, whether this submission already happened — have to live where they can be made deterministically and tested.

Part III: How it works

The choreography in text

Here is a first-time guest's checkout, annotated with what actually happens:

Customer: checkout

Model:  "Any special notes for your order?"          ← protocol: always ask first
Customer: no, that's it

Model → shopping_cart(submit, orderDetailsConfirmed=true)
Core:   resolve contact → nothing anywhere → NEED_CUSTOMER_INFO

Model → collect_user_info(show)                       ← a contact form renders in chat
Customer: fills in name, email, phone; taps Confirm   ← saved to their profile

Model → shopping_cart(submit, orderDetailsConfirmed=true)
Core:   contact was entered in THIS conversation → proceed
        → order engine → SUBMITTED, order #1042

Model:  "Order #1042 placed — 2 items, $34.50.
         A confirmation email is on its way."

Two things to notice. The model never asked for the customer's email in free-form chat — a structured surface was available, so the form collected it, validated it, and normalized it. (On a phone call no such surface exists and asking out loud is all there is — we'll come back to what stays structured even then.) And when the second submit went through, it still carried orderDetailsConfirmed=true: settled things stay settled.

Now the same customer, back next week:

Model → shopping_cart(submit, orderDetailsConfirmed=true)
Core:   contact found ON FILE from a previous visit
        → not yet a trusted repeat buyer
        → CONFIRM_CUSTOMER_INFO {name, email, phone}

Model:  "I have you down as Pat Lee, pat@example.com,
         (555) 010-3985 — should I change anything?"
Customer: Actually, use my work email

Model → collect_user_info(show)                       ← form opens, prefilled
Customer: edits the email, confirms

Model → shopping_cart(submit, orderDetailsConfirmed=true,
                       contactConfirmed=true)
Core:   → SUBMITTED

The on-file values traveled inside the tool result — the model read them back rather than remembering them. That detail seems small in text. It's the entire ballgame in voice.

Voice: same sheet, no hands

Nothing about the protocol changes for voice — that is the point. The signals are identical; the difference is how each side performs its role.

When the core answers CONFIRM_CUSTOMER_INFO, the result carries the name, email, and phone number for the agent to speak. When the customer wants a change, there's no keyboard, so the form fills by voice: the agent calls collect_user_info(update) with each field as the customer says it, then confirm to save and close. On our web and app surfaces, the customer actually watches the on-screen form populate as they talk — same form the text flow uses, different hands on it.

That gives us three regimes of one protocol:

Text — the customer types into the form and taps Confirm.
Voice with a screen — the agent fills the form by update calls while the customer watches.
A phone call — no screen exists; show renders nothing, and the whole exchange is spoken read-back and correction.

The third regime deserves a caveat, because it's where "the model never asks in free text" stops being literally true. On a phone call, collection genuinely is free-form — the agent asks for the email out loud, and the customer says it. What stays structured is the capture: each spoken answer is immediately pushed through collect_user_info(update), one field at a time, then read back for correction. Email gets the closest scrutiny — it's repaired ("john at example dot com" becomes john@example.com) and validated the moment it's captured, because a mangled address is unrecoverable later; name and phone are captured as spoken and normalized when the order is written. The channel changes how the question is asked; it never changes where the answer lives. The transcript is not the database.

One choreography sheet, three stagings. And because the voice prompt is static, the supplier-side agent configuration never needs to know who is calling — the caller's identity resolves server-side and rides the tool plumbing.

Inside the core

The decisions the core keeps for itself are where most of the design lives.

Trust is gated by provenance, not presence. When the core resolves contact information, it doesn't just ask Do I have it? — it asks Where did it come from? An explicit argument means the model was told this turn. A conversation session means the customer entered it minutes ago, in this very chat — don't insult them by re-asking. A record on file means it's from a previous visit — read it back and confirm before spending it. Same three fields, three different levels of consent. The presence of data is not permission to use it.

Earned trust has a number. Re-confirming contact on every order would punish regulars with friction. So there's an explicit bar, and it takes three things at once: the customer's three most recent orders all carry the same contact details, that triple matches the contact currently on file, and all three fields — name, email, and phone — are present. Everything is compared after normalization, because PAT@x.com and (555) 010-3985 have to match pat@x.com and 5550103985. Miss any of it — one of those three orders differing, a phone that was never collected — and we ask. The threshold isn't clever; the point is that it's explicit, and a unit test can hold it in place.

Evidence gets consumed. One subtlety earned its scars: that "entered this conversation" session is an evidence token — it vouches that the customer just told us who they are. We shipped a version where the token outlived the order it vouched for, and a second order in the same conversation sailed through with no contact interaction at all, riding the first order's evidence. The fix: a successful order spends the token (a failed one keeps it, so retries don't re-ask). Note that spending it never sends the customer back to a blank form — by then their details are on file, so the next order in the same chat gets the one-line read-back, not re-entry. Ten lines and two tests — and it was findable in one debugging session precisely because there was exactly one place to look.

An order is claimed, not just created. Submission begins with an atomic claim on the cart, so a timed-out request that gets retried replays the same order instead of creating a twin. The model can be as retry-happy as it likes; the core is idempotent.

Part IV: Two alternatives, for comparison

Alternative 1: the workflow wizard

The traditional answer: a fixed state machine — cart → details → contact → confirm → pay — with the LLM reduced to narrating each step. It's deterministic and safe, and it's how most checkout flows were built for twenty years.

It dies on contact with conversation. A customer who asks "wait, does that come in a larger size?" mid-checkout is off-script; the wizard has no state for it. "Use my work email instead" is an edit the flow didn't anticipate. And the wizard is coupled to its medium — a form-based state machine has no natural phone-call rendering, so each channel grows its own variant, and you're back to N copies drifting.

Alternative 2: the free-form agent

The fashionable answer: give the model the tools and the conversation, and let it decide. Prompt it with guidelines — "always confirm contact details before ordering" — and trust it.

The conversation is genuinely wonderful. The decisions are not. The model fills in an email it saw two hundred messages ago. It announces "your order is placed!" one tool-call too early. It confirms contact details on Tuesday and forgets to on Thursday, because guidelines in a prompt are sampled, not executed. There's no idempotency, because retries are a concept the model doesn't reliably hold. And you can't write a unit test for behavior that lives in sampling temperature — every prompt tweak becomes a release risk for your revenue path.

Side by side

	Workflow wizard	Free-form agent	Protocol over a core (ours)
Conversation quality	Scripted, brittle	Natural	Natural
Decision safety	High	Sampled — varies	High (decisions are never sampled)
Channel portability	One wizard per channel	High, but each channel drifts	One core, thin adapters
Testability	High	Poor	High — decisions live in one heavily unit-tested core

Our approach aims to preserve the wizard's decisions and the agent's conversation. The price is real: someone has to design the protocol — enumerate the outcomes, define the flags, decide what rides in results versus prompts. That design work is exactly what the other two approaches let you skip, and exactly why they fail.

Part V: What generalizes

If you're building an agent that transacts — orders, bookings, payments, anything with consequences — the shape of the argument travels:

Put decisions in code and conversation in the model. The boundary between them is the most important interface in your system; make it explicit.
Typed outcomes are that interface. The model should react to signals, not parse prose or improvise state.
Keep prompts static; let data ride the tool results. You may be forced into this by frozen voice configs, as we were — but it's worth doing anyway. It's what makes one protocol portable across channels.
Gate trust on provenance and recency. That you have a customer's details and that you may use them are different facts.
Give trust thresholds explicit numbers, so tests can pin them down.
Single-use evidence must be consumed. Anything that vouches for "the customer just did X" needs to be spent when it's used.
Make the decision layer testable. Push each rule into one place and unit-test it there. The sharpest rules — the trust threshold, the order-details check — fall out as pure functions you can test in isolation, even where the core around them still does its own database and cache I/O. Our conversational layer changes weekly; the decision layer, pinned by its tests, doesn't flinch.

The quiet payoff shows up later. The same choreography sheet runs a chat widget and a live phone line, unchanged. And when a real bug appeared in the trust logic, the fix was ten lines — not because we're careful, but because there was one place to be careless in.

Originally published at xinfer.ai — where there is also a 2-minute audio spotlight and a full narration of this essay.

I'm Sam Wen, founder of XInfer.AI. We build AI agents that transact — chat, voice, phone, and SMS — with one deterministic core underneath all of it.

The Model Writes, the Judge Measures: Anatomy of an LLM Judge

Sam Wen — Mon, 27 Jul 2026 01:18:04 +0000

Here is the kind of sentence that should keep a team up at night:

"At this clarity grade, you won't see a thing with the naked eye."

Our assistant produced it while explaining a diamond. The grade it cites is real — SI1, printed on
the stone's lab report. The promise attached to it is invented. A clarity grade is assigned under
10× magnification; it does not resolve what a naked eye will see. Two SI1 stones can look entirely
different — one clean at arm's length, the other with a visible speck right in the middle.
The model took a true fact and manufactured its consequence, because that is what a helpful
sentence sounds like.

Notice what the sentence is not. It is not a banned word. It is not a hallucinated stone. It
contains no wrong number. It breaks no rule you could have written down in advance. It is simply
fluent — and fluency is the problem. Fluent and true are different variables, and a language model
optimizes the first one.

We build AI assistants for exactly this kind of product. The one in this essay explains diamonds:
a short, two-voice spoken conversation about one stone — a host asks the questions a shopper
would ask, a gemologist answers — generated from that stone's evidence. A diamond is the
definition of a high-consideration purchase: thousands of dollars, bought rarely, chosen under
uncertainty. And it has a property that makes it an unforgiving test bed for generative AI —
nearly every claim about a diamond can be checked against its certificate from GIA or IGI, the
independent laboratories that grade the world's diamonds. Overselling is not just tacky here. It
is checkable.

In an earlier essay we argued that the language model should conduct the conversation while a
deterministic core makes every decision. This essay is the same worldview applied to quality. The
model writes. It does not get to grade.

At one generation you can catch a sentence like the one above by reading it. Across a catalogue of
stones you cannot. So before we made the writer better, we built an instrument: an independent LLM
judge that reads every generated claim against the evidence and returns a verdict we can count.
This essay is about how that judge is built, how we keep the judge itself honest, and what changed
once we had it — which is that the arguments stopped.

Part I — Three gates, and who owns which rule

The assistant explains one diamond at a time, from that diamond's evidence: the lab certificate,
the price, the retailer's published analysis of the stone. One rule governs everything: every
claim about this stone must trace to supplied evidence. If the evidence cannot answer — is it
eye-clean, meaning no marks visible to the naked eye? — the assistant says so, out loud: the
honest answer is a maybe, and it will not promise more. Then it points the shopper to a human gemologist.

Enforcing that rule is not one mechanism. It is three, layered, and they differ in kind.

The first gate prevents. It runs in code, inside the pipeline. When the assistant generates a
follow-up question about a stone, the question must name the evidence path it would be answered
from. A question that names no available path is rejected — by code, not by prompt hope. This is
what stops the system from asking "how does this price compare to the market?" about a stone
whose data has no comparable set. The model never gets the chance to improvise an answer, because
the question never survives.

The second gate catches. A deterministic validator runs over every finished script: a
hard-fail lexicon of the words the assistant must never say — amazing, must-have,
investment, the manufactured urgency, the "trust me." Word-boundary regexes, a few dozen tests,
no judgment involved. If a banned phrase appears, the script fails. Cheap, fast, and absolute.

The third gate judges. What remains after prevention and regex is the dangerous class: the
plausible-but-unsupported claim. "You won't see a thing with the naked eye" passes both gates. No
rule names it in advance, because it is not a phrase — it is a relationship between a sentence and
a lab report. Checking it requires reading both. That is a job for a reader, and at scale the
reader has to be a model.

One detail here taught us more than we expected: the boundary between the gates is itself a
design decision. When we extended the banned-word list from the domain experts' style guide, the
unambiguous terms went into the regex gate. But the context-dependent ones deliberately did not —
and the best example is flawless. In most products, "flawless" is empty praise. In diamonds it
is literally a grade: FL, the top of the GIA clarity scale. "It is not flawless" is a factual
statement about an SI1 stone, not a compliment withheld. Likewise "guaranteed": a real test
fixture proved that "that would need a guarantee the report doesn't give" is the assistant at its
most honest. A blunt regex would fail exactly those sentences. Those words went to the judge
instead, which can read the sense and not just the string. Which gate owns which rule is not a
default. It is a decision, and we made it with test evidence.

Part II — Anatomy of a verdict

"Use an LLM judge" usually means: ask a model to rate the output from one to ten. A score like
that is an opinion with a number attached. You cannot appeal it, you cannot audit it, and you
cannot tell whether a 7 means two small problems or one large one. It is not an instrument; it is
a mood.

Our judge is built differently, and every difference exists because a naive version failed first.

It returns a triage, not a score. For every sentence that asserts something beyond restating
the input data, the judge answers typed questions: Is this a claim about this stone, or about
diamonds in general? If it is about this stone — does the evidence establish it? If it is general
— is it accurate gemology? "Lab-grown and natural diamonds are chemically identical" is category
teaching: permitted, and wanted — the judge's instructions say teaching is never a violation.
"This stone will look white to the eye" is an item claim, and it had better trace to something. A real
violation is only the intersection: about this stone, and unsupported.

A claim also has an owner. Supported is not the only axis. GIA graded the stone; the retailer
analyzed it; the assistant speaks for the house. If it says "we grade this VS1," every word is
factually true and the sentence is still a violation — it credits the lab's work to the house.
The judge flags misattribution as its own class, separate from unsupported claims, because
provenance errors are how a trustworthy voice quietly claims authority it does not have.

It must report the claims it cleared. The judge is required to list the general statements and
the supported claims too, with the classification visible — and it is forbidden from letting them
inflate the violation count. This sounds like bureaucracy. It is the whole point. A verdict that
only lists problems is a number you have to trust. A verdict that shows every ruling — each quote
verbatim, each one naming the evidence path it was checked against — is an audit trail you can
re-check. When the judge is wrong, you can see exactly where.

Orthogonal signals stay orthogonal. The judge also measures tone: does the assistant sound
like a trusted expert across the counter or a closer working a commission? Does the conversation
sound like two people responding to a stone, or a script? But tone is a separate, subordinate
channel, and the judge's instructions are blunt: warmth never excuses an unsupported claim. A
formatting check rides in a third channel and changes nothing in the honesty count. There is no
blended score, deliberately. A blended score is where warmth hides dishonesty.

The writer and the judge read the same constitution. Every rule about what a lab report does
and does not support lives in one content-standard document. That document is spliced into the
writer's prompt and into the judge's prompt. The spec is a single artifact: the writer implements
it, the judge enforces it, and editing one sentence changes both sides of the contract at once.
There is no version of the rules that only the judge believes.

It never grades its own output. The writers run on Claude Sonnet 5. The judge's seat has been
held by two models — first GPT-5.5, a different lab entirely, and today Claude Opus 4.8, a
different model from the writers — and we verified the discipline held across the change. A model
asked to grade its own writing has a conflict of interest baked into its weights. Independence
has to be constructed.

It never touches production. The judge is a development instrument, not a runtime guardrail.
It grades output during the dev cycle to find where the writer's prompts leak; it adds zero
latency and zero cost to serving. Grading one diamond costs about thirty cents — four passes,
each re-reading the full evidence — which is trivially cheap in development and pointlessly
expensive in production, where the deterministic gates already stand. A judge is not a content
filter. It is how you find out what to fix.

Part III — Calibrate the instrument

The uncomfortable part: the judge is also a language model. It is an unreliable narrator hired to
audit an unreliable narrator. This is less absurd than it sounds — auditors are people too — but
only if the workflow plans for it. Ours does, in three ways.

The judge can be wrong, and the loop says so. Our written dev cycle is: the judge finds a
violation → we fix the writer's prompt, or we fix the judge if the judge is wrong → we log it →
we re-run. The same humility applies to the deterministic gates. We ban the word flaw — the
marks inside a stone are characteristics, never flaws. Then the assistant, explaining a modest luster
rating, said it was "not a flaw; just part of why the rating sits at four instead of five or
six" — and the regex failed the most honest sentence in the script. So the ban was narrowed:
the corrective "not a flaw" now passes, while "no flaws in this stone" — a perfection claim —
still fails. The exemption got its own tests. False positives are not noise; they are
calibration data.

The verdict is schema-checked, and malformed verdicts retry. In real runs the judge would
intermittently drop a required field, or return a field with the wrong type. We do not average
around that; the harness detects the malformed verdict and retries it, and every consumer of the
verdict is guarded against a bad shape. An instrument with unhandled failure modes doesn't stop
measuring — it starts lying quietly.

One sample is noise. The same stone, judged twice, does not score the same. We watched one
diamond score fifteen violations on one run and six to eight on reruns. If we had compared two
models using single samples, we could have concluded anything at all. The discipline is boring:
average at least two runs before believing any comparison, and never let a single verdict decide
anything important.

An instrument you never calibrate is just another opinion.

Part IV — Three arguments we never had to have

Here is what actually changed when the judge existed. Three decisions that AI teams normally
settle by taste, seniority, or vendor reputation got settled by reading the same number before and
after.

Which model. We ran two writer tiers over the same stones, with the same prompts and the same
judge. Claude Haiku 4.5 — the small, fast tier — averaged about ten real violations per diamond.
Claude Sonnet 5 averaged 0.8. A twelve-fold honesty gap, on identical inputs. And the direction of
the tradeoff surprised us: Sonnet also scored higher on specificity, so the honesty did not cost
richness. Every arithmetic blunder in the corpus came from the small model — including the run
where it declared one stone "59 carats more" than another. The real gap between those stones was
0.13 carats — and fifty-nine dollars. The model had turned a price difference into a weight.
Fifty-nine carats is not an engagement ring; it is a museum piece. A model that cannot keep
dollars and carats apart is not a bargain at any latency.

The judge also caught something no benchmark would have: the fast tier existed only to shave
seconds off time-to-first-word, yet it had quietly generated every record in our persisted
"quality" collection. The latency shortcut had poisoned the quality artifact. No unit test flagged
it, because every unit test passed. The judge flagged it, because the judge reads output.

The ending is the good part. The fast model's one advantage evaporated when we fixed a parser to
accept the larger model's output format — and measured end to end, the strong model at low
reasoning effort was fast enough to serve. Reputation said "the small model is fine for this." The
instrument said otherwise, and the instrument had receipts.

Whether RAG. Everyone's default architecture for grounding is retrieval. We shipped something
smaller: a deterministic key-join. The gemological knowledge — vetted definitions, approved
analogies, the boundaries of what a report can and cannot tell you — is keyed by characteristic,
and each stone's generation receives only the blocks matching what its report actually lists: a
cloud, a feather, a needle — the trade's names for the tiny marks inside a stone. An exact
lookup, not a search. So when the assistant explains a
cloud, it uses the vetted definition — a cluster of microscopic pinpoints — and keeps the vetted
boundary: the report cannot tell you whether it causes visible haze in this stone. The
knowledge base is content-hashed into the cache key, so editing one entry regenerates exactly the
output that depended on it.

Did we lose anything by skipping the vector database? That is normally a religious question. For
us it was an empirical one: the judge held the grounded output at zero real violations per stone.
Retrieval remained unnecessary for a bounded, one-stone-at-a-time task — not as an article of
faith, but as a measured result we can revisit the day the task stops being bounded.

There was a quiet confirmation along the way. A gemological standard written independently by the
domain's own experts — people who had never read our prompts — was later diffed against our
system. It landed on the same rules: the same evidence discipline, the same refusal to infer
eye-cleanness from a clarity grade, the same banned reassurances. Two instruments, built apart,
giving the same reading. That is what being right feels like from the inside: unremarkable.

Did the edit help. The prompts are the product, so we treat them like one. Every prompt edit
carries a changelog entry with four fields: the case (the concrete finding that forced it —
almost always a judge finding), the reason (the principle it serves), the change (what
actually moved), and the verification (a judged real run, with numbers). A typical case: the
judge caught the assistant volunteering why a stone had earned its value rating — a cause the
evidence never states — so the rule against manufactured explanations was widened, and the
verification showed the pattern gone across redraws with the honesty counts unchanged. About
sixty entries accumulated in the first week. No silent edits, ever. And because the prompt version
is folded into the cache key, no cached episode can ever have come from an unknown prompt: edit
the prompt, and everything it touched regenerates by design.

This is test-driven development for a non-deterministic system. The judge's verdict is the failing
test you drive to green. And the tests that matter run against the real API, on real output —
because in this project, every finding above came from a real run, and the mocked tests passed the
whole time. Over four hundred automated tests guard what can be asserted deterministically; not
one of them counts as evidence that the writing got better. Only the instrument says that.

Part V — What generalizes

Diamonds are the setting, not the point. This pattern transfers to any team shipping generative
output where being wrong is expensive. The short version:

Build the instrument before polishing the product. You cannot improve what you argue about. A judge you trust converts taste into experiments.
Layer the gates, and decide which gate owns which rule. Prevent in code what can be prevented. Hard-fail on regex what is unambiguous. Send to the judge only what requires reading. The boundary is a design decision — make it with test evidence.
Demand a triage, not a score. Typed questions, verbatim quotes, named evidence, cleared claims reported alongside violations. If you cannot re-check a verdict, you do not have an instrument; you have a mood.
Keep channels orthogonal. Honesty, provenance, tone, formatting — measured separately, never blended. A single number is where problems hide behind virtues.
Construct independence. A different model, ideally a different lab, reading from the same written standard as the writer. Never let a model grade its own homework.
Calibrate, and distrust single samples. Fix the judge when the judge is wrong, guard against malformed verdicts, average runs. Variance is real and it will lie to you.
Keep the judge out of production. It is a development instrument. The gates that serve customers should be deterministic, cheap, and boring.

The payoff is cultural as much as technical. Model selection stopped being a debate about vendor
reputation. Architecture stopped being a debate about fashion. Prompt engineering stopped being a
vibe. We stopped arguing about which model was smarter and started reading the same number twice.

One honest caveat, because our instrument would flag us if we skipped it: the judge measures
whether the assistant is honest about a diamond. It cannot measure whether a shopper wants to
listen. That is a different experiment, with a different instrument — and no model gets to grade
it for us.

Originally published at xinfer.ai — where there is also a 2-minute audio spotlight and a full narration of this essay.

I'm Sam Wen, founder of XInfer.AI. We build AI agents that transact — chat, voice, phone, and SMS — with one deterministic core underneath all of it.

The Factory Builds, the Evidence Admits: Anatomy of an Agent Certification

Sam Wen — Mon, 27 Jul 2026 01:16:04 +0000

Sooner or later, someone across the table asks the question. A client's security team. An auditor. An acquirer's diligence engineer. A regulator. Or you, at two in the morning:

"Why is this agent allowed to do that?"

Not does it work — demos answer that, and demos are cheap. Not how good is it — benchmarks answer that, about someone else's test set. The question is why it is allowed: what, concretely, stands behind the decision to let a language model operate near your customers and your money, and would that reasoning survive being read back to you a year later.

We build AI agents that transact — chat, voice, phone, SMS, one deterministic core underneath all of it. In an earlier essay we argued that the language model should conduct the conversation while that core makes every decision. In a second, that the model which writes must never be the model that grades. This essay is the third act, and it asks what the first two were building toward: when an agent is done, what actually admits it to production?

Our answer is unglamorous. Not a launch review, not a leaderboard, not a sign-off meeting. A file.

A note on what kind of essay this is, because the subject demands it. The first two essays reported what we had built and measured; every claim had a receipt. This one goes further than the receipts. The system it describes is real and in production — but parts of what follows are extrapolation from the work we have done: conclusions the built system points to, worked out on paper, not yet running. I will mark the line as we cross it. An essay about certification that blurred the line between measured and extrapolated would fail its own bar.

Part I — The question a demo can't answer

Strip the ceremony away and "this agent may run in production" asserts four things:

Its authority is bounded. There is a known, finite list of decisions it can cause, and everything off that list is impossible — not discouraged, impossible.
Its inputs are trusted by origin. Every piece of data it acts on has a provenance, and provenance — not mere presence — determines what may be done with it.
Its words are measured. Whatever the model still says freely has been evaluated against evidence, by something that is not the model.
All of it replays. Anyone, later, can reconstruct exactly what was approved, against what evidence, and check the reasoning themselves.

Notice what's missing from that list: the builder's confidence. "We built it and it works" is the builder grading their own homework — the precise conflict of interest the second essay spent five parts constructing independence against, now at the scale of a whole agent instead of a single sentence. A model must not grade its own writing; a team cannot certify its own agent. Not because teams are dishonest, but because the incentives point one way and the weights, so to speak, are baked.

The first essay separated the decider from the narrator inside one request. The second separated the evaluator from the writer inside one build. Certification is the same cut, one level up: it separates the party that builds from the thing that admits. And the admitting party, in our view, should not be a committee. It should be the evidence.

Part II — The evidence file

What follows is built and measured — this is the file we can produce for our own agent today.

Authority is a list, not a mood. When our model submits an order, the request lands in one server-side function, and that function answers with one of five typed outcomes — order placed, contact needed, contact to confirm, cart empty, or a failure with a reason code. That list is the agent's authority over money. Nothing else can happen, because there is nothing else the core returns. This is the first thing that belongs in an evidence file, and the cheapest to check: the enumerated decisions. If you cannot write the list down, your agent does not have bounded authority; it has tendencies.

Seriousness is not the criterion for what goes in code. Determinism is. The temptation is to move everything "important" into the deterministic core. That's the wrong test. What moves into code is what has a code-time-knowable correct answer — thresholds, lookups, idempotency, required disclosures. Force a genuine judgment call into code and you either reduce it to a crude proxy or smuggle a model in through the back door, destroying the very property that made the core trustworthy. The right move, when a judgment is needed, is the one the second essay described: the model emits a typed verdict, and the core decides the action. Judgment measured by a model; consequence owned by code. The same split, applied recursively.

There is a corollary we now treat as a rule: an agent may make a decision only where the decision rule already exists in deployable form — a written policy, an existing validated system, a governed model. Where the rule exists nowhere — where the call is genuinely discretionary — the agent does not get to manufacture it, and neither does your prompt. The agent does the perception: it reads the messy input, extracts the structure, drafts the summary. A person, or captured logic, decides. This sounds restrictive; in practice it's liberating, because an enormous amount of valuable work was never automated not because the logic was missing but because the inputs arrived unstructured. The rules were always written down. They were gated on reading. Reading is exactly what these models are for.

Provenance, and evidence that is spent. The first essay covered this machinery in detail, so a summary: our core trusts data by where it came from — supplied this turn, established this session, on file from before — and each origin authorizes something different. Session evidence is a single-use token, consumed by the order it vouches for, so a second order can't ride the first one's proof. Submissions claim the cart atomically, so a model that retries a timeout replays the same order instead of minting a twin. Every one of those sentences is a unit test somewhere; that's what makes them evidence rather than architecture slides.

The gate is not a dashboard. Our judge — a different model from the writer, reading the same written standard — returns typed verdicts, quotes each claim verbatim, names the evidence path it checked against, and reports the claims it cleared alongside the ones it flagged. But the detail that matters for certification is what happens downstream of a verdict: grounding failures block the merge; quality is tracked as a trend. One is a gate, the other is a gauge, and confusing them is how eval programs die. A check that cannot fail the build is not a control. It's a dashboard with opinions.

Everything replays. Every judged run persists what it saw: the prompt, the evidence supplied, the verdict, the model and version. Prompt versions and knowledge content are hashed into cache keys, so no cached output can ever have come from an unknown prompt — edit the standard and everything that depended on it regenerates by design. Every prompt change carries a changelog entry: the case that forced it, the reason, the change, the judged verification. The second essay said that a verdict you cannot re-check is not an instrument but a mood. One level up, the same law: a certification you cannot replay is not a decision. It's an opinion.

Part III — The map, and the open column

This part is an audit, not a system — real work, done on paper, against standards we didn't write.

Confidence in your own architecture is worth little; the essays above are, after all, the builder talking. So recently we did the adversarial thing: we took the two patterns — core-decides, judge-measures — and put them in front of twenty-five requirements drawn from NIST's AI Risk Management Framework and the EU AI Act's high-risk articles. Frameworks written by people who have never read our prompts and owe us nothing.

The map came back in three colors: ten requirements covered, ten partial, and the rest open.

Covered clustered exactly where you'd predict, which is itself a useful check. Accuracy and truthfulness — the judge's whole existence. Robustness and reliability — idempotency, cross-channel consistency, retry-safety. Logging and traceability — typed outcomes are auditable by construction; you don't add logging to a decision that is already a value. Independent measurement — a calibrated evaluator that is not the writer. In NIST's vocabulary, the judge is a Measure function and the core is a Manage function, and the two patterns turn out to be those two boxes, implemented.

Partial is where architecture helps but doesn't finish: data governance, transparency, human oversight. The core makes oversight possible — a transparent, logged, explainable decision that a person can review is strictly better than a model output to rubber-stamp — but possible is not the same as done. Disclosure to the end user sits here too, and it's an instructive case. Every agent we ship introduces itself as an AI in its opening line — "I'm Kai, your AI assistant. How can I help you today?" — across chat, voice, and phone alike. But that sentence lives in a configurable greeting, not an enforced gate: a merchant could edit it away. Handled by default is not the same as guaranteed by construction, and a map that blurred those two would be flattering itself.

Then the open column, and here is the finding that reframed the exercise for us. The open items were: accountability and ownership. Governance of third-party and vendor models. Bias and fairness examination. Privacy and PII handling. Runtime monitoring and incident response.

Read that list twice. Every open item is organizational. Not one of them is a gate you can design. No typed outcome produces an accountable owner; no judge verdict staffs an incident channel at 3 a.m.; no cache key reviews a vendor. The architecture ran out precisely where the org chart begins — which is not a failure of the architecture. It is the discovery of where certification stops being an engineering problem. An audit that came back all green would have told us nothing except that we'd audited ourselves gently.

One more honest finding from the same exercise, because it surprised us: "high-stakes claim" and "high-risk system" are different axes. Under the EU AI Act, a commerce agent is limited-risk — its legal obligation is disclosure, not truthfulness. The Act tiers by domain, not by how expensive a false sentence is. Our guardrail work is justified commercially — by what a false promise costs a merchant — not because a statute demands it. Teams over-claim compliance pressure in both directions; the map keeps you honest about which force is actually pushing.

And the audit exposed a seam we had not named before. The judge works in development — it grades output to find where the writer's prompts leak, then stays out of production, where the deterministic gates stand watch. Which means: at runtime, nothing independently verifies the narration. The words the model speaks to a live customer pass regex and locked facts, and nothing else. For our risk profile today, that's a reasoned trade. But naming the seam tells you exactly what the next mechanism has to be — and that is where the receipts end.

Part IV — Extrapolations

The line is here. Everything below is extrapolation from the work above — worked out on paper, argued from the built system, not yet running. Our own judge would flag us if we let you read it any other way.

First extrapolation: gate the consequence, not the utterance. You cannot batch-check a sentence after a customer has heard it — a slow check on spoken words is monitoring, not a guardrail. But you can gate the consequence. Capture the irreversible step immediately and idempotently — the order lands, marked pending review. Then run the full-strength judge in the window between capture and fulfillment, where it has no latency budget at all: the entire transcript against the entire evidence file, all the time it wants. Deterministic code reads the verdict and releases, holds, or corrects — before anything ships, charges, or becomes real.

Payments people will recognize this instantly: it is authorize-then-capture with a fraud hold, applied to conversation. Fast cheap prevention at the utterance; slow accurate judgment at the commit; the irreversible step gated in between. The pattern has one precondition — a reversible window must exist. Instant digital delivery has none; pure advice has nothing downstream to gate; both fall back to strong inline prevention. Our core is idempotent and claims orders atomically today; the hold-window with an asynchronous judge deciding release is design. It is, we think, the single most underused idea in agent safety — mostly because the people who know the payments pattern and the people building agents have not been the same people.

Second extrapolation: the admissions office. Scale the argument of Part I up from one agent to a catalog of them — the "agent factory" every large organization is now standing up — and the logic forces a structure. If a certification is a file, and the builder cannot self-attest it, then somewhere there is a party whose job is to read the file and say no. Separate from the builders. Owning the bar, not the backlog.

That party's product is not agents. It is the bar — and the credibility of the bar is measured by exactly one number: the rejection rate. A catalog with no rejection rate is not a control. It is a spreadsheet with better branding. Every organization that stands up an agent program announces a catalog; almost none can tell you what their bar has refused. Twelve months later there are four hundred entries, zero rejections, and no claim anyone outside the team believes.

The same file does one more job: it is how reuse gets enforced rather than encouraged. Once an agent's pattern is certified, the file is the reference — and the second team that needs the same capability doesn't get a second certification for a copy; they get pointed at the first one. The catalog becomes the place where duplicate work goes to be refused. We run this logic at merchant scale — one certified checkout protocol, reused across every channel rather than re-certified per channel. At enterprise scale we have not run it, and this essay won't pretend otherwise. The shape is extrapolation. The forces it answers — builders grading themselves, catalogs that only ever grow — are not.

Part V — What generalizes

The three essays in this series are one idea wearing three sizes. At every layer, one party proposes, and a different party — with different incentives, reading the same written standard — decides:

Layer	Proposes	Decides
One request	The model offers words	The core owns the action
One build	The model writes	The judge measures
One organization	The team builds	The evidence admits

If you are deciding what "certified" should mean for agents — yours, or a catalog of them — here is the shape of the argument, portable:

Certification is a file, not a ceremony. Enumerated authority, provenance rules, measured words, replayable runs. If you cannot replay the decision, you have an opinion with a date on it.
Check authority first — it's the cheapest gate. The list of decisions an agent can cause is knowable before a single behavioral test runs. No list, no bounded authority.
Determinism, not seriousness, decides what lives in code. And where judgment is genuinely needed: the model emits a typed verdict, code owns the action.
An agent decides only where the rule is captured. Everywhere else it reads, extracts, and drafts — perception — while captured logic or a person decides. Most of the value is in the reading anyway.
Audit against a standard you didn't write. Your own rubric will be kind to you. The open column — what your architecture cannot certify — is the most valuable output, because it is a precise list of what the organization must supply instead.
Gate the consequence, not the utterance. Where a reversible window exists, capture fast, judge slow, and let code release the hold. Where it doesn't, prevention is all there is — know which case you're in.
Separate the builder from the bar. And publish the rejection rate. It is the only number that proves the bar exists.

One honest caveat, in the house style: the file certifies the agent. It cannot certify the organization around it — no artifact we know how to construct measures whether an accountable owner will actually show up when the incident channel fires. That column stays open, and pretending a gate design closes it would be exactly the kind of fluent, unsupported claim this whole series exists to catch.

The extrapolations above are unbuilt, and we've said so. That is not a hedge; it is a roadmap with witnesses. When the hold-window ships, it will get its own essay — receipts first.

Originally published at xinfer.ai — where there is also a 2-minute audio spotlight and a full narration of this essay.

I'm Sam Wen, founder of XInfer.AI. We build AI agents that transact — chat, voice, phone, and SMS — with one deterministic core underneath all of it.