Here is the pattern, in the smallest form that still does the job.
from dataclasses import dataclass
@dataclass(frozen=True)
class Check:
name: str
score: callable # (output, ctx) -> float in [0, 1]
floor: float
hard: bool = False # hard failures never retry; they halt
def gate(generate, checks, ctx, attempts=3):
"""Return (output, trace). Never returns an output that failed a check."""
trace, feedback = [], None
for i in range(attempts):
out = generate(ctx, feedback=feedback)
scores = {c.name: c.score(out, ctx) for c in checks}
failed = [c for c in checks if scores[c.name] < c.floor]
trace.append({"attempt": i, "scores": scores,
"failed": [c.name for c in failed], "output": out})
if not failed:
return out, trace
if any(c.hard for c in failed):
raise HardFailure(failed[0].name, trace)
feedback = render_feedback(failed, scores)
raise Escalate(trace)
That is the whole idea. No model output reaches anything downstream until it has been scored, and a failure feeds the failure back into the next attempt rather than being logged and forgotten.
I build this in healthcare revenue cycle, where the output is an insurance
appeal and the hard check is protected health information. But nothing in the pattern is domain specific. Any time you want an LLM to do work nobody reads line by line, this is the shape.
Why bother, instead of a human in the loop
Because a human in the loop on every output is not a safety feature, it is a throughput ceiling, and usually the one you were trying to raise.
The task I care about is writing appeals against denied insurance claims. The published numbers make the case better than I can: KFF reported in July 2026 that skilled nursing denials are overturned 95 percent of the time when appealed, and appealed 18 percent of the time. Nobody skips an appeal they expect to win. They skip it because nobody is free to write it.
Put a person on every output and you have moved the work from writing to
reviewing. Faster, but the same ceiling, plus a licence fee. So the engineering question becomes: what has to be true for the output to be
safe to send unread?
Design note one: hard checks are not just checks with a high floor
This is the part people get wrong on the first pass, and I did too.
Most checks are quality checks. Fail one and retrying is correct, because the model can often fix it given the failure as context.
Some checks are not like that. In my domain, PHI leakage is one. If the output contains protected data it should not, retrying is exactly the wrong move: you have already produced the thing, and the correct response is to halt, raise an incident, and page a human. Retrying a safety failure is how you turn one incident into three.
CHECKS = [
Check("groundedness", score_groundedness, 0.90),
Check("accuracy", score_accuracy, 0.95),
Check("variance", score_variance, 0.88),
Check("phi_safety", score_phi, 1.00, hard=True),
]
Note the floor on the hard check is 1.00, not 0.99. There is no partial credit available on that dimension, and a floor of 0.99 is an admission that you expect to leak occasionally.
Design note two: the feedback is the retry
A retry that sends the same prompt again measures your temperature setting. A retry has to carry what failed.
def render_feedback(failed, scores):
lines = ["Your previous answer did not pass validation. Fix these and "
"return the corrected answer only."]
for c in failed:
lines.append(f"- {c.name}: scored {scores[c.name]:.2f}, "
f"needs at least {c.floor:.2f}. {HINTS[c.name]}")
return "\n".join(lines)
HINTS = {
"groundedness": "Every factual claim must appear in the provided source "
"documents. Remove anything you cannot point to.",
"accuracy": "Codes and identifiers must validate against the supplied "
"reference set. Do not invent plausible ones.",
"variance": "Answer at the level of specificity the source supports, "
"no more.",
}
The hints matter more than the scores. A model told "groundedness 0.71" does nothing useful. A model told "remove any claim you cannot point to in the source" usually fixes it in one pass.
In my system this loop corrects 87.2 percent of catchable issues without a person, in about 4.2 seconds and roughly 1,800 extra tokens per correction, at about 0.00054 dollars. That is measured on the runtime standalone rather than in a customer environment, and I flag that because a number without its measurement context is not a number.
Design note three: groundedness is the check that earns its keep
If you implement only one, implement this one. It is also the one people
implement worst, usually as an embedding similarity between output and context, which is close to useless because a fluent paraphrase of something false scores well.
Decompose instead.
def score_groundedness(out, ctx) -> float:
claims = extract_claims(out) # atomic factual assertions
if not claims:
return 1.0
supported = sum(is_supported(c, ctx.sources) for c in claims)
return supported / len(claims)
extract_claims is a cheap model call with a strict output schema. is_supported is another, per claim, asked as a yes-or-no with the relevant source span attached. It is more expensive than cosine similarity and it is the difference between a check and a decoration.
Two implementation notes that cost me time. Ask the support question in isolation per claim, because a model shown ten claims at once will pattern-match to "mostly fine." And log the unsupported claims, not just the ratio, because that list is the actual product of the check.
Design note four: the trace is the point
The trace returned above looks like debugging output. It is the most valuable thing the whole pattern produces.
Persist it. Every attempt, with the before and after text, every dimension
score, whether a correction was applied, how many attempts it took, whether it escalated, which model ran, the latency and the cost.
def persist(trace, ctx):
for row in trace:
write_append_only({
"request_id": ctx.request_id,
"attempt": row["attempt"],
"scores": row["scores"],
"failed": row["failed"],
"output_before": row["output"] if row["failed"] else None,
"output_after": None if row["failed"] else row["output"],
"model": ctx.model,
"latency_ms": ctx.latency_ms,
"cost_usd": ctx.cost_usd,
})
Append-only, and keep it as long as your regulator asks. Mine asks for seven years.
Two reasons this earns its storage. Operationally, failures cluster, and the cluster tells you what to fix long before an aggregate pass rate moves.
And this table is where your override rate lives: how often a human disagreed with the system, on what, and what happened next. In a regulated domain that is the only evidence that human review was real rather than a signature. A March 2026 discovery order in a US coverage denial case compelled production of internal AI review board materials. You cannot reconstruct that log retroactively.
What this does not do
It does not make the output correct. It makes it verifiable against what you supplied, which is a weaker and much more achievable property. If your source documents are wrong, a perfectly grounded output is confidently wrong.
It costs latency and tokens on every call that needs a retry. If your workload is latency-critical this trade may not be available. Mine is not: an appeal that takes four extra seconds is still days faster than the queue it came from.
And retry count is a parameter, not a principle. Three is where the marginal correction rate stopped justifying the latency for us. Measure yours rather than inheriting mine.
The one line I would take away
The interesting cost in an LLM system is not inference. It is verification, and whether you pay it in software or in people.
If you pay it in people, the system does not scale past their hours, which is usually the exact constraint you bought it to relieve.
The healthcare platform this pattern runs in is ARIA, which my team builds at Ambharii Labs. Performance figures above are internal, measured on the runtime standalone or on our own 197-case evaluation suite, and labelled as such. We have no independent benchmark, which is the honest gap.
Sources: KFF, 6 July 2026. Lokken v. UnitedHealth discovery order, 9 March 2026.
If you run a gate like this, I would like to know what your hard-check list contains. That list is a very direct statement of what an organization thinks is unrecoverable, and I have never seen two that match.
Top comments (0)