The one-line version
We ran five current frontier models over a set of documented-failure questions, twice each: bare,
and wrapped in a thin external layer (retrieved evidence + a rule that lets the model say "I don't
know"). We were not trying to make them smarter. We were asking whether confident fabrication can be
reduced from outside the model — no fine-tuning, no internals. On this corpus, single draws: three
models collapsed to near-zero, one resisted, one refused the wrapper outright. That unevenness — not a
universal claim — is the story, plus what the measuring rig caught its own authors doing along the way.
One framing note up front: the wrapper tested here is one deliberately-thin layer cut from a much
larger verification system we run in production on ourselves — chosen because fifteen lines is
what a stranger can copy and verify by dinner. Where the layer's gaps show below (they do, measured),
you're seeing the outline of the layers it was cut away from. We say which is which as we go.
What we are NOT claiming
Because a reader should know where the floor is before the numbers:
- Not a new phenomenon. That frontier models confabulate, and that grounding + abstention reduces it, is well-established (Stanford's Large Legal Fictions, 2024; the RAG-faithfulness literature).
- Not general failure rates. The corpus is built from documented failures — the bare rates are high by construction and say nothing about these models on ordinary traffic. The finding is the bare→wrapped delta on identical items, never the absolute numbers.
- Not "lying." Our metric is mechanical: a non-abstained answer graded wrong. No intent is measured or implied — "confidently wrong" is the precise claim everywhere in this post.
- Not benchmark-comparable scores. Our grader, not any lab's official scorer — the arms are comparable to each other, on our corpus, one sampled response per item. Read every number as "under this treatment, this corpus, single draw."
- Not "we made the model honest." The wrapper trades guessing for abstention. Correctness did not go up; it went down (more on that below). Honesty ≠ accuracy, and we lead with that so it can't be mistaken.
The numbers (280 paired factuality items — the corpus is 295: 280 factual QA pairs graded here, plus 15 fabricated-legal-citation probes reported separately below)
Confabulation = a confident wrong answer (non-abstained, graded wrong). Bare → wrapped:
| model | bare | wrapped |
|---|---|---|
| GPT-5.4 | 47.9% | 1.8% |
| Gemini 3.1 Pro | 16.4% | 2.1% |
| Grok 4.3 | 5.7% | 1.4% |
| DeepSeek v4 | 33.3% | 24.3% |
| Claude Fable 5 | 40.0% (bare) | refused the wrapped prompt, 295/295 |
Overall across the four that took the treatment: 25.8% → 7.4%. But correctness fell 43.9% → 21.3%
— because our retrieval (Wikipedia) often didn't contain the answer, and the rule then correctly drove
abstention. Read that drop correctly: this sample layer has no ability to go find more evidence —
that's deliberate. The full system's job is exactly the other half: seek wider, then make the model
use what it finds. The layer you can copy buys the honesty; the layers it was cut from buy back the
answers (registered at the bottom of this post, running next). The wrapper's accuracy is bounded by
retrieval quality, and stating that plainly is the whole point.
The two things worth your attention
1. DeepSeek keeps confabulating with the evidence in front of it. Every other model collapses to
near-zero wrapped; DeepSeek only moves 33%→24%. And on a separate set of 15 court-confirmed fabricated
legal citations, DeepSeek affirmed nonexistent cases as real 12 of 15 times (the others: 3/15).
Here's the part that makes it non-obvious: where these same models are co-benchmarked on summarization
faithfulness (Vectara's leaderboard), DeepSeek is mid-pack and better than Grok/Gemini. Same models,
opposite ordering — so this is a task-specific result, not a re-observation. A contemporaneous paper
(LegalCiteBench) even found abstention-prompting alone did NOT fix legal-citation correctness — so the
delta wasn't a given.
2. Claude Fable 5 refused the honesty wrapper — 295 of 295, and again on fresh retest. Every
wrapped prompt in the study run returned a refusal (stop_reason: refusal); every bare prompt
answered normally. A fresh 2-item retest days later reproduced it exactly: bare answered, wrapped
refused. A reworded prose version of the same instructions answered fine, which suggests the
prompt's structured shape matters — but one intermediate probe (the rules section alone) was
recorded as answering on one occasion, so we stop short of calling the trigger deterministic or
diagnosed. What we claim is what receipts show: on this exact prompt shape the refusal is strongly
reproducible across days, and the corpus is available for any safety team that wants to reproduce it.
Then the twist, same day as the retest: we handed the same honesty discipline to the same model
as an agent — a search tool plus seek-before-answer, ground-only-in-results, abstain-if-not-found
rules — and it accepted 620 of 620 agent-shaped calls across every arm of the tools experiment,
zero refusals, searching, grounding, and abstaining exactly as instructed. Same honesty content,
different packaging, opposite outcome. Whatever trips the refusal, it is not the honesty — it's
something about the wrapper's shape, and an agent harness sails past it.
Then we gave them hands: the tools experiment
The obvious objection to everything above: of course they make things up — they can't look anything
up. So we ran the experiment that objection demands. Same 200 factual items (the SimpleQA subset),
same five models, each given a search tool in an agent loop — search, read, search again, then
answer or abstain. Two arms again: bare-with-tools (search available, no rules) and harness-with-tools
(seek before answering, ground only in results, abstain if not found). Retrieval served a fixed
same-night Wikipedia snapshot — the same design choice KILT made for reproducibility: anyone can
rerun our exact experiment, which live web never allows. Judged answers, both-directions grader
control (zero overturns), ~$10 total, receipts for everything.
| model | bare+tools: correct / confab | harness+tools: correct / confab / abstain |
|---|---|---|
| GPT-5.4 | 31% / 51% | 5% / 8% / 87% |
| Gemini 3.1 | 70.5% / 25% | 4% / 0% / 96% |
| Grok 4.3 | 36% / 29% | 4.5% / 0% / 95.5% |
| DeepSeek | 34% / 29% | 4% / 0.5% / 95.5% |
| Claude Fable | 71% / 28% | 5.5% / 1.5% / 93% |
One reading note before the findings: the high abstain columns are the thin layer being honest
about a thin library — a fifteen-line sample can refuse to guess, but it cannot go find what the
library lacks or force a model to use what it holds. Those are the missing organs, they are the
other half of this system, and they're pre-registered at the bottom of this post. Judge the layer
on the confab column; judge the organism on what comes next.
Five results, in the order they surprised us:
1. The honesty discipline survives the move to agents — almost perfectly. With search in hand,
four of five harness models confabulated at 0–1.5%. Gemini and Grok: zero wrong answers in 200 each,
with receipts like "FINAL: James Basire — the search result states directly that Thomas Ryder 'was
a pupil of James Basire.'" Grounded, cited, correct.
2. Giving a model search made it MORE confidently wrong — unless the discipline came with it.
Grok bare, no tools: 11% confabulation (the cautious one). Grok bare with a search tool over a
thin library: 29%. Weak search results emboldened the careful model into guessing. The same
pattern hit every lab's model to some degree. If you ship agents: a search tool without an
abstention discipline is a confidence machine. This is the finding we'd want someone to have told
us before we built anything.
3. Given the answer in-hand, bare fabricates and the harness goes quiet — but not honest-and-loud.
On the 25 study items where the evidence pack demonstrably contained the answer: bare models were
confidently wrong at up to 60% (GPT), 56% (DeepSeek), 44% (Claude). Wrapped, on the same items:
confident-wrong collapsed to ~4% — but the models spoke the truth only about a third of the
time, staying silent with the answer in their hands the rest. Small cells, single draws, framed as
asymmetry, not precision — but the asymmetry is stark: the wrapper converts fabrication into
silence far more reliably than it converts silence into truth. That gap is the honest frontier,
and it is a retrieval-and-enforcement problem, not a prompt problem — see the refutation below.
4. Prompt pressure doesn't fix the silence — we tested it so you don't have to. The cheap theory
says the harness rule just needs a counter-pressure: abstaining when the results contain the answer
is as serious a failure as guessing. We added exactly that sentence, sealed predictions it would
work, and reran all five models. Verdict changes were noise. Words don't buy back what retrieval
doesn't deliver — the fix for honest silence is mechanical: a wider library and a layer that
refuses the "I don't know" when the evidence on the table contradicts it. The mechanism is the thing.
That's v3, registered below.
5. Model orderings reversed across settings. DeepSeek — the evidence-resistor of the no-tools
study — ran nearly perfect discipline with tools (1 confab in 200). GPT — the model the wrapper
fixed best without tools — was the one that kept guessing through the harness (searched twice, found
nothing, answered "FINAL: 1482" anyway). Hallucination discipline is setting-specific; a
universal model ranking is the wrong mental model.
Every prediction in this phase was committed to a public git hash before any result existed, and
graded after: 3 confirmed, 3 missed on the main board, 0 of 3 on the prompt-fix board — the
misses reported with the same prominence as the hits. Methods notes: fixed-snapshot retrieval
(disclosed above), single draws, and the LLM judge at temperature 0 still drifts ±1–2 verdicts per
200 on re-grade — an order of magnitude below every effect reported.
It holds on models you can run for free
We reran the same bare→wrapped test on five open models on a rented GPU (deepseek-r1:8b, gemma3:12b,
llama3.1:8b, mistral-small3.1, qwen3:8b). Every one cut confabulation under the wrapper — relative
reductions from 31% to 100% on the documented-failure set. The effect isn't a frontier-only
luxury; it reproduces on 8–12B models a laptop can run. (The two reasoning models truncated their
long "thinking" at the first pass's token cap; deepseek-r1's full re-run at a raised cap is complete
and its numbers will replace the first-pass figures before this table is cited anywhere; qwen3's
re-run is in progress — a config note, disclosed, not a result.)
The part that's actually ours
Here is the thing no benchmark table shows, and the reason we trust these numbers: the rig is
adversarial to its own operators, and during this experiment it caught us — on the record.
- It caught a poisoned run mid-flight: Wikipedia rate-limited us, our search code returned the failure as an ordinary "no results," and every abstention became ungradable — 512 receipts were quarantined and the search layer rebuilt to fail loudly before a single bad number reached a table.
- It forced the retraction of a committed diagnosis: we blamed a DeepSeek tool-format bug; a debug probe showed its tool calls were valid all along, and the real bug was ours. The retraction is in the same commit history as the claim.
- It killed the word "deterministically" from this very post: a fresh retest of the Claude refusal reproduced the behavior but an earlier probe on record contradicted the strong wording — so the claim shrank to exactly what receipts show.
Anyone can report that a model fails. What we're putting forward is the method — the failure caught in
the act, an external layer measured on current models, and a verification system that flags its own
authors before a claim leaves the building. The findings are this week's output. The instrument that
produces them is the actual work.
What's next — registered in advance
The over-silence finding names its own next experiment, so we're committing to it here, in public,
before it runs: v3 replaces the thin snapshot with real multi-source retrieval and adds a
mechanical use-of-evidence check — a layer that refuses an "insufficient evidence" verdict when
the retrieved text demonstrably contains the answer (the same architecture our own verification
system uses on us). Predictions will be sealed to a public git hash before the first call fires,
and the follow-up post grades them — hits and misses both, same as this one.
Reproduction corpus, grader, and the full self-catch record are on request. Numbers current as of the
run dates (study 2026-09-18; tools 2026-09-20/21); the 2026-era prior-art comparisons were read from HTML and should be
confirmed against source PDFs before anyone builds on them.
Top comments (0)