A viral post made the rounds recently: "How to Make Claude Code Stop Making Stuff Up When It Doesn't Know." It described a 4-layer setup — honesty rules in CLAUDE.md, verification protocols, linter hooks, and a fact-checker subagent — to catch Claude fabricating functions, faking test results, and confidently delivering nonsense.
The post was good. The problem it named is real. But the solution is a duct-tape stack, and every layer has a gap the next one is supposed to cover. Four moving parts, all configured by the user, all depending on the agent's willingness to follow the rules that are supposed to catch it breaking the rules.
I built a single hook that replaces all four. Here's why the layers don't hold, and what does.
What the 4 layers actually are
Layer 1: Honesty rules in CLAUDE.md. Tell Claude to admit uncertainty, cite file:line when referencing code, and ask before adding dependencies.
Layer 2: Verification protocol. More CLAUDE.md instructions forcing Claude to check files exist before importing them.
Layer 3: Linter/type-checker hooks. A PostToolUse hook that runs tsc or a linter after every file write. If Claude invented an import, the checker fails.
Layer 4: Fact-checker subagent. A second agent invoked before commits whose job is to verify claims made by the first.
Each layer is reasonable in isolation. Together, they share a structural problem.
The structural problem: layers 1, 2, and 4 ask the agent to police itself
CLAUDE.md rules (layers 1 and 2) work by telling Claude to behave differently. But Claude's fabrication problem exists because it's optimised to look helpful, and admitting uncertainty feels unhelpful. You're asking the same optimisation that causes the lie to also catch the lie. Sometimes it works. Sometimes Claude reads the rule, weighs it against the pressure to look competent, and guesses anyway. The viral post itself acknowledges this: "Punish honesty once and Claude goes back to guessing."
Layer 4 — the fact-checker subagent — is a second model reviewing the first. This is the "two AI reviewers" trap I've written about before: if both models share the same training, the same helpfulness pressure, and the same blind spots, the second one can be talked out of a finding just as easily as the first. A model reviewer is not an independent check. It's a correlated one.
Only layer 3 — the linter hook — is genuinely independent. tsc doesn't care what Claude claimed. It reads the file and either the types resolve or they don't. That layer works because it doesn't involve a model.
What a single deterministic hook does instead
I built Groundtruth around that same principle: nothing reads the code, so nothing can be talked out of the verdict.
It's a Claude Code Stop hook. When the agent says "done," Groundtruth reads three things from outside the agent's control — the request, the transcript (what the agent actually ran), and git diff HEAD (what actually changed) — and renders a verdict before the turn ends.
No LLM. No network. No API key. One hook, not four layers.
Here's what it catches, mapped against the 4-layer stack:
"Tests pass" — when nothing ran
The viral article's fix: a Stop hook that runs the test suite before Claude can declare done.
Groundtruth's approach: it checks whether a test command actually executed in the session transcript. If Claude claims "tests pass" but no test runner appeared in the Bash history, it fires:
🔴 false test/build claim — "tests pass", but no test command ran this session
The difference: the article's hook runs the tests (which costs time and can fail for environment reasons). Groundtruth checks whether they were run. If they weren't, the claim is false regardless of what the tests would have shown. Both are valid; mine is cheaper and catches the specific lie — "I ran the tests" when you didn't.
Made-up imports
The article's fix: tsc as a PostToolUse hook catches fabricated imports when the type checker fails.
Groundtruth's approach: for JS/TS (where imports are path-relative), it resolves the import path against the file tree. If the target doesn't exist, it fires:
🟡 phantom ref — new import of './utils/tokenHelper' but that file doesn't exist
For package-qualified languages (Python, Go, Rust, Java, C#), it abstains rather than false-flag — because a missing package import isn't provably wrong from the file tree alone. This is deliberate: a check that false-fires teaches you to ignore the card, which breaks everything.
"Done" — but the file isn't in the diff
The article doesn't cover this. Groundtruth does:
🟡 silent no-op — claimed src/upload.test.js, but it is absent from the diff
The agent said it created a file. The diff says it didn't. No model needed to catch that.
Rules your agent could see and overrode anyway
The article's layer 1 puts rules in CLAUDE.md and hopes Claude follows them. Groundtruth compiles rules from your docs into deterministic predicates — your doc says "use pnpm not npm," so a regex checks the diff for npm install. The agent can't rationalise past a regex.
These rules are proposed, never auto-armed. You review them with /groundtruth-rules and approve what's correct. If one fires wrongly, silence it by id. The permission gate is yours, not the model's.
Stubs and placeholders
The article doesn't catch // TODO: implement this or raise NotImplementedError left in added code. Groundtruth does:
🟡 stub/placeholder in added code: // TODO: real backoff — single attempt for now
If the agent claims a feature is done and the diff contains a placeholder, those two facts contradict. No judgment call needed.
Why "don't involve a model" matters
The viral article's own best insight — buried at the end — is that the linter hook works because it doesn't involve a model. That's the whole thesis of Groundtruth, applied to every check instead of just one.
A model reviewer (layer 4) can be reasoned out of a finding. Claude is very good at explaining why a seemingly-wrong thing is actually fine. "The test file wasn't needed because the function is trivial" sounds plausible to a second model that shares the same optimisation for helpfulness. It doesn't sound plausible to a diff that simply notes the file isn't there.
The 4-layer approach works partially because 25% of it is genuinely independent (the linter). Groundtruth works fully because 100% of it is.
What Groundtruth does NOT do
I'll be precise, because overclaiming is the exact failure mode this tool is built to catch.
- It does not judge code quality. It doesn't know if your algorithm is correct. It knows if the agent claimed it was tested when it wasn't.
- It does not do semantic evaluation. "Did the agent rationalise past a rule" requires understanding intent, which requires a model. That's roadmap, not shipped.
- It does not replace testing. It catches the lie that testing happened. You still need the tests.
- It runs in the same trust domain as the agent — tamper-evident, not tamper-proof. The honest security scope is in SECURITY.md.
The honest comparison
| Check | 4-layer manual setup | Groundtruth |
|---|---|---|
| "Tests pass" with no test run | Stop hook that runs tests | Checks transcript for test execution |
| Fabricated import |
tsc PostToolUse hook |
Path resolution against file tree (JS/TS); abstains elsewhere |
| File claimed but not created | ❌ Not covered | ✅ silent no-op detection |
| Stub/TODO left in "done" code | ❌ Not covered | ✅ placeholder scan on added lines |
| Project rules violated | CLAUDE.md (hope-based) | Deterministic regex from your docs |
| Fact-checking claims | Subagent (model-based, correlated) | Diff-anchored (no model, uncorrelated) |
| Config required | CLAUDE.md edits + hooks JSON + subagent file |
/plugin marketplace add + restart |
| Can the agent talk its way out? | Layers 1, 2, 4: yes. Layer 3: no. | No. |
Try it
/plugin marketplace add akahkhanna/groundtruth
/plugin install groundtruth@groundtruth
Restart Claude Code. Every turn gets a warn-only verdict card. No config needed. Arm your project rules with /groundtruth-rules approve-all when you're ready. Turn on blocking with /groundtruth-block on once you trust the precision.
MIT. No LLM. No network. No API key.
The 4-layer article was right about the problem. The fix is one layer, not four — and the layer that works is the one that doesn't involve a model.
I write these while building Groundtruth and EraPin. The 74% false-positive story — how we found it, how we killed it — is in FIXES.md.
Top comments (1)
The abstain-on-package-imports choice is the right call - a checker that false-fires gets ignored within a week, and then it catches nothing. Two edges I would test against the transcript check: does it verify the test command's exit code, and that the run happened AFTER the last write? "Tests pass" can be technically true from a run that predates the final edit, and presence-only matching would bless that stale green. Partial runs are the same shape - a filtered
npm test -- --grep fooshows up in Bash history and reads as "tests ran" while covering one file. Ordering (last test run postdates last mutation) plus exit status looks like the minimal pair that closes both without reintroducing a model.