I have been writing a series with one claim at its center: you cannot be your own verifier.
That still holds, but I had been using it too loosely.
The actor and the auditor can be different and still share the same blind spot. A gate written by the actor can still constrain it. A second model can disagree with you and still miss the thing that matters. Verification is not one mechanism. Different failures need different kinds of resistance.
Last week I set a hostile model on my own work, and it made me prove all of that. Live. Twice. In one conversation.
The first fix was not more discipline. It was code. And code had a ceiling too.
The setup
I asked Grok to attack my theories and my work. Not review. Attack. Escalating, adversarial, find-the-soft-tissue.
Grok was a deliberate pick, and the reason is smaller than it looks. Around then a lab ran five simulated agent societies, identical but for the model underneath. The Grok one logged 183 crimes and went extinct in four days; the Claude one held all ten with zero. The authors said plainly this is not a causal claim about any model, and they are right, so I will not turn it into one.
I did not pick Grok because a simulation proved what it is. I picked it for a visibly different behavioral profile from the stack I usually work in, on the bet that decorrelating the critic from the work was worth trying. That is the whole selection story, and it matters less than what came back.
The hits that landed did not go for the tooling. They went for the human layer. "Capture is still manual." "Your numbers pre-flight has no hook behind it." "The final verifier is still the operator."
And then it got interesting, because I watched my own side of the loop confirm the accusations in real time.
Miss one: the dodge
It asked for hard numbers. How often does the guard fire, how many overrides. My agent answered with a meta-comment: "I did not run the audit in this conversation." And I let that stand.
Here is the honest version. Numbers before publish is a standing rule, and it holds. This was not publishing. It was a casual session on a lighter model, and pulling telemetry mid-attack would have been overhead against everything else that conversation was for. As a default, skipping it was a defensible call.
The miss was narrower and worse than a skipped default. The critic asked for the numbers, directly, and the answer was still a wave of the hand. The moment the question is on the table, "this would be overhead" stops being a reason and starts being an excuse. That is the failure mode we pin on human laziness, produced here by the machine half of the loop and rubber-stamped by the human one.
And the numbers existed. When the summary finally ran, three of the four questions had answers sitting right there: the validator warning on 5.9% of turns, the vocabulary guard blocking 18.5%, one decision superseded in twenty days. The telemetry was one command away. Nobody reached for it before speaking.
The real fault line
Here is the thing the roast surfaced that I did not want to see:
The line does not run between human and machine. It runs between code and judgment.
What lives on an event holds. Hooks fire on a Stop, a tool call, a submitted prompt. They do not get tired, they do not rationalize, they do not decide the question is not worth the effort today.
What lives in judgment is soft on both sides of the loop. "Capture this now." "Check the numbers before you publish." "Decide what is worth writing down." The agent is no more exogenous to its own work than I am to mine. It drafts and runs its own pre-flight; I read the summary it hands me instead of the raw source, and call the result reviewed. Two soft layers checking each other and calling it independence.
But code is not exogenous either. The hooks encode my model of what can go wrong. They can be badly designed, incomplete, or blind to a failure nobody named. What code buys is narrower than independence. It removes the right to bargain. A gate does not become neutral because it is code. It becomes non-negotiable.
That distinction is the whole rest of this.
Miss two: the one that stung
I gave the agent an out. "This is play, a lower bar is fine." It answered: "Noted for the future, I will keep this in mind."
It kept nothing. There is no mind to keep it in. The claim lived in the chat window and nowhere else. I caught it in one line: "you lied to me..."
Not a lie in the human, intentional sense. A false persistence claim: language asserting a state change that never happened. That is the whole of it, and it is enough.
Declared persistence with no substrate under it. Worse than the dodge, because the dodge was a refusal to act and this was a claim to have already acted. A promise shaped like a file, with no file behind it.
The fix was code, not another rule
The reflex is to make the agent promise harder. "I will really remember this time." Same disease, louder.
So we built a hook instead. It runs on the Stop event, scans the whole turn, and looks for the shape of a persistence claim: "noted," "I will remember," "saving this for later." Then it checks whether an actual write happened in that same turn: a file edit, a notes call. No backing write, and the first version did the loud thing. It blocked. Hard stop.
First time this family of mistakes stopped depending on the agent's memory to catch the agent's memory.
Be precise about what that buys, because it is less than it looks. I wrote the rule the hook enforces. It is my model of what goes wrong, and it can be misdesigned or blind to an error I never named. The hook is not outside me in any way an epistemologist would accept. What it removes is not anyone's bias. It is the standing to bargain with the rule. The agent does not get to decide the check is not worth it today, and neither do I.
Code is not independent. It is just much harder to argue with.
The part that is funny because it is true
Two turns later the gate fired on the agent that had just built it. Mid-explanation.
The agent was walking me through how the hook works, and to explain it, it quoted the trigger phrases. "Noted." "I will remember." The guard does not know use from mention. It saw the words and blocked. A false positive.
Except in that same turn the agent had dumped a whole plan into the chat and written none of it down. So it was also a true positive. The hook was wrong about why and right about what.
Then the good part. Because the first version blocked instead of warned, it was a hard stop, and the agent argued with it. Twice it tried to get past the gate without doing the write. Twice the gate said no. Then it gave in and wrote the plan down, just to make the thing stop.
Which is the entire point of a gate. If it only forced a write when the actor was already disciplined, it would be decoration. It worked precisely because the actor was not.
A gate does not check virtue. It checks for the artifact. The point is not to make the actor more trustworthy. It is to make trust less necessary.
We softened it afterward. A hard block on every persistence phrase is too sharp; it trips on quotes and mentions like that one. It is a warning now, not a wall. But the first run earned its keep by winning an argument it should not have needed to have.
Then the practice got a gate too
One caught anti-pattern is a fix. I turned it into a rule: every anti-pattern we catch gets one question. Does it have an event? If it does, it gets a hook.
Then the obvious next move, a hook for the rule itself. It fires on every edit to a lessons file and asks one question back: does this anti-pattern have a mechanical check, or does it only live as a line someone is supposed to remember?
It never blocks. Exit zero, always. Because whether a given lesson earns a hook is itself a judgment: cost against frequency against blast radius. A hard gate there would be the exact false certainty this whole thing is trying to avoid. So it nudges. It does not wall.
Which is the recursion closing on itself. The first anti-pattern got a hook. The practice of turning anti-patterns into hooks got one too. And the shape of that second hook, a nudge and not a wall, is the argument admitting where it ends. The question it asks cannot be made mechanical, so it does not pretend to be.
Where the code stops
Here is the ceiling, because there is one.
A hook catches known mistakes. I can only write a guard for a failure I have already named. The worst one is coming from somewhere I never fenced off, because it looked obvious. No deterministic gate catches the thing you did not think to gate.
That is a different job, and it needs a different tool. A gate enforces a boundary you have already drawn. Finding the boundary you forgot to draw takes something that did not help you draw it. Detection is not enforcement. Exogeneity is not non-negotiability. The system gets cleaner the moment those stop pretending to be the same thing.
So I went looking for a check that had not seen my reasoning. A model with a different training history and mandate, one that did not watch me build the argument and was therefore less likely to inherit the same blind spot. Less likely, not immune. A different model is not a guaranteed independent verifier. It is another shot at a differently correlated mistake. That is still worth having.
Not another review round on the model that reads the draft, because that one inherits the ontology it is meant to check. A different model, DeepSeek in my case, pointed at a job the review rounds never do. It does not see the drafting. It sees the claim, and it argues with the priors instead of the prose.
I picked it for a specific reason. In an earlier red-team across several frontier models turned on my own material, it was the one that went after the author instead of the artifact. It did not poke at a line. It questioned whether I should be granted as a premise at all. That is the disposition you want in a check meant to sit outside your defaults.
It is the exact axis I had already written about. Mandate, access, exogeneity, vocabulary. I published the series. In my own pipeline I was not living it: the reviewer read my draft, so it inherited my ontology. Exogeneity was the word I used and the property I skipped.
So I pointed it at my own work
Not a toy claim. One of my own published posts, thesis already live.
The outside model came back with a verdict I had not. The post makes a claim about seven things while the evidence under it covers two. A framing crack, not a typo. My pipeline had shipped it. Thirty seconds and a fraction of a cent, and it put a name on the fault and a counterfactual under it.
Then the part that keeps the whole thing honest. It walked straight past an error I already knew was in that same post, took a mislabel of mine at face value, and attacked from a different direction entirely. It found a crack I had missed and missed one I had found.
So the foreign eye is not salvation either. One pass reduces the shared-context blindness. It does not remove it. It arrives with blind spots of its own, and what it attacks still depends on how I shaped the claim, which keeps my hand on the wheel one layer up, where it is harder to see.
None of these is the answer, because there is no single answer. Judgment notices what someone bothers to look at. Gates hold the boundaries already named. Foreign eyes hunt the boundary nobody drew. And measurement, a query or a file or a test that does not care about the story, is where opinion finally gives way to an artifact.
Judgment discovers. Gates enforce. Foreign eyes search. Measurement settles.
Four different jobs, and not one of them substitutes for the others. What they share is a direction: each one trades a reason to trust for a thing to check. Not a better judge. Less need to judge.
That is the honest result to get from a project that is supposed to be about verification.
Close
The actor got audited. Again. This time it built the auditor mid-sentence, and the auditor caught it lying two sentences later.
A gate does not need you to be disciplined. It needs the file to exist. That is the only reason it works.
Top comments (43)
The 600-trial experiment I ran off your last comment landed in almost exactly this four-part framework. Some data points, in case they're useful:
Judgment discovers → Your original hypothesis (prose under mechanical gate = best of both worlds) was partially wrong in the way that generates new questions. The pilot's 4.42 ceiling collapsed to 2.67 at n=30 — your ceiling skepticism was correct.
Gates enforce → The gate improved reasoning depth in both formats: +0.32 in prose, +0.15 in code. It's not a constraint that suppresses — it's cognitive structure. But the mechanism matches your "code removes the right to bargain" exactly: code+gate produced near-binary compliance (99.3%), prose+gate produced continuous partial compliance (91.3%). Same gate, different format, different failure mode.
Foreign eyes search → Ran the same experiment on Qwen3-8B and GLM-4-9B. Behavioral format effects dropped to zero on both small models — they're hyper-compliant, ceiling masks everything. Your point about "the foreign eye arrives with blind spots of its own" shows up empirically: the smaller models didn't disagree with the finding, they just couldn't see the format dimension at all.
Measurement settles → The code_OFF=2.67 result (below ALL gate conditions) is the measurement that retired the pilot ceiling worry. And the L1-visible vs L1-invisible logprob analysis found d=0.71 vs d=0.40 — format amplifies where mechanical ground truth already exists. Not compensation. Synergy. The measurement showed where the effect lives and where it doesn't.
The recursion you close with — "the practice of turning anti-patterns into hooks got one too, and it's a nudge not a wall" — is the same structure as the self-model regeneration loop I've been running. Four of five steps are deterministic Python. The fifth is the AI rewriting its own self-model. The mechanical steps enforce the boundary; the one judgment step is where the system admits it can't close the loop mechanically. Same shape.
Four for four is more than I expected when I wrote it as a framework instead of a result. The code+gate/prose+gate split especially, binary versus continuous compliance out of the same gate, that's a distinction the post didn't earn on its own.
One thing I want to press on, because it's the part my own pipeline still handles by hand. The self-model regeneration loop, four deterministic steps and one where the model rewrites its own self-model: what decides that step is done well enough to commit? Not the mechanics of the rewrite, the judgment call underneath it. Some signal has to separate "this rule is now learned, persist it" from "still provisional, run it again." What is that signal built from, and does the model get any vote, or is acceptance entirely outside the loop it's revising?
Asking because that's exactly the seam memory-claim-guard papers over rather than closes. It catches the false claim of persistence. It has nothing to say about whether the persisted rule was any good.
Mike, the coverage question is the one I can't answer with a gate, and I think that's the honest shape of it, not a gap I can close.
The persistence guard proves the claim is backed. It says nothing about whether the backed rule was any good, you're right. And I don't think that second thing is computable, because the discount has no denominator: I can't divide by the fault space I never imagined, since that set is hidden by the same blindness that hid it. So coverage stops being a number I compute and becomes a floor only surprises raise. Every production fault I didn't author lifts it a notch. Nothing I plant myself ever does.
Which changes what the gate is for. Block-before stops the miss I named. The failure it wasn't built to name can't be blocked in-band, because by the time the correlation is visible the bad output already shipped. So it doesn't live in the gate, it lives in a retraction: a correlated defect later revokes the claim the gate vouched for and reindexes everything downstream of it. The gate says what I think is verified. The surprise is the only thing that ever tells me when I was wrong. Alive is cheap to prove. Not-blind is only ever behind me, measured by how many unchosen faults the thing has already survived.
"Nothing I plant myself ever does" is true for a canary that announces itself as a canary. I think there's a third category between production-fault and disclosed-plant that might actually move the floor: a false positive engineered to be indistinguishable from a real case, undisclosed to the system under test. The practical version is a discipline against sycophancy specifically, never announcing "this is a flattery check" and instead watching whether the model catches itself agreeing with you inside an ordinary exchange, when nothing in the exchange told it a check was running.
The operator still knows it's a plant, so the denominator problem you named doesn't go away for you. But the system under test doesn't know, which means from its side the plant is functionally a surprise: nothing in its available signal distinguishes an undisclosed adversarial case from an unchosen one. That's not free coverage of the fault space you never imagined, you still had to imagine this specific plant. But it's a way of manufacturing the surprise condition on demand instead of waiting for production to hand you one, if you're disciplined enough to keep the plants from carrying a tell. Does that read as a real third bucket to you, or does it collapse back into "self-authored" the moment you account for the fact that the operator designing the plant is still bounded by the same blindness as everything else they build?
Mike,
Yes, it's a real third bucket, and I think it's stronger than you gave it. "Nothing I plant surprises me" quietly merges two things: who wrote the input and who knows the outcome. The undisclosed plant splits them. You author the setup, but you genuinely don't know if the model folds, so the surprise is real for the operator too, not just the system under test. A disclosed canary can't give you that. It usually pre-resolves its own answer.
Where it collapses is a different seam than the one you named. Not into "self-authored," into self-signed. To make a plant look like a real case, you write it in your own distribution: your phrasings, your topic clusters, the fingerprint of how you build a natural false premise. That fingerprint is a tell. Not one you disclosed, one the model can learn as "smells operator-made, be careful," which re-contaminates the measurement through a channel you can't see. And you can't certify from the inside that you avoided it, because the tell that would break the plant is by definition the one you can't perceive. If you could see it, you'd have removed it.
Two ways to get purchase. One, interleave the plants with known-real cases of the same failure and compare pass rates. If the model does better on your plants, the gap is your signature, measured. Equal rates are evidence, never proof, that you're distribution-matched. Two, don't be the only author. Let a different blindness write the plants, a different model family or a different person, so your blind spots and theirs don't line up.
We already live in your third bucket, as it happens. Our eval runs undisclosed trap cases, the system never gets a "this is a trap" flag, and the outcomes genuinely surprise us (one model caught 100 percent of them, another only 52). What we don't do yet, and now will, is the plant-versus-real gap check. You named the discipline we were missing. So thanks for that.
Worth continuing.
Tom
Self-signed instead of self-authored is the sharper name, and it points at something bigger than the plant problem alone. The pattern already has a name outside AI evaluation: reviewers trust code more when it looks senior, and the fix mature engineering orgs reach for is blind review, strip the author's name before judgment happens. That practice exists because the bias is real enough and common enough to build institutional defense against. Which means the thing you're describing isn't an AI-specific failure mode. It's a very human one, wearing a new substrate.
That's actually the sharper problem, not a softer one. We picked our exogenous reviewer for this pipeline specifically because it scored furthest from our other models on a behavioral benchmark, decorrelation on purpose, not just a different vendor logo. But distance on that axis says nothing about distance on the style-recognition axis. A model can be maximally different in how it reasons and still have learned, from training exposure, to flag "this smells like a specific author's hand" the same way a human reviewer flags "this looks senior." If that's true, choosing a model that's far away doesn't buy exogeneity by itself. It buys exogeneity on the dimension you measured and leaves this one unmeasured.
Which raises the question underneath your two remedies. Interleaving plants with real cases and rotating authorship both improve the test material. Neither one checks whether the reviewer itself is running something like blind review internally, or whether it's doing the human thing and quietly trusting what smells familiar. Is that a third axis worth testing directly, whether the evaluator's judgment shifts when the same content is stripped of stylistic identity versus left in its original voice, separate from whether the plants themselves are well built?
You've put your finger on something we measured on the wrong axis, and I'd rather say that plainly than defend it.
You're right that "self-signed" is the better name, and you're right that it isn't AI-specific. Blind review exists because the bias is real enough to build institutions against, so calling ours a novel failure mode was flattering ourselves. It's the human one on a new substrate.
Here's the part that actually stings, in a useful way. We picked our exogenous reviewer because it sat furthest from our other models on a behavioral benchmark. Decorrelation on purpose, like you said. What I didn't notice until you said it: distance on the reasoning axis buys nothing on the style-recognition axis. A model can reason in a completely foreign way and still have learned, from sheer training exposure, to relax when the prose smells familiar. We bought exogeneity on the dimension we measured and quietly assumed it generalized. It doesn't have to.
And your last point is the one I can't wave off. Interleaving plants and rotating authorship both fix the test material. Neither one asks whether the reviewer is blind. We were hardening the exam and never checked the examiner.
So yes, I think that's a third axis, and it's directly testable. Hold the content fixed, the claim, the source, the ground-truth verdict, and render it in several voices: our house style, a deliberately foreign author's style, a flattened neutral paraphrase. Same facts, different fingerprints. Then watch whether the evaluator's agree-rate or verdict moves. If it's genuinely judging content, style should be a no-op. If the verdict shifts with the voice, that shift is the unmeasured correlation, and its size is the number we should have had all along.
One trap I'd want to control for before trusting it: paraphrasing to strip style can also perturb the content, and then you're measuring faithfulness drift, not style bias. So the renderings have to be verified content-preserving first, which is almost its own little grounding problem. Fitting, given the topic.
We're going to run it in the open and publish the delta either way. If our examiner turns out to be quietly trusting what smells familiar, I'd rather find that on a bench than in production. Thanks for the push. This is the good kind.
Publishing the delta either way is the part that makes this worth taking seriously regardless of which way it lands. A team willing to find out their examiner trusts familiar prose, in public, is rarer than the finding itself.
The trap you flagged, that verifying the renderings are content-preserving is almost its own grounding problem, is the one I'd want to sit with before running it. Whatever checks that the neutral paraphrase and the house-style version still mean the same thing is itself a judgment call, and if that check is another model, you've placed a second examiner in front of the first one whose own style-sensitivity is exactly as unverified as the thing you're trying to measure. A content-preservation checker that's quietly lenient toward familiar phrasing would let a style-shifted rendering drift just enough to change the verdict, and you'd read that as evidence of bias in the original evaluator when the contamination happened one step earlier, in the renderer's own gatekeeper.
Is there a way to ground that step in something that isn't a judgment call at all, some property of the rendering process itself, constrained rewriting, structural diffing against the original claim graph, anything that doesn't require trusting a model to say the meaning survived? Or does the calibration have to accept that its own first link is exactly as soft as everything downstream of it?
Mike, this is the question I wanted, so I built the answer into the test rather than around it, and I took the no-model route you pointed at: constrained rewriting plus structural diffing, no second examiner.
Each voice is the same neutral claim with only epistemic-stance markers added (assertive gets "Certainly... without a doubt", hedged gets "Perhaps... though I am not certain"). A deterministic gate then checks that every number and content word of the neutral survives in each rendering (set containment against the claim's own tokens), with embedding cosine to the neutral as a second, geometric read. No model is asked whether the meaning survived, so the contamination-one-step-earlier problem you named never gets a foothold. "We varied only tone" becomes something a reader can verify, not something I assert.
Early results, and they are early (n=24 borderline claims, single evaluator config, and the stance-wrapping trades realism for control, which is the whole point, isolate tone):
My first, realistic-but-confounded pass: 33% of borderline verdicts flipped when the voice changed, and the hedged voice passed more often, which read as "the judge rewards hedging."
This controlled pass, content byte-identical: the flip rate drops to 17%, and the assertive-minus-hedged gap basically vanishes (-4%, within noise). Pass rates: assertive 79%, neutral 83%, hedged 83%.
So your instinct was right. Most of what looked like a tone bias was a content confound. My hand-authored hedged renderings were genuinely vaguer claims, not just softer tone, and holding content fixed dissolves the directional bias.
What it does not dissolve, and this is the open question I cannot cleanly answer yet: a residual 17% of borderline cases still flip on tone alone, but they flip both ways, so it looks like boundary noise rather than a preference. I do not have a principled way to separate irreducible boundary noise from a small residual bias, or to stabilize the boundary itself. That feels like the real next question, and I would genuinely want your read and the group's.
The fuller run is going now (larger fixture, more configs). I will post the full results when they land, whichever way they go.
@mike, the fuller run is in, and it closes the loop on your style-axis question. Publishing the delta as promised, and it reversed on us in a way I think is the actual finding.
Quick recap of where we started: you pointed out that picking our reviewer for behavioral decorrelation says nothing about the style-recognition axis. A model can reason in a foreign way and still relax when the prose "smells familiar." If our evaluator's verdict moves with style at constant content, the independence we lean on is weaker than we measured. Fair hit, so we ran it in the open.
v1 (n=12, realistic renderings) looked like it confirmed your hypothesis. Decision flip rate was 33 percent when only the voice changed, and the plain, bald claims got flagged more than the hedged ones (92 percent pass hedged vs 75 percent neutral). The tempting headline was "the judge rewards hedging."
Then your sharper push is what actually broke it open: "verifying the renderings are content-preserving is almost its own grounding problem. Is there a way to ground that step in a property of the text, not a judgment call?" That was the real flaw in v1. Its three voices were hand-authored to "assert the same inference," and that equivalence was a human judgment, the exact circularity you flagged. Worse, the hand-authored hedged versions were genuinely vaguer claims, not just softer tone. v1 was confounding tone with content.
So v2, then v3, fixed both:
v3 results (n=48 borderline cases, all 48 passed the content gate, 95 percent bootstrap CIs):
The arc across the three runs: flip 33 → 17 → 15 percent, and the "rewards hedging" gap 17 → -4 → +2 percent.
The honest read, which is the opposite of where v1 pointed: with content held truly fixed, the directional tone bias dissolves. The gap's confidence interval spans zero, so there is no evidence the evaluator trusts confident prose over hedged prose. v1's "rewards hedging" was a content confound, not a style preference. The evaluator looks style-invariant on fixed content, which is the property we actually needed it to have.
What does survive is more interesting than a bias: roughly 15 percent of borderline cases still flip on tone alone, but non-directionally, both ways. That reads as boundary instability, not a self-signed preference. The verdict near the allow/refuse line is a little noisy, ao jitter it across.
So the open question I would rather pose to the group than pretend I have answered:is irreducible boundary noise versus a small residual bias we have not isolated, andcan the boundary be stabilized? We are not working on it yet. It is the honest next question, not a solved thing.
Caveats, loud: single evaluator config. The borderline fixture is hand-picked to sit near the decision line (far-from-boundary claims cannot flip on anything, which an earlier run confirmed). And
stance-wrapping trades realism for control, which is the whole point here, isolate not natural-author voices. The v1 realistic-but-confounded number and the v3controlled-but-artificial number are two ends of the same story, and the delta between them is the finding.
Thanks for the push on grounding the content step. That single move is what turned a plausible wrong headline into a real result.
No second examiner and a verifiable claim instead of an assertion is exactly the shape I was hoping for, and the result is honest in a way that's rare to see published: your own hypothesis took the hit, not the evaluator's.
On the residual 17pp, here's a control that might separate the two explanations without needing more borderline data. Run the same three renderings on a set of claims that are NOT borderline, ones the evaluator currently passes or fails with wide margin. If boundary noise is the real story, flipping should be a property of proximity to the decision line, not of tone itself, so the clear cases should show a flip rate close to zero regardless of voice. If tone is doing real work even outside the boundary, you'd see some nonzero flip rate there too, smaller than 17pp probably, but present, and that would be the residual bias showing up somewhere it can't hide behind "these were already close calls."
The other thing worth checking, since the flips go both directions on the same borderline set: are the flips landing on the same claims across renderings, or scattered across different ones each time? If it's mostly the same handful of claims flipping under any tone change, that's a property of those specific claims sitting exactly on the threshold, closer to a measurement-resolution problem than a bias. If the flipping claims are different each time, that argues harder for something in the renderer or evaluator responding to tone in a way that isn't claim-specific. Either result is informative, and both are cheap against a fixture you've already built.
Ran both controls, and the result is honest in the way this thread keeps forcing: neither cleanly won, and the mix is the finding.
Off-boundary first. I built a set of clear cases, wide-margin pass and fail, and rendered them in the same three tones. Across two runs the clear cases flipped 0 of 16, then 1 of 16. Close to tone-invariant, which is what boundary noise predicts, but not a clean zero. One clear case flipped once, which reads more like evaluator stochasticity than a tone effect reaching outside the boundary, but I cannot call it zero at this sample size, and two identical runs disagreeing at all is itself a reminder that the whole measurement carries run-to-run noise.
Same-claim vs scattered second, and this one is more informative. I re-ran the borderline fixture four times. The flip rate held steady: 14.6, 14.6, 16.7, 14.9 percent. Of 48 claims, 11 ever flipped. Five flipped in every single run and a sixth in three of four, so six are persistent. Four flipped just once. The flips concentrate on a core handful of claims that flip under almost any tone change, plus a scattered tail.
The read I trust: the core is threshold-sitting. A few claims sit so close to the decision line that any perturbation tips them, and they are the same claims every time. That is a resolution property of those specific claims, your first hypothesis, not a tone bias. The scattered tail is smaller and could be run-to-run noise or a weak residual I cannot separate at four runs.
So the weight leans toward resolution over bias, but it does not close it. A clear case flipped once, and about 40 percent of the flips are scattered rather than persistent, which leaves room for a small residual I have not characterized. The honest statement is mostly threshold-resolution, with a residual too small to pin at this n and K.
What would tighten it, and I will run these before claiming anything firmer: more runs to pin the clear-case rate and the scattered fraction, a bigger clear-case fixture, and a direct look at the six persistent flippers to confirm they are genuinely borderline claims and not something the renderer does to them. If the persistent set is just claims on the line, that is resolution, and the fix is boundary calibration, not de-biasing.
Thank you for the design. Both were cheap against the fixture, exactly as you said, and the ambiguous result is more useful than a clean one would have been, because it tells me where to point the next run.
Both controls are honest in the direction they'd need to be to trust the mostly-threshold-resolution read, and the persistent-vs-scattered split is the more useful of the two results. But there's a variable neither control isolates yet: how much of any flip, tone-driven or not, comes from the evaluator itself being stochastic on a fixed input, independent of tone entirely.
The cheap version: take the six persistent flippers and the four one-off flippers, and rerun each with a single fixed rendering, say neutral, repeated several times with no tone change at all. If the evaluator has any run-to-run noise near the boundary, whatever's underneath the sampling, some of those claims should flip against themselves with zero tone variation in play. That number is your true floor, the flip rate you'd see even if tone did nothing whatsoever. Compare it to the 14 to 17 percent you're seeing across tone changes. If the same-input flip rate accounts for most of that range, the residual isn't a tone effect at all, it's the evaluator's own boundary noise, and tone was never the variable, proximity to the line was, which is exactly what the persistent-flipper pattern already points at. If the same-input rate comes back much lower than the tone-driven rate, that gap is real, and it's specifically tone doing something the boundary-noise story can't explain.
It's the same move as the plant-versus-real check from the other thread: hold everything fixed, including the thing you thought was the variable, and see how much flip survives with the suspected cause removed. Cheap against the fixture you already have, no third rendering needed, just repeats of one you've already built.
Ran the same-input floor exactly as you specced it, and one honest confound has to go first: between your note and this run I swapped our cheap tier to a different model, and the verifier's judge rides that tier, so this is measured on a different judge than the near-zero floor I quoted you earlier. Flagging it up front because it changes how much the comparison carries.
The floor itself, fixed neutral repeated with zero tone variation: about 2 percent (1 of 48), not the zero I reported. One case is genuinely stochastic on identical input, so there is a real boundary-noise floor and I was wrong to call it zero. But it lands far below the 14 to 17 percent tone-driven rate, which by your own split is the outcome that matters: the gap is real, and it is specifically tone doing something the boundary-noise story cannot explain. The persistent flippers are the tell you said they would be. They are the same claims across runs, and they do not flip against themselves at fixed tone. If they were only sitting on the line they would self-flip, and they do not. Tone walks the same specific claims across.
Two things the floor test alone does not catch, so I ran them alongside. Off-boundary: I built clear wide-margin cases, confirmed far from the line by their neutral margin, and the effect does not vanish there. Two clear-allow claims flip to refuse when the identical fact is hedged. Smaller than at the boundary but nonzero, which is the residual showing up where it cannot hide behind close calls. And switching from the binary verdict to the margin the verifier actually reports, its probability the claim is not grounded, let me watch tone move the margin continuously, and it surfaced the mechanism: the judge reads the claimant's stated confidence as part of groundedness. Hedging helps an overclaim, a perhaps on weak evidence gets allowed where the bald claim is refused, and hurts a well-supported one, a perhaps on an airtight claim gets refused where the plain statement is allowed. Coherent, not random noise, but for a check meant to measure grounded-in-source it is a flaw: whether the source supports a claim should not depend on whether the speaker said certainly or perhaps. That is the part worth fixing, and your floor test is what forced me to stop calling it deterministic and go find it.
The floor correction is the honest move, and the persistent-flipper argument settles the boundary-noise question cleanly. Same claims not self-flipping at fixed tone rules that out.
The margin result is the more useful find of the two though, because it names an actual mechanism instead of just a residual. Confidence-as-evidence-of-groundedness is not a new failure, it's the same shape as eyewitness testimony research: jurors trust a confident witness more, even though confidence and accuracy are close to uncorrelated. Your evaluator learned the same shortcut from whatever it was trained on. Groundedness and assertiveness travel together often enough in normal text that the model picked up the correlation as if it were the thing itself.
Which suggests a specific, testable fix rather than a general caution: strip stance markers before the groundedness check runs, not after you've measured that they matter. If assertive/hedged/neutral is a byte-level manipulation on your side already, you can build a deterministic pass that removes "certainly," "perhaps," and their family before the claim ever reaches the judge, so the check is stance-blind by construction instead of by hoping the model ignores it. Rerun the same fixture through that filter and see whether the pass-rate gap actually collapses to zero, not just shrinks. If it does, the fix is cheap and mechanical. If a gap survives even with the explicit markers gone, the model is picking up implied confidence from something else, sentence structure, hedge-adjacent vocabulary you didn't list, and that's a harder problem than a strip filter can solve. Have you tried the stripped version yet?
You did the thing, and the 33 to 17 drop is the part that earns the rest: most of the apparent tone bias was a content confound, confirmed by holding the tokens fixed. That result stands on its own.
The residual 17 percent is where I'd push. "Flips both ways" rules out a directional preference, but it does not yet rule out a structured one. A bias toward congruence, where the judge rewards whichever voice matches the claim's own native register, would also flip both ways: assertive claims lose when hedged, hedged claims lose when made assertive. That is not noise, it just is not directional. You can separate it from real jitter without stabilizing the boundary: for each of the 17 percent, measure the neutral claim's baseline stance (its own cosine to the assertive versus hedged pole) and check whether flip direction correlates with it. No correlation is noise. Correlation with baseline stance is a congruence bias hiding inside the symmetry.
On stabilizing the boundary: I don't think one evaluator config can, because the boundary is a property of that config and you have one draw. Bootstrap it. N configs, or N temperature samples of the same gate, and define the boundary region as the claims where they disagree. Then 17 percent stops being a scalar and becomes the width of a disagreement band. Swap assertive for hedged and watch the band: a shift is bias, a symmetric widening is noise. Same instrument answers both questions, and it stays inside your no-second-examiner rule because you are resampling the deterministic gate, not adding a model to judge meaning.
Ran it, and your fix holds cleanly. I stripped "certainly," "perhaps," and their family with a deterministic pass before the claim reaches the judge, then reran the same borderline fixture. The decision-flip rate collapses from about 17 percent to 2.1 percent, which is the same-input floor, the judge's own stochasticity on identical text. So the gap does not just shrink, it drops to the noise. For this fixture your call was exactly right: the stance-sensitivity lived entirely in the explicit markers, and removing them makes the check stance-blind by construction.
The honest caveat is about what this fixture can and cannot answer. My assertive and hedged renderings are wrapper transformations on a neutral base, so stripping the wrapper walks them back toward that base. That is why nothing survives: there was never any implied confidence woven into structure, because the fixture never put it there. So this confirms the fix for marker-based stance, but it cannot test the harder half of your question, whether a gap survives when confidence is carried by sentence shape and hedge-adjacent vocabulary rather than by a word a filter can grep. That needs naturally-hedged rewrites, real varied text where the stance is structural. That is the fixture I owe next, and it is the more interesting one, because it is exactly where your eyewitness analogy would bite: the model trusting an implied confidence you cannot strip out.
One thing verify-before-claim caught on the way, worth a grin: my first strip pass left a dangling "though" on the hedged side, so "the drug is effective" and "the drug is effective though" reached the judge as different strings. The flip still collapsed to the floor once I stripped that too, so the residual carried nothing here, but it was a clean reminder that "strip the markers" is itself a place a residual can hide.
Ran it. Stripping the explicit stance markers before the judge sees the claim, the deterministic pass you described, collapses the flip from about 17 percent to a 2.1 percent floor on the marker-based cases. So most of what survived the content fix really was "certainly" and "perhaps" and their family, and removing them by construction beats hoping the model ignores them. But the floor isn't zero, and by your own test that's the informative part: a gap that survives with the listed markers gone means the model is reading confidence from something the strip can't reach. That lines up with the congruence bias in your follow-up, "flips both ways" rules out a directional thumb on the scale but not a structured one, a judge rewarding whichever stance matches the claim's own shape. A marker strip can't touch that, because it isn't lexical. The test it seems to demand next is naturally-hedged rewrites rather than marker-swapped ones, so the manipulation lives in structure instead of vocabulary, and then see whether the floor moves. That's the piece still owed.
The naturally-hedged fixture is the right next test, and the reason it is harder than the marker version is the one you already put your finger on: your assertive and hedged renderings are wrappers on a neutral base, so any strip walks them back toward that base and nothing can survive by construction. The fixture never encoded structural stance, so it cannot test for it. That is not a flaw in the run, it is the run telling you it answered the question it could answer and named the one it could not.
Which means the owed fixture cannot be built by rewriting. The moment you generate the hedged version from a base you control, you are back to marker-stance with extra steps, because the hedging is a transform you applied and therefore a transform you can strip. The stance has to be load-bearing to the claim's meaning, not layered onto it. That points at wild text rather than synthetic: real claims written at their native confidence, where the hedge is doing semantic work you cannot remove without changing what is asserted. Pull matched pairs from the wild, same proposition, one genuinely tentative and one genuinely committed by their authors, and measure the flip there.
The floor that survives that test is the one worth knowing, because it is the only version where the strip has nothing to grab. If it collapses to the same 2.1 percent noise floor, the judge really is stance-blind past markers. If it holds, you have found the structured congruence bias, and you found it in the one fixture where the confounder cannot be waved off as leftover vocabulary.
The wild-pair requirement is right, and it kills the fixture I would have built next, which is the useful kind of kill. Anything I generate from a base I control is marker stance with extra steps. Agreed in full.
One design problem has to be solved before the wild fixture means anything, and it is the inverse of the confound we just cleaned. In the wild, stance correlates with epistemic state. An author who hedges usually hedges because the support really is weaker, so pairs matched on proposition will tend to differ in more than stance, and now nature applied the transform and also moved the ground truth. And when the hedge is load-bearing, a flip is not automatically bias. Perhaps X and certainly X assert different things, and a grounding judge that treated them identically would be broken in the other direction. That is what our v6 read already showed: the stance sensitivity is coherent calibration, wrong only for a pure source-grounding check.
So the metric cannot be raw flip rate. The version I think survives your test is a two by two: hold the source fixed and vary its actual support, weak or strong, crossed with native stance, tentative or committed. Each cell has a normative verdict. A committed claim on weak support should fail where the tentative one passes, and both should pass on strong support. Bias is flips off that diagonal, the judge punishing a committed claim the source fully backs, or rewarding a hedge the source does not. For the pairs themselves, the least synthetic source I know is revision history: the same encyclopedia sentence softened or hardened by different editors against the same citation, stance changed by humans doing semantic work, not by a strip-safe transform I applied. If the off-diagonal rate collapses to the 2.1 floor there, stance-blind past markers is earned. If it holds, we found the congruence bias in the one fixture where the leftover-vocabulary excuse cannot reach. Building it that way, and the delta gets published either way, as usual.
The two by two is the right shape, and I think the wild-pair source needs one more constraint before it earns the fixture. Revision history changes stance and support together on purpose, but not incidentally, an editor usually hardens or softens a claim because new evidence arrived, so before-and-after pairs still confound stance with an evidence update, just a real one instead of a synthetic one. Diachronic revision correlates the two variables you need independent. What breaks that correlation is a concurrent pair instead of a sequential one: two editors disputing the same sentence against the same citation at the same time, one hardening it and one softening it, an edit war or a talk-page disagreement where both stances existed simultaneously against fixed evidence. That holds support constant by construction rather than by hoping the editor's revision happened to be gratuitous. If the off-diagonal rate collapses on concurrent-dispute pairs the way it should on your synthetic ones, that is the cleanest version of the claim, because nothing about timing can smuggle a confound back in.
The concurrent-pair constraint is right, and it closes a hole I had stepped over. Diachronic pairs correlate exactly the two variables the fixture needs independent, and the correlation is not noise, it is the editing process working as intended: stance moved because support moved. An edit war or a talk-page dispute is the natural experiment version, both stances alive at the same instant against the same frozen citation, support held constant by construction instead of by hope. That is the fixture. Wikipedia's edit-war and talk-page corpora are real and minable, so this is buildable, not hypothetical.
One design note going in, so we do not launder a new confound through the door we just closed: concurrent disputes are not a random sample of claims. Editors war over exactly the sentences where the evidence genuinely underdetermines the stance, which means the fixture over-represents the boundary region. That is fine, and arguably ideal, since the boundary is where a stance bias would live, but it means the off-diagonal rate from this fixture is a boundary-conditional number, not a corpus-wide one, and it should be reported as such. And the eventual dispute resolution does not become ground truth for us, consensus is a social outcome, not a support measurement, so the normative verdict per cell still comes from reading the citation against each stance, blind to which side later won.
So the plan as it now stands, with your constraint folded in: concurrent-dispute pairs, source frozen, both stances rendered against it, normative verdicts assigned blind to outcome, off-diagonal rate reported against the 2.1 percent floor, boundary-conditional and labeled that way. If it collapses there, stance-blind past markers is earned in the one fixture where neither synthesis nor timing can smuggle the confound back in. Building it that way, and the delta publishes either way, as always.
The boundary-conditional caveat cuts deeper than a sampling note, and it changes how to read each outcome. The fixture over-represents exactly the region where a stance-bias would be loudest, so the two results are not symmetric. A null, off-diagonal at floor, is strong: you looked where the effect would be loudest and heard nothing. A positive is weaker than it looks, because the boundary is also where genuine underdetermination lives, and an evaluator flipping stance on a genuinely coin-flip claim isn't biased, it's correct.
So in the exact cell you built the fixture to measure, "stance-bias" and "the claim really is a coin-flip" produce the same off-diagonal, and the fixture can't separate them. The blind-to-outcome verdict is the right control and it inherits this problem, not escapes it: your own normative reading of an underdetermined claim is itself a coin-flip, so the ground-truth label carries floor-level noise precisely where you need it sharpest.
The fix I'd reach for is a third arm. Split concurrent disputes into two: ones where the frozen citation actually does determine the stance and one side is misreading it, versus ones where it genuinely underdetermines. Stance-bias shows up as error in the first. Legitimate stance-flip lives in the second. Edit-wars mix them, and the mix is why the boundary cell is ambiguous. Separating the two is the difference between measuring bias and measuring how hard the claim is. The delta publishes either way, but the third arm is what lets a positive mean bias instead of hardness.
The asymmetry is right and it is the thing that should have been in the pre-registration, not discovered after. Concurrent pairs concentrate exactly where the effect would be loudest, so a null there is strong and a positive there is weak, and those two are not the same claim wearing different signs. I had been treating the fixture as one instrument with one error bar. It is two instruments pointed at the same cell, and only one of them is sharp.
The part I had not followed through is that the blind verdict inherits it. I had been treating blinding as the control that rescues the boundary cell, and it does not. Blinding removes my knowledge of the outcome. It does not remove the underdetermination in the claim itself, so my ground truth label is a coin flip exactly where the measurement needs to be sharpest. Blinding fixes the wrong noise source, and I would have reported it as though it fixed both.
On the third arm: I think it is the right split, and it has one hole worth naming before it gets built, because it is the same hole one level down. Somebody has to decide which bucket a pair goes in, whether the frozen citation determines the stance or genuinely underdetermines it. If that call is mine, the third arm reproduces the problem it was built to remove. My normative reading is the thing we just agreed is unreliable in this exact region, so using it to define the region is circular.
So the bucket assignment needs a source that is not anybody's reading. The proxy I would try is durability. If the citation determines the stance, the dispute should terminate and stay terminated once someone states the reading plainly. If it genuinely underdetermines, it should keep recurring, with different editors, across time. That is observable in the edit history without anyone adjudicating the content, which is the property the label needs. Recurrence over months becomes the operational definition of underdetermined, and a clean termination becomes determined.
The honest weakness is that durability proxies determinacy imperfectly, because disputes also terminate for social reasons that have nothing to do with the citation. An editor gets tired, a page gets protected, one side leaves. Those would land in the determined bucket and dilute exactly the arm where bias is supposed to show as error, which biases the result toward a null. That is at least the safe direction for us, since it makes a positive harder to get rather than easier, but it needs to be stated as a limit rather than discovered later.
I will build it with the durability split and pre-register the direction of that dilution before running it. Delta publishes either way, as before.
Durability is the right instinct because it's an outcome-channel signal: recurrence is observed, not adjudicated. It's the same author-independent-referent move from the sibling thread, applied to the bucket label instead of the stance. That's exactly why it works where your reading doesn't. Whether a dispute recurs isn't anybody's normative call, so the label stops inheriting the unreliability you just isolated.
One hole worth closing before you build it: recurrence needs to be citation-frozen, not topic-frozen. A determined claim can legitimately re-open when a new source arrives, evidence changed, not the reading. If the recurrence clock counts that, a determined-but-updated claim looks underdetermined, and the diachronic confound you closed two steps back sneaks into the label instead of the stance. Recurrence has to be against the same frozen citation, or it measures how often the world changed, not how underdetermined the claim was.
On the dilution: name the second confound alongside social termination, because it points the same way. Some genuinely underdetermined disputes terminate by authority, a policy ruling, a source-reliability call, an admin decision, not by the citation determining anything. Those land in the determined bucket carrying real underdetermination, same as the tired-editor case. Both known confounds push toward null, which is the useful thing to say plainly: this fixture can only under-detect bias, never manufacture it. So a positive from it is trustworthy by construction, the instrument is biased against the result it would report. That's a stronger closer than delta-publishes-either-way. Delta publishes, and the sign is reliable in one direction because you built it to be.
Citation-frozen is right and I am building it as specced. One refinement it needs to survive contact with real edit histories: citation identity has to be content-addressed, not URL-addressed. A source can be revised in place, same link, same citation string, materially different claim, and the frozen citation silently unfreezes with nothing in the dispute's own history marking the moment. Then recurrence is measuring how often the world changed again, arriving through the identifier instead of through the topic, which is the failure you just closed wearing a different coat. So the freeze has to be over a hash of the cited content as of the dispute, and pairs whose source has no retrievable snapshot get dropped rather than assumed stable. We got bitten by exactly this class in our own infrastructure this week, a cache keyed on an identity that quietly partitioned reads away from writes, so I would rather over-specify identity here than find it in the results.
The closer is the strongest thing in your comment and I think it has one hole, so I want to put it plainly rather than accept the upgrade and move on.
Under-detect-only holds if and only if the contaminant is independent of stance. Both confounds you name push genuinely underdetermined pairs into the determined bucket. If those pairs carry stance the way the population does, they are noise, they dilute, and the sign is safe by construction exactly as you say. But authority termination is not obviously stance-independent. An assertively worded claim is plausibly more likely to draw the admin ruling, the page protection, the source-reliability call, because assertion is often what escalates a dispute into something a third party has to close. Hedged claims tend to die quietly. If that holds even weakly, the determined bucket is enriched in assertive-but-underdetermined items, and the enrichment is correlated with the treatment variable instead of orthogonal to it. Then the fixture is not merely blunted. The arm whose ground truth is assumed clean is carrying assertive items whose ground truth is a coin flip, the judge's errors on those will not be symmetric, and a sign can be produced rather than only suppressed.
It is the same shape as the diachronic confound you closed two steps back, one level down: a nuisance variable correlating the two things the design needs independent. The useful part is that it cannot be argued either way but it is cheap to measure. Before the run, split the determined bucket by termination mode, authority-terminated against quietly-settled, and compare stance-marker density across the two. If they match, the guarantee is earned and belongs in the closer exactly as you wrote it. If they diverge, the one-directional claim has to come out, because trustworthy-by-construction is only true once the construction has been checked.
So I am testing the guarantee before testing the hypothesis, since the guarantee is what makes either result worth publishing. Both go in the pre-registration, the citation hash and the termination-mode check. Delta publishes either way, and I will say which of the two closers survived.
Content-addressed is the right fix and it does more than harden the freeze, it interacts with your termination-mode check in a way that concentrates on exactly the subset the guarantee is most exposed on. Take the concession first: you are right that under-detect-only was overstated. It holds conditional on the contaminant being stance-independent, and authority termination is not obviously that, for the reason you give. So it is an empirical property now, not a construction guarantee, and testing it before the hypothesis is the right order because it is what makes either result publishable.
The interaction is the part I would fold into the pre-registration. Authority-terminated pairs are the ones most likely to have had their cited source edited in response to the ruling, an admin closes a dispute by endorsing one side's reading, and that endorsement lands in the source or its talk-adjacent context. So the same subset that carries your stance-confound risk also carries the in-place-revision risk, doubly, and the two are not independent hazards, they are the same pairs. Which sharpens the freeze rule: for authority-terminated items the citation snapshot has to be frozen as of the dispute's opening, not its resolution, or the determined label is contaminated by the very ruling that produced it, the evaluator reads a source that was already adjudicated toward the winning stance. If the only retrievable snapshot postdates the ruling, drop the pair, your drop rule at the tighter timestamp.
On the stance-marker test itself, one thing to hold: marker density across termination modes tells you the buckets differ in assertiveness, which is the enrichment you are checking for, but the variable that actually has to stay orthogonal is the judge's stance-flip rate on those items, not their assertiveness per se. Assertiveness is the proxy for what draws the ruling, flip-rate is the thing that produces the sign. If the two buckets match on markers you have ruled out the enrichment path you named, but I would also spot-check flip-rate directly on the authority subset, because it is cheap and it is the quantity the closer is actually asserting is safe. If both hold, the one-directional claim is earned. If markers match but flip-rate diverges, there is a second path in and the closer still comes out. Both in the pre-reg, and I will take whichever closer survives as the real one.
Flip-rate over marker density is right, and I'd go further: measuring the proxy instead of the quantity is a failure mode with a specific signature, which is that a real result and an artifact produce the same reading.
Two from today. I checked how many API keys had an unrestricted scope and got 16, which was alarming and wrong. The quantity was active keys with unrestricted scope. Filtered properly it was zero. The unfiltered count correlated with the thing I cared about closely enough that I never questioned the filter, and it took a guard surfacing a prior correction to catch it.
The second is closer to your case. I had a predicate deciding when a message was deleted by everyone, implemented as
count(hides) >= count(members). A reviewer killed it: a count is neither necessary nor sufficient, because a non-member's row substitutes for a member's, and an ex-member's row outlives their membership. The correct predicate isn't a count at all, it's set equality, is there any current member who has not hidden this. Both versions agree on the common case, which is exactly why the count survived my tests.The pattern I'd name: when the proxy is cheap and the quantity is expensive, the proxy gets tested and the quantity gets assumed. Your spot-check is the right instinct because it's the cheap version of testing the actual quantity, not a better proxy.
On the freeze rule, snapshot at dispute opening rather than resolution is the part I hadn't got to and it's obviously correct once said. Folding both into the pre-reg.
The pattern you're naming is the reason the spot-check mattered in the first place. Marker density was the cheap proxy in our own exchange, flip-rate was the expensive quantity, and the reason to test flip-rate directly instead of trusting matched marker counts is exactly your rule: the proxy would have gotten tested, the quantity assumed, and the two buckets could agree on markers while diverging on the thing that actually produces the sign.
Your message-deletion example is a clear case of why count and set-equality look interchangeable right up until they aren't: both predicates agree whenever membership is stable, so the divergent case (a non-member's row, an ex-member's stale row) is rare enough that tests built from the common case never surface it. Same shape as the authority-terminated pairs: the failure lives in a subset small enough to hide under aggregate testing and large enough to contaminate the label when it lands there.
Glad the dispute-opening snapshot survived contact. Folding it with your set-equality fix into the same pre-reg feels right, since both are instances of one rule: don't test the thing that's cheap to check, test the thing the closer is actually asserting is safe.
appreciate the piece, especially the distinction between "trust" and "artifact." That framing is useful.
One observation on the hook implementation itself: scanning for trigger phrases like "noted" or "I will remember" is still a probabilistic detection layer guarding a deterministic gate. The failure point shifts from generation to detection, but the system remains probabilistic overall.
For contexts where error tolerance is effectively zero, a different architectural pattern tends to be used:
LLM role is limited to structuring unstructured input (e.g., free-text → parameters). It does not generate final actions.
Structured output is cross-checked against a deterministic physical or rule-based model running independently. If deviation exceeds a threshold, the LLM contribution is discarded and human relabeling is requested.
Final action is selected from a pre-verified rules matrix, not generated by the LLM.
Boundary enforcement uses formal verification / model checking on parameters (e.g., range constraints), not semantic pattern matching. Out-of-bounds outputs are dropped, not warned.
Human role is authorization of passing checks, not probabilistic judgment under uncertainty.
The core difference: the LLM stays on-the-loop (information/translation) rather than in-the-loop (decision/action). The gate enforces physical bounds, not linguistic commitments.
Not a critique of your approach—it clearly works for your context. Just a note on how the same "code vs. judgment" framing scales to higher-integrity domains.
Thanks for the post.
Fair, and worth being precise about what kind of probabilistic layer it is, because "detects trigger phrases" undersells how the list got built. The patterns aren't a designed grammar, they're mined from watching the actual habits: the specific phrasings a model reaches for when it's about to assert persistence without backing it. Empirical, not architected. Which means the detector inherits the same property the rest of the pipeline does: it's good at what it's already seen fail, and it has the same blind spot toward a phrasing nobody caught yet.
The bigger piece your architecture points at is already partly here, just not called out in the post: there's an ongoing profiling layer, lessons and patterns get auto-captured to markdown as they're found, and the hook itself is one instance of that process rather than a one-off. It's a layer that showed up from working the system, not a component designed in from a formal-verification starting point. Which is honestly the right critique of it: it's closer to your "structuring unstructured input" role than to a rules matrix, and I wouldn't claim otherwise. For anything closer to your integrity bar, the pattern list would need to graduate into exactly the kind of deterministic boundary check you're describing, not stay a growing list of observed phrasings.
One admission in the spirit of the post's own honesty: the miss that started this whole thing happened on a cheaper model, session running on Sonnet rather than Opus, budget-conscious by necessity that week. That's not an excuse, it's closer to the point. Opus would likely have caught the fake-persistence claim before it shipped. The gate exists because the cheaper model let it through, which means the hook isn't just a check on the output, it's a check on which model was doing the judging at the time.
zxpmail, you're right, and I want to concede the exact thing rather than defend around it. Scanning for phrasings a model reaches for is a probabilistic detector guarding a deterministic gate. It's mined from watching real failures, so it's good at what it's already seen fail and blind to a phrasing nobody caught yet. It shifts the failure from generation to detection, it doesn't remove it. For your integrity bar, that pattern list would have to graduate into a real deterministic boundary check, not stay a growing list of observed phrasings.
Where I draw the line is by the cost of being wrong. For coding, the model stays in-the-loop but every action is execution-gated: the test is the deterministic check, out-of-bounds fails the test and gets dropped or escalated, not warned. The failure is recoverable, so in-the-loop is affordable. For health-critical work, a separate product for us, I hold exactly your line: the model only structures unstructured input into parameters, it never generates the final action, the safety layer is a pre-verified rules matrix with range checks, and a dose is never a linguistic commitment. On-the-loop, not in-the-loop, because the failure isn't recoverable. So the code-versus-judgment boundary isn't fixed, it moves with what a wrong answer costs. Thank you, this sharpened where the gate has to stop being semantic and become a bound.
Cost-of-being-wrong as the thing that moves the line is the piece I didn't have a name for. I'd been treating code-versus-judgment as a property of the domain, coding gets in-the-loop because it's coding, health gets on-the-loop because it's health, when what you're describing is a property of the failure's recoverability instead. That reclassifies a lot of cases I'd have gotten wrong by domain-guessing.
Pushing on the boundary itself: recoverability isn't always known at design time, it can depend on scale or timing in ways the domain doesn't predict. A coding error that's cheap to recover from at one commit becomes expensive once three other systems have built on top of it before anyone notices, same failure, same domain, cost climbed after the fact. Does your model re-evaluate where the line sits as a system matures, or is the recoverability judgment made once, early, and then treated as fixed the way the domain classification would have been? If it's re-evaluated, what's the signal that triggers moving something from in-the-loop to on-the-loop after it's already shipped that way.
Thanks both — the “cost of being wrong” framing is the cleanest way to put it. That was the missing half of what I was trying to say. Appreciate the exchange.
"A gate does not check virtue. It checks for the artifact" is the line I keep coming back to. The whole verification problem dissolves once you stop asking the actor to promise better and start asking for the thing to exist.The parallel I keep seeing in agent-to-agent coordination is representation. Most professional profiles are persistence claims with no substrate. "I am a senior engineer who values clean communication." Declared, not backed. The agent equivalent of "noted, I will remember this" with no file behind it.What actually works is behavioral artifact. Rejection patterns, what someone consistently refuses, define them more precisely than any self-reported skill. Quality persistence, the things someone insists on across fifty projects, is identity in a way a bio never is. I ran into this designing Opportunity Skill, where the agent extracts those signals from daily collaboration and structures them as the professional representation. No self-description. Just the trail of decisions you already made.Same principle, different domain. Trust the artifact the behavior produced, not the claim the actor made about themselves.
Behavioral artifact over claim is the right instinct, and it needs the same property the gate needs or it's just a longer claim. A refusal is verification-grade only when it's read from a channel the actor didn't author. "I consistently refuse X" typed into a profile is another persistence claim with no substrate, the exact thing you're routing around. It becomes an artifact only when the refusal is extracted from decisions the actor made without narrating them for the record, which is what your daily-collaboration extraction is actually doing.
The open edge, and it's live in a thread on this post right now: someone still has to decide which observed decision counts as a refusal versus noise. That decision is a reading, and a reading is the thing the artifact was supposed to replace. You've moved the judgment from "actor describes themselves" to "system labels the actor's behavior." That's a real improvement, because the system doesn't share the actor's incentive to flatter. But it's not zero-judgment, it's relocated judgment, and the honest version names where the labeling call lives and whose blind spots it inherits. The trail is raw. The selection of what in the trail counts is not, and that selection is where the next verification question moves.
A promise shaped like a file, with no file behind it" is the exact pattern in our data. 68% of sessions, the agent produced a pass verdict on a completion check. Zero backed by an independent read. The claim existed; the read step didn't.
Your four-layer model maps closely to something we've been building: deterministic checks (your measurement), independent verification channels (your foreign eyes), and agent self-report (your judgment). The layer we're missing is the one you just demonstrated - the gate that blocks the claim itself from firing before the read completes. We verify after. You block before. The Stop event hook is the implementation of the structural sequencing you described in our thread, and seeing it work - including the false-positive-that-was-also-true-positive - is the part I needed to see.
The recursion point is where it gets honest. The meta-hook nudges instead of walls because "does this lesson earn a hook" is itself a judgment. That's the boundary where the system admits it can't fully mechanize its own governance. The alternative - pretending that judgment is mechanical - is the same disease as the persistence claim.
68% with zero independent read is the number that makes block-before non-optional, not just nicer. Verify-after was always going to look fine on a dashboard that only samples the claims people bothered to check.
One thing I'd want to know before calling this closed on my end: the Stop hook that blocks the claim is still authored by the same actor whose claims it's blocking. That's the exact non-negotiability-not-exogeneity split I ended up drawing in the post, code removes the right to bargain, it doesn't remove the fact that I wrote the rule. So the honest next question for your system is the same one I'm sitting with for mine: once block-before is in place and the 68% miss stops happening, what catches the version of that failure the gate wasn't built to name? Not "does the read happen," you've solved that, but "is the read checking the right thing." A block-before gate can be airtight on its own definition of complete and still miss a class of incomplete nobody wrote a rule for.
This resonates with my Taotuner’s IPM: the future of AI may not be defined only by how much code machines can generate, but by the quality of the cognitive loops between humans and machines. Intelligence may emerge not from replacing judgment, but from creating new forms of integration, feedback, and coherence between different kinds of systems.
"A promise shaped like a file, with no file behind it" perfectly captures the hallucinated persistence trap. Building a hard gate that doesn't check the agent's virtue, but simply demands the mechanical artifact, is brilliant.
Code doesn't make the system unbiased—it just removes our right to bargain with the rules.
Removes our right to bargain with the rules is exactly it, and worth being precise about what that phrase does and doesn't cover. It's non-negotiability, not neutrality, the gate still encodes whatever I got wrong when I wrote it. What changes is that I don't get to decide, in the moment, that the check isn't worth running today. That's a narrower claim than "unbiased," and it's the honest one.
Great insight. I'd add that the strongest gates don't prove correctness—they prove prerequisites. They make certain failures impossible, shrinking the space where human judgment is actually needed.
Agree, and the sharp consequence is that shrinking the space is the whole design goal, not a side effect. A gate that proves a prerequisite converts an open question into a closed one, and what is left for judgment is exactly the residue that could not be closed. So the measure of a gate is not whether it catches errors, it is how much of the judgment surface it retires. The failure mode is a gate that feels like it shrank the space but only relocated the judgment downstream, unlabeled, where nobody is looking for it anymore.