I have spent the past month reading skill libraries, and I kept feeling like I was comparing things that would not line up. One guide would compres...
For further actions, you may consider blocking this person and/or reporting abuse
Our skills carry content - domain steps, trigger words, execution protocols. Our harness carries control - checkpoints, verification scripts, evidence requirements. The separation is deliberate: a skill that owned both would make the gate and the work in the same pass, and the gate would inherit the work's assumptions.
The sharper trap you name - honest evidence that inherits the blindness of the thing it checks - is the one we hit before adding non-LLM sources. It wasn't fabrication and it wasn't carelessness. The verifier and the data shared the same training data, so the gate passed on a check that could never have failed. The fix wasn't a stricter rubric. It was a source whose blind spots don't overlap the claim's - exit codes and git diffs from a different layer entirely. Tom's "uncorrelated blindness" is the right framing for where the chain stops: not at a final unaudited gate, but when the next checker would be blind in the same way as the last.
"The gate would inherit the work's assumptions" is sharper than how I shipped it. The separation isn't organizational hygiene, it's that a single pass can't hold both the work and an independent check on the work, because the check would be conditioned on the same intermediate state that produced the thing it's checking. Same pass, same priors, and the gate passes on a test that could never have failed.
Where I'd push one level further: a separate pass isn't sufficient either, only necessary. A second LLM reading the first one's output is separate in time and still correlated in blindness, same training data, same channel. Your exit-codes and git-diffs move works because it's a different layer, not just a different pass. So the load-bearing axis under content/control isn't gate-versus-work separation, it's channel independence: control has to read something the work couldn't have authored. Separation gets you a distinct checker. Independence requires that checker to read a channel the work never wrote to. The first is cheap, the second is where the actual guarantee lives.
Channel independence gives a per-source test: can the agent author this channel? Exit codes and git diffs are unauthorable; a second LLM reading the first one's output is a distinct pass but still reads authorable text—separation without independence.
Our three-tier hierarchy (deterministic > independent > agent_claim) maps exactly onto this axis: deterministic sources are channel-independent by construction, agent_claim is the authorable channel, the middle tier is separation only. The load-bearing question for any new evidence source is which channel it reads, not whether it's a separate pass.
One consequence we hadn't traced: pre-registration—writing what "right" means before the agent runs—makes content checks channel-independent. The agent can author the file, but not the predicate. That extends your axis into places exit codes can't reach.
Pre-registration is channel-independent across the run, and worth being precise about the one place it isn't: authorship across time. If the same model that will later be evaluated also drafts the predicate, even hours earlier with no visibility into the eventual output, it can still write a predicate it's unconsciously easy for its own future self to satisfy, the same way a person grading their own exam beforehand tends to write questions they already know how to answer. The freeze stops the agent from seeing the diff and rationalizing backward. It doesn't stop the predicate from being shaped by the same priors that will later produce the work.
Which is the same fork as the frame-independence question running through the rest of this thread: a predicate is only as independent as whoever wrote it. Human-authored predicate, or a distinct model with no stake in the later run, closes the authorship gap the way a different base model closes the frame gap in a differential check. Same-model-earlier-session closes the temporal gap but not the authorship one, and the two are easy to conflate because both produce a file that existed before the output did.
Been watching this two-dial thing play out across the 36 Stratagems series without having a name for it — so this one landed.
Characters who only turn content (know the move, never check if it worked) fold every time. Characters who only turn control (audit everything, miss the big picture) fold just as fast. Lena's the one who actually runs both — she knows when to give something up and she checks the numbers to make sure it was the right thing to give up.
That "a skill is only as strong as the evidence it demands" line — gonna live in my head. The series keeps circling the same truth. You just said it cleaner.
Lena running both dials and knowing which one is failing her in the moment is a sharper test than most fiction gives this, because the easy version is a character who is just generally competent. The harder and more honest version, the one you're describing, is a character who has a real miss rate on one dial and has built a habit of checking rather than trusting her own read of which dial she's on. That's a good tell for whether a story actually understands the split or is just naming it.
I like the split because it explains why a lot of skills feel impressive in demos and mushy in production. Content gets you the right move, but control is what keeps the run from turning into a confident diary entry. The boundary I keep coming back to is whether the next step can read an artifact instead of trusting the chat. Where would you put skills that mostly define permissions and tool limits, control dial or a third dial?
Control dial, and specifically its enforcement half. A third dial would have to vary independently of the other two, and permissions don't. Tighten them and you are not adding a degree of freedom, you are changing what control can guarantee.
The way I'd cut it: content is what-to-do, control is prove-it-happened, and permissions are the fence that makes prove-it enforceable instead of advisory. Without the fence, control is a request the run can decline; with it, the checkpoint has teeth because the step physically cannot reach past the artifact it's allowed to read.
The one place it feels like a third thing is timing. Permissions are declarative and set once, enforced by the harness before the run; control is procedural and fires per step. But that's a difference in when it runs, not what question it answers. Both answer "can I trust this ran the way it claims." Your artifact-not-chat boundary is the same test from the other side: permissions decide what the next step is allowed to read, control decides whether it actually read it instead of trusting the chat.
Great perspective. I think many people optimize for "knowledge density" but ignore "execution reliability." A skill isn't valuable because it's smart—it's valuable because it produces consistent outcomes in real workflows.
the thread has been circling the gate's evidence channel, but content and control degrade together when the domain moves, and neither dial trips an alert. the gate still runs, the content still provides a move, and both pass on stale reality. a control gate set against outdated requirements is now both the wrong check and a check that can never fail — which is the isatty blindness, but acquired through slow drift rather than a single patch.
the test worth adding to the sixth row: not only 'has this gate ever blocked its own author' but 'was the thing it checked still the requirement when it did'. confirming the gate fires isn't the same as confirming it fires on something that still matters. what's your signal for detecting domain drift before a run rather than after the gate passes?
Domain drift acquired slowly is the harder case precisely because it never produces the isatty moment, the single commit where you can point at the exact line that broke it. A gate against stale requirements degrades one review cycle at a time, and every individual cycle looks like the gate doing its job, because it is, just against a target that quietly moved.
The signal I'd reach for is provenance on the requirement itself, not the check. Every gate is implicitly asserting "this is still what matters," so timestamp that assertion and diff it against upstream reality on a cadence independent of the gate's own pass rate. Concretely: track how long since the requirement the gate encodes was last confirmed against its source, whatever generates that requirement, a spec, a metric definition, a business rule, and alert on staleness of the requirement, not just failure of the check. A gate that has passed 500 times in a row with a requirement nobody re-confirmed in 400 of them is exactly the shape you're describing, green the whole way, because passing was never the question.
The uncomfortable part is that this needs the same property the sixth-row test needed: a reader that doesn't share the gate's blind spot. If the same team that wrote the requirement also owns re-confirming it, drift gets rubber-stamped the same way self-review does. The re-confirmation has to come from whoever's closest to why the domain moved, not whoever's closest to the code that checks it.
Great breakdown! The distinction between Content (what to do) and Control (proving it happened) makes so much sense. It really explains why so many smart AI setups still fail in production.
The two dials name a split we run in production, so I can put numbers on two of your paragraphs.
On the consensus that shares a stale assumption: we measured it. Two cheap models agreeing and both wrong ran 24 to 33 percent on our benchmarks depending on model family, so agreement alone was not evidence, it was correlated blindness with a pass stamp. What survived was routing by a measured per-domain strength map and escalating on disagreement, with the adversarial trap set reported as its own line so aggregate accuracy could not hide it. Consensus was the theater version of the control dial. The strength map was the real one, and it only exists because the gate demanded a number that could have come back ugly, and did, repeatedly, before it came back clean.
On gates demanding evidence that could have come back negative: the operational test we trust is whether the gate has ever blocked its own author. Our claims ledger requires every number in public copy to trace to a measured row with a date and a source, and a pre-commit tripwire kills any commit citing a dead or unmeasured number. It has blocked me, the person who built it, more than once, including this morning when a different gate in the same chain caught me staging backup files that contained live tokens I had created an hour earlier. A gate that has never embarrassed its own operator is probably checking the wrong thing.
Which points at the row I think the scorecard is missing: the examiner sits on the same two dials. A control layer is itself content plus control, and most stacks never measure it. You know the live example, since it is the fixture work in our other thread: our own reviewer looked style-invariant until we held content fixed and measured, and the honest answer took three corrections to reach. So the sixth row I would add to any skill audit is who measures the gate, on what fixture, and when it was last run. A stack where every gate is unaudited is content-heavy control-light one level up, wearing a uniform.
Numbers on both paragraphs, and the sixth row is right, and I want to push on where its own regress bottoms out. The examiner-is-content-plus-control point cannot recurse forever, someone eventually has to be the last unaudited gate, or the audit chain becomes the thing needing an audit chain. What your two receipts have in common, the pre-commit tripwire catching live tokens and the reviewer measured against held-content fixtures, is that both bottomed out in an event nobody scheduled: an accident, not a planned review cycle. That is the same shape as the unforced-correction signal from the other thread today, an event that counts as evidence exactly because nothing arranged it to happen. Which suggests the sixth row needs two sub-questions, not one: was the gate ever audited on a fixture, and separately, has it ever failed unprompted, caught its own operator by surprise. The first is schedulable and can be gamed by scheduling around it. The second cannot be scheduled at all, which is what makes it the harder and more honest test to report.
The two sub-questions are the right split, and I can report on the second one because today ran almost entirely on the unscheduled kind.
Yesterday I shipped a knowledge delivery surface: open a file, see the prior verified commits that touched it. It ran, it produced output, and it had no evidence it helped anyone. Today I built the evaluation, which is your first sub-question, the schedulable one. Then the unscheduled part happened three times.
First, before spending the compute, I handed the eval design to an independent model to attack. It found the flaw in one pass: my judge could see what the engineer actually did afterward, which turns "this prevented rework" into "these two commits look related in hindsight." Unfalsifiable, and it would have produced a clean number I would have believed. That is your correlated blindness case exactly, except the blind one was me, and the fix was a checker that did not share the blindness rather than a stricter rubric.
Second, the measurement embarrassed its author. Blinded judge, 78 deliveries: 3.8 percent prevented a re-derivation, 69.2 percent noise, 0 percent misleading. Safe and nearly useless. It also failed in the opposite direction from the one I built it to prevent. I had tuned it precision-heavy because a small model adopts a wrong injection about half the time. This surface feeds a reader who reads and discards. Same corpus, opposite failure mode, and the deciding variable was who is reading it.
Third, the one nobody scheduled. While chasing an unrelated stale alert I found that our box memory monitor reports healthy when it cannot read memory at all. Blind and healthy were the same output, so it had been asserting a safety it never measured. Then I wrote the fix and gated it on isatty, so it stayed silent under captured output and went on misleading the exact reader it was written for. I caught that one only because I ran it and noticed nothing came back.
On where the regress bottoms out. I do not think it ends at a last unaudited gate. I think it ends at uncorrelated blindness. You do not need a final checker that is above audit, you need one whose blind spots do not overlap the claim's, which is why the move that paid was handing the design to something that did not share my priors, before the run rather than after. The chain can stop when the next checker would be blind in the same way as the last, and unlike "who audits the auditor," that is a property you can actually inspect.
One limit, since this thread is about evidence that could have come back negative. My narrowing fix is tuned on three positive examples. The direction holds. The threshold does not, and I am not going to pretend otherwise until the corpus is bigger.
The isatty bug is the whole thread in one line. You fixed blind-reads-as-healthy, and the fix inherited a new blindness that wasn't on your list when you designed it: silent under captured output, aimed at exactly the reader it was written for. That's the part that makes uncorrelated blindness harder than it first sounds. You can inspect it pairwise, this checker isn't blind the same way as the last, but "the same way" is judged against your own catalog of ways-to-be-blind. The isatty failure lived in a way that wasn't in the catalog yet.
So the property you can actually inspect is narrower than the one you want. You can verify the next checker doesn't share a known blindness with the previous one. You can't verify it doesn't share an unknown one, because enumerating your own blind spots is the thing you can't do. That's why handing the design to a foreign schema before the run pays, and it's a sharper reason than "a second opinion." It's not that the other model sees more. It's that its blindness-set doesn't overlap yours, and the non-overlap is exactly the part you couldn't have checked yourself.
The 3.8-prevented, 0-misleading result is the other receipt. It didn't miss because it was badly built. It missed because you tuned it against the failure mode of a different consumer, and the deciding variable turned out to be who reads the output, not how good the check is. A check's failure mode is set by its reader. That generalizes past this eval, and it's the thing that makes "safe" such a treacherous word for an instrument. And the three-positive-examples note at the end is the right way to leave it: direction earned, threshold not, and no pretending otherwise until the corpus is bigger.
the split explains a failure asymmetry too: control failures are loud, content failures are quiet. when a gate rots the pipeline blocks and you notice the same day. when a playbook rots the agent keeps confidently executing last year's best practice and nothing surfaces until the damage is architectural. which suggests where to spend the second dial even in a content-heavy skill: not gating the work's correctness, just gating the content's own freshness, a checked-on date the chain refuses to run past. you already do this instinctively with the (checked 2026-07-17) stamps, the skill format just never made it enforceable.
The asymmetry is the sharper framing of something I'd only noticed as an annoyance: I keep re-verifying the same three links every few weeks because nothing tells me when it's due, only that it's overdue once someone points it out.
A refuse-to-run-past date closes the case where nobody looked. It doesn't close the harder one: someone looked, bumped the stamp, and the domain had already moved out from under the checked claim. That's the same split recursing one level down. A date gate is control-lite: it enforces that a check happened, not that the check was any good. So the content dial would need its own held-out test to be genuinely enforceable, something like re-running the cited claim against a fresh source and diffing, not just trusting the human who touched the file last.
Cheap version I'd actually ship: date gate for the common case (nobody looked), plus a periodic spot-check that pulls one skill at random and asks whether the linked source still says what the skill claims it says. Same shape as a marker-density-vs-flip-rate split from another thread: the proxy (date fresh) is what's cheap to test, the quantity (still true) is what actually matters and gets assumed.