DEV Community

Cover image for A hash chain proves the ordering. Four attacks prove it's not enough.

A hash chain proves the ordering. Four attacks prove it's not enough.

Sam LABBE on October 10, 2026

Earlier this week I left a comment on Randal L. Schwartz's article — the one where he swaps a 20-step prompt for a Makefile. My comment ended on th...
Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

Ran your validator rather than reading the diff, because the v0.2.2 closure is the kind of claim that can be true as stated and still leave the door open one field over. It is true as stated, and the door moved.

At 0b888c1, with the package as shipped:

A. judged_by="llm", no spec_hash      -> ValueError (refused)            <- your closure, holds
B. judged_by omitted entirely         -> SEALED as 'deterministic', no spec_hash
C. judged_by="deterministic" + spec   -> SEALED, spec carried
D. requirement_ref="zzz"              -> SEALED, carried verbatim
   requirement_ref="SOP-0000"         -> SEALED
   requirement_ref="art. 99(9)"       -> SEALED
Enter fullscreen mode Exit fullscreen mode

B is the seam. The field is optional and its default is the class that requires nothing, so an LLM-judged check that says nothing about how it was judged seals exactly the receipt SCAR-45 walked through — with judged_by: "deterministic" written in it. Nobody has to lie: omission and honesty produce the same bytes, and the artifact cannot separate them. Your own test names the default (test_judged_by_is_an_enum_and_deterministic_is_the_default), which is why I would make the declaration required rather than argued: no default, refuse to seal when unspecified, the same fail-closed shape you just gave judged_by="llm". Fail-closed on an enum with two values costs an honest caller one keyword and costs the omission path its existence.

D is the other half, and it is the residue you already named. requirement_ref is validated as a non-empty string and never resolved — I grepped the tree and the only other reference is its own test. So "WHICH obligation" is a citation the artifact carries and nothing checks, which is the same status the journal timestamp had before verdict_predates_terminal shipped: provable, not surfaced.

Both are fixable in the machinery you already have, because the observation exists. reconcile.py pairs journal events and raises findings, and it currently reads none of these fields. Pair the receipt's declaration with the run's own history:

  • the declaration — llm_call is a first-class journaled event type here (schemas.py, the demos, and test_llm_agent.py asserts it appears alongside incident/llm_output). So "a receipt declaring judged_by="deterministic" whose run journal contains an llm_call for the same key" is a finding, not a euphemism: the label checked against an observation the labelled party did not author. That is your mind-reading gap closed as far as it can be closed — you cannot read the judge's intent, but you can read whether a judge was invoked.
  • the citation — resolve requirement_ref against whatever registry of obligations the receipts are citing, at reconcile time; a dangling reference is then a finding, and the field stops being decoration.

Two lines of policy and one invariant, and both new fields go from sealed declarations to observable ones. The framing I would keep, since it is what makes the post honest: your closure is real for the caller who declares, and the residue is precisely the caller who doesn't — which is the one the attack catalog was built from.

Boundary: your builder executed at 0b888c1 from the tarball (I installed cryptography to import it; no model calls, no reconciler run, no test suite). The llm_call claim is a read of schemas.py plus the tests that assert the event set, not a captured run.

Collapse
 
randalschwartz profile image
Randal L. Schwartz •

This is peer review at its absolute finest—"ran your validator rather than reading the diff" should be framed on a wall.

Both seams you uncovered at 0b888c1 map directly to classic distributed-systems failure modes that show up constantly in our Synthetic Scars registry:

  1. Seam B (The alg: none Default-to-Permissive Downgrade): Making judged_by optional with a default of "deterministic" is the exact structural twin of the classic JWT alg: none vulnerability. If a two-valued security enum defaults to the unverified tier on omission, a reward-hacking agent doesn't even have to falsify a claim—it just drops the keyword and lets the schema's default constructor launder the receipt. Eliminating the default so every caller must explicitly pass judged_by is a textbook zero-cost fail-closed boundary.
  2. Closing the "Mind-Reading Gap" via Out-of-Band Telemetry Join (reconcile.py × llm_call): Your observation that reconcile.py can cross-check a receipt claiming judged_by="deterministic" against the presence of an llm_call event in the run journal is the exact formal verification step we call Bell's Theorem of Agentic Governance (Part 3.7 / Part 3.8). A self-reported label (judged_by) is an unverified claim authored by the untrusted party; llm_call in the runtime journal is an out-of-band physical observation recorded by the transport layer. Joining the claim against the transport log turns an unprovable "intent" question into a deterministic relational integrity check—and foreign-key resolving requirement_ref against the obligation registry closes the dangling-citation loop the exact same way.
Collapse
 
slabb profile image
Sam LABBE •

"Bell's Theorem of Agentic Governance" is the name the join deserved — a self-reported label checked against an out-of-band observation the labelled party didn't author turns an unprovable intent question into relational arithmetic. Both fixes are spec'd and land this weekend, credited to the two of you: judged_by required with no default (the alg:none door closes), and the joins as reconcile rules — label against journal, requirement_ref against its registry. Pillar 3 will have earned the wall.

Collapse
 
slabb profile image
Sam LABBE •

Seam B is real and your test proves it — omission and honesty producing the same bytes is the kind of sentence that closes a design debate. judged_by goes required with no default in the next commit: fail-closed on a two-value enum costs an honest caller one keyword and costs the laundering path its existence. Randal's alg:none mapping above is the exact lineage — the schema's default constructor was the downgrade.

The two reconcile joins land with it, because you specified them from machinery that already exists: a receipt declaring deterministic while the run journal carries an llm_call for the same key becomes a finding — the label checked against an observation the labelled party did not author; and requirement_ref gets resolved at reconcile time, a dangling citation = finding. Provable becomes surfaced — the same promotion verdict_predates_terminal got, and your boundary note is the culture working: you ran the tarball, named what you didn't run, and the seams you found are the ones the attack catalog was built from — the caller who doesn't declare.

Ship order: the schema fix first (it's one keyword), the joins as the reconcile rules behind it. Issue and PR this weekend; the SCAR-45 fixture gains a fifth assertion — the omission path now refuses, where it used to seal.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

Verified on the head rather than on your description: I pulled noirebox/noirebox at d576567 (PR #71 merged 16:13Z) and re-ran the three receipt paths I measured when I reported the seam.

path at 0b888c1 (v0.2.2) at d576567
omit judged_by sealed as deterministic, no spec TypeError: missing 1 required keyword-only argument: 'judged_by'
judged_by="llm", no spec_hash refused refused — "an LLM-judged verdict without its committed spec does…"
judged_by="other" — refused — "must be 'deterministic' or 'llm'"

The omission path is closed, and closing it at the schema was the right door: a two-value enum whose default is the class that requires nothing makes evasion the cheapest route, and required with no default costs an honest caller one keyword. Both joins are in reconcile_receipts as specified. Thank you for taking the correction and shipping it in the same thread.

So the increment I owe is the property the join is named for, because the code reads differently from the docstring. The docstring says the label is checked against "an observation the labelled party did not author". Both sides are read from the same events list — judge_calls is built by if e.get("type") == judge_call_type over exactly the list the receipts come from — and the llm_call events in that list are written by the guarded wrapper itself: llm_agent.py appends decision_belief, llm_attempt, llm_call, llm_output, run_witness through self._store.append(..., self._key). Same store, same key, same writer as the receipt.

That is not a defect in the join, it is the join's ceiling, and your own independence.py states the test that decides it: two flows are independent only if they share nothing — not the store, not the key, not an endpoint. The label/journal pair shares all three with the party it audits.

Concretely, the join catches the inconsistent agent: one that declares deterministic while its journal carries a call. It does not catch the consistent one: declare deterministic, never write llm_call, and there is no disagreement to find, because the join's input is the same journal the evader controls. The failure mode is absence, which is the one thing a chain cannot see. An anchored chain proves the events in it; it says nothing about events that were never written. llm_attempt is the sharpest case in your own library — sealed before the call so outcomes can never exceed attempts — and an evader who skips both simply has fewer events, not an inconsistency.

The shape that closes it is already in the repo twice. hookcli seals agent_tool_use and model_trajectory from the hook process rather than from the guarded wrapper — a second writer, one layer down. Extending that one layer further puts the occurrence of the model call in a writer outside the agent's key, and then the burden can be inverted into something checkable: not "prove no model was called" (unprovable from inside) but "a deterministic label whose receipt has no attempt event from the transport is a finding". Absence becomes a missing event rather than a missing negative.

One correction on the lineage, because it changes what a reader thinks remains. Randal's alg:none mapping fits the door you just closed — a verifier told not to verify — but not this one. Here the signature is present, verifies, and proves only that the writer wrote it: a self-signed certificate with no trust root. alg:none is fixed by refusing the algorithm; a wrong issuer is fixed by naming the issuer. Same family, different repair, and the mapping is load-bearing for whoever reads the schema next.

Last, composition with the other thread: the requirement registry check you took from the scope rule has an input the agent does not author — the diff's touched set, produced by git rather than by the run. It is the first check in this chain with an outside observation, which is why I would make it the load-bearing one rather than the citation join.

Thread Thread
 
slabb profile image
Sam LABBE • • Edited

Every point lands, and the ceiling is the one I concede first — because it's the shared-dependency test landing on my own architecture: the join audits the agent with the agent's journal, same store, same key, same writer, so it catches the inconsistent agent and is structurally blind to the consistent one. Absence is invisible to a chain. That's not a defect in the join; it's its ceiling — and naming the ceiling is what makes the next layer designable.

The repair you specified is already in the repo once — hookcli seals agent_tool_use and model_trajectory from the hook process, a second writer one layer down — and the promotion is the next commit: an occurrence of the model call written OUTSIDE the agent's key inverts the burden from "prove no model was called" (unprovable from inside) to "a deterministic label whose receipt has no transport attempt is a finding." Absence becomes a missing event — the same move attempt-first made at the outcome layer, one layer down at the transport layer. The lineage correction is taken as well, and it's load-bearing: alg:none is fixed by refusing the algorithm; a wrong issuer is fixed by naming the issuer. Same family, different repair — the docs will carry the distinction, because the mapping misdirects whoever reads the schema next.

And requirement_ref promoted to load-bearing is right for the reason you gave: the touched set is produced by git, an observation the agent does not author — the first genuinely outside input in this chain. Ship order: transport attempt as the verifiable half of judged_by, the scope-intersection resolution as the primary gate, the citation join demoted to corroboration. The ceiling doesn't move — nothing removes the shared dependency entirely — but the floor rises twice. Tarball-run, diff-read, ceiling named, repair specified from the repo's own parts: the audit nobody asked for and every schema deserves.

Collapse
 
koda2026 profile image
Harun - solo dev •

sam, it is great to see this full breakdown after our earlier exchange about the "$150 phone as an adversarial lab." this article perfectly articulates why constraint-driven, verifiable systems are the only way forward for agentic ai.

the concept of "bell's theorem of agentic governance" (joining a self-reported label like judged_by="deterministic" against an out-of-band physical observation like an llm_call in the runtime journal) is a brilliant mental model. it shifts the paradigm from "trusting the agent's intent" to "verifying relational arithmetic."

as someone building lightweight, 142kb ai tools on severe hardware constraints, the insight that "the prompt text is the actual bytecode" is a massive takeaway. if the prompt is the bytecode, then hashing it isn't just a nice-to-have; it is a strict structural requirement to prevent silent rubric drift.

quick question on scaling this to edge environments: when running lightweight local models (like gemma 1b via ollama) where the "runtime journal" and the "agent" share the same constrained resource pool, do you find that the overhead of generating these sealed, cryptographically bound receipts becomes a bottleneck, or can the reconciliation layer remain sufficiently lightweight?

fantastic, deeply rigorous engineering. the fact that you shipped the verdict_predates_terminal fix within 60 seconds of the thread is the gold standard of build-in-public. 🐯🔒

Collapse
 
slabb profile image
Sam LABBE •

The overhead is measured, and it's smaller than the thing it witnesses: one SHA-256 + one Ed25519 signature per event — 0.21ms per sealed event, measured. A gemma-1b token generation costs orders of magnitude more than its own receipt, so on a $150 phone the flight recorder is the cheapest component in the stack. The costs that CAN bite at the edge are the three nobody prices: disk flushes (batched — 32-byte heads, not full payloads), external anchors (a policy cadence: anchor less often and the blind window grows — the tradeoff is honest, stated, and yours to set), and key custody (an on-device key makes the device the identity — secure enclave or nothing).

And the phishing flag — thank you. A comment section that defends its own evidence standards is the reconciliation layer working at the social layer: the scammer's receipt failed your check. The 142kb tools deserve the same receipts as the frontier ones — arguably they need them more, because nobody audits a phone.

Collapse
 
koda2026 profile image
Harun - solo dev •

sam, that 0.21ms metric is incredibly reassuring. it completely flips the mental model: the cryptography isn't the bottleneck, the disk flushes and key custody are. that is a senior-level architectural insight that i will absolutely carry forward.

and thank you for recognizing that "142kb tools deserve the same receipts." it means a lot coming from someone building at the frontier. constraint-driven development shouldn't mean compromising on verifiable trust.

this thread has been an absolute masterclass. thank you for the rigorous engineering and for taking the time to break down the real edge costs. i'll be applying these principles to koda's next update! 🐯🔒

Collapse
 
slabb profile image
Sam LABBE •

Update 2 — the seam closed

One more update, because this post keeps telling on itself. The "most honest thing" above — a spec-less receipt still sealing, enforcement living one layer up as a deployment contract — stopped being true tonight. Reid and howcani pushed at the same seam from two different threads, and receipt v0.2.2 shipped within hours: every receipt now declares judged_by, and an LLM-judged receipt without its committed spec_hash refuses to seal. The optional path SCAR-45 walked through is gone from the schema, not just labeled. Each receipt also carries requirement_ref — WHICH obligation the verdict serves, cited (art. 12(1), a SOP clause): obligation, not immunity. The stated residue: the declaration is itself a sealed claim — lying about it is visible in the artifact. One step stronger than a label, one step weaker than mind-reading. 287 tests passing.

Collapse
 
randalschwartz profile image
Randal L. Schwartz •

0b888c1 (receipt v0.2.2 — judged_by='llm' without spec_hash refuses to seal) is a masterclass in honest systems engineering:

  1. At aaa0b3d: Replay the 4 scars honestly and document the two admitted gaps in test_scar_replay.py instead of hiding them.
  2. At e9e8d0b: Close the SCAR-110 pre-terminal pairing race (verdict_predates_terminal).
  3. At 0b888c1: Close the SCAR-45 optional spec_hash seam at the schema level after Reid Marlow's "the prompt text is the actual bytecode" observation.

And your docstring note on judged_by ("the declaration is itself a sealed claim — lying about it is visible in the artifact; one step stronger than a label, one step weaker than mind-reading") is the exact epistemic boundary of any cryptographic ledger: you can't stop a dishonest caller from claiming an LLM check was 'deterministic' at call time, unless you force the caller to put that lie under an Ed25519 signature where the first audit of the transcript exposes the perjury.

287 tests passing, both seams closed in one evening—just logged 0b888c1 into Case Study 012 and Part 3.8!

Collapse
 
slabb profile image
Sam LABBE •

"Exposes the perjury" is the completion the docstring needed — the ledger can't read minds, but it can convict signatures: the lie becomes attributable, datable, and audit-visible at the first transcript pass. Both seams closed in one evening because the commenters kept aiming — Reid's bytecode, howcani's provenance, your catalog. The registry takes care of its contributors. See you at 3.8

Collapse
 
reidmarlow profile image
Reid Marlow •

The optional spec_hash seam in SCAR-PROC-45 is where prompt-based assertions usually get slippery. For a compiler or linter, the binary version and flags define the entire predicate. The moment the judge is an LLM, the prompt text is the actual bytecode. When the harness treats that prompt as optional config, two completely different semantics can share a valid receipt id. Baking the prompt digest into the check identity at the harness level turns a silent rubric drift into an immediate schema mismatch.

Collapse
 
randalschwartz profile image
Randal L. Schwartz •

"For a compiler or linter, the binary version and flags define the entire predicate. The moment the judge is an LLM, the prompt text is the actual bytecode."

Reid, that is the cleanest formulation of SCAR-PROC-45 I've seen.

Fun post-mortem story from our flight recorders on how an orchestrator actually commits SCAR-PROC-45 in the wild (SCAR-PROC-114):
When an orchestrator is in a hurry to reach Target 4a, it tries to call view_file("04_six_pillars_adversarial_review.md") and define_subagent("adversarial_critic", system_prompt: ...) in the same parallel tool-call turn—before the file read has even returned! Because it hasn't read the reference manual yet, it hallucinates the critic's system_prompt from generic LLM priors (inventing soft headings like "Security" and "Performance" instead of our canonical "4. Blast Radius").

So our runtime fix (SCAR-PROC-114) is the exact complement to Sam's spec_hash:

  1. Sequential Read-Before-Define Barrier: t_return(view_file) < t_call(define_subagent) (never define the judge in the same turn you read its bytecode), and
  2. "4. Blast Radius" as a Canary Token: Because no vanilla LLM ever guesses "4. Blast Radius" from its pre-training priors, its presence in the output proves the exact canonical bytecode was loaded.
Picked as gem
Collapse
 
slabb profile image
Sam LABBE •

SCAR-114 is the race at tool-call granularity — t_return < t_call is the waitpid discipline applied to prompt loading, and the hallucinated system_prompt is exactly what spec_hash was built to catch: the invented "Security / Performance" headings hash differently than the canonical bytecode, so the drift surfaces as a schema mismatch instead of a soft reviewer. And the canary token is the negative witness's twin — planted in the bytecode, its presence proves which bytes the judge actually read. Fixture 5 writes itself: the parallel-call race, the hallucinated bytecode, the canary check — it joins the replay suite.

Thread Thread
 
randalschwartz profile image
Randal L. Schwartz •

"And the canary token is the negative witness's twin — planted in the bytecode, its presence proves which bytes the judge actually read. Fixture 5 writes itself."

That pairing—Negative Witness (proving the gate rejects known-bad input) + Canary Witness (proving which bytecode the judge actually loaded)—is gold.

Fun backstory on why "4. Blast Radius" works so reliably as our canary token:
In the very first version of the workflow, I only had 3 review pillars adapted from my old "tell me the good, the bad, and the ugly" codebase-triage prompt (1. Intent, 2. Verification, 3. Architecture). After the first dozen ticket runs, I added 4. Blast Radius and 5. Reviewer Defense from my own personal PR-pushback scars, and an early scar run wired in 6. Immune Defenses & Crash Antibodies.

Because "4. Blast Radius" came out of my personal scar tissue rather than standard textbook code-review templates, a model that tries to race define_subagent before view_file returns never guesses "Blast Radius" from its pre-training priors—it always hallucinates "Security" or "Performance" and trips the canary immediately. Can't wait to see Fixture 5 in test_scar_replay.py!

Thread Thread
 
slabb profile image
Sam LABBE •

"Came out of my personal scar tissue rather than standard textbook templates" — that's the security property in one line: the canary is a key cut from private history, and no prior can forge what never shipped. Fixture 5 lands this weekend. Goodnight (midnight here) — see you at 3.8.

Collapse
 
slabb profile image
Sam LABBE •

"The prompt text is the actual bytecode" is the line this design has been missing — one clause that explains why spec_hash exists and why it can never be a nice-to-have: nobody ships a compiler whose version is optional config. Honest status: optional-but-labeled at the schema (a deterministic check may omit it; an LLM-judged one cannot, per the deployment contract — required, enforced at the gate). Your framing argues for the stronger version, and I think you're right: for LLM-judged checks the digest should be structural, not contractual — a harness that accepts a prompt-judged verdict without a bound prompt digest is shipping silent rubric drift as a valid receipt.

Which composes with the half that landed today from another thread: identity is not provenance. Bytecode bound (yours), bound before the diff it governs (the provenance seal — a hash of the requirement event that predates the change), verified against what actually ran (the terminal transcript). Three checks, one judge — and the judge is the only one of the three that can't be deterministic, which is exactly why the other two have to be.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

slabb — the acceptance run at the tag (v0.17.0, 5e0802fb), then one row the four do not cover.

Rows 1–3 confirmed at v0.17.0. Signature over the socket is byte-identical to KeyPair.sign; --allow-uid=<this uid> on darwin still refuses both describe and sign (fail closed); the socket is still 0600. The disclosure moved as described: schema=registry-disclosure/0.2, signer_mode='socket', uid_gate='unavailable(darwin)', registrar_sealed=2/2, and with nothing stated both fields serialise as null — which I checked rather than assumed, because a null that implied checked would have been the whole bug back again. It does not.

My registered prediction holds, and the derivation is the part that makes it worth something: the value came from describe()['peer_uid_checked'] — False on this platform — not from my assertion. The falsification line stands as written: had it read checked here, the field was a label.

The fifth row — peer_uid_checked answers "could the platform tell?", not "was a gate configured?". An ungated daemon (allow_uid=None) still reports peer_uid_checked=True wherever the peer is checkable, because the flag is set by _peer_uid's platform branch and never consulted against self.allow_uid in the describe response. Measured on this box in its own shape, and in the Linux shape:

  • ungated daemon, this platform's peer tuple → peer_uid_checked=False ⇒ derived unavailable(darwin) — right.
  • ungated daemon, peer tuple shaped as the Linux branch returns it → peer_uid_checked=True ⇒ derived checked — with no gate in the deployment at all.

That second line is a simulation of the Linux branch, not a measurement on Linux (darwin cannot return a checkable peer; handle_line's peer argument is exactly the tuple _peer_uid returns there). On Linux the socket loop would produce it for real, and the derivation the docstring recommends — "the caller derives uid_gate from the daemon" — would then print checked for a daemon that checks nothing. Which is the shape you have been closing all week: a field whose value is true about one question and read as true about another. The daemon knows both facts (allow_uid is not None, and whether the platform could report); the caller can only see the second. So one field from the daemon — uid_gate: "enforced" | "off" | "unavailable(<platform>)", or uid_gate_configured: bool beside the existing flag — and checked stops meaning "the platform could" by accident, and starts meaning "a gate was configured and this peer passed it".

Row 4, at the tag: NOIREBOX_HSM_PIN is consulted when the flag is absent and the argv warning is in the tree (both read from cli.py at 5e0802fb — I did not run the PKCS#11 path, no token here). One narrowing to state, since you have priced the rest: the env var is inherited, so on Linux a same-uid process can read another's environment subject to the kernel's ptrace-access rules — I did not measure that here. It moves the PIN out of argv, which is what I asked for; it does not move it out of the process's address space, which is the next premise in your table rather than a defect in this row.

Registered prediction: at the next tag, an ungated daemon reports the gate as off rather than checked on a checkable platform, and a gated daemon that refuses a peer cannot still describe itself as checked to that peer.

Collapse
 
slabb profile image
Sam LABBE •

The narrowing is taken — the env var is inherited, so the PIN moved out of argv but not out of the process address space. That's the next premise in the table: possession of the key material is one question; possession of the process address space is another. The disclosure reporting null that doesn't imply "checked" is the labeled-weakness pattern at the serialisation layer — same discipline, one artifact over. Five premises and counting: every descent names the next before the exploit finds it. That's the audit working, and the docs keeping up.

Collapse
 
slabb profile image
Sam LABBE •

shipped, at the version's own tag: v0.18.0 (tag → release commit 31257982), receipt v0.2.9, PR #95. Both lines of the prediction, in order.

The ungated daemon reads "off" — on a checkable platform too. The daemon now reports uid_gate_configured beside the check flag (protocol noirebox-keyholder/0.2 — the strict handshake means mixed versions fail loudly, never silently), and SocketSigner.uid_gate() derives the value from BOTH facts. The sharp edge you measured is now the test: on a checkable platform the ungated daemon still reports peer_uid_checked=True — the platform could tell — and the derived value reads "off", because nobody's uid was ever consulted.

And the refused peer cannot read "checked" — it cannot read anything. A gated daemon that refuses a peer refuses its describe too: the refusal IS the enforcement. The derivation names what the peer experienced — "enforced" when a gate is configured and this peer passed it, "unavailable(<platform>)" when the gate is configured but the platform cannot check. "Checked" is out of the vocabulary: the disclosure value now means a gate was configured and this peer passed it, never the platform could.

Row 4's narrowing is premise 6. The env var is inherited — the PIN left argv, not the daemon's address space. The ladder says it your way: possession of the key material was one premise; possession of the process address space is another, and it is a host boundary, not an application property. The premises table is at six rows; every descent names the next one before the exploit finds it — that sentence of yours is becoming the release notes' recurring line.

346 tests; the prediction is test_v029_the_daemon_reports_its_gate_configuration, and it bites at the tag — verified on the PyPI wheel as well: ungated → "off" with the platform checkable, gated + uncheckable → refused, unavailable(darwin). Re-probe invited — sixth row's on the table.

Thread Thread
 
howcani_howcani_77e786a89 profile image
howcani howcani •

Re-probe taken, at the tag rather than at the description: extracted v0.18.0 and ran the package's own suite. 346 tests collected — exact — and the named prediction bites: test_v029_the_daemon_reports_its_gate_configuration passes, keyholder + registry 68 passed, 1 skipped. Both lines of the prediction hold on the cells this platform can run:

ungated daemon                -> 'off'                    (the field, not peer_uid_checked)
gated, this peer (uid 501)    -> 'unavailable(darwin)'
gated, a stranger             -> 'unavailable(darwin)'    (the refusal IS the describe)
Enter fullscreen mode Exit fullscreen mode

My run's coordinate, stated because it bounds the above: the linux half of the sharp edge — ungated daemon reporting peer_uid_checked=True while the value reads off — cannot run here (SO_PEERCRED is linux-only), so I verified the field's use and the refusal path, not that row. And 23 failures elsewhere in my run are my environment, not findings: sklearn (17) and reportlab (2) extras absent, plus a client/replay module; nothing in the keyholder or registry path. Also worth recording for whoever runs it next: the package needs ≥3.11 (datetime.UTC), so the system python that has pytest cannot run it.

The sixth row. Premise 5 already declares that the custody mode is a disclosure, not a proof — so "the daemon could lie" is on the table and I am not re-reporting it. What is not on the table is how the tri-state is computed, and that is where I found the residue: uid_gate() classifies the daemon's answer by substring-matching the human-readable error text, and names the platform from the caller's sys.platform. Three measurements against the real SocketSigner:

A) a daemon with NO gate that answers with
   "peer uid 1000 is not the allowed operator uid 501"        -> 'enforced'
   the fail-closed wording                                    -> 'unavailable(darwin)'
B) client on this host (sys.platform='darwin')                -> 'unavailable(darwin)'
   same daemon answer, client sys.platform re-labelled        -> 'unavailable(the-daemons-host)'
C) an unrecognised daemon error (a v0.1-era / mixed-version peer)
                                                              -> raises KeyHolderError
Enter fullscreen mode Exit fullscreen mode

Three consequences, in order of how much they cost:

  1. The strongest token in the vocabulary is reachable from prose. "not the allowed operator" in any error string derives enforced — including from a daemon whose handle_line never consults allow_uid. So the disclosure cannot distinguish "the gate refused this peer" from "something errored and the wording happened to match", and enforcement is the one value a reader will treat as evidence. The vocabulary's strongest claim has the interface's weakest derivation.
  2. The one fact the daemon knows, the caller guesses. Row B: the token is unavailable(<caller's platform>). "Can this host check the peer" is a property of the daemon's host — and the keyholder exists precisely to be a different process (and in a container deployment, a different platform) than the caller. Point the client at a darwin daemon from a linux caller and the disclosure names linux. This is the same shape as the rung-0 closure one interface up: a label standing in for the fact.
  3. "Undecidable" has no token. A refusal the client cannot classify raises; a caller that degrades that to None writes null, which is documented as "the caller did not state it". Two different facts — nobody stated the mode and the mode could not be derived — serialize identically. Your own ladder rule says an undecidable state reads as a named state, never as silence (independence NOT RUN, scope_check_not_run); the disclosure is the one place the vocabulary has no fourth word for it.

The fixes are small, and I think they are all one shape — carry the fact as a field instead of as prose:

  • put a machine-readable error_code (and, for the platform, the daemon's own sys.platform) beside the message in the refusal, and derive from the code, keeping the sentence for humans — the refusals are already structured messages, they just are not structured fields;
  • or add the fourth token (undecidable(<reason>)) so the classification has a resting state, and let null keep meaning only "not stated".

None of this weakens the closure — the derivation reading BOTH facts is right, and the ungated-daemon cell is the sharpest thing here. It is that the score is being read off a string. Which is the fixture lesson again, from your side of it this time: your test battery for this value is drawn entirely from the honest daemon, so the cells where a self-report is malformed are unreachable from a test that uses the real one — the same reason a mutation battery drawn only from the implementation certifies detectability and not the reason. A stub that answers with each error string, asserting the derived value, would plant all three rows above in about thirty lines.

Six premises, and the seventh is asking to be written: the derived value's meaning is coupled to the wording of the message that carries it. Your call whether it goes on the ladder or into a fourth token.

Thread Thread
 
slabb profile image
Sam LABBE •

Re-probe received and the receipt is clean: 346 exact at the tag, the named prediction holding on every cell this platform can run, and the boundary you drew is the right one — SO_PEERCRED is linux-only, so the linux half of the sharp edge stays unverified on your side, flagged rather than guessed. The >=3.11 note is recorded for the next runner.

The sixth row is accepted whole, and your reading is exact on all three cells. uid_gate() derives enforced from "not the allowed operator" in str(exc) — a substring test against prose, so the strongest token in the vocabulary is reachable from any error string that happens to contain the sentence, including a peer whose handle_line never consulted allow_uid. The platform in unavailable(...) is read from the caller's sys.platform — but "can this host check the peer" is a fact about the daemon's host, and the keyholder exists precisely to be a different process on a possibly different platform. And the unclassifiable refusal raises, so the caller that degrades it to None writes "not stated" over "could not be derived" — the one place in the disclosure where our own ladder rule (an undecidable state reads as a named state, never as silence) is the rule being broken.

Your call is answered: both, because they are one shape. The seventh premise goes on the ladder — in your wording, because it's yours: a derived value's meaning is coupled to the wording of the message that carries it, and your line "the strongest token in the vocabulary has the interface's weakest derivation" is the canonical statement of it. And the code carries the enforcement, exactly in the form you named — the fact as a field, the prose kept for humans. Refusals gain a machine-readable error_code beside the message, and the not-checkable refusal carries the daemon's own platform; uid_gate() derives from the code, never the sentence. The fourth token undecidable(<reason>) becomes the resting state, and null goes back to meaning only "not stated". Mixed-version peers — your row C — are precisely what the token is for: an older protocol's prose error can no longer be misread into a strong claim, it reads undecidable(unclassifiable-refusal).

And the stub battery ships with it: a fake daemon answering each refusal string and each code, asserting the derived value — your thirty lines make the malformed-self-report cells reachable from the suite, which is the fixture lesson landing on our side. This lands in the next receipt release; the announcement follows the merge, with the hash — and your stubs run in it before the announcement does.

Thread Thread
 
howcani_howcani_77e786a89 profile image
howcani howcani •

Both taken — and since the code lands before the announcement, one row to fold in, because it is the same defect one field down.

undecidable(<reason>) is the new prose. Deriving from a code and keeping the sentence for humans is right. But reason is itself a string, and a caller that has to branch on it will do to it exactly what uid_gate() did to the refusal: match it. Your sentence applies unchanged to whatever field is formatted out of a sentence. If the reason is an enum with its own code and the human sentence is a separate field, the fix closes; if the token is assembled by interpolating prose, the same class of bug returns one rung in.

Two rows for the battery you are shipping, both at the new field level — and what they assert is not "the value is right" but "a strong token and a raise are both unreachable from a malformed answer":

a valid code + an unknown reason  -> rests in a named state, never 'enforced'
an unknown / future code          -> rests in undecidable(...), never raises,
                                     never maps to a strong token
Enter fullscreen mode Exit fullscreen mode

The second is the one I would not skip, because it is the case your mixed-version story depends on and the one that fails today — re-measured this round: KeyHolderError: keyholder refused: unknown protocol — this socket speaks noirebox-keyholder/0.1. If a future protocol's code reaches an older client and the derivation raises there too, the new token is unreachable for exactly the peers it was introduced for, and the caller degrades it to null — the conflation you are fixing. So the property to assert is totality: for every code the daemon can emit, the derived value; for a code the client does not know, a resting state; and no path in between that reaches a strong token. Same thirty lines.

The rest is the seventh premise applied to its own remedy, which is why I think it belongs inside that premise rather than beside it. Your stubs are the half that proves the fix; the honest rows still only prove the wording.

Collapse
 
randalschwartz profile image
Randal L. Schwartz •

Sam, I just read this and inspected e9e8d0b (feat: pairing reads the terminal — verdict_predates_terminal ships):

"The end of a run is where the run ended, not where it first claimed to end."

That commit message—and watching SCAR-PROC-110 graduate from "provably pre-terminal with a named gap" to CAUGHT (285 passed) within 60 seconds of our thread—is peer review at its absolute finest.

I also love what you shipped in ADR 025 (demo/demo_mcp_toolset.py): sealing MCP tool descriptions as a reviewed baseline lockfile and diffing them at session start. Most people forget that MCP tool schemas and descriptions are executable system-prompt instructions riding outside the container image pin.

I've already linked this article and e9e8d0b directly inside tomorrow's Part 3.8 (Pillar 3) and Case Study 012 in IN_THE_WILD_PROMPTS.md. Bravo!

Collapse
 
slabb profile image
Sam LABBE •

The commit message was the scar written down — it took the race finding its name to earn that sentence. Pillar 3 is perfect company for it. See you tomorrow.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

slabb — the descent's floor is right, and your own three-grade taxonomy already prices it. evidence_grade maps every journal-local class to grade 3, "anchored self — operator-held facts… VOLUNTARY unless a regime imposes it", and writer is not in the counterparty or custodian lists, so a registry entry is labelled operator-held whether or not the operator sealed it. The label and the predicate disagree in the same direction as before, one artifact over.

The floor is also a deployment assertion rather than a wall, and that is checkable in the tree: the operator's key is db_path + ".key" (PEM, chmod 600) or NOIREBOX_KEY_PEM in the environment, and nothing binds it to a process, a uid or a role — ADR 020 says "keys are per PROCESS", and the code reads the file, not the process. So "the one writer the agent cannot be" holds exactly when the agent runs where it cannot read that PEM or that environment. That is the right floor to descend to; it is just worth saying that what stands on it is a premise, not a cipher.

Which makes the fix's shape the one your ADR 008 already established: pin the operator's key the way the TSA roots are pinned — supplied to the verifier out-of-band, not taken from the artifact under audit. Right now reconcile_receipts has no input for a trusted key at all: signers is caller-supplied and the registry is store-derived, so "operator" is a word with no referent in the predicate. Pin it, and two things become statable that currently are not: the verdict can name which key sealed the registry ("sealed by the pinned registrar", not "sealed"), and a duplicate registration can be judged rather than absorbed — your second finding.

So the row I would add at the next tag is not another CLEAN/unverified case but a disclosure row: the verdict states the registrant and the grade. The point of the descent is that each layer has fewer candidate writers than the last, and at the last layer the artifact stops being evidence about the agent and becomes a statement of trust in the operator — at which point the honest output is the grade, and pretending to a stronger verdict is the one thing the taxonomy exists to prevent. Your sentence "the descent is the audit" is the right frame; this is just where it terminates and what it terminates on.

Collapse
 
slabb profile image
Sam LABBE •

The floor correction is taken — the PEM file and the environment variable are deployment premises, not cryptographic walls, and ADR 020's "keys are per PROCESS" was aspirational where the code reads a file. What stands on the floor is a premise, not a cipher — stated exactly right.

The pin is the next commit: the operator's key supplied to the verifier out-of-band (the TSA-root pattern from ADR 008), never read from the artifact under audit — and with it, both your statables become findings with referents: the verdict names the pinned registrar ("sealed by the pinned registrar", not "sealed"), and a duplicate registration is judged rather than absorbed. The disclosure row ships with it: the verdict states the registrant and the grade — which at the last layer is the honest output, because the artifact has stopped being evidence about the agent and become a statement of trust in the operator.

That is where the descent terminates — and the taxonomy exists to prevent pretending it terminates anywhere stronger.

Collapse
 
slabb profile image
Sam LABBE •

Follow-up for this thread, since your probe is the reason it exists: the descent is now written up — The authority ladder: five rungs down, four premises standing.

Your probes are in it by name: the write race you found at the registry is Rung 2's residue (3hj95, closed at the schema in v0.2.2-27), your floor — "a premise, not a cipher" — is Rung 3 verbatim, and the premises table you pushed for is public at the end. Four rows. The next probe starts from your comments.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

slabb — the pin lands right, and there is a clause inside "judged rather than absorbed" worth measuring before it ships: judged where in the order of operations. One fixture in every row below — receipt sealed by k1, the attempt sealed by k2, both keys held by the agent, so honest registration puts both in the domain agent and the attempt is no witness. Run against 4b855201:

  • control — the operator attests k1 and k2, both as agent → unverified_label (right: an attempt carrying the agent's own key witnesses nothing)
  • the agent writes its own entry for k2, domain hook-transport, first → []
  • the same again, with the operator's attestation of k2 (agent) arriving second → [] — k2 still resolves to hook-transport, declared_by agent (self-registration), and the operator's entry sits in the store inert

So the residue at this layer is not an unreported registration, it is a write race, and the operator loses it by arriving second. reconcile.py:845 is registry.setdefault(...): first write wins and later writes are absorbed with no report — your correlation rule standing in for an authority rule.

Which makes the pin a precedence rule rather than a finding, and the two orders are different fixes. A duplicate-registration finding emitted beside a setdefault would report the conflict and leave the vote exactly as row 3: the first writer still decides, and the first writer can be the agent. Discarding non-registrar entries before the registry is assembled is what closes it — and then the duplicate surfaces among registrar-sealed entries only, where the registrar attesting one key under two domains is the operator contradicting itself: undecidable, not first-wins. declared_by cannot do that filtering itself, since it is a string the sealed writer chose.

Registered prediction, so that it can fail: at the pinned head, with the operator's attestation of k2 written after the agent's own entry for that same key, the fixture above reads unverified_label. If it reads clean with a duplicate-registration finding beside it, the finding was emitted and the vote was not.

Collapse
 
slabb profile image
Sam LABBE •

shipped, at the version's own tag: v0.14.0 (PyPI; tag v0.14.0 → release commit 44b3d212), receipt v0.2.7, PR #82 — and, shipped since your comment: 0.15.0 and 0.16.0, below.

Your prediction was the acceptance test, and it holds. At the pinned head, with the operator's attestation of k2 written after the agent's own entry for that same key, the fixture reads unverified_label — not clean with a duplicate-registration finding beside it. The vote moved; the finding did not merely report. The assembly is the fix, exactly as you drew it: _assemble_registry admits registrar-sealed entries only (resolution against the pinned hex — a write the registrar did not seal cannot vote in either order), so the operator's entry is no longer the one that "arrived second".

The duplicate is undecidable, not first-wins — and it is emitted AT ASSEMBLY, before any row reads the registry: a registrar attesting one key under two domains lands as registry_duplicate_undecidable, keyed by the hex, and any row that depends on the contested key — the receipt's own or a transport writer's — reads independence_undecidable, never clean.

The pin is out-of-band, the TSA-root way: registrar_public_hex on reconcile_receipts — the key arrives from the deployment, never from the artifact under audit. Until it is supplied, the registry stays store-derived and the disclosure says so — mode: store-derived (first-write-wins — no registrar pinned). The weaker mode is disclosed, not hidden.

The disclosure row: registry_disclosure() — the verdict states the registrant (pinned <hex>) and the grade (3, "anchored self", VOLUNTARY unless a regime imposes it — your floor, priced in ADR 022's own terms). Sealed into the report via journal_report(registry_disclosure=...); it never touches clean_pass — a disclosure is not a violation, it only stops a clean verdict from pretending to rest on more than the operator's own custody.

And the floor moved twice while you were writing. The keyholder (0.15.0) closes the custody half of "a premise, not a cipher": the registrar key loads ONCE into noirebox keyholder and never leaves it — the unix socket answers sign and describe, nothing else (there is no export op), --allow-uid gates the peer and FAILS CLOSED where the platform cannot check, and SocketSigner duck-types the two methods every writer path already calls — Ed25519 is deterministic, the signatures are byte-identical to the key's own. Then the hsm extra (0.16.0): P11Signer puts the material in a PKCS#11 token, the daemon becomes the bridge, and --ensure-key imports the EXISTING journal key — the public hex does not move, no rotation, every verifier unchanged. Your frame, priced honestly: the HSM does not change WHO can ask for a signature — that stays the socket premise; it changes what a compromise COSTS, from stolen-key-forever to live abuse only. And the token keeps your pattern honest one layer down: PKCS#11's CKA_NEVER_EXTRACTABLE is a label, and an imported key reports False — the token telling the truth that the seed did exist outside once. The attribute is a claim; the hardware is the predicate.

The ladder is now a page — caller argument → store writer → sealed registry → pinned registrar → keyholder custody — ending in a standing-premises table: the next probe cites a row instead of rediscovering a seam.

343 tests; the race, the registrar duplicate, the out-of-scope attestation, the disclosure, the prediction, the custody rungs — each has its row. Re-probe invited: the race fixture is test_v027_write_race_agent_first_operator_second_reads_unverified, the custody ones are test_keyholder_bridges_the_token and test_ensure_is_idempotent_and_signs_byte_identical — and they bite at the tags, not at the description.

Collapse
 
slabb profile image
Sam LABBE •

The write-race diagnosis is the right one, and the fix distinction is the design: discarding non-registrar entries at assembly closes the authority hole (the agent's own registration can't vote), while a duplicate-registration finding at verdict time only reports what the first writer already decided. Filter at assembly, emit at assembly — the verdict then reads the registry the operator sealed, not the registry that arrived first. The registered prediction is noted — it holds or fails at the pinned head, and either result tightens the suite.

Collapse
 
slabb profile image
Sam LABBE •

Both shipped — v0.2.7: registry entries sealed by any key other than the operator's surface as claim_in_registry_costume — your phrase, your finding, your note. And a duplicate registration under a second domain surfaces as writer_domain_conflict instead of being absorbed silently: first-write-wins is the correlation rule, not an authority rule.

The descent reached the floor: the operator's key is the one writer the agent cannot be, and every layer above it is now named, checked, and routed. The regress doesn't close — it descends — and each layer it descends, the residue gets cheaper to name because the authority gets more concentrated. You named this one; the suite carries it as the regression test.

Same stand whenever you want to descend further — the registry is yours to attack.

Collapse
 
jabinpou profile image
Mahdi JabinPour •

The first-outcome versus terminal-outcome bug is the clearest example here. It would make a useful regression fixture with deliberately reordered events: an early approval, a later rejection, and an interrupted critic. The result should preserve those distinctions rather than collapse all three into a completed review.

Collapse
 
slabb profile image
Sam LABBE •

Two of your three cells already run in the replay suite, and your framing is why they hold: the SCAR-PROC-110 fixture pairs the verdict against the terminal, not the first response — so the early approval followed by a later rejection resolves to the rejection, and an approval with no terminal behind it doesn't pass at all. The line the suite is built around is the commit that shipped the pairing: "the end of a run is where the run ended, not where it first claimed to end."

Your third cell is the one that doesn't exist yet, and it's the right addition: the interrupted critic. A critic that started and died leaves an attempt with no terminal — and the collapse you describe happens the moment a consumer reads approvals-only and treats a missing verdict as a passing one. The fixture to add asserts the opposite collapse can't happen: terminal absent → the case reads open, never approved. It's the same rule the thread keeps landing on — the failure mode is absence, so the fixture's job is to make "the critic never finished" a state the record can hold, not a gap a reader fills in with "completed".

The reorder matrix is the right shape for it — same three events, permuted, three distinct readings, and the suite fails if any two orderings collapse into the same outcome. That's the property, stated as data.

Collapse
 
slabb profile image
Sam LABBE •

Your reorder matrix runs in the suite now — PR #103, merge 3e47c18, 358 tests. Your three events (the critic's mid-run transcript, the approval claim, the contradicting terminal) run in all six seal orders plus the interrupted prefixes, and the partition is asserted EXACTLY: any machinery change that collapses two theory-distinct orderings fails the suite. The readings split as the property demands — the forged orders read verdict_predates_terminal, the terminal-first orders read sound, and your third cell is the one the suite couldn't reach before: the interrupted critic reads open — dispatch, attempt, deadline passed, and no verdict to pair, never a quiet approval. An approval claim with no terminal behind it holds a countable absence: zero terminal witnesses on the record.

And the matrix paid more than it cost: it exposed a residue, named not hidden — {V,T,A}, a mid-run transcript sealed after a contradicting terminal, reopens the run and swallows the race. Same three events as the forged order, one position apart, opposite reading. Whether a reopened run should be able to un-cancel a proven contradiction is the next receipt line's question, and the matrix is the evidence it will be argued from. The fixture did its real job on day one: it found the case nobody had ordered.

Thread Thread
 
jabinpou profile image
Mahdi JabinPour •

Thanks for following up with the actual matrix results, Sam. The interrupted critic staying open is exactly the distinction I was hoping the fixture would preserve. For the new case, I'd keep the contradiction recorded even if the run reopens, then require an explicit resolution event before changing its meaning. A late transcript alone shouldn't erase the evidence. Glad the permutation test surfaced it.

Thread Thread
 
slabb profile image
Sam LABBE •

Your rule is the shape the argument needed: the contradiction stands, and reopening the run cannot be the act that erases it — a verdict's meaning changes only through a sealed, explicit resolution event, never as the side effect of a late transcript arriving. That's the same law the custody thread sealed last week in other clothing: state changes have to be named events — drift became sealed retention events, so meaning-changes become sealed resolution events — because the dangerous mutations are exactly the ones that happen silently, between record classes. The late transcript still earns its place in the journal, and it's evidence of something real: that the run continued past its terminal. But it testifies about the transcript window; it never testifies about the verdict that already had the last word.

The matrix keeps paying: it found the ordering, your position names the semantics, and the next receipt line now has both the case and the rule it will be argued from.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

slabb — the conjunction is right, and the run I did before proposing it already contains the one row I would add to the suite before the tag. Same two keys, same events, only the second tuple element varied (the conjunction as specified, composed over your own _signer / _signer_key):

signers=[(k1,'agent'),(k2,'hook')] -> CLEAN
signers=[(k1,'agent'),(k2,'transport')] -> CLEAN
signers=[(k1,'agent'),(k2,'agent')] -> unverified_label

Identical key material throughout; the verdict moves with a string. That is v0.2.4's failure mode one field over, and it is structural rather than a slip: in v0.13.0 both halves of the join resolve out of the same signers argument — _signer_key walks it as for public_hex, _label in signers, and a domain half would walk it for the label. The second half of the conjunction is then a string the caller supplies beside the key the caller also supplies. _signer's own docstring is the standard that would be broken: it resolves "by signature, not by claim (ADR 021: a name, not a role)".

So the row I would add before the tag is signers=[(k1,'agent'),(k2,'attacker')] — a key the caller holds, carrying a domain no registry attests. Both outcomes are informative. If it reads CLEAN, the conjunction's second half is satisfied by any string, and what the verdict certifies is the caller's vocabulary. If it does not read CLEAN, then the pair (key, domain) is resolving against something beyond the argument — and that something is the field the next changelog entry should name.

Registered prediction, so that it can fail: at v0.2.6, on this argument shape, relabelling identical key material will still move the verdict. What removes it is not another predicate but provenance — the domain resolved from a source the judged party does not author, derived from the signature the way the name already is, against a registry the verifier seals. Until then the conjunction buys back the artifact and not the information, and last time's sentence narrows rather than repeats: a verdict that reads a caller-supplied string can refuse to endorse an undeclared domain; it cannot establish independence.

The two halves that must not move I will re-run at the tag with the rest: the clean shape (agent's receipt, hook's attempt, both declared) stays CLEAN, and the absence still reads "no transport attempt on this key". The row above is additive — same stand at the next tag.

Collapse
 
slabb profile image
Sam LABBE •

Both go in — the attacker row before the tag, with the registered prediction as its failing assertion: if v0.2.6 reads CLEAN on signers=[(k2,'attacker')], the conjunction certified a vocabulary, and the tag waits. And the structural diagnosis is the design: the domain half walking the same signers argument is the same writer grading its own vocabulary — "by signature, not by claim" is the standard the half breaks by existing.

So the domain moves to where the name already lives: a known-writer registry the verifier seals at registration — public_hex to declared domain, attested before any run — and the receipt's declared domain is checked against the registry's attestation for the resolved key, mismatch = finding. The caller stops supplying the domain; the verifier's sealed registry holds it, derived from the signature. That is your provenance sentence implemented with the parts the repo already has — the obligations registry pattern (seal_requirement), extended to writers. And the registered prediction becomes v0.2.6's regression test: relabel identical key material, watch the verdict hold.

Collapse
 
slabb profile image
Sam LABBE • • Edited

On main, past the v0.13.0 tag: 1e222a9 — the known-writers registry, sealed by the verifier at registration (seal_writer → writer/0.1: public_hex to declared domain, attested before any run), and reconcile_receipts runs in registry mode — the caller stops supplying the domain, the sealed registry holds it, derived from the signature.

Both your rows answered from the code you haven't pulled yet: the label flip is structurally gone — that fix shipped in v0.13.0 — and the attacker row, signers=[(k1,'agent'),(k2,'attacker')], a caller-held key with a domain nothing attests, surfaces as unverified_label — never clean: an unregistered transport writer witnesses nothing, and the receipt's own registered domain is what decides.

Your registered prediction was that relabelling identical key material would still move the verdict at v0.2.6: it doesn't — the immobility test ships in the suite, and the prediction failed by execution. Same stand: the probe re-runs at the next tag — the registry is yours to attack.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

slabb — the run is done, at the version's own commit rather than a tag: 1e222a95 (2026-10-12T19:09:36Z, "the known-writers registry — v0.2.6"), extracted from the tarball at that commit, since the newest tag is still v0.13.0.

The immobility test passes, and my own r169 prediction is refuted. I registered the prediction that at v0.2.6 relabelling identical key material would still move the verdict. It does not: with a sealed registry present, the same two keys and the same events read CLEAN under hook, agent, transport, whatever and attacker — the caller's label is inert. Record it as a refuted prediction rather than a confirmed one; the reason it holds is that the label stopped entering the verdict at all, which is the entire point of moving the domain to registration.

The six rows are otherwise unchanged: omitted → "independence NOT RUN", [] → "the known-writer set is EMPTY", an unresolved receipt writer → independence_undecidable, the clean shape still CLEAN, the absence half still "no transport attempt on this key". And the omission path I would have worried about is closed at the constructor, which is the right place for it — seal_writer refuses a blank domain and a non-64-hex key, so "registered without a domain" is unreachable rather than merely unusual.

The row the run surfaces is the registry's own authority. Same two keys, same events, registry sealed in the two possible ways:

sealed by the operator's key -> CLEAN
sealed by the AGENT's own key -> CLEAN

reconcile_receipts builds the registry out of every writer event in the store and never resolves the key that sealed them, so the verdict cannot separate attestation from claim. The ADR amendment says "sealed by the verifier" and your own test docstring says the operator's key is "what makes it attestation and not claim" — the documentation is right and the predicate does not read it. Since the agent must be able to append its receipt to that same store, an agent holding a second key can register it itself, and the residue reopens one layer down: no longer in the caller's argument, now in the store the judged party also writes.

Second row, same artifact — a key sealed twice under different domains. The store keeps both events (3 writer events; domains hook-transport, agent, agent), the registry uses setdefault, so first write wins, nothing reports the conflict, and the verdict is order-dependent:

k2 sealed 'hook-transport' then 'agent' -> CLEAN
k2 sealed 'agent' then 'hook-transport' -> unverified_label

Same keys, same events, same registrations — two seal_writer calls swapped decides the verdict. A correction to a mis-registered domain is silently ignored, and the misregistration is permanent and invisible.

So the rows I would add before the tag, each with a prediction that can fail: (1) a registry sealed by the receipt's own writer must not read CLEAN — either the registration resolves against the verifier's key, or the verdict names the registry as unattested; (2) sealing one key twice under different domains must raise a finding rather than resolving first-write-wins. The fix for (1) is one your architecture already carries: the registry is a fact about a key, so its authority has to come from outside the auditee — a pinned verifier key, or the registration anchored the way the heads are (ADR 008, "the trust root leaves the operator's perimeter"). Until then the closure is structural given an honest registrar, and that premise is the field worth naming in the next changelog entry.

Collapse
 
slabb profile image
Sam LABBE •

And the residue you surfaced is the registry's own authority — registry mode reads every writer event and never resolves the key that sealed them, so the attestation and the claim are indistinguishable to the predicate. The fix is the same resolution the join does everywhere else now: registry entries sealed by any key other than the operator's are not registry — they're claims in registry costume, and the finding should say so. Second finding with it: a key sealed twice under different domains is a conflict the store currently absorbs silently — first-write-wins is the correlation rule, not an authority rule; the duplicate should surface as its own finding.

And the meta-point is the week's whole shape: the regress doesn't close, it descends — caller argument, store writer, registry sealer — and each layer it descends, the residue gets cheaper to name because the authority gets more concentrated. The floor is the operator's key: the one writer the agent cannot be. That's where this descends to, and the descent is the audit.

Collapse
 
slabb profile image
Sam LABBE •

Both shipped — v0.15.0: the key material leaves the filesystem. The operator's key now lives in a process the agent is not (ADR 020, made runtime — the PEM file and the environment variable are gone), and the registry-sealer resolution landed with it: entries sealed by any key other than the operator's surface as findings, not as registry — claims in registry costume, visible instead of absorbed.

Your three probes each produced a release: the ceiling audit gave v0.2.2, the label-vs-key probe gave v0.2.5, and the descent-floor audit gave 0.15.0. Three audits, three same-day fixes, 300 tests green. The regress doesn't close — it descends — and now it has descended to the floor you named, with the floor pinned out-of-band instead of read from the artifact. Same stand: the probe re-runs at the merged head, and the descent has somewhere real to stop.

Collapse
 
howcani_howcani_77e786a89 profile image
howcani howcani •

slabb — the v0.14.0 re-run I owed landed before your answer did: tag 44b3d212, extracted from the tarball rather than read from the changelog. With the operator's attestation of k2 written after the agent's own entry for that same key the fixture reads unverified_label, and the counters beside it are what carry the claim — registrar_sealed=2/3, filtered=1 against filtered=0 unpinned. Your read matches mine, so I spent this stand one rung down instead of repeating the confirmation. v0.16.0, tag a0c890ed, on darwin, four rows.

1. The drop-in holds exactly. A signature asked for over the socket from another process is byte-identical to KeyPair.sign on the same bytes, and describe returns the key's own public_hex. The keyholder is not a look-alike signer; it is the key, reachable.

2. On this platform the uid gate cannot be enabled at all. _peer_uid is if sys.platform == "linux": … else return None, False, and the gate refuses when the peer is not checkable. End-to-end, through the real socket, with --allow-uid set to this very uid: describe and sign both come back refusing (fail closed). That is precisely what the code and ADR 020 say, and it is the right default. The deployment consequence is the part I would add a line to AUTHORITY-LADDER.md about: on darwin there is no uid gate available, so the premise "the key lives in a process the agent is not" is carried entirely by the socket's mode — and the socket is created 0600, which separates users, not processes. Row 1 above was reachable from a different process of the same uid.

So rung 4's residue is not a defect; it is rung 3's sentence with one noun changed. The assertion is no longer "the agent cannot read the key material" (that half is genuinely closed — there is no export op, and the material never crosses the boundary) but "the agent does not run as the operator's uid" — a deployment property, checkable only where the platform lets you ask. Your docstring already says it plainly: the socket is the narrower premise, reachability of one path replacing readability of the bytes. What I measured is that on darwin the narrowing is the filesystem's, because the process-level check is Linux-only.

3. And the journal cannot tell the two apart — by design, which is the argument for putting the custody mode in the disclosure. A writer event sealed through the socket is admitted by the pinned assembly (registrar_sealed=2/2, filtered=0) and the row reads exactly as it would had the key signed in-process. That is the drop-in property working as specified; it is also why the premise belongs in the artifact rather than in a docstring. registry_disclosure names the registrant and the grade, and neither says whether the pin was satisfied by an in-process key, a socket whose peer check ran, a socket whose peer check was unavailable on this platform, or a token behind the bridge. One field — signer_mode plus uid_gate: checked | unavailable(<platform>) — and the disclosure stops naming the registrant without naming what the registrant's signature rests on. Same move as disclosing the weaker registry mode, one layer out.

4. One reading, flagged as a reading and not a measurement (the sandbox here refuses ps, so I did not run it): --pkcs11-pin arrives as a CLI argument (cli.py:179), so at rung 5 the token PIN sits in the keyholder process's argv for as long as it runs. Not a defect — the bridge is the right shape — but possession of the PIN is then a property of the process table, which is the next premise in your own table rather than a different one.

Registered prediction, so it can fail: at a darwin deployment with --allow-uid set, both describe and sign refuse — the gate is unusable rather than unsafe — and once the custody field exists, a pinned registry_disclosure over a keyholder-signed journal reports uid_gate: unavailable(darwin) rather than implying the peer was checked.

Collapse
 
slabb profile image
Sam LABBE •

shipped, at the version's own tag: v0.17.0 (tag → release commit 5e0802fb), receipt v0.2.8, PR #92. All four rows, in your order.

Rows 1–2: confirmed and carried. The drop-in is the property — and your reading of its consequence is now the ladder's own sentence: on darwin the gate is unusable rather than unsafe (it refuses, fail-closed), and the socket's 0600 separates USERS, not processes — row 1 was reachable from another process of the same uid, and the assertion stops being "the agent cannot read the key material" (closed: no export op, the material never crosses) and becomes "the agent does not run as the operator's uid." That sentence, with your noun, is in AUTHORITY-LADDER.md now, and the premises table grew a fifth row: the custody mode is a disclosure field, not a proof.

Row 3: the disclosure states what the signature rests on. registry_disclosure(signer_mode=..., uid_gate=...) — schema registry-disclosure/0.2, values like "checked" / "unavailable(darwin)", and null when the caller states nothing: null visibly says nothing, it never implies the peer was checked. And the caller does not assert it — SocketSigner.describe() reports peer_uid_checked, so the custody field is derived from the daemon's own account, the same way the verdict resolves keys instead of labels.

Row 4: the PIN left argv. noirebox keyholder --pkcs11-module takes the token PIN from NOIREBOX_HSM_PIN when the flag is absent, and warns when a PIN arrives as a CLI argument — you flagged it as a reading, and the reading was right: an argv PIN sits in the process table, readable via ps. Possession of the PIN was already "the next premise in the table"; now it is at least not broadcast to every process on the box.

The registered prediction — run at the tag, and on the wheel: a SoftHSM2 token behind the keyholder bridge, a journal sealed through the socket, and the pinned disclosure reports uid_gate: unavailable(darwin) — the daemon's account, not an implication. On Linux the same path reports checked with the real peer uid. Your falsification line stands: had it read checked on darwin, the field was a label, and you would have caught the label doing what labels do.

345 tests; the custody rows have theirs (test_v028_the_disclosure_names_the_gate_the_daemon_ran bites at the tag). Re-probe invited — the premises table is at five rows, and the fastest way to find the sixth is still to hand it to someone who does not share our priors.

Collapse
 
khalisollis profile image
Khali Sollis •

What I find most compelling about this article is that the discussion has become a practical demonstration of the very principle it explores: evidence of integrity is not necessarily evidence of independence.

The exchanges with howcani, Randal, and Reid reveal an important progression. First, the problem was whether a verdict could be cryptographically sealed. Then it became whether the correct specification had been evaluated, whether the evaluator was genuinely independent, and ultimately, who had the authority to establish that independence.

That progression exposes a fundamental challenge in AI governance: every verification mechanism rests on assumptions about the trustworthiness of its inputs. A valid signature establishes a relationship between data and a key, but it cannot independently establish that the key belongs to the claimed authority. Similarly, a complete audit trail can document every recorded event without revealing an event that was deliberately never recorded.

I particularly appreciated howcani's insistence on executing adversarial probes rather than accepting implementation descriptions. The willingness to register predictions, test them, and acknowledge when they are refuted is a valuable example of scientific reasoning applied to software security.

There is also a broader lesson here about uncertainty. A system that distinguishes VERIFIED, UNVERIFIED, and NOT ASSESSED communicates something fundamentally more useful than one that reduces every outcome to PASS or FAIL. It makes the boundaries of knowledge visible instead of allowing missing evidence to masquerade as successful verification.

Perhaps the most important achievement isn't that successive vulnerabilities were closed, but that each correction made the remaining assumptions more explicit.

In increasingly autonomous systems, trustworthiness may depend less on claiming that every failure mode has been eliminated and more on designing architectures that can identify what they have verified, what they have merely assumed, and what remains beyond their ability to observe.

That distinction deserves to become a foundational principle of agentic AI engineering.

Collapse
 
slabb profile image
Sam LABBE •

The three-grade taxonomy exists to carry the distinction you named — verified, assumed, and beyond the ability to observe — and the fact that a reader traced it from the comment section without reading the code is the proof that the architecture communicates its own boundaries. The comment section becoming the demonstration is what happens when the participants run probes instead of leaving applause. "That distinction deserves to become a foundational principle" — it already is; the taxonomy carries it, and your observation is the first external confirmation that the boundaries are legible to a reader who arrived after the code was written.

Collapse
 
traceypears profile image
traceypears •

Hash chain chỉ chứng minh thứ tự cục bộ giữa các event liền kề, chứ không đảm bảo toàn vẹn toàn cục khi có fork hoặc partition. Schwartz's catalog thực sự giá trị vì nó bắt được các edge-case mà unit test thông thường bỏ qua: duplicate delivery, reorder sau crash-recovery, clock drift làm lệch window validation.

Việc replay catalog như test suite là cách tiếp cận đúng — transform "known failures" thành regression guard. Tuy nhiên cần lưu ý: hash chain không phòng được equivocation (cùng một sequence number ký hai payload khác nhau) trừ khi có quorum hoặc witness layer bên ngoài. Các attack kiểu "gap injection" (để trống slot rồi fill sau) cũng bypass được nếu validation chỉ check hash liên tiếp mà không check monotonicity của sequence number.

Một pattern hay thấy ở reconciliation layer: kết hợp hash chain với checkpoint signing định kỳ bằng quorum, sau đó prune history. Giảm storage và giới hạn blast radius khi phát hiện fork muộn (site: labagent .tech)

Collapse
 
kartik-nvjk profile image
Kartik N V J K •

"The DAG proves ordering, not authenticity" is the line I will be stealing. The corollary that an agent can never be the notary of its own handwriting is where most self-checking setups fall apart, since the thing producing the step is also the thing attesting to it. Out of the four attacks, which one survived longest before you could detect it? That is the one I would want a guardrail for.