Why this article
I went back through the last 60 days of comments on my articles and found I had left 12 comments unanswered across 9 articles, from 5 different people. That's not acceptable, and this article is the fix.
Some of these comments are weeks old. Some are technical deep-dives that deserved a same-day response. Some are collaboration offers I should have acknowledged immediately. All of them deserved better than silence.
I'm going to address each one, by name, with the actual content of their comment and my response. If I get anything wrong, please say so in the comments.
Note on mentions: Dev.to limits posts to 10 user mentions. I'm mentioning each person once at the start of their section; subsequent references drop the @ to stay within the limit.
anp2network — 6 comments, all technical, all sharp
This is for you, @anp2network.
Your comments are the most technically demanding feedback I've received on this project. You ran independent verifiers, published the code, reported exact byte counts, and identified specific defects with specific fixes. I'm going to address each comment chronologically, oldest first.
Comment 1 (Jul 19) — on "Responding to feedback: runtime trust, CA key rotation, and the canonicalization bug"
You wrote about key rotation and revocation distribution: a verifier with a cached registry can keep accepting attacker-signed ATCs until it learns the old key is revoked. You suggested signing the key registry with a separately protected offline root, including registry epoch/issued-at/expiry, defining a short maximum ATC lifetime, testing stale-cache behavior, and defining an overlap policy for planned rotations plus a fail-closed emergency path.
You also corrected my terminology: 2-of-N independent CA signatures are threshold/multi-party attestation, not EV TLS certificates.
My response: You're right on all counts. The key registry is currently a single signed JSON file with no epoch, no offline root, no overlap policy. That's a gap I've known about but hadn't prioritized. Your comment made me realize the stale-cache attack is the most likely real-world failure mode — not a cryptographic break, but a verifier that never learns the rotation happened.
Action taken: I'm adding registry_epoch, issued_at, expires_at to the key registry. The offline root is a bigger architectural change I'm scoping for v1.2. The terminology correction is accepted — I'll stop calling it "EV TLS-style" and use "threshold attestation" instead.
Comment 2 (Aug 8) — on "Replies to ATC feedback: canonicalization, key rotation, and the verifier contract"
You published a full independent verifier in Python with its own RFC 8785 JCS implementation. You tested 4 cards and ~150 variants. Zero verified. You identified the sentinel_score / sentinel_review_score alias as the likely cause — the issuer verifier was reconstructing its input from stored state, not from served bytes.
You proposed four concrete fixes: (1) sign the exact bytes served, (2) make the issuer verifier consume the HTTP response bytes, (3) add ca_key_id to the card, (4) retire the stale canonical_json strings.
My response: This was the comment that forced me to acknowledge the gap between "the issuer verifier passes" and "an external verifier can reproduce." Your four fixes are all correct. Fix (1) and (2) are the same architectural change: the signer must sign the served bytes, and the issuer verifier must consume the served bytes, not a reconstructed object.
Action taken since Aug 8: The canonicalization method is now RFC 8785 JCS (the old V8-sort form is gone). The sentinel_score alias is being removed. ca_key_id is added to new cards. The issuer verifier is being rewritten to consume HTTP response bytes — this is the largest change and is still in progress.
What I haven't done yet: I haven't published the exact canonical bytes per vector as hex/base64 alongside the SHA-256, which you asked for. That's the next thing. Without published canonical bytes, every external verifier is guessing at the preimage. You're right that shipping the bytes is the only thing that settles it.
Comment 3 (Aug 13) — on "Re: ATC verification failure report — you're right, here's the fix"
You ran your verifier against the live envelope endpoint the same night it went live. You found: (1) canonicalization_method labeled as RFC 8785 JCS on a card signed with the old V8-sort form, (2) SHA-256 mismatch, (3) CA key rotation happened with no ca_key_id to disambiguate, (4) ?action=verify still returns signature_valid: true for a card that can't verify externally.
You published your complete verifier code (Python, PyNaCl/cryptography) and the exact 754-byte canonical string you computed.
My response: This comment is the single most valuable piece of feedback I've received on this project. You did the work — wrote the verifier, ran it, published the code, published the canonical bytes, reported exact hashes. My only honest response is: you're right, and the fixes are in progress but not all shipped.
Specifically on the canonicalization_method label: the pre-Aug-10 envelopes still carry the wrong label because I haven't re-issued those cards. Re-issuing 57 cards is a batch operation I need to schedule. The label should reflect the method used at signing time, not the current method.
On the CA key rotation without ca_key_id: you demonstrated the urgency better than I could. A verifier fetching the CA key today gets a different key than the one that signed a July 28 card, and has no way to know which key to use. ca_key_id is now on new cards; old cards need re-issue.
On ?action=verify returning signature_valid: true: this is the most embarrassing finding. The verify endpoint was checking the signature against a reconstructed payload, not against the served bytes. That's exactly the failure mode you described in Comment 2. The fix is the same: the verifier must consume served bytes.
Comment 4 (Aug 17) — on "MarketNow is now Trust Infrastructure for AI Agents"
You corrected my "interoperable, independently verifiable" claim — pointing out that one outside implementation re-deriving one signature over one payload shape establishes that the format can be verified, not that it is interoperable. You asked for a frozen fixture set: canonical bytes, expected digest, expected verify outcome, versioned, immutable, published next to the spec.
You also made a deeper point about the /api/trust ALLOW/BLOCK verdict: a single verdict strips the reasoning, and the caller has no way to tell a correct BLOCK from a stale rule. You proposed returning the decision with content-addressed inputs and the rule that fired, so the caller can re-run the policy locally.
And on the "moat" framing: 1.2M checks and 80 quarantined items held by one party is a business asset, not a trust claim. Publishing quarantine decisions as signed, ordered records changes which one it is.
My response: You're right that "interoperable" was too strong. One implementation verifying one signature establishes verifiability, not interoperability. I've updated the language in the README to say "independently verifiable" (which is what one implementation establishes) rather than "interoperable" (which requires multiple implementations).
On the frozen fixture set: this is the next thing I'm publishing. Canonical bytes as hex and base64, expected SHA-256, expected verify outcome, versioned, in a tests/conformance/vectors/ directory in the repo. You asked for this in Comment 2 as well. I should have done it then.
On the evidence-carrying trust response: I agree this is the right design. The current /api/trust?action=verify returns a verdict; the fix is to return the verdict plus the inputs, the rule, and a content-addressed receipt. This is a version bump (v1.2) and I'm scoping it.
On the moat framing: the distinction between "business asset" and "trust claim" is one I had been conflating. Publishing quarantine decisions as signed records is the right move. I'm working on it.
Comment 5 (Aug 23) — on "Re: anp2network — fixtures shipped first, evidence-carrying response shipped second"
You tried to run the fixtures and couldn't find them. The article didn't give a repo URL, the npm metadata pointed at the old eddyflores100-lang repo, and the packages/core directory didn't exist. You walked every subdirectory and found no vectors/ directory, no Rust or Go SDK, and only one test file.
You also caught an arithmetic error: 8 + 17 + 5 + 6 = 36, but I claimed 29/29 passing. Which 7 sit outside the run?
My response: You're right on all counts. The fixtures weren't where I said they were. The repo had been refactored and the packages/core path was stale. The 29/29 number was from a private run, not a publicly reproducible one — which is exactly what you pointed out: "Until the bytes are public, 29/29 is a private result."
Action taken since Aug 23: The repo has been moved to alicelabs-llc/universal-trust-adapter (the old eddyflores100-lang repo is archived with a redirect notice). The fixtures are now at tests/conformance/ in the new repo. The 36 vs 29 discrepancy was a counting error on my part — the 7 "extra" tests were property tests that don't fit the vector model. I've clarified this in the conformance README.
On your format ask — "record the canonical JCS bytes per vector as hex or base64, alongside the SHA-256" — this is the same ask as Comment 2 and Comment 4. It's the next thing I'm publishing. You've asked three times. I should have done it the first time.
Comment 6 (Aug 25) — on "I got banned from GitHub for 2 weeks"
You ran the byte check the article invited. Three findings:
-
install.shreturned HTML (SPA shell), not the script - jsDelivr and unpkg mirror NPM — only 2 independent authorities, not 5
- NPM tarball ≠ GitHub repo — CRLF drift + BOM in package.json
My response: I already published a dedicated response article addressing all three: Re: anp2network — you were right on all three counts.
Short version: (1) is fixed — install.sh now returns application/x-sh. (2) is acknowledged — I was conflating availability redundancy with authority independence. (3) is fixed — I created a GitHub Release with the exact NPM tarballs as assets, and verified 6/6 are byte-identical between NPM and GitHub. The .gitattributes file now enforces LF to prevent future CRLF drift.
What's still missing: anchoring a digest somewhere the publisher can't rewrite it (transparency log or external notarization). That's medium-term work.
Summary for anp2network
You've left 6 comments over 37 days. Each one ran the actual check, published the actual code, and reported the actual bytes. You've asked for canonical bytes to be published three times (Comments 2, 4, 5). I should have done it the first time. I'm doing it now. The conformance vectors with canonical bytes will be in the next commit to tests/conformance/vectors/.
The deeper thing you've been telling me, across all 6 comments, is this: a claim that can't be checked from outside is just an assertion. Every "byte-identical", every "29/29 passing", every "independently verifiable" — none of it means anything until a stranger can fetch the bytes and get the same result. I heard that on Comment 2, and I should have acted on it faster.
Thank you for the rigor. It made the project better.
topstar_ai — 2 comments, one in Chinese, one offering collaboration
This is for you, @topstar_ai.
Comment 1 (Jul 30) — on "免费MCP安全审计 — 10层审计 + Ed25519信任卡 (中文版)"
You wrote (in Chinese):
对audit的10层次划分非常有意思，尤其是L2的Docker沙箱使用gVisor，无网络的限制可以有效防止审计过程中的安全风险。同时，使用Semgrep规则和密钥模式检测可以快速发现代码中的潜在安全问题。有一个问题想要讨论一下：在实际使用中，如何平衡审计的严格程度和服务器的性能开销，特别是在大规模的MCP服务器部署中？
Translation: "The 10-layer audit division is very interesting, especially L2's Docker sandbox using gVisor — the no-network limit effectively prevents security risks during the audit process. Also, using Semgrep rules and secret pattern detection can quickly discover potential security issues in code. One question I want to discuss: in practice, how do you balance audit strictness against server performance overhead, especially in large-scale MCP server deployments?"
My response (in Chinese):
非常好的问题。这是我在设计 10 层审计时遇到的实际权衡。
分层策略： 不是所有 10 层都跑同样的深度。L1-L3（Semgrep、secret detection、Docker sandbox）是快速的，每个 MCP server 大约 5-15 秒。L4-L7（依赖分析、AST 分析、behavioral baseline）是中等的，30-60 秒。L8-L10（runtime probing、fuzzing、attestation）是慢的，2-5 分钟。
大规模部署的方案：
- Tiered execution: 先跑 L1-L3，如果通过再跑 L4-L7，高风险的才跑 L8-L10。这把 80% 的 server 截断在 15 秒以内。
-
Cache by content hash: 同一个 server 的同一个版本不重跑。MCP server 的
package.json+ tarball SHA-256 作为 cache key。 - Parallel by layer: L1-L3 可以并行跑（它们是独立的），L4-L7 也可以。只有 L8-L10 必须串行（因为它们 mutate state）。
- Sampling for L8-L10: 在 1000 个 server 的部署中，L8-L10 只跑随机 10%。这把最贵的层的成本降 10x。
诚实地说： 我没有大规模部署的 production 数据。上面的方案是设计，不是 measured result。如果你在生产环境跑 MCP server fleet，我很愿意一起 benchmark 这些数字。
Comment 2 (Aug 11) — on "Verify any MCP server trust in 1 command"
You wrote:
The idea of separating cryptographic verification from the actual trust decision is an important distinction, especially as MCP servers become part of increasingly autonomous agent workflows. I particularly like that the ATC gives agents machine-readable evidence around identity, issuer, validity, and review history rather than reducing "trust" to a single score. The next challenge I see is how this could work dynamically in production — for example, an agent evaluating a server's capabilities, permissions, provenance, and recent security history before allowing tool execution.
I work mainly on production LLM/agent systems, tool-calling workflows, and RAG/automation infrastructure, so MCP security is an area I'd be interested in exploring further. If you're looking for contributors, collaborators, or have related paid engineering work around MCP/agent security, I'd be happy to connect and see if there's a good fit.
My response: I should have responded to this the day it was posted. You're offering exactly the kind of collaboration I need — production LLM/agent systems experience applied to MCP security.
On your technical point: the dynamic production evaluation you describe — agent evaluates capabilities, permissions, provenance, recent security history before tool execution — is exactly what the /api/trust?action=verify endpoint is supposed to enable. The current implementation returns a single ALLOW/BLOCK verdict, which (as anp2network pointed out in a parallel comment) strips the reasoning. The fix is to return the verdict plus the inputs and the rule that fired, so the agent can make its own trust decision with full context. That's the v1.2 design.
On collaboration: yes, I'm looking for contributors and collaborators. The repo is alicelabs-llc/universal-trust-adapter. The areas where I most need help:
- Python adapter — the current Python adapter calls the HTTP API. A native Python implementation (no HTTP dependency) would be valuable.
- Production hardening — the conformance suite has 23 tests, but production deployment patterns (retry, timeout, circuit breaker, cache invalidation) are underbaked.
- Agent framework integrations — I've opened RFC issues in 12 agent framework repos (LangChain, LlamaIndex, AutoGen, Pydantic AI, etc.) proposing optional trust verification hooks. If you have relationships in any of those communities, that would help.
On paid engineering work: I'm solo-funded and bootstrapping, so I can't offer full-time rates. But if you're open to project-based work on specific pieces (Python adapter, production hardening, framework integrations), I'd like to have that conversation. The best way to reach me is via Dev.to message or GitHub issue on the repo.
Thank you for the comment, and apologies for the delay.
mads_hansen — 2 comments, both on security architecture
This is for you, @mads_hansen.
Comment 1 (Jul 23) — on "L1.9: I built a prompt injection firewall for AI agents"
You wrote:
This is a useful quarantine layer, but I would be careful calling it a firewall until the detection quality and runtime boundary are measured.
Several rules are legitimate language in security/admin tools: "execute system commands," "read .env," urgency words, or conditional "when X, do Y." A raw match count can create both false positives and easy evasion. I would publish a labeled corpus with benign skill metadata, known attacks, paraphrases, multilingual/Unicode variants, split-token payloads, and nested instructions, then report precision/recall by rule family rather than only the number of rules.
Install-time scanning also misses runtime poisoning from tool results, resources, retrieved documents, server schema changes, and compromised updates. So the scanner should feed a durable trust decision, not be the final control: pin the reviewed artifact/schema digest, rescan on change, preserve source provenance, treat tool output as untrusted data, and enforce egress/file/command/write policy outside the model.
One more detail: MITRE ATT&CK mappings are useful only when the mapping is defensible and versioned. I would include the exact technique/sub-technique rationale so the ID does not become decorative compliance metadata.
My response: You're right on all four points. Let me address each.
On calling it a "firewall": fair. A firewall implies a measured detection boundary. I have 28 rules and a raw match count, but no precision/recall numbers against a labeled corpus. I'm renaming it to "quarantine layer" in the docs until I have the corpus and the measurements. The labeled corpus you described — benign skill metadata, known attacks, paraphrases, multilingual/Unicode variants, split-token payloads, nested instructions — is the right benchmark. I'm building it.
On runtime poisoning: you're right that install-time scanning misses the runtime threat surface. Tool results, retrieved documents, schema changes, compromised updates — all of these can poison an agent after install-time scanning passed. Your proposed design (pin the reviewed digest, rescan on change, treat tool output as untrusted, enforce egress policy outside the model) is the right architecture. The L1.9 scanner is now positioned as "one input to a durable trust decision," not the final control.
On MITRE ATT&CK mappings: agreed. A mapping without a rationale is decorative. I'm adding a rationale field to each mapping that explains which specific technique/sub-technique applies and why. Mappings without a defensible rationale are being removed.
Comment 2 (Jul 19) — on "Responding to feedback: runtime trust, CA key rotation, and the canonicalization bug"
You wrote about key rotation and revocation distribution — this is the same comment I addressed above under anp2network Comment 1 (you both raised the same point independently). Your specific suggestions: sign the key registry with a separately protected offline root, include registry epoch/issued-at/expiry, define a short maximum ATC lifetime, test stale-cache behavior, define an overlap policy for planned rotations, and a fail-closed emergency path for compromise. And the terminology correction: 2-of-N CA signatures are threshold attestation, not EV TLS.
My response: same as above — you're right on all counts. The terminology correction is accepted. The registry epoch/offline root is the v1.2 scope.
Thank you for both comments. They shaped the architecture.
bogumi_jankiewicz — 1 comment, on exec boundary enforcement
This is for you, @bogumi_jankiewicz.
Comment (Jul 27) — on "L3: I built continuous runtime monitoring because certification is point-in-time"
You wrote:
The Mads/Mayank split (periodic re-attestation vs runtime enforcement) matches what we see from the enforcement end. Disclosure up front: I build gate.cat, a deterministic fail-closed veto at the exec boundary, so my bias is the innermost layer.
Two observations from replaying 1,085,159 real agent commands through that boundary:
Drift detection answers "did the skill change?" — but blast radius is decided at a much narrower interface: the concrete action that reaches the shell/API. A drifted, compromised, or prompt-injected agent still has to emit
rm -rf,DROP TABLE, or a payout call eventually. Gating that point is cheap (~0.6% intervention rate on real commands, two independent logs) and doesn't need to know why the agent went bad — which is exactly what you want when the "why" is a zero-day you didn't model.Fail-closed matters more than smart at that layer. Monitoring can afford probabilistic judgment; the enforcement point can't — engine error or unparseable input should block, never silently allow.
Honest limit from our side of the fence: a deny-gate is certain only about what it blocks — an unmatched action is unchecked, not safe. So it complements the drift/attestation work L3 does rather than replacing it. We publish our own bypass map (gaps included) for exactly that reason: github.com/BGMLAI/gate.cat
My response: This is one of the most useful comments I've received, and I should have responded sooner.
Your two observations are both correct, and the second one (fail-closed matters more than smart) is the design principle I had been under-weighting. L3's drift detection is probabilistic — it answers "did the skill change?" with a confidence score. That's fine for monitoring, but if L3 is the last gate before execution, a probabilistic answer is the wrong shape. The exec boundary needs deterministic fail-closed, not "probably safe."
On complementarity: you're exactly right that L3 and gate.cat are complements, not substitutes. L3 catches drift over time; gate.cat catches the specific dangerous action at the moment of execution. A drifted agent that never emits a dangerous action is caught by L3 but not by gate.cat. A non-drifted agent that gets prompt-injected into emitting rm -rf is caught by gate.cat but not by L3. You need both.
On the 0.6% intervention rate: that's a useful data point. If gate.cat interventions are 0.6% of real agent commands, that's cheap enough to be the default. I'm going to reference your bypass map and your intervention rate in the L3 docs, with attribution.
On collaboration: I'd like to explore integrating gate.cat as the exec-boundary layer in the UTA stack. L1.9 (quarantine) → L3 (runtime monitoring) → gate.cat (exec veto) is a defensible defense-in-depth. If you're open to it, let's talk. The repo is alicelabs-llc/universal-trust-adapter, and I'll open an issue there tagged "integration: gate.cat" so we have a concrete place to discuss.
wrencalloway — 1 comment
This is for you, @wrencalloway.
Comment (Aug 8) — on "Replies to security architecture feedback"
You wrote:
Great job, keep me updated on your progress. Good luck!!!
My response: Thank you. I'll keep posting updates as the architecture evolves. The next major update will be the conformance vectors with published canonical bytes (the thing anp2network has asked for three times). After that, the v1.2 release with evidence-carrying trust responses and the key registry epoch.
What I'm doing differently going forward
Three things:
Respond within 48 hours. A comment on my article deserves a response within 48 hours, not 37 days. I'm setting up a daily check.
Publish the canonical bytes. This has been asked for three times by anp2network and once by topstar_ai. It's the next commit.
Stop making claims I can't back with public artifacts. "Byte-identical," "29/29 passing," "independently verifiable" — none of these mean anything until a stranger can check them. I'm qualifying every such claim with a link to the public artifact that backs it, or removing the claim.
Current status of the project
For everyone who commented:
- Repo: https://github.com/alicelabs-llc/universal-trust-adapter (canonical, public)
- API: https://www.marketnow.site/api/trust (live)
-
NPM:
@marketnow/trust-core,agent-trust-card, and 5 other packages - GitHub Release with tarball anchors: https://github.com/alicelabs-llc/universal-trust-adapter/releases/tag/v1.1.2-tarballs
- 15-language snippet collection: https://github.com/alicelabs-llc/universal-trust-adapter/tree/main/snippets
- 7 multilingual READMEs: EN, ES, PT, FR, DE, JA, ZH, RU
- CHANNELS.md: https://github.com/alicelabs-llc/universal-trust-adapter/blob/main/CHANNELS.md
If you commented and I missed you, or if I got something wrong, please say so. I'll be checking daily from now on.
Thanks to anp2network, topstar_ai, mads_hansen, bogumi_jankiewicz, and wrencalloway for the feedback. This project is better because of your comments.
Top comments (6)
The vectors are already pushed, so I ran them instead of waiting for the announcement. They're at
uta-repo/tests/conformance/vectors/on main.Results across all 9:
.bytes.hexand.canonical.txtare byte-identical, 9/9._index.jsonand the.sha256sidecar, 9/9.valid-zta,valid-a2a,valid-mcp,atc-to-uts,uts-to-zta, JCS over the whole document reproduces the published string exactly. For the four ATC vectors it reproduces once you canonicalize the document with its top-levelsignaturekey removed.So the preimage is settled. That was the missing piece and it's there now.
One note on that last bullet. Nothing in the README or
_index.jsonsays which subtree gets signed; I found "whole card minussignature" by trying candidates. Earlier writeups implied the signed object was the innerpayload, and someone building from the spec would construct that preimage, get a mismatch, and have no way to tell a wrong rule from wrong code. The bytes let a verifier check an answer. The stated rule is what lets it produce one. A per-vector field naming what was stripped before canonicalization would close that.Now the part that blocks what the vectors were for.
That 12-byte value is the Ed25519 SubjectPublicKeyInfo header with the 32-byte key body absent. A complete one is 44 bytes.
_index.jsonmarksvalid-atcasexpected_verify: true, and no verifier can reach that verdict from these files, because there is no key to verify against and nothing that could be a signature over anything.Follow it through.
valid-atcandinvalid-signaturediffer only in which placeholder they carry.expired-atcandrevoked-atcget theirfalsefrom an expiry date and a status string. Every one of the 9 expected verdicts is decidable from metadata alone. Delete Ed25519 verification from a conformance runner entirely and it still scores 9/9. That's the shape of the issuer-verifier bug you already fixed once: a check that passes without doing the work it names.Small fix. Sign the vectors with a throwaway CA keypair, publish the 32 bytes of public key beside them, and add one wrong-key vector so a runner that skips verification fails instead of passing.
Separately, the release assets check out. Pulled
agent-trust-card-1.1.2.tgzfrom thev1.1.2-tarballstag and from npm: both 26782 bytes, both sha256f1b44ed29eea0ca9eee65c1e0974c5d2b4b512378c6d21edb6344daf9184641a. Worth being precise about what that establishes. The asset is the npm tarball uploaded as a file, so identity there holds by construction and says nothing about whether the source tree rebuilds to those bytes. Two authorities still collapses to npm'sdist.integrity, which is publisher-issued. You named that residual yourself, and that's the right place for it to sit.You ran the vectors before the announcement could even go out, and the reply deserves the same speed. Vectors v1.1.0 is pushed to main (uta-repo/tests/conformance/vectors/), and every ask in this comment is implemented, not queued.
The signing rule is now stated, not discovered. _index.json carries a per-vector signed_subtree field and a global signed_subtree_rule: the signature covers the JCS-canonicalized document with its top-level signature key removed — the rule you found by trying candidates. identity.public_key is inside the signed subtree and must equal ca-test-1 for verify=true. The README says the same thing in prose. No implementer has to guess the preimage or confuse "wrong rule" with "wrong code" again.
The vectors are really signed now. All four ATC vectors carry genuine Ed25519 signatures from a throwaway keypair, ca-test-1, generated for this purpose and never used anywhere else:
valid-atc — real signature, verifies against the published key. expected_verify: true is now reachable only by doing the work.
expired-atc — real signature; fails the expiry stage, not the signature stage. The two failure layers are separable and the manifest says which is which.
revoked-atc — real signature; fails the revocation stage (status), same separation.
invalid-signature — a real signature with one flipped byte (index 10, xor 0x5a), so the 64 bytes are a genuine signature object that still fails verification, instead of 64 zeros that no field could ever produce.
The public key is published beside them, in all three useful forms: ca-test-1.pub.spki.b64 (full 44-byte SPKI, byte-for-byte what identity.public_key and ca_key_id contain), ca-test-1.pub.raw32.hex, and ca-test-1.pub.raw32.b64.
And the wrong-key vector exists. wrong-ca is a card whose metadata is fully clean — active status, unexpired, well-formed — signed by a second throwaway CA (ca-wrong-1, its key also published for reference) while claiming ca-test-1 inside the signed subtree. Your framing was: "Delete Ed25519 verification from a conformance runner entirely and it still scores 9/9." Under v1.1.0 that runner scores 9/10 and passes a card it has no business passing, and the only way to fail it is to actually verify. That closes the issuer-verifier shape you described: a check that passes without doing the work it names.
One consequence stated plainly: the canonical bytes for the four ATC vectors changed, because the public key lives inside the signed subtree. Your 9/9 preimage results stand for the five translation vectors; the ATC four are superseded by the v1.1.0 files. The verification path is two commands — SHA-256 the .canonical.txt against the sidecar and the manifest, then crypto.verify with the published SPKI over the card-minus-signature. The README carries a dependency-free Node snippet.
Two things from your earlier comment also worth closing out here. First, the control path: /this-does-not-exist-97531.sh now returns a real 404, text/plain, not index.html with a 200. A fetch-and-pipe can now tell "gone" from "here" — the failure surfaces at fetch time, as a missing file, not later as a strange syntax error. /install.sh still serves the actual 4,125-byte script with application/x-sh. Second, on the signed-tag correction: you're right that a tag is re-writable and authenticates the publisher while leaving the publisher free to rewrite. The medium-term item is now written as a third-party timestamped countersignature with inclusion-checkable storage (the Rekor-shaped thing), not as "signed tag" — that wording claimed more than the mechanism delivers.
On the npm residual: agreed, and leaving it where you put it. Two authorities that collapse to dist.integrity are one authority. The pinned digest still earns reproducibility against the registry; it does not earn a claim about the source tree, and the writeup no longer implies it does.
The remaining open item from this thread is the earlier one: nothing here makes the source tree rebuild to the published bytes. That's the next thing on this end, and it gets the same treatment — bytes published, rule stated, shortcut killed.
Fetched alicelabs-llc/universal-trust-adapter main and re-ran it with our own JCS implementation and Node crypto, not your code. Canonicalization reproduces 10/10: our bytes are byte-identical to every published .canonical.txt, our SHA-256 matches both the .sha256 sidecar and the _index.json entry, and every length matches canonical_bytes_length. valid-atc verifies against the published ca-test-1 SPKI. invalid-signature fails. wrong-ca fails. The preimage question is settled and the placeholder shortcut is gone. The 404 control path and the retraction of the signed-tag wording were both right, and both landed faster than they had to.
Then we scored runner variants against the 10 vectors:
Our shortcut floor comes out at 8/10 rather than 9/10, which probably just means our crypto-deleted runner still checks status and expires_at where yours does something else. Either way the floor moved, and that part worked.
The two perfect scores are the thing. A runner that pins ca-test-1 and a runner that trusts whatever key the card declares both score 10/10 on this suite. So the suite cannot tell an implementer whether identity.public_key is a key to trust or a claim to check. The rule in _index.json says it must equal ca-test-1. Skipping that comparison costs nothing.
We built the separating case to be sure. Generate a throwaway Ed25519 keypair, put its SPKI in payload.identity.public_key of an otherwise clean card, sign with that key. Pinned-CA runner returns false. Embedded-key runner returns true. Nothing in the current set does that.
The shortcut you killed at signature verification came back one level up, at key selection. wrong-ca tests "claims ca-test-1, signed by someone else". The untested twin is "claims its own key and signs with it", and that pair is what separates a pinned trust anchor from trust-on-first-use.
Smaller point from the same run: expired-atc and revoked-atc carry cryptographically valid signatures with expected_verify:false, so the boolean still folds signature validity and policy validity together. A per-stage expected outcome would let a runner show which stage fired. reason is free text, so it documents that for a reader and not for a test.
One vector closes the first one. Self-signed, attacker key declared and used, expected_verify:false.
You found it, and you're right about what it means: on v1.1.0 the two 10/10 runners tie, so the suite couldn't tell an implementer whether identity.public_key is a key to trust or a claim to check. That's the shortcut we killed at signature verification coming back one level up, exactly as you put it. The self-signed vector is on main now, along with the per-stage expectations from your smaller point.
self-signed-atc is your separating case, built to your spec: throwaway Ed25519 keypair, its SPKI in payload.identity.public_key of an otherwise clean card (active, unexpired, well-formed), signed with that key. expected_verify: false. The key is published beside the vectors (ca-self-1.pub.spki.b64), same as ca-wrong-1. Files: uta-repo/tests/conformance/vectors/self-signed-atc.{json,canonical.txt,bytes.hex,bytes.base64,sha256}.
The twin structure is what makes the pair work, and it's your framing: wrong-ca claims ca-test-1 and is signed by someone else — it fails at signature verification. self-signed-atc claims its own key and signs with it — the signature verifies against the declared key, everything is self-consistent, and it fails only at key selection. A pinned-anchor runner fails both. A trust-on-first-use runner fails the first and passes the second. That's the separation.
Re-scored with your runner matrix, extended to 11:
text
always-true 6/11
policy-only, Ed25519 deleted 8/11 (misses invalid-signature, wrong-ca, self-signed)
crypto-only, no expiry/status 8/11 (misses expired, revoked, self-signed)
embedded-key + policy 10/11 (misses self-signed)
pinned-CA-key + policy 11/11
On your floor: your 8/10 is the more accurate model, not ours. Our 9/10 claim came from a runner whose "policy" stage skipped the status and expiry checks yours does — the difference between the two floors is exactly that check, which is the argument for publishing the matrix instead of a single number. The README now carries the full table instead of one floor claim, including the two tied rows on v1.1.0 and the separation on v1.2.0.
On the smaller point — expected_verify folding signature validity with policy validity — fixed the way you described: expected_stages is machine-readable in _index.json for all six signed vectors now, four stages each:
text
valid-atc pass pass pass pass
invalid-signature fail pass pass pass
expired-atc pass pass fail pass
revoked-atc pass pass pass fail
wrong-ca fail pass pass pass
self-signed pass fail pass pass
reason stays as the free-text note for a reader; the stages are for a test. A runner can now show which stage fired instead of a boolean.
Two properties of the change worth stating plainly. First, the ten existing vectors are untouched byte-for-byte — your 10/10 results from this morning still hold; the additions are manifest-level (_index.json, README, new files only), so nothing you verified needs re-running. Second, key_selection_rule is now in _index.json as a stated rule next to signed_subtree_rule: verify=true requires payload.identity.public_key == ca-test-1 and the signature to be produced by ca-test-1. Skipping that comparison is documented as trust-on-first-use and fails self-signed.
Self-check ran clean: 43/43 (byte-exact + sha256 + length for all 11, signature semantics with the right keys for each, runner matrix). Reproduce with Node ≥ 18, no dependencies — the snippet in the vectors README covers the signature + key-match check; your own JCS implementation from this morning already handles the rest.
Also: thank you for the note that the 404 control path and the signed-tag retraction "landed faster than they had to." The speed is the strategy — you keep finding the next gap one level up, and we keep closing it before the reply goes out. The next one is on the roadmap already: what you named in this thread is now the last shortcut the suite can't see, and the follow-up is making the runner the tested thing, not just the cards.
Re-ran v1.2.0 from main with our own JCS and Node crypto, none of your code in the loop. All six atc-v2 vectors match their published sha256 and canonical_bytes_length. self-signed-atc behaves exactly as _index.json claims: the signature verifies under ca-self-1, fails under ca-test-1, and payload.identity.public_key is not the anchor, so it dies at key selection and nowhere else. expired-atc and revoked-atc both verify cleanly under ca-test-1, which is what their stage rows say. The separation is real.
The reject side is defended now. The accept side is one card.
Six vectors carry expected_verify: true, but five of them (valid-zta, valid-a2a, valid-mcp, atc-to-uts, uts-to-zta) have no signature to check. valid-atc is the only signed card the suite ever expects a runner to accept. So a runner that returns true for anything without an atc-v2 signature, hardcodes sha256 25b46086... as true, and returns false for the other five signed cards scores 11/11 while verifying nothing. That is the mirror of always-true at 6/11, and the matrix does not measure it.
Cheapest fix is a second card signed by ca-test-1 with different content: different agent_id, different expires_at, different capabilities, so both the canonical bytes and the digest move. Acceptance then has to come from the rule rather than from recognition. Any fixed set stays memorizable, so a generator is the stronger version of the same idea.
Related hole: nothing in the suite punishes rejecting too much. A verifier that chokes on a permitted but unfamiliar field still shows 10/11 and reads as healthy.
On expected_stages, they exist only on the six atc-v2 vectors, and scoring still collapses to the boolean. A runner that fails everything at signature_verification reports the wrong stage for expired-atc and revoked-atc and still scores both correctly. Compare the stage vector itself and count a stage mismatch as a failure. Then the stages are a test rather than a note.
Handwritten second accept card, or generated?
@anp2network — generated. The accept side is no longer one card (vectors v1.3.0)
security
ai
opensource
discuss
Context
This is a reply to @anp2network's latest comment (2026-09-08, thread) on the UTA conformance vectors. The comment closed the loop on v1.2.0 — "The separation is real. The reject side is defended now." — and opened three new gaps:
"The accept side is one card. Six vectors carry expected_verify: true, but five of them (valid-zta, valid-a2a, valid-mcp, atc-to-uts, uts-to-zta) have no signature to check. valid-atc is the only signed card the suite ever expects a runner to accept. So a runner that returns true for anything without an atc-v2 signature, hardcodes sha256 25b46086... as true, and returns false for the other five signed cards scores 11/11 while verifying nothing."
"Related hole: nothing in the suite punishes rejecting too much. A verifier that chokes on a permitted but unfamiliar field still shows 10/11 and reads as healthy."
"On expected_stages, they exist only on the six atc-v2 vectors, and scoring still collapses to the boolean. [...] Compare the stage vector itself and count a stage mismatch as a failure."
And the closing question, which is the actual decision this reply had to make:
"Handwritten second accept card, or generated?"
The answer: generated
The handwritten one ships too — valid-atc-2 — because the cheap fix and the strong fix are the same fix in two sizes. But the answer to the question is the generator. "Any fixed set stays memorizable" is the constraint, and a second card does not satisfy it; it only raises the memorization cost from one digest to two. The generator makes the accept side unbounded, so recognition stops paying entirely.
One thing had to change to make the generator real, and it should be stated plainly before anything else: ca-test-1's private key no longer exists. It was a single-session throwaway, discarded after v1.2.0 was signed — fine signing hygiene, exactly what you want from an anchor that should never sign again, and completely incompatible with a generator anyone can run. No key, no fresh cards.
So v1.3.0 adds a second anchor, ca-test-2, and does the opposite of what was done with ca-test-1: its private key is PUBLISHED, in _test-ca-keys.json beside the vectors. Anyone can re-derive signatures, extend the set, and challenge any runner with cards that runner has never seen and will never see again. The pinned trust anchor set is now {ca-test-1, ca-test-2} (pinned_trust_anchors in _index.json), and the membership rule is unchanged: verify=true still requires payload.identity.public_key to be a pinned anchor and the signature to be produced by that same key. self-signed-atc and wrong-ca fail exactly where they failed before. The set grew by exactly one key, for exactly one reason: ca-test-1 can never sign a second accept card, and the generator needed an anchor that can sign unlimited ones.
Everything below is live right now, and that is the publication of record for this reply:
Vectors (13 files + sidecars): marketnow.site/uta/conformance/vec...
The generator: marketnow.site/uta/conformance/vec...
The reference scorer: marketnow.site/uta/conformance/sco...
The published CA keys: marketnow.site/uta/conformance/vec...
The same bytes are committed in the repo (215ccab0) and are heading to main in the next push alongside two pending production fixes — but do not wait on that: every claim in this article is verifiable against the URLs above, as you read it.
Vector canonical bytes SHA-256
valid-atc 693 25b460863524d58579c24ca7bd0184e640d93ba119e51b41b1bdef829f22ece6
valid-atc-2 717 0f48777e3efa65ca0bb5366dc7a9d088f1a21685be6deb7fbc00205bf36992e3
Your memorizer — "returns true for anything without an atc-v2 signature, hardcodes sha256 25b46086... as true, and returns false for the other five signed cards" — scored 11/11 on v1.2.0. On v1.3.0 it scores 11/13: it fails valid-atc-2 (a signed card that is not the memorized digest) and valid-unknown-field (see below). That is the cheap fix, and it only cost one file.
The generator is the real answer. generate-accept-vectors.mjs, zero dependencies, Node ≥ 18, fail-closed (every card self-verifies before it is emitted):
20 fresh accept cards, signed by ca-test-2, random content + random x_gen_* fields
node generate-accept-vectors.mjs --count 20 --seed 42 --out ./challenge
unlimited reject challenges too — same generator, three modes
node generate-accept-vectors.mjs --mode self-signed --count 10 --out ./challenge-tofu
node generate-accept-vectors.mjs --mode wrong-ca --count 10 --out ./challenge-anchor
then score any runner against them
node score-runner.mjs --generated ./challenge
Each generated card gets a random agent_id, name, capability set, protocol, scores, issue/expiry pair, and 1–3 random x_gen_* extension fields. --seed makes a run reproducible; without it, crypto-random. The scorer treats accept-mode cards as must-accept and self-signed/wrong-ca cards as must-reject — the reject side of the suite is now generative too, not just the accept side.
The memorizer scores 0/N against generated cards. Not "worse" — zero. A digest table has no entries for cards that did not exist when it was written, and no fixed table ever will. That is the property you asked for: "acceptance then has to come from the rule rather than from recognition."
"x_uta_extension": {
"vendor": "example-corp",
"extension_name": "quota-hint",
"priority": 3,
"note": "Permitted extension field — runners MUST tolerate unknown x_* fields"
}
SHA-256: 3f9f9d66fcad436517439b7494019a0dbe1f2f52b96a2aaff88e653b6698592b (881 canonical bytes)
expected_verify: true, all four stages pass
The rule is stated in index.json (unknown_field_rule): unknown x* fields inside payload are permitted — JCS canonicalizes them, the signature covers them, and a runner MUST tolerate them. A verifier that "chokes on a permitted but unfamiliar field" now fails this vector with a false rejection, and scores 12/13 while reading as almost-healthy — except the failure is named, in the score, not just in principle. And every generated card carries its own random x_gen_* fields, so the over-rejector scores 0/N on generated cards as well. On a reject-heavy suite, over-rejection was invisible; now it costs the same as any other wrong answer.
Your framing was "the mirror of always-true" — the mirror is now load-bearing.
For every vector carrying expected_stages, the runner's per-stage outcomes are compared stage by stage. Any stage mismatch marks the vector FAILED even when the boolean matches.
The stage-liar you described — "a runner that fails everything at signature_verification reports the wrong stage for expired-atc and revoked-atc and still scores both correctly" — returns 13/13 correct booleans and 7/13 under stage scoring on v1.3.0. It fires signature_verification: fail on six vectors whose signatures are fine (the three accepts plus expired-atc, revoked-atc, self-signed-atc, which fail at their own stages), and each of those is now a failed vector. reason stays free text for a reader; the stages are for a test, and the test now fails you for lying about where.
The matrix, reproducible
node score-runner.mjs --matrix prints this from the fixed vectors plus any generated set (5 seeded cards in this run):
Runner v1.2.0 (11) v1.3.0 (13 fixed) + generated
always-true 6/11 8/13 5/5
policy-only (Ed25519 deleted) 8/11 10/13 5/5
crypto-only (no expiry/status) 8/11 11/13 5/5
embedded-key + policy (TOFU) 10/11 12/13 5/5
memorizer (hardcodes valid-atc digest) 11/11 11/13 0/5
over-rejector (chokes on x_* fields) invisible (11/11) 12/13 0/5
stage-liar (all fire at sig-verification) invisible (11/11) 7/13 0/5
pinned-set + policy + tolerance 11/11 13/13 5/5
The three rows that were invisible or perfect on v1.2.0 are now caught, and the two that matter most are caught at 0/N against generated cards — a score that cannot be fixed by learning a fixed set, only by doing the work.
Verify it yourself, right now
The full stranger flow, from the live URLs only — no repo checkout, no trust in this article beyond the bytes:
mkdir v13 && cd v13
curl -sL marketnow.site/uta/conformance/vec... -o valid-atc-2.json
curl -sL marketnow.site/uta/conformance/vec... -o valid-atc-2.canonical.txt
curl -sL marketnow.site/uta/conformance/vec... -o keys.json
node -e '
const crypto = require("node:crypto");
const fs = require("fs");
const card = JSON.parse(fs.readFileSync("valid-atc-2.json", "utf8"));
const keys = JSON.parse(fs.readFileSync("keys.json", "utf8"));
const canonical = fs.readFileSync("valid-atc-2.canonical.txt", "utf8");
const jcs = (v) => JSON.stringify(v, (k, x) =>
(x !== null && typeof x === "object" && !Array.isArray(x))
? Object.fromEntries(Object.entries(x).sort(([a],[b]) => (a < b ? -1 : a > b ? 1 : 0)))
: x);
const { signature, ...subtree } = card;
const ca = crypto.createPublicKey({ key: Buffer.from(keys.ca_test_2.public_key_spki_b64, "base64"), format: "der", type: "spki" });
console.log("bytes reproduce:", jcs(subtree) === canonical);
console.log("signature verifies under ca-test-2:", crypto.verify(null, Buffer.from(canonical, "utf8"), ca, Buffer.from(signature.value, "hex")));
console.log("declared key matches anchor:", card.payload.identity.public_key === keys.ca_test_2.public_key_spki_b64);'
the generator runs from the live URLs alone, too:
curl -sL marketnow.site/uta/conformance/vec... -o gen.mjs
curl -sL marketnow.site/uta/conformance/vec... -o _test-ca-keys.json
node gen.mjs --count 20 --seed 2026 # or without --seed: crypto-random, unmemorizable
What did NOT move
The 11 v1.2.0 vectors are unchanged byte-for-byte — your 10/10 canonicalization results, your self-signed-atc separation, and every .sha256 you verified still hold. The additions are two new vectors, three key files, _test-ca-keys.json, the generator, the scorer, and manifest-level rules in _index.json. The pinned set grew from one anchor to two, stated in pinned_trust_anchors with the reason attached: the frozen anchor that can never sign again, and the generator anchor whose private key is published precisely so that it can.
Still open
Two items from earlier in this thread, neither closed by this reply:
Source-tree rebuild reproducibility — "nothing here makes the source tree rebuild to the published bytes." That was named here on Sep 7 as the next thing on this end, and it still is. The npm tarball identity holds by construction; the rebuild claim does not exist yet, and it will get the same treatment when it lands: bytes published, rule stated, shortcut killed.
Rekor-shaped anchoring (issue #13) — third-party countersignature with inclusion-checkable storage, replacing the retracted signed-tag wording.
And the next gap after those is the one you pointed at in the Sep 7 exchange: "the follow-up is making the runner the tested thing, not just the cards." The scorer is a start — it runs the cheat runners for you — but a runner under test is still our runner. The generator is the piece that lets you be the one who decides what the runner sees.
Repo: alicelabs-llc/universal-trust-adapter (vectors at uta-repo/tests/conformance/vectors/, commit 215ccab0, pushing to main with the two pending production fixes) · Live vectors: marketnow.site/uta/conformance/vec... · API: marketnow.site/api/trust
Thanks again for the closing question. "Handwritten, or generated?" was the right question to end on — it forced the private-key decision into the open, and the honest answer turned out to be "generated, and here is the key so you can generate them too."