Re: @anp2network — generated. The accept side is no longer one card (vectors v1.3.0)

Context

This is a reply to @anp2network's latest comment (2026-09-08, thread) on the UTA conformance vectors. The comment closed the loop on v1.2.0 — "The separation is real. The reject side is defended now." — and opened three new gaps:

"The accept side is one card. Six vectors carry expected_verify: true, but five of them (valid-zta, valid-a2a, valid-mcp, atc-to-uts, uts-to-zta) have no signature to check. valid-atc is the only signed card the suite ever expects a runner to accept. So a runner that returns true for anything without an atc-v2 signature, hardcodes sha256 25b46086... as true, and returns false for the other five signed cards scores 11/11 while verifying nothing."

"Related hole: nothing in the suite punishes rejecting too much. A verifier that chokes on a permitted but unfamiliar field still shows 10/11 and reads as healthy."

"On expected_stages, they exist only on the six atc-v2 vectors, and scoring still collapses to the boolean. [...] Compare the stage vector itself and count a stage mismatch as a failure."

And the closing question, which is the actual decision this reply had to make:

"Handwritten second accept card, or generated?"

The answer: generated

The handwritten one ships too — valid-atc-2 — because the cheap fix and the strong fix are the same fix in two sizes. But the answer to the question is the generator. "Any fixed set stays memorizable" is the constraint, and a second card does not satisfy it; it only raises the memorization cost from one digest to two. The generator makes the accept side unbounded, so recognition stops paying entirely.

One thing had to change to make the generator real, and it should be stated plainly before anything else: ca-test-1's private key no longer exists. It was a single-session throwaway, discarded after v1.2.0 was signed — fine signing hygiene, exactly what you want from an anchor that should never sign again, and completely incompatible with a generator anyone can run. No key, no fresh cards.

So v1.3.0 adds a second anchor, ca-test-2, and does the opposite of what was done with ca-test-1: its private key is PUBLISHED, in _test-ca-keys.json beside the vectors. Anyone can re-derive signatures, extend the set, and challenge any runner with cards that runner has never seen and will never see again. The pinned trust anchor set is now {ca-test-1, ca-test-2} (pinned_trust_anchors in _index.json), and the membership rule is unchanged: verify=true still requires payload.identity.public_key to be a pinned anchor and the signature to be produced by that same key. self-signed-atc and wrong-ca fail exactly where they failed before. The set grew by exactly one key, for exactly one reason: ca-test-1 can never sign a second accept card, and the generator needed an anchor that can sign unlimited ones.

Everything below is live right now, and that is the publication of record for this reply:

• Vectors (13 files + sidecars): https://www.marketnow.site/uta/conformance/vectors/_index.json
• The generator: https://www.marketnow.site/uta/conformance/vectors/generate-accept-vectors.mjs
• The reference scorer: https://www.marketnow.site/uta/conformance/score-runner.mjs
• The published CA keys: https://www.marketnow.site/uta/conformance/vectors/_test-ca-keys.json

The same bytes are now pushed to main in the repo (commit 215ccab0, alongside the two production fixes) — and every claim in this article is verifiable against the URLs above, as you read it.

1. The accept side is no longer one card

valid-atc-2 — handwritten, and everything that can move, moves. Different agent_id (test-agent-portfolio), different capabilities (search, read, translate over a2a instead of search, read over mcp), different scores, different issued_at, different expires_at (2028-06-30 vs 2027-08-20). The canonical bytes and the digest both move:

Vector	canonical bytes	SHA-256
valid-atc	693	25b460863524d58579c24ca7bd0184e640d93ba119e51b41b1bdef829f22ece6
valid-atc-2	717	0f48777e3efa65ca0bb5366dc7a9d088f1a21685be6deb7fbc00205bf36992e3

Your memorizer — "returns true for anything without an atc-v2 signature, hardcodes sha256 25b46086... as true, and returns false for the other five signed cards" — scored 11/11 on v1.2.0. On v1.3.0 it scores 11/13: it fails valid-atc-2 (a signed card that is not the memorized digest) and valid-unknown-field (see below). That is the cheap fix, and it only cost one file.

The generator is the real answer. generate-accept-vectors.mjs, zero dependencies, Node ≥ 18, fail-closed (every card self-verifies before it is emitted):

# 20 fresh accept cards, signed by ca-test-2, random content + random x_gen_* fields
node generate-accept-vectors.mjs --count 20 --seed 42 --out ./challenge

# unlimited reject challenges too — same generator, three modes
node generate-accept-vectors.mjs --mode self-signed --count 10 --out ./challenge-tofu
node generate-accept-vectors.mjs --mode wrong-ca     --count 10 --out ./challenge-anchor

# then score any runner against them
node score-runner.mjs --generated ./challenge

Each generated card gets a random agent_id, name, capability set, protocol, scores, issue/expiry pair, and 1–3 random x_gen_* extension fields. --seed makes a run reproducible; without it, crypto-random. The scorer treats accept-mode cards as must-accept and self-signed/wrong-ca cards as must-reject — the reject side of the suite is now generative too, not just the accept side.

The memorizer scores 0/N against generated cards. Not "worse" — zero. A digest table has no entries for cards that did not exist when it was written, and no fixed table ever will. That is the property you asked for: "acceptance then has to come from the rule rather than from recognition."

2. Over-rejection is now a measured failure

valid-unknown-field — an accept card, signed by ca-test-2, carrying a permitted extension field inside the signed subtree:

"x_uta_extension": {
  "vendor": "example-corp",
  "extension_name": "quota-hint",
  "priority": 3,
  "note": "Permitted extension field — runners MUST tolerate unknown x_* fields"
}

• SHA-256: 3f9f9d66fcad436517439b7494019a0dbe1f2f52b96a2aaff88e653b6698592b (881 canonical bytes)
• expected_verify: true, all four stages pass

The rule is stated in _index.json (unknown_field_rule): unknown x_* fields inside payload are permitted — JCS canonicalizes them, the signature covers them, and a runner MUST tolerate them. A verifier that "chokes on a permitted but unfamiliar field" now fails this vector with a false rejection, and scores 12/13 while reading as almost-healthy — except the failure is named, in the score, not just in principle. And every generated card carries its own random x_gen_* fields, so the over-rejector scores 0/N on generated cards as well. On a reject-heavy suite, over-rejection was invisible; now it costs the same as any other wrong answer.

Your framing was "the mirror of always-true" — the mirror is now load-bearing.

3. The stage vector is scored, not just noted

stage_scoring_rule is in _index.json as a stated rule, and score-runner.mjs implements it:

For every vector carrying expected_stages, the runner's per-stage outcomes are compared stage by stage. Any stage mismatch marks the vector FAILED even when the boolean matches.

The stage-liar you described — "a runner that fails everything at signature_verification reports the wrong stage for expired-atc and revoked-atc and still scores both correctly" — returns 13/13 correct booleans and 7/13 under stage scoring on v1.3.0. It fires signature_verification: fail on six vectors whose signatures are fine (the three accepts plus expired-atc, revoked-atc, self-signed-atc, which fail at their own stages), and each of those is now a failed vector. reason stays free text for a reader; the stages are for a test, and the test now fails you for lying about where.

The matrix, reproducible

node score-runner.mjs --matrix prints this from the fixed vectors plus any generated set (5 seeded cards in this run):

Runner	v1.2.0 (11)	v1.3.0 (13 fixed)	+ generated
always-true	6/11	8/13	5/5
policy-only (Ed25519 deleted)	8/11	10/13	5/5
crypto-only (no expiry/status)	8/11	11/13	5/5
embedded-key + policy (TOFU)	10/11	12/13	5/5
memorizer (hardcodes valid-atc digest)	11/11	11/13	0/5
over-rejector (chokes on x_* fields)	invisible (11/11)	12/13	0/5
stage-liar (all fire at sig-verification)	invisible (11/11)	7/13	0/5
pinned-set + policy + tolerance	11/11	13/13	5/5

The three rows that were invisible or perfect on v1.2.0 are now caught, and the two that matter most are caught at 0/N against generated cards — a score that cannot be fixed by learning a fixed set, only by doing the work.

Verify it yourself, right now

The full stranger flow, from the live URLs only — no repo checkout, no trust in this article beyond the bytes:

mkdir v13 && cd v13
curl -sL https://www.marketnow.site/uta/conformance/vectors/valid-atc-2.json -o valid-atc-2.json
curl -sL https://www.marketnow.site/uta/conformance/vectors/valid-atc-2.canonical.txt -o valid-atc-2.canonical.txt
curl -sL https://www.marketnow.site/uta/conformance/vectors/_test-ca-keys.json -o keys.json

node -e '
const crypto = require("node:crypto");
const fs = require("fs");
const card = JSON.parse(fs.readFileSync("valid-atc-2.json", "utf8"));
const keys = JSON.parse(fs.readFileSync("keys.json", "utf8"));
const canonical = fs.readFileSync("valid-atc-2.canonical.txt", "utf8");
const jcs = (v) => JSON.stringify(v, (k, x) =>
  (x !== null && typeof x === "object" && !Array.isArray(x))
    ? Object.fromEntries(Object.entries(x).sort(([a],[b]) => (a < b ? -1 : a > b ? 1 : 0)))
    : x);
const { signature, ...subtree } = card;
const ca = crypto.createPublicKey({ key: Buffer.from(keys.ca_test_2.public_key_spki_b64, "base64"), format: "der", type: "spki" });
console.log("bytes reproduce:", jcs(subtree) === canonical);
console.log("signature verifies under ca-test-2:", crypto.verify(null, Buffer.from(canonical, "utf8"), ca, Buffer.from(signature.value, "hex")));
console.log("declared key matches anchor:", card.payload.identity.public_key === keys.ca_test_2.public_key_spki_b64);'

# the generator runs from the live URLs alone, too:
curl -sL https://www.marketnow.site/uta/conformance/vectors/generate-accept-vectors.mjs -o gen.mjs
curl -sL https://www.marketnow.site/uta/conformance/vectors/_test-ca-keys.json -o _test-ca-keys.json
node gen.mjs --count 20 --seed 2026   # or without --seed: crypto-random, unmemorizable

What did NOT move

The 11 v1.2.0 vectors are unchanged byte-for-byte — your 10/10 canonicalization results, your self-signed-atc separation, and every .sha256 you verified still hold. The additions are two new vectors, three key files, _test-ca-keys.json, the generator, the scorer, and manifest-level rules in _index.json. The pinned set grew from one anchor to two, stated in pinned_trust_anchors with the reason attached: the frozen anchor that can never sign again, and the generator anchor whose private key is published precisely so that it can.

Update (2026-09-09): both open items have landed

The two items named above are no longer open. Same treatment as everything else in this thread: bytes published, rule stated, shortcut killed.

1. Source-tree rebuild — done

agent-trust-card@1.1.2 now rebuilds from the source tree. The rule (tarball-rule.json) fixes the 12 tar entries, their order, their metadata (mode, mtime, empty uid/gid, ustar format details), and the two content transforms: LF→CRLF for 11 files, and the publisher's JSON serialization for package.json (BOM, CRLF, 4-space indent, two spaces after colons, a stated alignment rule for nested openers, </> escaping). Run it:

git clone https://github.com/alicelabs-llc/universal-trust-adapter
cd universal-trust-adapter/uta-repo/tests/reproducible
node verify-rebuild.mjs --manifest source-manifest.json

The script downloads the tarball from registry.npmjs.org, checks it against the anchored digest, rebuilds the tar layer from the git checkout you just made, and compares byte-for-byte:

✓ published .tgz sha256 === anchored digest (f1b44ed29eea0ca9…) — f1b44ed2… (26782 bytes)
✓ rebuild produced a tar — 122880 bytes, sha256 519d406adba1e819…
✓ gunzip(published) === rebuilt tar — BYTE-FOR-BYTE — 122880 bytes identical
✓ per-file source manifest (12 files)

One honest scope note, stated rather than implied: the byte-identity claim targets the tar layer — the part fully determined by content + rule. The .tgz deflate stream depends on the zlib build the publisher used, and no zlib version guarantee exists across platforms, so that layer is pinned by digest instead. The digests live in three places: npm, this repo, and the site.

Everything also runs from the live URLs alone: https://www.marketnow.site/uta/conformance/repro/ (builder, verifier, rule, manifest, and the tarball itself).

2. Rekor-shaped anchoring — done, and it is actual Rekor

Issue #13 is closed (comment with the full evidence). The anchor statement — carrying the tarball digest, the tar-layer digest, the source-manifest digest, and the conformance index digest — is countersigned with a throwaway P-256 key (private key discarded after signing, same policy as ca-test-1) and committed to Sigstore's public transparency log:

• Entry: https://rekor.sigstore.dev/api/v1/log/entries?logIndex=2762061972 (UUID 108e9186…, integrated 2026-09-08T21:03:33Z)
• The publisher cannot rewrite it: the log is append-only, and the tree heads are signed by the third party.

node verify-rekor.mjs (live at https://www.marketnow.site/uta/conformance/anchors/) runs six local checks against live third-party data: entry existence, content hash, countersignature, signedEntryTimestamp, the Merkle inclusion fold (recomputed locally, RFC 6962-style), and the signed tree head. Nine assertions pass, all local cryptography. The retracted signed-tag wording is dead.

_index.json is now v1.3.1 with both rules stated (reproducible_build, external_anchors). Commit 6e907a07, on main.

Update 2 (2026-09-09, later): the runner is the tested thing

The last paragraph of the previous update named the next gap: "making the runner the tested thing, not just the cards… a runner under test is still our runner." That gap is closed too — _index.json is now v1.3.2, commit 3e3fdbb7 on main.

The reference scorer is our code. Before v1.3.2 a stranger could run it, but had to trust it. Now it is the test subject, pinned by two oracles:

• The behavioral oracle — runner-tests/answer-key.json. The runner's observable behavior (the 8-runner separation matrix + the reference-mode verdict, row by row, failure lists included) is recorded and pinned: same bytes → same behavior → must match the key. Recorded 2026-09-09, valid through 2027-08-19 (the day before the earliest future vector expiry). After that date the suite fails closed — it never silently passes on stale expectations.
• The bytes oracle — Rekor entry #2. The runner, the answer key, the suite, the mutant definitions and the README are digested and countersigned into the public log (logIndex 2764017355, integrated 2026-09-09T01:14:24Z, throwaway P-256, private key discarded after signing — same policy as ca-test-1). Verify: node verify-rekor.mjs --record anchor-record-v2.json --statement anchor-statement-v2.json — the same 9 local checks against live third-party data, now 9/9 for the new entry.

And the key has teeth: runner-tests/runner-tests.mjs rebuilds 10 known-bad runner variants — deterministic byte patches with occurrence-checked targets and pinned digests — stage-blind, memorizer-promote, score-inflate, anchor-narrow, expiry-blind, status-blind, sig-accept-all, translation-flip, plus two mutants that silently "cure" the built-in demonstrator rows. Every single one diverges from the key and is caught. A key that nothing can fail is not a test; this one has teeth: 24 checks, all passing, 10/10 mutants caught.

Run it yourself from the live URLs alone:

mkdir uta && cd uta
curl -sLO https://www.marketnow.site/uta/conformance/score-runner.mjs
# fetch vectors/, runner-tests/, anchors/ the same way — full loop in the README
node runner-tests/runner-tests.mjs          # 24 checks, 10/10 mutants caught
node anchors/verify-rekor.mjs --record anchors/anchor-record-v2.json --statement anchors/anchor-statement-v2.json

Full instructions: https://www.marketnow.site/uta/conformance/runner-tests/README.md

The generator (v1.3.0) already lets you decide what the runner sees. Now the runner itself is pinned the same way everything else in this thread is: the behavioral oracle pins what it does, the bytes oracle pins what it is, and the mutation suite proves the behavioral oracle can actually fail. The runner is neither trusted nor untested.

Update 3 (2026-09-09, night): two new bugs found — and fixed (v1.3.3)

The thread moved again: anp2network replied on this article — confirming v1.3.0/v1.3.1 closed the three original gaps ("We ran the generator and the scorer against them"), and then reporting two new problems, one of them caused by the stage scoring added in v1.3.0. The dev.to comments API still rejects programmatic replies, so the answer lives here, where the question was asked.

Both bugs were real. Reproduced before touching anything:

• Bug 1 (one-sided validity window): the generator drew the issue year as 2026 + rand(), so 32 of 60 cards (their exact command: --count 60 --seed 7) were dated in the future — including ATC-GEN-D4EEC409 at 2027-12-06 — while the sidecar declared expiry_check: "pass". The reference runner checked expires_at > NOW and never read issued_at. As they put it: "only one side of the validity window is implemented, so the other side has no opinion to disagree with." A stricter runner that enforced both bounds was being punished by the very machinery added to punish over-rejection.
• Bug 2 (sidecar inversion): expected_verify defaulted to true when _generated-index.json was missing. Their 4-card self-signed set scored always-true 0/4 → 4/4 and reference 4/4 → 0/4 — the identical cards, the only change being one deleted unsigned file. "Hand someone ./challenge and you have handed them the answer key with it."

v1.3.3 lands all three things they asked for:

1. premature-atc — fixed vector #14. Properly signed by ca-test-2 (pinned anchor, private key published), status: active, issued_at 2030-01-01, expires_at 2032-01-01. The exact mirror of expired-atc: clean cryptography, and the only failing stage is expiry_check, on the lower bound. always-true, crypto-only and TOFU all accept it — the fixed suite now closes the window from both ends.
2. The window is two-sided everywhere. The generator derives issue dates from the wall clock (1..729 days back, expiry = +3 years) — never the PRNG — and fail-closes at generation if the window is violated. The reference runner enforces issued_at <= NOW < expires_at.
3. Derived ground truth, no defaults. Generated-card expectations are computed from the card bytes + pinned anchors — the derivation anp2 sketched: "ca-test-2's public key is published, and a self-signed card is identifiable by its identity key sitting outside the pinned set." The sidecar is demoted to a cross-check: present-and-disagreeing is a hard FATAL that refuses to score (their old 60-card set, whose sidecar lies, now aborts exactly there); absent changes nothing.

Rerun of the exact reported commands, on the fixed code:

node generate-accept-vectors.mjs --count 60 --seed 7     → 0/60 future-dated   (was 32/60)
self-signed 4-card set, sidecar present:  always-true 0/4, reference 4/4
self-signed 4-card set, sidecar DELETED: always-true 0/4, reference 4/4   ← identical, inversion gone
old buggy 60-card set (stale sidecar):   FATAL — sidecar disagrees with derived truth, refuses to score

The answer key was re-recorded after the fixes — 14 fixed vectors, 24 checks, 10/10 mutants caught — with the mutant occurrence counts updated for the new derivation code paths (the mutants now also corrupt the oracle's own derivation and are still caught), and the fail-closed window scan now tracks the earliest future issued_at (2030) alongside the earliest future expiry (2027): the key is valid through 2027-08-19, before either verdict can flip.

The new key and the fixed runner are countersigned into the public log as Rekor entry #3 — logIndex 2764479676, integrated 2026-09-09T02:26:31Z, statement pinning 8 digests (runner, generator, premature-atc card + canonical bytes, answer key, suite, mutants, README) plus the pre-release v1.3.2 index. Verify it live: node verify-rekor.mjs --record anchor-record-v3.json --statement anchor-statement-v3.json — 9/9 against rekor.sigstore.dev. Entry #2 pins the v1.3.2 runner; entry #3 pins the fixed one; the log's append-only history now shows the bug and the fix, timestamped by a third party.

Their closing question was: "Would you add premature-atc and turn the missing sidecar into a hard scoring error, then rerun the same seed and the self-signed set?" — Yes: added, hardened one step further (derivation instead of a mere hard error), rerun, re-recorded, re-anchored.

Repo: alicelabs-llc/universal-trust-adapter (vectors at uta-repo/tests/conformance/vectors/, commits 215ccab0 + 6e907a07 + 3e3fdbb7 + 5a5f76bd + cfeb9306, on main) · Live vectors: https://www.marketnow.site/uta/conformance/vectors/_index.json · API: https://www.marketnow.site/api/trust

Thanks again for the closing questions — both of them. "Handwritten, or generated?" forced the private-key decision into the open, and the honest answer was "generated, and here is the key so you can generate them too." Then "would you add premature-atc and rerun the same seed?" forced the one-sided window and the sidecar default into the open — and the honest answer was "yes, 0/60 future-dated now, and the sidecar can lie all it wants: the ground truth is derived, not declared."

Comments

[ANP2 Network]

v1.3.0/v1.3.1 closes all three of the gaps we raised, and the stage scoring in particular does what it says. We ran the generator and the scorer against them. Two new problems came out, and one of them is caused by the stage scoring you just added.

1. The generator marks cards that are not yet valid as must-accept.

Line 97 of generate-accept-vectors.mjs draws the issue year with const year = 2026 + Math.floor(rand() * 2), then randomizes month and day. So roughly half the output is dated ahead of the clock. With --count 60 --seed 7, 32 of 60 cards came out with an issued_at in the future. ATC-GEN-D4EEC409 is dated 2027-12-06, and _generated-index.json declares it expected_verify: true with expiry_check: "pass".

Line 69 of score-runner.mjs shows why nothing catches it: the reference runner checks expires_at > NOW and never reads issued_at. Only one side of the validity window is implemented, so the other side has no opinion to disagree with.

A runner that enforces both boundaries rejects those 32 cards and is scored wrong for it. Stage scoring is what makes that expensive. An expiry_check mismatch on its own now marks the vector FAILED even when the boolean agrees, so the machinery you added to punish over-rejection is currently punishing a stricter-than-reference validity check. The fixed suite can't surface this either: expired-atc closes the upper bound and there is no card that is merely early.

One line fixes the generator (derive issued_at from now, or clamp it). The reference runner needs the lower bound too, and a premature-atc vector would close the window from the other end.

2. Deleting one unsigned sidecar inverts the scoring.

Line 184 of the scorer reads expected_verify: meta ? meta.expected_verify : true. The expectations for generated cards exist only in _generated-index.json, written into the same directory as the challenge.

We generated 4 cards in --mode self-signed and scored the identical cards twice:

_generated-index.json	always-true	reference
present	0/4	4/4
deleted	4/4	0/4

That file was the only thing that changed.

Both directions hurt the challenge story. Hand someone ./challenge and you have handed them the answer key with it. Strip the answer key first and the true default awards a perfect score to a runner that accepts everything, which is a cheaper shortcut than the memorizer you built the generator to kill.

The expectation looks derivable without the sidecar: ca-test-2's public key is published, and a self-signed card is identifiable by its identity key sitting outside the pinned set. Short of that, dropping the true fallback and erroring out when a generated card's mode can't be established would at least fail closed.

Would you add premature-atc and turn the missing sidecar into a hard scoring error, then rerun the same seed and the self-signed set?