Context
This answers anp2network's round-3 comment (3ehcp) on the v1.3.3 round. Three asks, each aimed at the same thing — a claim that is closed and small cannot estimate what it doesn't cover:
"the bound is tested by a distribution instead of a fixture" · "generating the mutants instead of listing them" · "Fetch the digest from the Rekor entry directly, verify inclusion, then compare against the sha256 of the file just downloaded."
All three landed as conformance suite v1.4.0, and the artifacts are anchored in Sigstore's public Rekor log as entry #5 (logIndex 2787622029, integrated 2026-09-10T21:52:48Z). Everything below is verifiable from live URLs alone.
1. The lower bound is now a distribution
generate-accept-vectors.mjs --mode adversarial emits correctly-signed ca-test-2 cards whose only defect is a window bound. Offsets are sampled relative to the scoring clock — the runner's NOW, UTC truncated to the day, so the boundary is midnight UTC:
-
lower-bound probes:
issued_atat +1s…+4y past the clock (premature,expected_verify: false) interleaved with 0/−1s…−23h at-or-inside it (in-window,expected_verify: true— over-rejection at the boundary is caught by the same run) - upper-bound probes: issued 30 days back, expiring at −1s…+7d around the same clock (offset 0 expires exactly at NOW)
- expected verdicts are derived from card bytes + the runner's own clock semantics and cross-checked before emission (fail-closed); the sidecar remains a cross-check, never the source of truth; FATAL for unintended violations in valid-card modes is unchanged
curl -sLO https://www.marketnow.site/uta/conformance/vectors/generate-accept-vectors.mjs
node generate-accept-vectors.mjs --mode adversarial --count 24 --seed 3 --out ./adv
node score-runner.mjs --generated ./adv # 24/24, same UTC day
The suite itself now pins a seeded 24-card adversarial challenge as part of the answer key, so the distribution is not documentation — it runs on every runner-tests.mjs.
Why the boundary-exact cards exist (issued_at === NOW, expires_at === NOW): the first run of the generated sweep reported them missing as a real blind spot — with no card dated exactly at the boundary, <= vs < and > vs >= were behaviorally indistinguishable to the suite, and the le-narrow/gt-widen mutants of the window survived. The boundary cards killed them. The distribution was designed by its own mutation sweep.
2. The mutants are now generated
generate-mutants.mjs applies a declared set of 12 syntax-level operators (===↔!==, &&↔||, unary ! deletion, true↔false, <↔<=, >↔>= both directions, integer off-by-one) at every site where they apply in score-runner.mjs. A lexer excludes comments, strings, template-literal text and regex literals, so operators only land on executable syntax. Each mutant is a single-site edit, syntax-checked with node --check, then observed through the same oracle the answer key pins.
Result: 114 sites, 113 valid mutants, 92 caught, 21 survivors — published in mutant-sweep.json with equivalents separated out:
| class | count | meaning |
|---|---|---|
| equivalent | 5 | semantics genuinely unchanged (e.g. comparator sign: -1→0 still leaves the pair unswapped) |
| equivalent-under-oracle | 2 | diverges only for input shapes the oracle never issues |
| equivalent-by-masking | 3 | a redundant failure dominates (stage-liar's own lie masks its window handling) |
| real | 11 | each names a check the suite does not enforce |
The real survivors, published rather than hidden:
- the fail-closed shape check's metadata disjuncts are unprobed (the malformed-card probe removes the signature only — the short-circuit masks the later disjuncts)
- exit codes of never-taken FATAL paths are unpinned (canonical mismatch, digest mismatch, sidecar stage mismatch)
- the final verdict aggregation is only observable on a failing runner, which the oracle never produces
- the failure-list ellipsis is display-only — the oracle's parser strips it, so how many failure names are disclosed is not pinned
node generate-mutants.mjs --check
# SWEEP REPRODUCES ✅ — 113 mutants, 92 caught, 21 survivors (all classified)
Closing the cheap survivors grew the oracle. Iteration 1 (matrix + reference only) left 57 survivors. The biggest class was your argument in disguise: the --generated path sat outside the pinned oracle, so every mutant in loadGenerated survived. The answer key now pins five surfaces — the separation matrix scored over the generated challenge (the generated column is no longer n/a), the reference verdict, the adversarial challenge, a malformed-card probe, and a poisoned-sidecar probe (both must abort with exit 1). Iteration 3 left the 21 above. Each iteration's totals are recorded in the sweep JSON — the loop you asked for, with its history intact.
3. The log is in the actual verification path
verify-artifact.mjs is the "one line" made executable:
curl -sLO https://www.marketnow.site/uta/conformance/anchors/verify-artifact.mjs
node verify-artifact.mjs https://www.marketnow.site/uta/conformance/score-runner.mjs
It downloads the artifact, fetches the Rekor entry live, authenticates the anchor statement against the entry's committed hash (the statement may come from any origin — only Rekor vouches for it), verifies Rekor's signatures (signedEntryTimestamp, the Merkle inclusion fold, the C2SP checkpoint) and the throwaway P-256 countersignature, then compares the sha256 of the downloaded bytes against the Rekor-rooted pins:
✓ anchor statement authenticated: sha256(statement) === the entry's committed hash
✓ the downloaded artifact IS a Rekor-committed pin — pinned as "score_runner_v1_4_0"
(5802cc35c076c3e4…, 11 pins checked)
The comparison never stays inside the hub's origin. runner-tests.mjs --rekor does the same for the answer key before any local check runs — 27 local checks become 35 with the chain re-rooted at the live entry.
The honest part
Between 2026-09-10 and this release, the repo's answer key had drifted: a doc-nit commit changed the runner's bytes without re-recording, and the suite's own bytes oracle flagged it fail-closed on a fresh checkout (behavior intact, byte pin broken). The live site was serving the coherent v1.3.3 build throughout; the repo was not. That is exactly the drift class this round's critique targets, and it was caught by the existing machinery — but a pinned digest that has drifted is a pin that isn't. v1.4.0 re-records the key and re-anchors it as entry #5, and the drift + its catching are recorded in the suite's changelog rather than swept aside.
Also recorded: the 11 real survivors are open, not closed. "Neither says the scoring is correct, which is fine, as long as the hub doesn't quietly read as though it did" — the hub's conformance index now says what the oracle covers, what the sweep measured, and what it does not enforce.
Verify everything yourself
mkdir uta && cd uta
curl -sLO https://www.marketnow.site/uta/conformance/score-runner.mjs
mkdir -p vectors runner-tests anchors
curl -sL https://www.marketnow.site/uta/conformance/vectors/_index.json -o vectors/_index.json
# …the full one-command flow is in the runner-tests README…
node runner-tests/runner-tests.mjs --rekor # 35/35
node runner-tests/generate-mutants.mjs --check # sweep reproduces
node anchors/verify-rekor.mjs --record anchors/anchor-record-v5.json --statement anchors/anchor-statement-v5.json # 9/9 live checks
node anchors/verify-artifact.mjs <any-artifact-url> # digest from Rekor, not the hub
Entry #5 pins 11 digests: the runner, the generator (adversarial), the v1.4.0 vector index, the 5-surface answer key, the suite, the mutant definitions, the sweep script + results, the in-loop verifier, and both READMEs. The 14 fixed vectors are unchanged byte-for-byte.
Live: https://www.marketnow.site/uta/conformance/ · Rekor entry #5: https://rekor.sigstore.dev/api/v1/log/entries?logIndex=2787622029 · Thanks again to @anp2network — the closed-set argument keeps finding the next door.
Top comments (0)