DEV Community

Edison Flores
Edison Flores

Posted on

v1.7.0 shipped: adversarial window sampling, generated mutants, and the survivor list published

v1.7.0 shipped: adversarial window sampling, generated mutants, and the survivor list published

This is the follow-through on "Re: @anp2network — all three critiques land", which committed to the changes queued as v1.4.0. They are now live on main as manifest v1.7.0 / package 1.5.0 — the suite had advanced to 1.6.0 (third-party interop evidence) while the work was queued, so the numbering continues from there. The content is what was promised, and CI now runs all of it on every push, including mutation testing under --strict.

Everything below is verifiable from the repo with Node ≥ 18 and no dependencies.

1. The lower bound is a distribution now

generate-accept-vectors.mjs --mode adversarial emits correctly-signed, anchored, active cards whose only defect is a violated validity window — so expiry_check must fail and everything else must pass:

Bucket Share Range
future-issued 55% +2..7d (boundary band), +8..90d, +91..730d, +731..1460d
already-expired 35% including expires_at == NOW (the > boundary itself)
empty window 10% issued_at == expires_at in the future

The fail-closed mirror rule: in valid-card modes a window violation is a FATAL; in adversarial mode a card that lands in window is a FATAL — the generator failing to produce its own defect. Accept mode additionally issues 20% of cards today (boundary-in band), so a runner using < instead of <=, or a clock-retarded runner, is caught by the sidecar cross-check pinning the generation-time clock.

One honest hole, named rather than hidden: adversarial future offsets floor at +2 days so a generated set stays re-scoreable across midnight (a +1d card flips in-window after midnight while the sidecar pinned the old truth). The clock+1d mutation therefore survives, by design. The trade-off is written into the generator's header and the survivor list — catching a +1d clock shift would cost portability of every generated set.

2. Generated mutants, survivor list published

mutate-runner.mjs applies a declared operator set — every comparison on the scored path (cmp-swap), every single-line guard (guard-drop), every block condition (branch-negate), and the clock ±1d — mechanically at every applicable site, and executes every mutant against the full suite. No hand-picking, which was the closed-set failure you named: our memorizer argument pointed at our own catalogue.

Classification is mechanical: caught (suite fails), equivalent-on-suite (exit 0, byte-identical --json output — suite-equivalence, not true equivalence), observable-escape (exit 0, output changed). Current state on 46 mutants: 40 caught, 6 equivalent, 0 observable. The survivor list is committed as an artifact, and each survivor names something real:

  • clock+1d — the deliberate midnight-portability trade-off above.
  • The meta-integrity check that guards the other checks can itself be disabled. The runner now counts its own checks (byte-exactness, sidecar and stage cross-checks each exactly once per entry — a skipped check is a FATAL), and mutation testing immediately found that the counter assertion itself can be negated. Self-verification bottoms out somewhere. That is not a defeat; it is the demonstration of exactly your Rekor point — internal checks cannot certify their own execution, and the exit is external cross-anchoring (same digest republished from origins that don't share control), which stays queued after this release.
  • The rest are dead-path-under---json, error-message-only, and a provably order-neutral JCS comparator mutant on this suite's key sets.

Three of the tightenings shipped in this release were found by the mutation run itself, then closed: --generated on the command line that isn't parsed or yields zero cards is now a FATAL (a silently-ignored input used to be a green run); sidecar cross-checks are sha256-bound to the card bytes (on a uniform set, cross-checking the wrong row still agreed on expected_verify); and those completion counters above. The survivor list shrank as the derived-truth checks tightened — the loop you asked about works.

3. The matrix, with the window asked distributionally

| Runner                                  | fixed | adversarial (40) |
|-----------------------------------------|-------|------------------|
| always-true                             | 8/14  | 0/40             |
| policy-only (Ed25519 deleted)           | 11/14 | 40/40            |
| crypto-only (no expiry/status)          | 11/14 | 0/40             |
| embedded-key + policy (TOFU)            | 13/14 | 40/40            |
| memorizer (hardcodes valid-atc digest)  | 12/14 | 40/40            |
| over-rejector (chokes on x_* fields)    | 13/14 | 40/40            |
| stage-liar                              |  7/14 | 0/40             |
| reference (pinned + policy + tolerance) | 14/14 | 40/40            |
Enter fullscreen mode Exit fullscreen mode

crypto-only at 0/40 is the row that didn't exist before: a runner with intact cryptography and no window enforcement now fails against a distribution of window violations, not one frozen 2030 timestamp. The memorizer's 40/40 on adversarial is honest too — always-false happens to match always-reject; its weakness shows on the accept side, which is why the CI job scores both a fresh adversarial set and a fresh accept set on every push.

What to re-run

node vectors/generate-accept-vectors.mjs --mode adversarial --count 40 --seed 7 --out .gen-adv
node score-runner.mjs --generated .gen-adv
node mutate-runner.mjs --generated .gen-adv --out survivors.json
Enter fullscreen mode Exit fullscreen mode

The two questions from the last article are now answerable by anyone with the repo: the runner's rejection rate holds across the sampled boundary (not one timestamp), and the survivor list — not a caught-count — is the published claim. Rekor cross-anchoring remains queued after this release, still not dressed up as done.


I'm Edison Flores, founder of AliceLabs LLC — we build open-source security infrastructure for AI agents. Independent verification of anything cited here is not just welcome, it's the point.

Top comments (0)