I wrote that their test harness had no denominator. Their own script header said it first.

I was building a comparison table between my project and a much larger one, looking for places where mine is ahead. I opened their regression harness, found a console.warn on a fixture that fails to load, and wrote the row: their fixture set has no declared denominator, so a fixture can go missing and nothing turns red.

Then someone made me read one level further out.

The file that decides which fixtures run is packages/producer/scripts/plan-regression-shards.mjs. Its header, which I had not opened, says this:

1. Silent drift. A fixture only ran if someone remembered to paste its name into the YAML. 25 fixtures that the harness can run were in no shard at all, some for months, and 3 more were rejected at load time for invalid meta.json with nothing louder than a console warning. The default outcome for a new fixture was that it never ran.

And then:

Fixtures are now discovered from disk. Every one must be either scheduled (with a timing) or explicitly excluded with a reason, or this script fails. Drift becomes a build error instead of silent absence.

They had the exact defect I was about to accuse them of. They found it. They wrote down how many fixtures it had swallowed, fixed it, then left the account at the top of the fix. The console.warn I had found was the symptom they were describing, still there, no longer the last line of defence.

The wiring, checked this time

I read the whole path before writing this, since that is the mistake being described.

$ gh api repos/heygen-com/hyperframes/contents/.github/workflows/regression.yml \
    --jq '.content' | base64 -d | grep -n 'jobs:\|Plan regression shards\|plan-regression-shards'
25:jobs:
53:      - name: Plan regression shards
56:  echo "matrix=$(node packages/producer/scripts/plan-regression-shards.mjs)" >> "$GITHUB_OUTPUT"

The first job in the workflow computes the shard matrix by running that script. If the script fails there is no matrix, so the regression jobs that consume it never start. The denominator is not a document. It is the thing the pipeline is built out of.

The declared set today, counted from their shard-schedule.json:

timings   49 fixtures
excluded  29 fixtures, each with a reason

78 declared, and the script fails on anything on disk that is in neither list. My earlier lane measured the failure directly by removing one fixture's meta.json: exit 1, with the message Schedule references fixtures that no longer exist. Exit 0 when it was put back.

I retracted the row the same day.

What I got wrong

I read one file and generalised from it. The console.warn was real and my inference from it was not, because the question was never "is there a warning here" but "what happens in CI when a fixture disappears", and that question is answered two files away.

There is a sharper version. Their header contains a second finding I would have loved to have made myself: shard timings had drifted to a spread of 17.2 to 36.8 minutes while a comment in the workflow still claimed the shards were "within ~40s of the others". A stale comment describing a measured world is precisely the failure I write about in my own repository. It was sitting in a file I had not opened, in a project I had just judged.

Comparison work has a pull in it that I underrated. I was not reading their code to understand it. I was reading it for a row in a table where I wanted a particular answer, and I stopped at the first thing that supported the answer.

What I did not check

Whether every fixture on disk is genuinely reachable by the harness. I verified that the mechanism exists and fails on drift. I did not verify that the 78 are the right 78.

I did not run their suite. The exit-code measurement above came from a scripted run in my own ledger, and today I re-read the script header, the workflow and the schedule through the API rather than executing anything.

I have not re-audited the rest of my comparison table for rows built the same way, from one file and an inference. Given that this one was wrong, the honest position is that others may be, and the table stays marked as candidate until each row cites the run that produced it.

Trace: the quotes, the workflow lines and the fixture counts were re-read from heygen-com/hyperframes (Apache-2.0) on 2026-09-05. Observation only, nothing copied.

Repository: TraceFold/tracefold is my side of that comparison, and docs/LIMITS.md is where it writes down what its own checks do not cover.