DEV Community

sk8ordie84
sk8ordie84

Posted on • Originally published at falsify.dev

Lock #2: the first thing PRML falsified was its own distribution hypothesis

The mechanism worked. The outreach didn't.

On 2026-05-08 I locked a public commitment using PRML, the pre-registration format my own project is built on. The claim: at least 3 independent contributors would file RFC engagement (issues or PRs against the rfc-v0.2 tracks) within a two-week window. Public hash, public target, public resolve date.

Today it resolved at 0 / 3. Zero independent contributors. Not partial, not "close" — zero.

An honest account:

1. The mechanism worked

Lock #2 exists precisely so that a missed commitment can't be quietly re-framed after the fact. The fail is public now, automatically, via the same registry, with no admin intervention from me. If it had soft-failed silently, the whole pre-registration thesis would be theatre.

2. It's a distribution problem, not an interest problem

The v0.2 RFC is technically sound: the JSON Schemas validate, the conformance vectors pass byte-equivalently across four reference implementations. What didn't happen is the RFC reaching rooms where someone is paid to read RFCs in this domain.

3. v0.2 still freezes on schedule, founder-only

The version doesn't get postponed because the engagement target was missed. That would be moving the goalposts.

The second thing it falsified: its own counting bug

Two days before the lock resolved, a registry-side audit found the unique-producer-count routine was over-counting: it showed 8 when the correct number was 2 (a quoting/regex bug that misread other fields as producer IDs). Fixed and deployed; the lock manifest itself was untouched, only the observing code. This is what running the mechanism on yourself looks like in practice: if a critic finds the next bug, that's PRML doing its job; if I find it first, that's also PRML doing its job.


PRML (Pre-Registered ML Manifest) is an open spec for committing an ML evaluation claim to a SHA-256 hash before the run. Spec and four reference implementations: https://falsify.dev

Top comments (0)