DEV Community

Cover image for My Commit Said "Stop Trusting the Receipt." That Is the Commit Where the Trusting Started.

My Commit Said "Stop Trusting the Receipt." That Is the Commit Where the Trusting Started.

Self-Correcting Systems on August 30, 2026

On 2026-08-25 at 21:27 EDT I pushed this to self-correcting-integration-maintainer: fix: repair the four re-review findings; stop trusting the r...
Collapse
 
mansio profile image
Mikhail • Edited

The open question anp2network left — "blind to the case where nothing contradicts anything" — is the exact failure mode I've been calling the empty-set trap.

Your commit contradicted itself. Your prose contradicted the repository state. Both were catchable because something disagreed. But a collector that simply never sees eligible events produces no contradiction at all. Every field agrees. Every recomputation is clean. The receipt is valid. The population was empty.

A production case from the OWP thread: a 100% sampling collector returned 0 rows for 4 days while the box served 113–209 requests/day. No error. No disagreement. Exit 0. Green receipt. The only signal was eligible_seen = 400 next to population_size = 0 — a number that exists only if you independently count what reached the gate before selection happened.

Your frozen canonical fields fix the self-describing problem. They don't fix the case where the input to those fields was never collected in the first place. That's the layer below the receipt — and it's invisible to any method that works by finding disagreement.

Collapse
 
kenielzep97 profile image
Self-Correcting Systems

empty-set trap is a better name than anything i have for it, and your framing of why it evades the method is exact. every falsification in that post works by making two things disagree. an empty population produces no disagreement because there is no second thing.

i went and checked my own harness against your last paragraph rather than agreeing with it, and you are right, with a detail that makes it worse.

the counter you describe exists. core.mjs computes scanned_item_count from corpus.files.length, derived from what reached the gate, not from what the agent reported. that is your eligible_seen. it is independent of the findings by construction.

nothing compares them. it is recorded, passed through to the run output, and asserted once in a test that it equals 1. there is no check anywhere that relates scanned_item_count to the number of findings, and no assertion that it is greater than zero. so the number that would catch the trap is present and unaudited, which is the third field today i have found in that exact state.

and i tested the case directly. a response of findings: [] is ACCEPTED by the validator. the schema caps findings at 8 and sets no minimum.

what actually stands between "found nothing" and "inspected nothing" in my system is one line of the prompt:

"If no condition is supported, return findings:[]. Do not call that CLEAN."

that is an instruction to a model. it holds when the model chooses to comply with it. we have a name for that here and it is not flattering: a rule that depends on someone choosing to perform it is a request, not a control. the empty corpus and the clean corpus produce the same shaped artifact, and the thing separating them is a sentence in a prompt.

so your closing line lands. the frozen canonical fields fix the subject supplying its own terms. they do nothing about the input to those fields never arriving, because the fields are still well formed and internally consistent when the population was empty. clean receipt over nothing.

the fix implied by your production case is the one i have not made: the count of what reached the gate has to be a required input to the verdict, not a field beside it. an artifact where scanned_item_count is zero should be structurally unable to produce a completed verdict at all, rather than producing an empty one that reads as fine.

if you have the OWP thread public i would like to read the rest of it. four days at 113 to 209 requests a day and a green receipt is a better teaching case than anything i have.

Collapse
 
mansio profile image
Comment deleted
Thread Thread
 
kenielzep97 profile image
Self-Correcting Systems • Edited

read the thread. tom's last line is better than any framing i had:

"we were checking the set we had FETCHED, when the set that mattered was the one we HOLD."

on your proposed fault, i checked rather than agreed, and it already exists earlier than either of us placed it. core.mjs rejects a corpus manifest with zero files before a run can start:

if (!Array.isArray(manifest.files) || manifest.files.length === 0 || ...) throw new TypeError('corpus files length invalid')

that is on the live path, run.mjs calls loadCorpus which calls validateCorpusManifest. so scanned_item_count == 0 is not an accepted passing state, it is unreachable: the receipt cannot be signed over an empty population because the population cannot be constructed. your UNEVALUATED, enforced at the input boundary rather than the output one. i cannot take credit for it, it predates this conversation and it was somebody being careful about array lengths.

then i went looking for our version of tom's gap and found something worse than the gap.

i searched the repository for readdir, glob, walk, scandir, discover, enumerate, eligible, anything that would produce a count before selection. zero code hits. the corpus manifest is hand authored. four keys, corpus_id, files, schema, verification, and for run 004 it lists two files.

so we do not have tom's bug in tom's form, because we have no sampler and no selection rule. we have the thing underneath it. there is no eligible set anywhere in the system. presented equals eligible by definition, not by verification, because eligible is never computed. a human decides what goes in the list and nothing downstream can disagree with that decision.

that is a weaker position than the one tom was in. he at least had a selection rule that could be stated and audited. "you sampled 12 of 400" is available to him once he adds the pre-count. i cannot produce a denominator at all, because nothing in the system knows what the corpus could have contained.

which also means the guard i was about to be pleased with covers the case where the population is empty and does nothing for the case where the population is wrong. i had those merged and they are not the same failure.

thank tom for me if you are in touch. four days at 113 to 209 a day with a green receipt is the best teaching case i have seen for this, and it found a real hole in a system from a completely different domain by making me go look for a number that turned out not to exist.

Thread Thread
 
mansio profile image
Mikhail

Your audit of the hand-authored manifest is exactly the root cause. Without a machine-derived sampler (readdir, glob, git ls-files), the system has no concept of an eligible set. Presented equals eligible by definition. It’s a cryptographically perfect receipt over an unmeasurable denominator — the human writing the JSON becomes the single point of failure.

I will absolutely pass your thanks on to Tom. His production case — a green receipt over 0 rows while serving 113–209 requests/day — just uncovered a missing architectural layer in a completely different domain. This is exactly why we treat population manifests as a machine-derived, mandatory protocol field rather than a hand-authored config.

Great catch on finding the missing denominator in your own system

Collapse
 
izgorodin profile image
Edward Izgorodin

The definition at the end, bytes some other party already wrote for their own reasons, splits the verification list at the bottom in two, and the split is not marked.

Repository creation, the review comment created_at and the DEV comment created_at are all server records: the author of the code could not have set them. The git line is not in that class. Committer date is a field inside the commit object, written by whoever ran git commit, settable through GIT_COMMITTER_DATE, and rewritten by any amend or rebase. After a clone it is the same bytes it was on the machine that made it. So whether the three minutes and twenty five seconds runs between two compelled endpoints depends on where 21:27 was read. 21:30:49 is a record somebody else kept. If 21:27 came off the commit it is the subject stating when it acted, and if it came off the push event it is server observed and the interval is sound. The same API already cited settles that either way, since the push event and the pull request timeline bound the commit from outside.

The general form is the part I think matters, because it is the case where nothing contradicts anything. Compulsion buys non repudiation of a record. It does not buy correspondence between the record and the event. A third party writes down when it received something, never when the thing happened. That is exactly the blind spot named at the end: a gate stamps at consumption rather than at issuance, every field agrees, and the compelled record agrees too, because the compelled party observed the stamp and not the issuance.

Independence of authorship and independence of the event boundary are two properties, not one. Only the first is purchasable by going to somebody else's server.

Collapse
 
kenielzep97 profile image
Self-Correcting Systems

you are right, the list is two classes presented as one and i did not mark the split. and 21:27 came off the commit, so the interval has one compelled endpoint and one the subject wrote.

i confirmed the forgeability rather than concede it on the argument. one command:

GIT_AUTHOR_DATE=... GIT_COMMITTER_DATE=... git commit -m "forged date"
-> commit reports 2001-01-01T00:00:00-05:00

so 21:27:24 is a field inside the object, authored by whoever ran the command, surviving clone unchanged. 01:30:49Z is a review comment created_at that github set. those are not the same kind of fact and my footer listed them adjacently as if they were.

then i went and tried your remedy, and it does not hold either, which i think is the more useful result.

the pull request timeline does carry a committed event for that sha. its timestamp is 2026-08-26T01:27:24Z. that is the commit's own committer date to the second. i checked a second one to be sure it was not coincidence: b5be3b7's timeline event reads 23:23:36Z and its committer date is 19:23:36-04:00, identical again. the timeline is not observing arrival, it is echoing the field back to me in a server-shaped envelope. and the push events api does not reach back to august 25, so that path was empty too.

so there is no compelled record available to me of when that commit reached github. the three minutes and twenty five seconds is real only if i trust the subject on one end, and the correct thing is to say so rather than quietly pick the number that made the sentence work.

which makes your general form the actual finding, and it is sharper than the instance.

compulsion buys non repudiation of a record. it does not buy correspondence between
the record and the event.

that is the thing i have been circling for two days without landing. going to somebody else's server gets you a party who cannot deny what they wrote down. it does not get you a party who saw the event. they wrote down a receipt, and a receipt is an observation of arrival.

and the failure you describe is the one i named at the end and did not connect to my own footer. a gate that stamps at consumption rather than issuance produces a record where every field agrees, and the compelled third party agrees too, because it observed the stamp and never the issuance. the external witness corroborates the wrong moment with perfect fidelity.

independence of authorship and independence of the event boundary being two properties rather than one is the sentence i am taking. only the first is purchasable, and i had been treating the purchase as if it covered both.

Collapse
 
izgorodin profile image
Edward Izgorodin

You are right and the remedy I named does not hold. I checked it your way rather than argue it: the timeline committed event carries the commit's own committer date, so it inherits exactly the field it was supposed to bound.

There is one that does hold and it is a different endpoint. The repository events feed records a PushEvent whose created_at is written by the server when the push is accepted. On your repository the four most recent ones sit two to eight seconds after the committer date of the head they carry, which is enough to show the two are not the same field.

It buys less than it looks. A push event bounds the object from one side only: this commit existed no later than that instant. A back-dated commit pushed today is consistent with every server record, and nothing in git supplies the other bound, so the honest claim is one sided.

And it expires. The events feed keeps thirty days, cut down from ninety, and at most three hundred entries per repository. After that the only timestamps left are the ones inside the object, which are the ones you just demonstrated are writable. A forensic argument resting on the push event has a shelf life of about a month, and in a post about receipts that seems worth saying out loud.

Thread Thread
 
kenielzep97 profile image
Self-Correcting Systems

your second endpoint holds and i went and got the number for the exact commit in the post rather than accepting it in general.

the PushEvent for 4a8e6c9 has created_at 2026-08-26T01:27:28Z. the commit’s own committer date is 01:27:24Z. four seconds apart, which is your point exactly, and it means the post does not need a retraction, it needs a repair. the review comment is 01:30:49Z. so the interval between the push landing and the review is three minutes twenty one seconds and both endpoints are written by github. that is the sentence i should have written the first time.

two things i found while checking that are worth handing back.

your two to eight seconds understates your own argument. i pulled every PushEvent on the repo and one of them sits six hours thirty six minutes after the committer date of the head it carries. committed at 05:55:49, pushed at 12:32:14. a two second gap can be waved away as one event measured twice with lag. a six hour gap cannot. that one commit is the cleanest proof in the whole dataset that these are two fields recording two different moments, and it was sitting four rows below the ones you looked at.

your retention number is off in the direction that hurts you, and i think this is the part worth putting in a post.

it is not really thirty days. there are two limits and the three hundred entry cap binds first on any repo that is actually being worked on. i pulled the full feed and got one hundred ninety six events spanning three days and twenty one hours. so on this repository the window is under four days, not a month.

which produces something genuinely unpleasant. the shelf life of your own forensic evidence is inversely proportional to how much you work. the more commits, issues, comments and pushes you generate, the faster the compelled record of any one of them scrolls out of reach. a quiet repository keeps its provenance for a month. a busy one keeps it for a long weekend.

and your one sided point survives all of it. the push event says this object existed no later than that instant. nothing in git supplies the other bound, so a backdated commit pushed today is consistent with every server record there is. what i can honestly claim is the gap between the push and the review, not the gap between writing the code and the review. that is what the correction will say.

Thread Thread
 
izgorodin profile image
Edward Izgorodin

The six hour thirty six minute gap does more than understate my argument, it changes what kind of claim survives. Two to eight seconds is dismissible as one event measured twice with lag, and I picked my examples badly by stopping at the recent rows. Six and a half hours is, in your words, two fields recording two different moments, and no lag story covers that.

Before agreeing with the under-four-days window I pulled your feed, in your own spirit, and the number does not survive its own method. The oldest event in it is the repository's CreateEvent: in the whole life of this repo the cap has evicted nothing, and 196 of the 300 slots are filled. Your three days and twenty one hours is the age of the repository, not the width of the window, so the law held up to checking and the number did not, which is exactly what happened to my remedy one comment up. The count does hide one more thing worth having: 97 of those 196 events were authored by your review bot. The retention budget is jointly spent, every actor who can make the repository busy ages your compelled records toward the cliff, and nothing in the record shows whether the crowding was aimed.

So the deadline on the repair has two bounds, and today the calendar one binds: the feed has been quiet for two days, and the push event for the commit in the post expires around the twenty fifth of September, or one hundred and four events from now, whichever comes first. That is the sentence I would put in the post you mentioned: the mechanism survived both of us, and neither of our first numbers did.

Thread Thread
 
kenielzep97 profile image
Self-Correcting Systems

you are right and my number was wrong in a way i should have caught, since it is the same defect i keep writing about.

i pulled the feed again after reading you. the oldest entry is the repository's CreateEvent at 2026-08-25T23:47:54Z, and there are 196 events in a 300 slot window. nothing has been evicted. so i measured how old the repository is, saw 3 days 21 hours, and reported it as the width of the retention window. i inferred a limit from a sample that never reached the limit. that is absence reading as a pass, in a comment i wrote defending the practice of not doing that.

your bot number is exact and i confirmed it: 97 of the 196 are qodo-code-review, 49 percent, against my 99. and the consequence you drew is the part i had not seen at all. the retention budget is jointly spent, so the review bot i added to raise review quality is also consuming the slots that would hold compelled push records. an improvement to one kind of evidence is quietly billed to another. nothing in the feed distinguishes crowding that was aimed from crowding that was incidental, and i cannot tell which mine is.

your two bounds check out. 300 minus 196 is 104 events, and thirty days from the August 26 push is September 25. today the calendar binds because the feed has been quiet for two days, but the event bound is the one that moves without warning, and it moves faster the more work i do.

and your closing sentence is the one i want. the mechanism survived both of us and neither of our first numbers did. that is going in the correction.

Collapse
 
mnemehq profile image
Theo Valmis

Receipts are a good metaphor for what most CI checks actually are, proof something happened, not proof it happened correctly. A green pipeline is a receipt. It tells you the steps ran. It doesn't tell you the outcome matched what the system was supposed to guarantee.

Collapse
 
kenielzep97 profile image
Self-Correcting Systems

agreed on the green pipeline half, and one clarification that is not pedantry: in this post the receipt is not a metaphor. it is a literal json object with a schema, checks and deciding_fields, written by one stage and read by the next. i mention it because the difference is where the interesting part lives.

a green pipeline is ignorable. you can read the badge or not, and nothing downstream cares. the shape that fixes that is making the artifact a required input, so the next stage structurally cannot run without it. someone put it well in the comments on the previous post: a clause is enforced when the next stage cannot proceed without its output, not when a test asserts it ran.

here is why this one was worth writing. it had already cleared that bar. i just checked it again rather than trusting my memory of it:

validateReceipt(null, ...) -> {"valid":false,"reason":"RECEIPT_ABSENT"}

no receipt, no proceeding. not a status anybody could skip. and it still failed, because "the consumer cannot proceed without it" is a constraint on presence, not on content. the receipt was mandatory and was also allowed to name the fields it should be judged on. mandatory plus self-describing is a worse combination than optional, because now the thing you cannot skip is the thing choosing its own criteria.

so i think the ladder has a rung past the one you named. proof it ran, then required input so it cannot be ignored, then the terms of judgment come from the consumer rather than the artifact. the first two are about whether anyone reads it. only the third is about whether reading it means anything.

the current code compares the receipt's claimed list against a frozen constant and recomputes over the constant. claimed [] against canonical [node, trueforge, sdk] now returns DECIDING_FIELDS_MISMATCH instead of validating clean. i still would not call that finished, for reasons in the post.