Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
The part of this design I would put at the top is the one you buried near the bottom: UNKNOWN.
"When evidence is incomplete or the control does not match, the protocol can return UNKNOWN rather than manufacture confidence." That single sentence is doing more work than the signatures around it, and I would argue it is the load-bearing piece.
A signature proves who said it and that it was not altered. It cannot prove that anything was measured. So a signed receipt over an unmeasured verdict is not neutral - it is a certificate. It converts "I don't know" into a durable, portable, cryptographically attested claim, and the next system downstream has no way to tell the difference.
Three from my own week, all the same shape:
A harvesting tool ran across sixteen public repositories. Five returned 260–277 results. The remaining eleven returned zero, reporting "no issue has a linked pull request" - for repositories with tens of thousands of them. The cause was an hour rate limit and one line: } catch { return null; }. Every error became the statement "this one has nothing". It then wrote a valid, empty file and exited 0.
A production watchdog judged a fleet of 68 from a random sample of 8 and reported the verdict as a fact about all 68. It had no state for "I cannot know this from this sample", so it said something else.
A health endpoint answered database: true because select 1 succeeded - while three screens were returning 500 on a column that did not exist. The check was true. It was answering a different question than the one asked.
None of those would have been caught by an authority layer, because no boundary was crossed. In all three the system was entitled to act and its output was well-formed. What was missing was a third state. There was "found" and "found nothing", and silence had to become one of the two - so it became zero. And a zero looks like a result.
So the question I would put back to your design: is UNKNOWN a first-class signed outcome, or is it the absence of a receipt? Those are very different. If a verifier that could not measure simply emits nothing, then downstream "no receipt" is ambiguous between not run, not measurable, and lost in transit - and the ambiguity lands exactly where you built the protocol to remove it.
One more, offered as a shared risk rather than a criticism, because you already drew the evidence boundary yourself: 4,265 tests with zero failures and zero skips is the shape that should prompt "can this suite fail?" I had a 17-item benchmark that scored 92.3% and ranked two rankers in exactly the reverse order from the 499-item one. It was not a weak eval. It was a painting of one, and it held up under review because nothing about it was inconsistent.
Part 2 of your article has the sentence I would put on the wall: "A reserved action needs a safe timeout. Silence should not become implicit permission."
That is the same failure with the sign flipped. Yours: silence becomes permission. Mine, three times this week: silence becomes zero — a rate limit booked as "this repository has no linked pull requests", a sample of 8 booked as a verdict about 68, a select 1 booked as "the schema is fine".
Both are the same missing state. There is "yes" and "no", and nothing said has to become one of them. So it becomes whichever one the code path reaches first — and in an authority system that is permission, which is the worse of the two defaults.
Heinrich, I think you are right to make UNKNOWN the load-bearing part.
OpenWorkProof v0.5 already treats UNKNOWN as a signed, first-class decision. Missing, mismatched, or incomplete population and control evidence can produce a signed VerificationDecision with explicit reason codes, and downstream acceptance gates must not treat that result as success.
But your examples expose a remaining gap in the model.
Today, OWP can sign the statement: "The evidence available is insufficient for this claim." It does not yet cleanly distinguish between:
the check never ran
the check ran but could not measure the target
the check measured only a sample
execution was interrupted
a receipt was created but lost in transport
Those states should not all collapse into the absence of a receipt. Absence is a transport or execution ambiguity. UNKNOWN is an epistemic conclusion.
I think the next schema needs two separate fields:
The signed result should also bind what was actually measured: the declared population, observed population, sampling rule, coverage, verifier version and commit, control ID, and negative-control result.
Your 8-of-68 example makes the distinction concrete. "Eight items were measured under this sampling rule" may be a valid statement. It is not a valid conclusion about all 68 unless the verification contract explicitly permits that inference.
Your criticism of the "4,265 tests passed" claim is fair as well. That number proves only that the registered suite passed in the recorded environment. It does not prove that the suite is representative or that every guard can fail.
OWP already uses reason-bound negative controls for conclusion-bearing verification, but raw test count is still the wrong headline metric. We should report falsifiability coverage alongside it: how many conclusion-bearing guards were challenged with known-bad inputs, and how many rejected them for the expected reason.
The protocol rule I would write now is:
Every completed verification attempt should produce a signed verdict and a signed measurement status. No receipt should never be interpreted as UNKNOWN. It should mean that no attestable outcome exists.
Would you be open to reviewing a small schema proposal for this split? Your three examples would make excellent acceptance cases.
Building cachly (MCP): your coding agent remembers your codebase and every decision made — not just the chat. Follows you between editors, stays when someone leaves. I post what I learn shipping it.
Yes, gladly - send the schema and I'll review it against real cases rather than opinions.
Your split is right, and I want to hand you the one case that survives it, because it's the one that cost me the most today.
Two of my three examples are status problems. The third is not.
The harvest: the check ran, got an HTTP error, and booked it as data. Under your schema that is not_measurable - if the producer can tell. It couldn't, and that is the whole bug. So the schema is correct and the hard part moves one step earlier: a producer that cannot distinguish "error" from "empty result" will emit measured with full confidence. The field is necessary; it is not sufficient.
The watchdog, 8 of 68: incomplete plus a sampling rule. Your schema closes it cleanly.
The health endpoint is the one that gets through. select 1 succeeded. Status: measured. Verdict: VERIFIED. Both honest. And three screens were returning 500 on a column that did not exist.
Nothing in measurement_status × verdict catches that, because the measurement was real and the verdict was true - of a different claim. The control was "can I reach the database". The claim was "the schema matches the code". A signed receipt for the first reads, downstream, as evidence for the second.
You already have the fix in your list and I'd promote it to the same rank as the two fields: control ID. My sharpening would be that the acceptance gate must compare the control to the claim, not just read the verdict. Otherwise the receipt is honest, complete, signed - and answering a question nobody asked. That is a harder failure than UNKNOWN, because every field in it is true.
So the third field I'd argue for is not a status. It is: what claim does this verdict license? If that isn't in the signed object, an acceptance gate has to infer it, and inference is where the ambiguity comes back.
On falsifiability coverage - you're right that raw test count is the wrong headline, and I'd rather demonstrate that than agree with it. Our test suite has a convention: every guard that draws a conclusion carries a counter-probe, marked KONTROLLE:, that feeds it a known-bad input and asserts it rejects it for the expected reason. It exists because a guard that finds nothing and a guard that has nothing to find produce identical output.
I have never measured the ratio. I'll count it this week and post the number here, whatever it turns out to be - including if it's embarrassing. If it's useful, "conclusion-bearing guards with a passing negative control / conclusion-bearing guards" might be a workable definition for your metric, because it's mechanically countable rather than a judgement call.
One caveat on my own three examples before you build acceptance cases on them: they are one repository, one week, and I found all three because someone from outside asked why a number was zero. They're real, but they're not a sample.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
The part of this design I would put at the top is the one you buried near the bottom: UNKNOWN.
"When evidence is incomplete or the control does not match, the protocol can return UNKNOWN rather than manufacture confidence." That single sentence is doing more work than the signatures around it, and I would argue it is the load-bearing piece.
A signature proves who said it and that it was not altered. It cannot prove that anything was measured. So a signed receipt over an unmeasured verdict is not neutral - it is a certificate. It converts "I don't know" into a durable, portable, cryptographically attested claim, and the next system downstream has no way to tell the difference.
Three from my own week, all the same shape:
A harvesting tool ran across sixteen public repositories. Five returned 260–277 results. The remaining eleven returned zero, reporting "no issue has a linked pull request" - for repositories with tens of thousands of them. The cause was an hour rate limit and one line: } catch { return null; }. Every error became the statement "this one has nothing". It then wrote a valid, empty file and exited 0.
A production watchdog judged a fleet of 68 from a random sample of 8 and reported the verdict as a fact about all 68. It had no state for "I cannot know this from this sample", so it said something else.
A health endpoint answered database: true because select 1 succeeded - while three screens were returning 500 on a column that did not exist. The check was true. It was answering a different question than the one asked.
None of those would have been caught by an authority layer, because no boundary was crossed. In all three the system was entitled to act and its output was well-formed. What was missing was a third state. There was "found" and "found nothing", and silence had to become one of the two - so it became zero. And a zero looks like a result.
So the question I would put back to your design: is UNKNOWN a first-class signed outcome, or is it the absence of a receipt? Those are very different. If a verifier that could not measure simply emits nothing, then downstream "no receipt" is ambiguous between not run, not measurable, and lost in transit - and the ambiguity lands exactly where you built the protocol to remove it.
One more, offered as a shared risk rather than a criticism, because you already drew the evidence boundary yourself: 4,265 tests with zero failures and zero skips is the shape that should prompt "can this suite fail?" I had a 17-item benchmark that scored 92.3% and ranked two rankers in exactly the reverse order from the 499-item one. It was not a weak eval. It was a painting of one, and it held up under review because nothing about it was inconsistent.
Part 2 of your article has the sentence I would put on the wall: "A reserved action needs a safe timeout. Silence should not become implicit permission."
That is the same failure with the sign flipped. Yours: silence becomes permission. Mine, three times this week: silence becomes zero — a rate limit booked as "this repository has no linked pull requests", a sample of 8 booked as a verdict about 68, a select 1 booked as "the schema is fine".
Both are the same missing state. There is "yes" and "no", and nothing said has to become one of them. So it becomes whichever one the code path reaches first — and in an authority system that is permission, which is the worse of the two defaults.
Heinrich, I think you are right to make UNKNOWN the load-bearing part.
OpenWorkProof v0.5 already treats UNKNOWN as a signed, first-class decision. Missing, mismatched, or incomplete population and control evidence can produce a signed VerificationDecision with explicit reason codes, and downstream acceptance gates must not treat that result as success.
But your examples expose a remaining gap in the model.
Today, OWP can sign the statement: "The evidence available is insufficient for this claim." It does not yet cleanly distinguish between:
Those states should not all collapse into the absence of a receipt. Absence is a transport or execution ambiguity. UNKNOWN is an epistemic conclusion.
I think the next schema needs two separate fields:
measurement_status: measured | not_run | not_measurable | interrupted | incomplete
verdict: VERIFIED | REFUTED | UNKNOWN
The signed result should also bind what was actually measured: the declared population, observed population, sampling rule, coverage, verifier version and commit, control ID, and negative-control result.
Your 8-of-68 example makes the distinction concrete. "Eight items were measured under this sampling rule" may be a valid statement. It is not a valid conclusion about all 68 unless the verification contract explicitly permits that inference.
Your criticism of the "4,265 tests passed" claim is fair as well. That number proves only that the registered suite passed in the recorded environment. It does not prove that the suite is representative or that every guard can fail.
OWP already uses reason-bound negative controls for conclusion-bearing verification, but raw test count is still the wrong headline metric. We should report falsifiability coverage alongside it: how many conclusion-bearing guards were challenged with known-bad inputs, and how many rejected them for the expected reason.
The protocol rule I would write now is:
Every completed verification attempt should produce a signed verdict and a signed measurement status. No receipt should never be interpreted as UNKNOWN. It should mean that no attestable outcome exists.
Would you be open to reviewing a small schema proposal for this split? Your three examples would make excellent acceptance cases.
Yes, gladly - send the schema and I'll review it against real cases rather than opinions.
Your split is right, and I want to hand you the one case that survives it, because it's the one that cost me the most today.
Two of my three examples are status problems. The third is not.
The harvest: the check ran, got an HTTP error, and booked it as data. Under your schema that is not_measurable - if the producer can tell. It couldn't, and that is the whole bug. So the schema is correct and the hard part moves one step earlier: a producer that cannot distinguish "error" from "empty result" will emit measured with full confidence. The field is necessary; it is not sufficient.
The watchdog, 8 of 68: incomplete plus a sampling rule. Your schema closes it cleanly.
The health endpoint is the one that gets through. select 1 succeeded. Status: measured. Verdict: VERIFIED. Both honest. And three screens were returning 500 on a column that did not exist.
Nothing in measurement_status × verdict catches that, because the measurement was real and the verdict was true - of a different claim. The control was "can I reach the database". The claim was "the schema matches the code". A signed receipt for the first reads, downstream, as evidence for the second.
You already have the fix in your list and I'd promote it to the same rank as the two fields: control ID. My sharpening would be that the acceptance gate must compare the control to the claim, not just read the verdict. Otherwise the receipt is honest, complete, signed - and answering a question nobody asked. That is a harder failure than UNKNOWN, because every field in it is true.
So the third field I'd argue for is not a status. It is: what claim does this verdict license? If that isn't in the signed object, an acceptance gate has to infer it, and inference is where the ambiguity comes back.
On falsifiability coverage - you're right that raw test count is the wrong headline, and I'd rather demonstrate that than agree with it. Our test suite has a convention: every guard that draws a conclusion carries a counter-probe, marked KONTROLLE:, that feeds it a known-bad input and asserts it rejects it for the expected reason. It exists because a guard that finds nothing and a guard that has nothing to find produce identical output.
I have never measured the ratio. I'll count it this week and post the number here, whatever it turns out to be - including if it's embarrassing. If it's useful, "conclusion-bearing guards with a passing negative control / conclusion-bearing guards" might be a workable definition for your metric, because it's mechanically countable rather than a judgement call.
One caveat on my own three examples before you build acceptance cases on them: they are one repository, one week, and I found all three because someone from outside asked why a number was zero. They're real, but they're not a sample.