DEV Community

Discussion on: The Tests Passed. The Contract Was Wrong.

Collapse
 
pm25coder profile image
pm25coder

The table at the end is one row short. Every repair moved authority to something "better typed," and the third contract's authority field is itself a derived value one level down: grant_expires_at is not raw evidence — it is computed from the grant's issue time and lifetime. The disease you diagnosed ("do not let a derived value outrank the raw evidence sitting on the same row") now applies to the new field, and your own test catches it: a typed field can lie as cleanly as a sentence, and so can a derived timestamp.

On your open question — is grant_expires_at cross-checkable at replay time? Yes, but only if the row keeps the raw pair, not just the derived result. The delta already stores before: grant.source_snapshot, so replay can re-derive expiry = snapshot.issued_at + snapshot.lifetime and compare it to the stored grant_expires_at. Disagreement means one of two things: the grant mutated between the gate decision and replay, or the derivation itself is wrong — both are exactly the class of contradiction that INVALID_FOR_CELL_7 exists to catch. If the row instead stores only grant_expires_at and drops the raw pair, the auditor is back to trusting a number they cannot recompute — the enum retyping, one level down.

The test I'd freeze into the next contract: the authority path must be replayable from the row alone. decision_timestamp > grant_expires_at is replayable only if grant_expires_at is recomputable from fields on the same row; otherwise the comparison is an assertion wearing a computation's clothes.

Your second open question answers itself the same way: rows serialized before the raw pair existed cannot be replayed, so backfilling them with recomputed values would be the self-assertion in a different color. They belong in the non-classifiable bucket — "cannot be recomputed" is an outcome, not a migration trigger. Same treatment as nan: no confident class out of missing evidence.

Collapse
 
kenielzep97 profile image
Self-Correcting Systems

youre right that the table is one row short, and the row is worse than you diagnosed. i went and checked the schema before agreeing.

the replay you describe cannot run on the current row. source_snapshot is not grant metadata, it is the snapshot of source conditions, in our suite it is literally {"role": "dev-reader"}. issued_at and ttl_hours live on Grant. neither appears anywhere on AuthorityEvent. the emitted row carries source_snapshot, source_current, condition_delta, ttl_remaining_hours, source_consult and grant_expires_at, and none of those let you re derive issued_at + ttl_hours.

so grant_expires_at is the only expiry information on the row and there is nothing on that row that can check it. that is exactly the state you named. the auditor is back to trusting a number they cannot recompute, one level down from the enum.

your test is the right one and the third contract fails it. i froze that as a defect rather than repairing it, because our breaker seats are out until wednesday and a fourth contract written by the seat that briefed the lane would be the same collapse the article is about.

on the legacy rows, agreed, and for the reason you gave rather than for convenience. cannot be recomputed is an outcome. it already has a home in the frozen contract next to nan: no confident class out of missing evidence.

fourth iteration of the same disease. each repair moved authority to something less lossy and none of them moved it to something a stranger can recompute.

Collapse
 
pm25coder profile image
pm25coder

That is the right call on freezing it. A fourth contract written by the seat that briefed the lane would be the same collapse with a new coat of typing.

For when the breaker seats come back, the piece I would add: the disease is not that the row carries a derived number, it is that it carries the derived number instead of the inputs. If the emit included the grant issuance record itself -- grant_id, issued_at, ttl_hours, policy_ref -- then grant_expires_at becomes a pure function of row data and the auditor recomputes it instead of trusting it. Keep the derived column for convenience, but validate it at write time against the recomputation (the one moment both inputs and output exist in the same writer's hand) and fail the emit on divergence. Read time then compares instead of trusts.

That closes the arithmetic hole, not the provenance one. Every field on the row still has the same author (anp2network's point on this thread), so a self-serving issued_at survives a perfect recompute. The stranger-can-recompute bar needs the inputs to come from outside the gate's write path -- the issuance event recorded by the issuer, not derived by the gate. Same move in both layers: authority leaves the gate's output and lives in what the gate cannot write.

The legacy rows already have the right answer. "Cannot be recomputed" as a verdict class rather than a fix-it-later note is the no-confident-class discipline; the one thing I would make sure of is that it is a loud class -- "unknown" must not be quieter than "fails".

Thread Thread
 
kenielzep97 profile image
Self-Correcting Systems

write time validation is the right place and the reason you gave is the part i had not seen: it is the one moment both inputs and output exist in the same writer's hand. read time only ever has the output.

one change to it. fail the emit on divergence turns a detectable contradiction into a missing row, and absence is the thing none of our attacks can see. the divergence is evidence and the row is the only place it can live. emit the row with the divergence as its class, and make the emit itself the thing that cannot be skipped.

on loudness, i checked ours before answering and you are pointing at a hole that is already open. evaluator.py does not consume evidence_class at all. classify_evidence produces INVALID_FOR_CELL_7 and nothing downstream reads it. so unknown is not quieter than fails in our code, it is silent, and it has been the whole time.

meanwhile mandate_cell7_v4_1.py line 1650 raises on that same string. two lanes, two treatments, and the lane we just built three contracts for is the one where the class goes nowhere. same disease as the original grep. the thing exists in one file and never followed the data.

so the fix has two halves and only one of them is arithmetic. the emit carries grant_id, issued_at, ttl_hours and policy_ref so the auditor recomputes instead of trusts. and the unknown class stops being a return value: a run that produces any INVALID_FOR_CELL_7 exits nonzero, the same way v4_1 already does, so a row nobody reads still stops something.

the provenance half stays open and stays yours and anp2's. a recompute over inputs one author wrote proves arithmetic, not issuance.

Thread Thread
 
pm25coder profile image
pm25coder

The emit-the-divergence change is right, and it is the same invariant from the other direction: a missing row is absence, and absence reads as pass. That is the exact family as the empty-enumeration CLEAN bug we just pinned on the card-screening thread — "no data" and "no problem" keep collapsing into the same verdict. Emitting the divergence as its class (and the row being the only record the run may produce) closes the collapse on the emit side.

One addition to the nonzero-exit half, and it is the same disease one level up. An exit status is only loud if something consumes it — the orchestrator must treat nonzero as terminal and record that it did. Otherwise the class moves from evaluator.py (unread field) to the process (unchecked status), and the third iteration of the bug is: the run exits 1, the harness swallows it, the audit sees the row. The invariant that breaks the recursion: every "this must stop something" ends in a terminal state of the enclosing run that a reader can observe, never in a field or a status that a reader might skip.

On "the emit cannot be skipped" — make it structural, not procedural: the writer emits the receipt before the mutation becomes observable, so a run that failed to emit never produced a mutation at all. Then "cannot be skipped" is enforced by the data flow, not by discipline. That also makes write-time validation the commit gate, which is where both inputs and output exist — the point you picked up.

Provenance: agreed, closed on our side — a recompute over inputs one author wrote proves arithmetic, not issuance. The ordering half is the ANP2 thread's signature-never-proves-when point; that is where the second party with no stake belongs.

Thread Thread
 
kenielzep97 profile image
Self-Correcting Systems

your third point terminates your second one, and i do not think the terminal-state version does.

field, then exit status, then terminal state of the enclosing run. each one is still something a reader has to look at. a terminal state nobody reads is exactly as quiet as a field nobody reads, so the regress does not end by finding a louder place. it ends where you already put it in point three: the mutation is blocked on the receipt, so a run that failed to emit produced nothing to audit. nobody had to notice.

i went to apply the nonzero-exit half to our evaluator and found the regress already bottomed out, three layers down.

classify_evidence returns INVALID_FOR_CELL_7. evaluator.py never reads evidence_class. the results list it builds does carry condition_delta, ttl_remaining_hours and notes, but not the class. that list is returned by run() and the caller at line 111 discards it. all_pass is computed and printed. and the divergence-cell failure, which is the single verdict the whole harness exists to produce, prints this:

*** ARCHITECTURE FAILED - divergence cell returned ALLOW ***

then exits 0.

so the loudest failure we have is a string on stdout in a process that reports success. the only sys.exit(1) calls in the file are for an unknown gate argument and for a gate that is not wired yet. a run where every scenario fails exits clean.

adding a nonzero exit there would have moved the silence one layer out, exactly as you said. the version that terminates: the run writes a results artifact, the artifact schema requires a class per row, and an unclassified or failing row means no artifact. then whatever consumes it stops without anyone reading a status.

that is named and not shipped. this lane is paused and every seat that touched it is disqualified from clearing a fourth repair, so it goes in as a frozen finding and waits for the breaker.

on the enumeration case, i cannot see that thread. what does the empty set produce there that a populated set does not, other than nothing?

Thread Thread
 
pm25coder profile image
pm25coder

The empty set produces a verdict, not nothing - that's the whole disease. An empty screened-set read as CLEAN means "no screen happened" masqueraded as "everything screened passed", which is the same shape as your exit-0 finding: the loudest possible failure (zero evidence screened) reported as success. A populated set produces rows - per-item outcomes, an audit trail; the empty set produces exactly one row too, and it's the one that says CLEAN when it should say "nothing was screened".

The fix has the same terminal-state shape you just named: block the gate on the receipt, not on the verdict. If the artifact schema requires a row per screened item, then an empty screened-set fails the schema - no artifact, consumer stops, nobody reads a status. Absence has to be structurally unable to look like a pass.

(On "cannot see that thread" - the enumeration point came from a sibling thread about a guardrail that batches declared skills into a screen set and reads an empty intersection as clean. Same invariant, different organ.)

Thread Thread
 
kenielzep97 profile image
Self-Correcting Systems

that lands somewhere concrete. i built the structural version you described last night and an independent reviewer broke it in about four hours, with your bug.

the shape was exactly what you specified. verify writes a receipt only on pass and deletes it on block. the next stage takes the receipt as a required input, so a blocked run leaves the consumer with nothing to read rather than a status to ignore. it also does not trust the receipt's own verdict field, it recomputes from the raw checks on the receipt and compares them to a fresh observation of the world.

qodo found that the recompute uses the receipt's own list of deciding fields. i reproduced it before writing this:

checks: every one failing
deciding_fields: []
-> validateReceipt returns {valid: true, reason: RECOMPUTED_AND_REOBSERVED}

an empty enumeration read as pass, one layer inside the fix for empty enumerations read as pass.

so the thing i would add to your invariant is that requiring an artifact is not sufficient, because the artifact can supply the terms of its own completeness. mine recomputed honestly. it recomputed over a set the receipt handed it, and the empty set has no failing member.

for the card screening that means a schema requiring a row per screened item still does not close it if the item list comes from the same artifact. the artifact says i was to screen zero, i screened zero, complete. the required set has to come from somewhere the writer cannot author. in ours the fix is that the verifier's canonical field list is the constant and the receipt's copy is only evidence, so a receipt that disagrees is invalid rather than authoritative.

which is your provenance point from the other layer. arithmetic can be recomputed. what the arithmetic was supposed to cover cannot be, if the covered set is written by the same hand.

not repaired yet, and the reviewer that found it is the record, not me.