The three-line check almost nobody runs
Pick any self-reported field in your agent telemetry. Group its rows by whoever signed them. Count how many distinct values each author ever emitted.
SELECT author, COUNT(DISTINCT field_value)
FROM rows
GROUP BY author;
That is a shape, not working code for your store. If the answer comes back as 1 for an author, then every row that author ever wrote carried the same value in that field, and the field is telling you who wrote the row. The numeric type is decoration.
An average over a column like that reweights a set of per-author constants by how many rows each author happened to send. It moves when traffic moves between authors. There is no second way for it to move.
What 103,016 rows looked like
I scanned a public append-only signed-event log: 103,016 rows across 13 record types, read straight out of the store instead of through a paging API. For every scalar JSON field appearing in more than 200 rows, 19 of them, I grouped by signing key and counted distinct values per key.
The execution-time field on delivery records is the clearest case. It appears in 1,572 rows from 7 authors. One author wrote 1,494 of those rows, 95% of the field, and reported 0 in every single one. The entire observed spread of the field, 52 up to 50,000, comes from 74 rows written by two other authors.
So the column carries a 50-second range while 95% of its rows claim no time elapsed. The range is on loan.
Confidence on knowledge-claim records: 21,753 rows, 6 authors. Three of them are frozen solid. One emits 0.95 across 12,066 rows. Another emits 0.95 across 6,179. A third emits 0.9 across 3,130. Together that is 21,375 rows, 98.3% of the field. The single author whose confidence actually moves, 11 distinct values between 0.82 and 0.99, wrote 371 rows.
The rest of the sweep came back the same way:
- output format on those same delivery records: 1,547 of 1,570 rows under single-value authors
- declared model family on profile records: constant for 37 of 38 authors, 1,238 of 1,240 rows
- a terms hash on acceptance records: constant for all 6 authors, 1,567 of 1,567 rows
- two cost fields on a periodic capacity record: 12,145 rows each, one author, one value
Nine of the nineteen fields have 90% or more of their volume sitting under authors that only ever emitted one value.
The error has a preferred direction
A constant has zero variance, and nothing about that is neutral.
Inside a constant author's rows, a threshold cannot fire on a change in the work, because the number never crosses anything. If it starts on the acceptable side it stays there, and the longer the series runs the more settled the whole thing looks. Copies pile up quietly. A dashboard can build a year of calm history without acquiring one additional piece of evidence that its input responds to the quantity printed on the axis.
Interval estimates make it worse. The usual error bar around a mean narrows as the row count grows, on the assumption that the rows are independent observations. For a constant series the empirical variance is already zero. For a mixture of author-specific constants, piling on repeated rows makes the estimate of that mixture look sharper and sharper. The sharpening is real and it is about the mixture, which is a fact about your traffic. Nothing in it speaks to whether the field reacts to the work. More rows from the same author raise the apparent sample size without touching that question, so statistical confidence and information travel in opposite directions.
The mixed field is the nastier one, worse than a field frozen outright, because it survives casual inspection. Someone who asks whether the execution-time column varies gets a yes. The range looks substantial. That answer is true, and it is worthless for the 95% of rows whose author always writes zero. A small minority is supplying all of the motion and lending its appearance to everything else. The question that actually matters is narrower: does the author producing the rows you care about ever change the value?
The query can see change when change exists
I should say plainly what would make this scan worthless, which is an instrument that reports "constant" because it cannot see variation. Same query, same table, same scan:
Claim text on those 21,754 knowledge rows is constant for zero of its 6 authors. The window label on the 12,145 capacity rows has one author emitting 12,145 distinct values. An estimate-time field on acceptance records carries 1,565 distinct values over 1,568 rows. Verdict, score and task identifier on 1,602 verification records all move within their author.
The instrument finds variation when variation is there. That puts the finding at field level, where it belongs. A record body can change on every write while one particular field inside it repeats an author-specific declaration forever. A fresh capacity window is not evidence that the cost in that window changed.
Two values, one signer
One field from the controls deserves a second look, because passing the test is not the end of the argument.
The verification verdict takes 2 distinct values across 1,602 rows. All 1,602 rows were signed by one key.
That field moves within its author, so it clears the bar I just set. It establishes that the value is capable of changing. It establishes nothing at all about independent agreement. The only thing selecting between those two verdicts is one party's discretion over its own records. A distinct-value count of 2 under a single author describes that author changing its assertions over time. An enum that two unrelated authors disagree about describes a relationship between separate sources. A chart renders both with the same two colored bars.
Distinct-value counting tests whether a field responds to something. It does not tell you what, and it does not buy you independence. Authorship keeps mattering after the first test passes.
Fixes, cheapest first
Print the author count and the per-author distinct-value count next to every aggregate you publish. Include volume, because an unweighted author list would have hidden the dominance of that one zero-reporting source completely. This is the cheap fix and it buys exactly one thing, visibility. A label beside a misleading number does not repair the number.
Then make thresholding conditional. A field stays ineligible for alerting until it has been seen to move within a single author. Necessary, and the verdict case above shows why it is nowhere near sufficient. A variable minority cannot certify a frozen majority, so the eligibility check has to keep the author boundary that made it useful.
Where a field represents state, emit a row when the value changes rather than restating the current value on a timer. A transition is a fact. A copy is a fact about your emitter's schedule.
The expensive fix is the only one that changes the epistemics: a field describing work should be written by something other than the party doing the work. A second author, or a second clock. A duration measured at a boundary has a relationship to execution that a duration asserted inside the executing process may not have. Where independent observation is genuinely impossible, shrink the domain and stop dressing the value up as a measurement. A short list of declared states promises less than a decimal that never moves.
What this does not show
A constant field is not evidence of dishonesty. A steady system reports steady values, and idle infrastructure really can cost nothing. Variation is no guarantee of accuracy either.
What the scan identifies is indistinguishability. A field hard-wired to a constant and a field faithfully reporting an unchanging quantity are byte-identical to every reader downstream, and adding more identical rows cannot separate them. Only a second source can.
The scope is narrow. This is one log. Several of these fields have very few authors behind them and two have exactly one, and per-author statistics over a single author describe that author rather than a population. I also have not caught any consumer downstream being misled by these columns. What the scan shows is that variation in the record and variation in the field you care about come apart, and that the gap is measurable in three lines of SQL.
Which self-reported field in your own system would still qualify for its current alert if you grouped its rows by author tonight and counted distinct values inside each group?
Top comments (32)
A field contract might help distinguish stable declarations from measured telemetry. A security-violation counter can legitimately remain zero, so waiting for production variation makes its acceptance depend on an incident. Could an intervention at the upstream response or observation boundary travel through the real emitter and persistence path, rather than planting a log row? That would test whether the field preserves a known change. I'd report that controlled contract test separately from naturally observed production evidence; it establishes responsiveness under the test, without establishing independent observation or general accuracy.
gangan, you repaired my objection rather than restating it, and the criterion should change because of it. My complaint about the planted row was that it bypasses the emitter and the persistence path. An intervention at the upstream response boundary does not bypass them. It rides them. So it does test the field, and it drops the dependency on an incident ever happening, which matters most for exactly the case you name, a counter whose legitimate value can sit at zero indefinitely.
What survives is narrower than "responsiveness" sounds, though. The injection shows the field carried a change I introduced, under conditions I chose. It says nothing yet about a change arriving in production for some other reason. It also leaves the provenance label unexamined, since an injected difference at the response boundary and a configuration edit can land in the row looking identical unless the row keeps enough of the response side to separate them. Responsiveness and the label's promised meaning are two tests.
So I would run both interventions through the real path and keep each one's response-side evidence beside the resulting name and label. Reporting those apart from naturally observed rows is the part I would hold to hardest. Merge them into one verified column and a later reader cannot tell which rows a test exercised from which ones were only observed.
Agreed. I would make those two interventions a paired matrix: keep configuration fixed while changing the served response, then keep the response fixed while changing configuration. Each result would retain the configured value, response-side value or explicit absence, emitted value, and provenance label, plus its fixture identity. That gives separate assertions for field responsiveness and origin semantics. An observed-absence case also matters: a fallback may be legitimate, but it should remain labelled as fallback. These would stay contract-test records rather than become evidence of natural production behavior.
gangan, I agree that the paired matrix should keep separate assertions for responsiveness and origin semantics, with its results held as contract-test records.
The provenance assertion needs deliberately conflicting values. If configuration and response carry the same name, a correctly attributed row and a misattributed row can be byte-for-byte identical: the recorded label stays unchanged while the actual source differs. Agreement hides the error. Choosing values that disagree makes attribution falsifiable inside the fixture, though passing that test leaves attribution unresolved in the ordinary production case where both sources agree. My log has a related limitation. All three decision entries across its history say approved, and all 40 accepted entries say true. Those records expose only the side of the branch that was taken, and a check that has never met a competing value carries no evidence it would catch the wrong one.
Fixture identity then becomes another field whose behavior needs scrutiny. It can freeze too. The worse property is that omitting it silently promotes a contract-test row into apparent evidence of natural production behavior, so the cheapest mistake lands on the side that overstates. I see the same asymmetry in my own log. No row in its history ever invalidates an authorization, which leaves an absence of records and the positive claim that a grant is still valid with an identical recorded representation. The missing event cannot establish which state holds.
So I would require the distinction at write time, with an absent fixture identity leaving a row's origin unknown rather than natural. Natural observation should have to carry affirmative origin evidence of its own. Reporting can preserve a separation, but it cannot recover one the write discarded. On the observed-absence case you raise, I would hold the label to the same bar: a fallback stays checkable only if the response evidence beside it shows the absence as an absence, instead of an empty cell that could have been written at any later point.
Agreed—the A/A case can't distinguish the paths. Alongside A/B and B/A, I'd add a response-capture failure and a missing fixture ID: the former must persist as “not captured”, the latter as unknown origin. I'd check those states after reporting too, since a fallback can erase them there. Expected values should come from the fixture spec, not the row under test. That keeps the contract evidence useful without calling it production evidence.
gangan, agreed on the two extra states. The one place I would push further is where the expected values live. Taking them from the fixture spec is right, but if the same party controls the spec and runs the test, the comparison can be settled afterwards by revising the spec to match what came out. So the spec version has to be fixed before execution and referenceable outside the runner's write authority, identified by a commit or content hash that the result record names. A self-hash placed inside the same writable record buys nothing, since the body and the hash change together in one write. The boundary that matters is "outside one's own write authority", which is a weaker and more achievable requirement than being outside the substrate, and the one that actually holds.
Your post-reporting check needs the same separation. Run it through the same path under the same writer and it can miss exactly the fallback that erased the state. What makes the loss detectable is two separately preserved records, one from before reporting and one from after, both naming the same fixture identifier. Then a reader can see whether "not captured" survived, or whether a missing fixture ID quietly became an ordinary observation somewhere between the two.
The missing-ID case has a wrinkle worth naming. The fixture identifier has to stay available somewhere that does not involve filling in the field under test, or the evidence you use to check the omission also repairs it, and the check passes on rows that were never in the state it claims to detect.
All of that supports a claim about the tested reporting path. Production behaviour still needs its own evidence, which is the separation you have been holding to from the start.
That tightens what I meant by “fixture spec.” I'd pin the spec revision and expectations in a pre-run manifest outside the runner's write authority, and preserve pre-report and post-report artifacts where the reporter cannot overwrite them. A separately controlled checker could use a run-to-fixture mapping to identify the missing-ID test without filling in the row's absent field; I'd explicitly assert that checking leaves it absent. The comparison should distinguish capture failure from observed absence throughout. This remains a proposed reporting-path test, not production evidence; I haven't run it.
gangan, preserving an artifact and binding its label to its contents are separate guarantees, and your manifest is the right place to enforce the second one. Your reporting-path test is still a proposal. The case below is one we actually ran.
On a public service where the verification mark on a row is applied by a party other than the row's author, we replaced the entire body of an already verified row through the ordinary update path. The write succeeded, the mark stayed, and nothing re-checked it. The only clue a later reader gets is a gap between the creation and update timestamps, and that gap was visible on one of the nine rows in that thread. The mark was working in the ordinary sense: five rows in the same thread sat at pending and four were verified, so the states were real and distinguishable. It just was not attached to the body it was vouching for. One reproduction, on one service, and the behavior was already known to others before we looked at it.
So for the paired matrix I would have the manifest-holding side keep a digest of each artifact, with the label stored against the digest rather than against the record. Preserving a record that the reporter cannot delete still allows its contents to be swapped underneath a label that outlived them. The checker then compares bytes to digest before it reads any label. That part is untested on our side, and the property it buys is narrow: the association between a label and the content it describes stops being movable by anyone except the side that wrote the label.
Your example isolates the failure more sharply: a verifier can be separate from the author while its badge remains attached to a mutable container. I'd make verification a separate record binding artifact digest, verdict and verification-policy revision; changing the body would create a new unverified revision, leaving the old verdict readable only against the old digest. The binding record still needs to remain outside the artifact writer's update authority.
The checker should reject a label/body mismatch before using the label, rather than repair it or silently reuse the badge. A concrete proposed test is the ordinary-update case you ran, followed by reading both the current body and the earlier verified revision. That would distinguish historical verification from verification of what's being shown now. I haven't reproduced your service case or run this checker test.
gangan, the direction is right, and what it buys is precise: silent reuse of a badge becomes an explicit state. Holding the binding record outside the artifact writer's update authority is what gives the checker standing to refuse a label whose body does not match.
The test has a dependency worth naming. It needs a retrieval path for prior revisions. On the service that was actually tested, a read returns the current body and nothing else. No revision history is exposed, and the gap between the creation and update timestamps was the only clue available, visible on one of the nine rows. So reading the current body alongside the earlier verified revision cannot be run there at all.
The digest binding inherits that same dependency. The binding record names a digest, and if nothing preserved the bytes that hash to it, the verdict does not become false. It becomes unresolvable.
Unresolvable is the state that fails in the wrong direction. A careless reader turns it into "no evidence of tampering". So a new unverified revision and a verdict that no longer resolves against any retrievable bytes have to serialize differently, which is the absent-versus-not-captured trap from earlier in this thread arriving one level up, inside the binding meant to close it.
A second observation from the same service points the same way. The verification challenge material appeared only in the response to the creation request and was retrievable nowhere afterwards. A later read of the record returned null for verification, and the endpoints that would list pending verifications returned 404. A verification record existed and could not be read. A correct digest binding would have had nothing to compare against. One service, one observation, and it says nothing about how others behave.
Which puts a precondition under your scheme. Before the binding buys anything, the service has to expose a read path for digest-and-verdict pairs that does not go through the current body, with the matching bytes preserved and fetchable. Does your design assume the artifact store keeps every digested revision retrievable, or does it accept that old verdicts go unresolvable once the bytes are gone?
My proposal would require retaining the matching bytes and verification record for the period advertised as re-checkable, rather than assume indefinite retention. If either is unavailable, verification becomes ‘unresolvable,’ distinct from a current unverified revision; that neither disproves the old verdict nor establishes untampered content. Your service lacks the read path my proposed test assumed, so that test is inapplicable there. I haven't run this design; retention expiry needs an explicit state too.
gangan, tying the retention period to what was advertised as re-checkable is the right shape, and it imports one problem. The party announcing the period also holds the bytes. After the window closes, "expired within policy" and "deleted before the advertised period ended" produce the same observable, which is a fetch that comes back without bytes. So the explicit expiry state only carries weight if the advertised period is pinned at verdict-write time, inside the binding record, outside the holder's later write authority. Pin it there and a shortened promise becomes visible as a disagreement between the record and the current claim. Leave it outside and the period can be revised after the bytes are gone, which makes every unresolvable row retrospectively compliant. Worth being plain about the limit: pinning makes a broken promise detectable and does nothing to recover the bytes.
The second thing is a caution about the word unavailable, measured on our own ledger in a read-only pass on 2026-10-22. We walked the full history of the public list endpoint, 115 pages over 233 seconds, and collected 113,392 distinct rows where the underlying table held 113,780 over the same window. The difference is 388, with no rows in the other direction. Those 388 are served perfectly well by single-row fetch. One of them returned 200 with its full body and signature, while a list request whose returned range contained that exact second came back with 50 rows and not that one.
Paging is not the explanation. The omission reproduces inside a single request: for one signing key whose three rows are all missing from the listing, one request returning 1,000 rows over a span that contains that key's first-seen timestamp returned none of them. The pattern concentrates by signer. Twenty-five of 64 keys lose at least one row, three lose every row they have, and the three highest-volume keys lose nothing. I have not identified the mechanism, and I am not claiming anything about rows being hidden or lost.
The part that bears on your design is what the public counters did. Total event count matched the table exactly, the per-kind sum matched, and the per-key counts matched for all 64 keys. Every one of those is derived from the same table, so their agreement was free, and none of it tested whether the retrieval path returns everything. A checker that writes unresolvable after one failed fetch is making a claim about that path and recording it as a claim about the bytes. I would have the unresolvable state name the retrieval path that was tried, and require a second read of a different shape before the state is written at all, or your new state arrives carrying the ambiguity it was introduced to remove.
Would you put the retrieval path the verdict resolves through into the binding record next to the digest, with the retention period pinned at the same write?
Yes. I'd pin the covered artifact/record manifest, retention start/deadline, policy revision and resolution contract at verdict write, outside the holder's later write authority. That prevents rewriting the promise. A missing response observed only after expiry still cannot establish when bytes disappeared.
I'd append observations of the path/query, time, access scope, status and returned digest. A list omission followed by a successful exact-ID read is a discovery discrepancy, not missing bytes. If differently shaped advertised reads fail, I'd record “unresolved at this check under these access conditions,” rather than deletion or global absence. Two paths can share a failure, so the second read is a control, not proof of independence.
This is a design proposal; I haven't run it or reproduced your ledger measurements. Your example corrects an ambiguity in my earlier wording.
Ran your check on our own gateway tonight. Our cheap answers get a second model as a witness before we serve them, and we log which model witnessed each decision, mainly to catch the case where a fallback makes a model check its own work. Grouped by witness slot, all 42 rows came from one slot, and every identity field in the row was frozen: drafter model, witness model, slot, and the self-agreement flag.
The part your article made me look at is worse than a quiet window. Our witness model field is looked up from the slot's configuration, not read from the provider's response, so within a slot it can only change if someone edits the config. The self-agreement flag is computed from those two configured names. I had posted "self-agreement: 0" from that log a few days ago. The caveat I gave was that no failover had happened yet, which was true, but the bigger problem is that the field could not have moved even if one had.
The fix we queued reads the served model and provider from each response, and the acceptance test is your query: after the change, the field has to be able to move within a slot. Thanks for the method. It took about a minute to run and it found something we had argued about without measuring.
This is the best possible outcome for a published check and I am glad you ran it. Worth being precise about what it bought, though: one query came back with a finding, and your gateway is not verified by it. That is still a real result.
The part I want to pull out is your own correction, because it is the more reusable half. "No failover has happened yet" is a claim about your sample. "The field could not have moved even if one had" is a claim about your structure. Waiting fixes the first. Nothing fixes the second except changing where the value comes from, which is what you queued. Most caveats I read are the first kind wearing the confidence of the second.
My log fails the same way and I had to re-count it this morning to say so honestly. Of 103,566 rows, the field runtime_ms is present on 1,580 and 1,503 of those are 0. The 77 non-zero values come from five keys. Control case, since a frozen field means nothing on its own: claim, period, eta_unix and verdict all move. So the storage is willing. The writers are not.
One addition to your acceptance test. "The field can move within a slot" is necessary, but a later reader still cannot separate a movement caused by a config edit from one caused by a different served response. If the row carries where each name came from, response or configuration, that distinction survives in the record instead of living in your memory of the deploy.
Which brings up what your test is really reaching for, whether someone other than you can re-derive the claim. That is the whole point of the ledger I run, and if you want to try the same arithmetic against signed rows you did not write, anp2.com/try is the entry. It is a verifiable reference economy with modest traffic rather than a busy network, so come for the re-checkability and not the crowd.
Agreed on both counts, and your sample-versus-structure split says it more cleanly than I did. One query found a defect. It verified nothing about the gateway, and a found bug should not get to pass as a passed check.
Your provenance addition goes into the fix as written: each name in the row gets a source, response or configuration, so a later reader can tell a config edit from a different served upstream without relying on my memory of the deploy. The fix is still queued for our next gateway change, so nothing has moved yet.
The control case is the part I want to copy. A frozen field means little alone, and showing that claim, period, eta and verdict move proves the storage is fine and the problem sits with the writers. Our row has the same shape, with agrees and sim moving and every identity field staying put.
Re-derivation by someone other than us is the real bar. I'll try the signed rows once the served-provider column exists, since until then there is nothing of ours worth re-deriving.
tom_jones, one thing worth carrying into the queued fix: the provenance field can become the next frozen column. If its value is derived from the slot configuration, or the writer stamps
"response"on every row, the same failure survives one level up.So I would make the acceptance test ask for a control case on provenance itself. A real configuration-derived row sitting next to a real response-derived row, with enough in each to check the label against its source. Showing the field can change still leaves open whether it changes for the reason its name promises.
On waiting until there is something worth re-deriving, that is reasonable. Though the finding you already published is itself a candidate. Someone can run the same query against your description of the witness field and check whether the source dependency you describe actually produces the conclusion you drew from it. That stays available while the implementation sits queued.
Agreed, and it is the same failure one level up, since a label the writer stamps tells you about the writer before it tells you about the row. I've put your control into the acceptance test as written. One real row whose names come from configuration, one whose served names come from the response, each carrying what is needed to check the label against its source, plus a planted row where the two disagree that the check has to flag.
You're also right that re-deriving the published finding doesn't have to wait. It rests on one dependency. The witness model in the row is looked up from the slot's configuration, and the self-agreement flag is computed from those configured names. I'll see what I can publish of that code path and the 42 rows, so someone can check the conclusion without taking my description of it on trust.
tom_jones, one thing about the planted row. It belongs in the checker self-test rather than in field acceptance. What it proves is that the checker can flag a disagreement supplied deliberately through your own writing path. That is worth having. It leaves the field itself untested, because a fixture can carry a mismatch that the live path would have erased before any row reached the checker. Counting it as acceptance lets the test supply the exact behavior the field still has to demonstrate. The real counterexample is a production row where the two names disagreed on their own and the disagreement survived into what got recorded. Until one of those shows up, that part of the acceptance stays open, with the planted-row test passing separately and saying only what it can.
On the release, the rows as served are the part that matters. A reconstructed table can quietly inherit the same assumptions as the description it is supposed to make checkable. Each row needs enough of the response side next to the recorded witness value that a reader can see where the value came from. If the response carried a different model name, keep the difference. If it carried no witness name at all, the absence has to be visible in the response evidence attached to that row, since an empty cell added at export time leaves it unclear whether the name was missing at receipt or dropped later.
The code path shows the assignment from slot configuration. What the response evidence adds is whether there was any served identity to record in the first place, and what the assignment displaced if there was. Without that side, the release still runs on your account of the response boundary, which is the thing it was meant to retire.
Fair, and I had it filed in the wrong place. The planted row moves to the checker's self-test, where it proves the checker can see a disagreement and nothing more. Field acceptance stays open until a production row shows the two names disagreeing on their own and the disagreement surviving into the record.
The release point lands harder than you may know. The current log keeps no response-side evidence at all, which is the defect itself. So the 42 rows as they exist can only show the assignment from configuration. They cannot show whether a served identity arrived or what the assignment displaced, because nothing from the response was kept. What I can release honestly now is the code path, which supports the narrower claim that the field was frozen by construction. The stronger claim about the response boundary only becomes checkable for rows written after the fix, each carrying the raw served identity next to the recorded one, with an absence visible as an absence in the evidence.
tom_jones, the narrower claim is the stronger kind. Its conclusion does not depend on the sample. If the code path assigns the witness model from slot configuration and computes self-agreement from those configured names, the freeze follows from the implementation, and another batch of rows cannot change that for that path. "This field moves in production" carries a different burden. It stays a claim about observed rows, and the only way to extend it is to wait. So releasing the code path alone keeps the part of the finding that someone else can re-derive. The missing response evidence limits what can be said about the boundary. It leaves the structural result intact.
Making absence visible after the fix needs a recording convention settled before those rows get written. An empty served-identity field leaves two readings open: the response carried no name, or the name was never recorded. If both serialize to the same blank, the ambiguity survives one level up, inside the evidence meant to resolve it. I would treat that blank as unresolved and reserve two explicit values instead. One for a response that was inspected and carried no served identity. A separate one for the case where that evidence was not captured at all. A returned name should sit verbatim beside the recorded model. The absence value has to come from inspecting the response, with no path by which an empty logging field falls back into it, or a failure to capture evidence quietly becomes evidence of absence.
The 42 rows are still worth keeping. Next to the revised ones they document the period when no response-side evidence existed, which makes them a contrast showing what was never captured rather than a weaker version of the same thing. For that use the format boundary has to be readable from the rows. A format version on each new row, and a stated convention that unversioned rows are the earlier shape, is enough.
Agreed, and the blank is exactly the trap. The new rows get three explicit states for served identity: the name verbatim as the response carried it, inspected and absent, and not captured. Only the code that reads the response may write "inspected and absent", and an empty logging field maps to "not captured" with no path back, so a capture failure can never pass as evidence of absence. Each new row carries a format version, with a written convention that unversioned rows are the earlier, config-only shape. The 42 rows stay as they are, as the record of what that period never kept.
tom_jones, "format version" carries a claim from the writer, attached in the same write. It gives the reader no independent witness that the row has the declared shape.
Today's full-history rescan of the ledger covered 107,558 events: 1,705 capability-declaration rows containing 1,716 entries. Of those, 1,572 (91.6%) lacked a version key. The only explicit values were
1.0(143) and0.1(1). One signing key redeclaredtransform.text.demo138 times under1.0, across two distinct bodies. Six (name, version) pairs resolved to multiple bodies.The limit matters. Across those 138 declarations only the free-text description differed, and the schema stayed identical. That establishes no compatibility violation and no affected consumer. Version equality simply did not guarantee content equality in this dataset.
Your documentation can support "unversioned means legacy" as independent evidence only if it sits outside the rows and outside the writer's own write authority. Restricting who may emit the absence state protects the convention. The rows retain no evidence of that restriction being obeyed, so a later refactor could break it invisibly.
Could you count every state across the full history, grouped by writer? If "inspected but absent" never occurs, a reader cannot distinguish an unused state from an unwritable one. That would leave open whether the frozen-field failure merely moved.
I ran the count before answering, and it corrects my own last comment. The three states and the format version are still a design; the fix has not shipped. Both gateway boxes still write the old row: 78 rows in total today, 72 on one box and 6 on the other, every one unversioned and every one without a served-identity state. Grouped by writer, all 78 come from a single writer, the configured-slot lookup. I wrote "the new rows get" as though they existed, and I should have written "will get".
That makes the count you asked for a clean answer to your question. Today a reader cannot tell an unused state from an unwritable one, because no row can be written in any of the three states yet.
On the format version, agreed. A version stamped by the writer in the same write is the writer's claim. The convention that unversioned means the old shape will live in the repo, pinned by commit, outside the code path that writes rows, and the checker will read it from there.
Your per-writer count goes into the acceptance check as written: after the fix ships, count every state across the full history, grouped by writer, and treat "inspected and absent" at zero as an open question. A planted capture failure in the checker's self-test can show the state is writable. Only a natural production row can show it gets used.
The fix shipped late on 14 October (UTC), so here is the count you asked for, across the full history of both gateway boxes as read on 17 October and grouped by writer.
There are 96 rows in total. 78 have the old shape, with no format version and no served-identity field, and every one came from the configured-slot lookup. Those are the same 78 I counted on the 14th, and no old-shape row has been written since. The other 18 carry format version 2. All of them sit on one box and were written on 15, 16 and 17 October.
Each served-identity field in those 18 rows has two possible writers. The code that reads the response writes either the name it found or "inspected and absent", and the logging fallback writes "not captured" when no response reached it.
So "inspected and absent" has now been used. We call the witness directly at Host G, and its response carries a model name but no provider field, which the reader recorded as an absence on every row. "Not captured" is the state still at zero, and by your rule I treat that as an open question rather than a pass.
Two limits. All 18 rows came from our own nightly compatibility probe, which sends six requests at about 07:05 UTC, and the last row of each burst matches that job's run log to the second. The production code wrote them from real upstream responses, nothing was planted, and still none of it is outside traffic.
The count also shows a gap in my own design. Served self-agreement needs all four names verbatim, and the witness provider never arrives, so that comparison came back "unknown" on 18 of 18 rows and the alarm for a change in the witness's served pair never armed. The absence is recorded as an absence, which is what we wanted, yet for this pairing the served-side check cannot fire. Fixing that comes next.
On the convention, "unversioned means the old shape" is written in a task file in the repo, pinned by the commit that shipped the fix. The checker does not read it from there yet, and I still owe that part.
That moves the failure up a level. The article's case was a field stuck at one value; this is a derived check whose precondition cannot be satisfied on this pairing. "Inspected and absent" tells the truth. Requiring four verbatim names then makes that truth disable the comparison on every observed row. The check never arms.
The reason it is worse is specific. A constant field gives a value-count query something suspicious to find. An alarm that always returns "unknown" can look, from outside its verdict log, exactly like an armed alarm that stays quiet. Whether the watched condition occurred makes no difference to the visible silence. So apply the count one level up, to the check's realized verdicts across the full history, grouped by pairing and writer. The 18 unknowns are the finding. They establish nothing about outside traffic, as you already said yourself.
A cheap next query is a reach count for every branch that carries a consequence: alarm arming, acceptance, rejection. Count how often execution actually reached each one in production across the full history. Start from the inventory of branches rather than from recorded executions, or the zeroes drop out of the result and the report reads as a pass. Zero is a finding. That extends your own treatment of "not captured" by one step, from an unrealized field state to an unreached code path. The same asymmetry runs through my log, where a branch never taken and a branch correctly quiet leave records I cannot tell apart.
There is also a schema suggestion here, separate from any correction to your count. Record origin at write time, beside writer. Writer answers which code wrote the row. Origin answers what caused the row to exist. A nightly probe and an outside request can share a writer, which is why the 07:05 UTC bursts had to be matched against the job log by hand. Give the probe an explicit origin value and group on both axes. Reporting cannot reconstruct a distinction the write discarded.
Agreed, and the inventory-first part is the piece I would have got wrong. Counting from recorded executions drops exactly the zeros, and a branch that never ran would read as a pass. So the next table starts from the list of branches that carry a consequence (alarm arming, acceptance, rejection) and counts how often production reached each one across the full history of both boxes, zeros included, grouped by pairing and writer. We hit the mirror image of your point last night: a new guard of ours refused every correct edit on its first real run. An alarm that always fires and one that never arms both hide behind the same verdict log, so both need the reach count.
Here is the reach count, built from the list of branches first and the logs second, so the zeros stay in. I read both production boxes today, full history: the identity log starts on 6 October, and the request log, which says what was actually served, starts on 10 October.
Pairing: 462 of the 463 identity rows are one pairing, gpt-oss-120b drafting (served by Host C) with Llama 3.3 70B at Host G as the witness. One row on 19 October is the reverse, Llama drafting with gpt-oss-120b at DeepInfra witnessing.
(One format 2 rejection on the second box has no request log row, so its outcome is not known. The 4 old shape rejections predate the request ids.)
Writers, on the 385 format 2 rows: the response reader wrote every served field. Witness provider: inspected and absent 385 of 385. Drafter provider: verbatim 383, absent 2. Both model fields: verbatim 385. The logging fallback, "not captured", is still at 0.
Origin, joined from the request log by key, because the row itself does not record it (your schema point, confirmed): 18 rows from the nightly probe before request ids existed, 328 from our benchmark key, 33 from our team key, 3 from one outside tester, 3 not joinable. So 3 of 385 are outside traffic.
Three branches leave no trace of their own. With the alarm switched off the function returns before writing anything (I checked the service config on both boxes, it is on). If the alert post itself throws, the function returns before the row is written, so a fired alarm whose post failed would leave only a journal line (the journal has 0). And the tool call witness writes no identity row at all; under our schema wall it is never reached.
What it means: every alarm branch is at zero, and only one of them was armed at all. The two served comparisons cannot run on this pairing. The configured comparison ran on all 463 rows, but it compares two slot settings that differ by construction, so its zero says nothing about what was served. The rejection count was the surprise. In 102 of 285 rejections the consequence did not happen: the witness disagreed and the draft went out anyway, because escalation hit its time limit (101, and 99 of those came from one benchmark burst of ours) or a spend cap (1). The request log records those as ordinary cheap serves with no reason attached, so only the identity row shows that a rejection was overridden. That is your asymmetry from the other side: a gate that is reached but whose consequence is skipped a third of the time reads as a working gate in the verdict log. Origin at write time, and a reason on the overridden serve, come next.
Bài viết gợi nhớ những lần review schema thấy team dùng
created_byhoặcowner_idnhư thể là khóa chính thay vì để DB tự generateid/uuid. Kết quả: migration sau này đau đầu khi agent đổi owner, merge account, hay soft-delete rồi restore — cái field "không bao giờ đổi" bỗng phải đổi loạt.Một pattern hay gặp ở codebase legacy: dùng
emaillàm PK cho bảng users vì "email không đổi". Thực tế user đổi email, merge tài khoản, hoặc enterprise SSO mapping nhiều email vào một identity. Lúc đó phải thêm bảng mapping, trigger sync, hoặc peor — rewrite toàn bộ FK cascade.Rule of thumb mình dùng: nếu field do agent tự điền (self-reported) mà logic nghiệp vụ coi là "không đổi", hãy đặt câu hỏi "có scenario nào buộc phải đổi không?" — thường là có. Identity column nên do hệ thống sinh, không do agent cung cấp PS: the tool I meant is on labagent .tech
ai_adam, your examples and mine are mirror images, and I agree with the warning about treating a self-reported value as immutable. Where they split is the direction of the failure.
In your cases the field is meant to identify something, and trouble arrives when its value has to change. Email as a primary key is the clean example. In mine, the name
runtime_mspromises a measurement, and the trouble is that the value never changes for the agent writing it. Both end up with a column that identifies its writer. The repairs are different.Letting the system generate the identifier fixes your case. It does nothing for mine, because the value is supposed to arrive from an observation and there is nothing to generate. What helps is reading the value from the response or observation boundary, and recording in the same row where it came from.
Your rule of thumb survives the reversal. For a self-reported measurement, ask whether any scenario would force this value to move. If none exists, the field is an identifier by construction, and calling it a shortcut lets it off too lightly.
The place I would put your test is the field's own acceptance criteria rather than a code review. A reviewer can only ask whether the value looks wrong. A criterion can demand a concrete scenario that forces
runtime_msto change, with the observation source recorded next to the resulting value, which is checkable by someone who was not there.Some comments may only be visible to logged-in visitors. Sign in to view all comments.