DEV Community: Self-Correcting Systems

My Health Check Watched the Wrong File

Self-Correcting Systems — Sun, 04 Oct 2026 18:05:19 +0000

I wrote a rule for my own agents:

Liveness is not usefulness. Artifact age beats PID.

A job that is loaded, looks healthy to the supervisor, and is producing nothing is the failure worth catching. Anyone can detect exit 127.

Then I applied the rule to my own machine and got the wrong answer.

What I saw

Three listener jobs. launchctl lists them. pgrep -f sentinel_listen.py returns a PID. Their log files:

sentinel_listen.log    8103 bytes   mtime 2026-08-11
courtside_listen.log   8127 bytes   mtime 2026-08-11
eye_listen.log         8149 bytes   mtime 2026-08-11

Last written August 11. By my own rule, that is three dead jobs wearing a costume. I had the sentence written.

A word about what these are, so nobody has to guess. They are small Telegram pollers, 148 to 243 lines each. They wait for a message and answer it. They are not the interesting part of anything I build, and the failure below is not a failure of theirs — it is a failure of how I checked them. I am using them as the specimen precisely because they are simple. If a health check cannot get a 148-line polling loop right, it will not get anything harder right either.

What the script actually does

Before publishing that, I opened the thing that writes the file. There are three print() calls in it:

scripts/sentinel_listen.py:120    print("SENTINEL LISTEN: no rail configured ...")
scripts/sentinel_listen.py:128    print("SENTINEL LISTEN: awake, polling every 20s ...")
scripts/sentinel_listen.py:142    print(f"SENTINEL LISTEN error: {type(exc).__name__}: {exc}")

Startup, banner, error. Courtside and eye are the same shape at :144/:152/:175 and :198/:201/:236.

The successful path prints nothing. There is no "still polling" heartbeat. On the normal path it calls getUpdates with a 25-second long-poll, writes a small state file, sleeps 20, and says nothing at all. So a quiet pass is the poll plus 20 seconds, not 20. A caught exception sleeps 30 and then takes that same sleep on the way round. The interval is not uniform across passes.

So silence in that log does not mean the job stopped. The file is only written on startup and on a caught exception, and I was reading its silence as death.

I cannot go further than that, and the reason matters. These three run without -u, so stdout is block-buffered. An exception printed one minute ago could still be sitting in a buffer instead of on disk. An unchanged file does not prove no errors occurred. It proves the file has not been modified. Those are different statements, and the second one is the only one I can support.

The 8103 bytes are an old error storm — one banner and eighty caught exceptions: 76 URLError, 3 timeout, 1 HTTPError. That is what the file records. What it has failed to record since is a question the file cannot answer about itself.

The file that was actually moving

While I was checking whether these jobs were dead, they were writing. I watched all three state files advance. Sentinel, for example:

$ stat -f '%Sm %N' -t '%H:%M:%S' scripts/sentinel_listen_state.json
  11:13:03   sentinel_listen_state.json
$ stat -f '%Sm %N' -t '%H:%M:%S' scripts/sentinel_listen_state.json
  11:15:19   sentinel_listen_state.json

The log had not been touched since August 11 and the state file was seconds old. Both belong to the same listener job, but only one of them was designed to move on ordinary loop iterations — and they live in different directories. I was watching agent_outputs/. The living file was in scripts/.

I picked the artifact that only moves on startup and on failure. The last startup line in that file is from August 11. The process itself came back on September 18, and at the September 22 check, the banner emitted by that restart had not reached the log file on disk.

The better signal is also not proof

Here is where I have to stop myself a second time, because "check the state file instead" is the obvious lesson and it is not quite right either.

This is the end of courtside's loop:

while True:                                           # :153
    try:                                              # :154
        for upd in updates.get("result", []):         # :156  (body elided)
            ...
            STATE_PATH.write_text(json.dumps(state))  # :173  only after a handled message
    except Exception as exc:                          # :174
        print(f"COURTSIDE LISTEN error: {type(exc).__name__}: {exc}")
        time.sleep(30)
    STATE_PATH.write_text(json.dumps(state))          # :177  every pass, either way
    time.sleep(POLL_SECONDS)                          # :178

There are two writes. The one at :173 happens inside the try, after a message is handled. The one at :177 is at loop level, outside the except, so it runs on every pass — including a pass that just caught an exception and slept.

That second write is the one that keeps courtside's timestamp fresh, and eye is built the same way. So for those two, a fresh state mtime proves the loop is turning. It does not prove a poll succeeded, and it cannot tell a good pass from a failed one.

Sentinel is not the same. It has no loop-level write — only :140, inside the try, after getUpdates returns. So sentinel's timestamp is a stronger signal than the other two. It shows that the API call returned parseable JSON and the loop reached its write. It does not separately record API-level success, whether messages arrived, or whether a reply was delivered. An empty successful poll is normal. Three jobs I had been treating as one row are three different write sites.

And the contents are not one story either. I checked all three:

sentinel    offset 0            history 0      13 bytes
courtside   offset 51848716     history 4     545 bytes
eye         offset 63652330     history 8    2988 bytes

I had looked at sentinel, seen {"offset": 0}, and was about to write that all three had never consumed a message. Two of them hold a non-zero saved cursor and a non-empty history. Three jobs that look identical from launchctl are in three different states, and the only reason I know that is that I opened all three instead of one.

Even sentinel's zero is smaller than it looks. It is the saved cursor right now. It is not a history of everything that job has ever done.

What each thing actually establishes

Observation	Supports	Does not establish
`pgrep` returns a PID	a process exists at this moment	that it polls or delivers
log file unchanged	the file was not modified	that no exception occurred
state mtime advances — courtside, eye	the loop reached its unconditional write at `:177`	that a poll succeeded; it writes after a caught error too
state mtime advances — sentinel	`getUpdates` returned and parsed, reaching `:140`	API-level success, that anything arrived, or delivery
saved offset / history	the file currently holds those values	lifetime behavior, or delivery
a successful-poll receipt	—	I do not emit one
a delivered-result receipt	—	I do not emit one

The last two rows are the point. Every signal I have is incidental — something the program happens to leave behind while doing its job. None of them was written to record that the job succeeded. Those are the two I would have to build, and until they exist I am inferring operation from debris.

My rule was an improvement on checking the PID. It was not the end of the ladder. "Artifact age beats PID" needs a harder second question: which operation am I checking, what evidence does that operation emit, and under what conditions does absence of that evidence mean failure?

A newer file is not automatically a better signal. You have to know what causes it to change. A quiet listener with no incoming work needs a different expectation than a queue with messages waiting, and a timestamp cannot tell those apart on its own.

The buffering part, which is real and separate

There is a second thing going on with those files, and it is worth separating from the write semantics. Two properties compound. The successful path barely writes to stdout at all, so the stream produces very little. And because stdout is block-buffered, the few lines it does produce may not reach disk promptly. Either one alone would be survivable. Together they make log silence ambiguous rather than informative.

I had a live example of the second half. These processes restarted on 2026-09-18 and the startup path prints a banner at :128. On September 22 the file still contained exactly one banner, with an mtime of August 11. The September restart ran the print, and the line was nowhere on disk. Either it was waiting in a buffer or it was gone. The result at the end says which.

These three plists run /usr/bin/python3 script.py. Four sibling jobs on the same machine run /usr/bin/python3 -u script.py. When Python's stdout is non-interactive, as it is when these jobs write to a file, it is normally block-buffered. The exact size is a runtime detail rather than a guarantee: Python commonly uses the file's block size and falls back to io.DEFAULT_BUFFER_SIZE, which on this interpreter reports 8192. -u and PYTHONUNBUFFERED=1 disable that buffering for stdout and stderr.

finance    -u   2026-09-22 19:50   52479 bytes
health     -u   2026-09-22 09:22   55759 bytes
kairos     -u   2026-09-22 09:30   55029 bytes
organizer  -u   2026-09-22 09:21   61883 bytes

sentinel   no   2026-08-11 16:52    8103 bytes
courtside  no   2026-08-11 16:52    8127 bytes
eye        no   2026-08-11 16:57    8149 bytes

On September 22, four out of four with the flag had logs that moved that day. Three out of three without it were frozen. That is a seven-job cohort on one machine, not a law of nature, and I am reporting it as what it is.

Both things are true at once: those programs have almost nothing to say, and without -u, the little they do say is not guaranteed to reach disk promptly.

A prediction, dated before I know

Frozen 2026-09-17. The ledger entry also carried my buffering explanation and an ESTABLISHED socket condition; the socket clause was removed by an amendment on 2026-09-17 and the script-name test added on 2026-09-21. This is a summary of the observable part that remains:

Those three logs will still carry mtime 2026-08-11 on 2026-10-01, while the processes are still alive.

Wrong on 2026-10-01 if any of the three logs has an mtime on or after 2026-09-18 while its plist still lacks -u and PYTHONUNBUFFERED, or if any of the three listeners is absent when tested by script name on that date. An intermediate restart is not a falsifier; the prediction is about what is true on the resolution date.

Not by PID. My ledger recorded 86150 / 86156 / 86157. The machine rebooted on 2026-09-18 and they came back as 859 / 860 / 853. On September 21, ps -p 86150,86156,86157 returned no rows. Resolving against those numbers would have written WRONG against a prediction that was still holding at the time.

That is an observation, and it is the only part I am committing to. The buffering account above is my explanation for it, and a correct prediction would not prove the explanation. If the October 1 check fails, I will say so here.

I am not adding -u until then. Changing the launch configuration now would invalidate the observation window I froze for the prediction.

October 1: it failed

By September 25, the logs had changed.

              frozen               Sep 29 snapshot (mtime Sep 25)   Oct 3 reading
sentinel   8103 B  2026-08-11    16276 B  2026-09-25 06:32    65095 B
courtside  8127 B  2026-08-11    16315 B  2026-09-25 20:39    65322 B
eye        8149 B  2026-08-11    16333 B  2026-09-25 21:27    57130 B

The plists still run /usr/bin/python3 script.py, no -u, no PYTHONUNBUFFERED, and their modification times are July 5 and July 11, before the prediction was made. So the launch configuration did not change across the window, and the September 25 log changes satisfy the frozen falsifier retrospectively, even though I failed to run the resolver on October 1. That first falsifier alone makes this WRONG. The process condition was never observed on October 1 itself. Checked later, on October 2 and again on October 3, all three were alive by script name, the same PIDs that started 2026-09-18 06:28:01. Eye's last change was October 2 at 20:20; its Oct 3 number is a reading, not a third move.

The new content is what the write sites said to expect: caught-error lines. As of October 3, sentinel's log holds 673 error lines. 594 are URLError, and 459 of those are this one:

SENTINEL LISTEN error: URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known>

That is a name-resolution failure. The error alone does not tell me whether the name was invalid or whether resolving a valid name failed. Between the frozen sizes and the September 29 snapshot, the files had grown by 8,173, 8,188 and 8,184 bytes. Together with the caught errors and the buffered configuration, those increases are consistent with accumulated output being flushed. I did not trace the writes, so I cannot identify the exact flush count or trigger from these snapshots, and a consistent result does not prove the explanation.

The bet was the part that lost. I bet the logs would sit still until October 1. Enough new error output accumulated that the logs had moved before October 1.

The banner question has an answer too. The sentinel log now has two startup banners. The second is line 82, directly after the 81 lines that were there on August 11, and the only startup these processes have had since August 11 was September 18. That banner is the September 18 startup line. It was not lost. It eventually reached disk. That is consistent with the buffering explanation above. I had a second dated prediction that said the opposite, that the restart's output was destroyed and the file would still be 8103 bytes with one banner on October 5. As of October 3 the file is 65,095 bytes with two banners. It is already not that file, so Monday can only confirm a result that is already visible, unless the file shrinks. Under my own rule it stays pending until then.

One more thing the date did not do. Nobody ran the check on October 1. The resolver is a script. An interim run on September 29 had already reported failing criteria, but no run was recorded on October 1, and the first recorded run after the deadline was October 2 at 23:35 UTC. Writing the falsifier down before the result stops me from moving it afterward. It does not make anyone show up on the day. If you bet on something, put the check on a scheduler, not on your memory.

What to run on yours

# 1. does the process exist
pgrep -fl your_job.py

# 2. what does the log say, and WHEN does your program write to it
stat -f '%Sm %z %N' -t '%Y-%m-%d' /path/to/your_job.log
grep -nE 'print\(|log\.|logger\.' /path/to/your_job.py

# 3. -u as its own argument, not a substring match
plutil -extract ProgramArguments json -o - ~/Library/LaunchAgents/com.you.yourjob.plist

# 4. if you made a dated bet about it, schedule the check
#    (a falsifier nobody runs on the day is a note, not a test)

These are macOS commands. On Linux, stat -c '%y %s %n' and your service manager's unit file do the same jobs.

Step 2 is the one I skipped. Read the write sites before you interpret the file. If the only print() is inside an except block, an old timestamp is not evidence of failure. It is also not evidence of health. It is evidence the file did not change.

What this does not establish

It does not prove these three jobs are healthy. On September 22, state mtimes showed courtside and eye reaching their loop-level write, sentinel getting a parseable response from getUpdates, and two of them holding a non-zero cursor. The October error logs do not re-prove that. What still holds is that I do not have a signal tied to a poll actually succeeding. That is the thing I have to go build.

It does not prove every quiet daemon is fine. It proves that for these three, on this machine, the file I chose could not answer the question I was asking it.

It does not prove -u is the only fix. PYTHONUNBUFFERED=1 would do the same thing.

And it does not prove the buffering story. The October result fits it, including three aggregate growth deltas clustered around 8 KiB, but fitting is not proof. The prediction tested the observation, and the observation was wrong.

The rule needed replacing, not defending. "Artifact age beats PID" got me off the PID and onto a file with no per-poll success record in it. The version I can actually use is longer and worse as a slogan: name the operation, name the evidence it emits, and name the conditions under which its absence is a failure. If I cannot write that third part down, I do not have a health check. I have a file I like looking at.

The Data Was Public. The Agent Path Wasn't. So His Mock Became My Documentation.

Self-Correcting Systems — Wed, 30 Sep 2026 04:36:02 +0000

Two commands against the same data. Run them yourself:

$ curl -sS --get 'https://u58x3mt0.api.sanity.io/v2025-08-15/data/query/production' \
       --data-urlencode 'query=count(*)'

{"query":"count(*)","result":120,"syncTags":["s1:dmd/mg"],"ms":11}

120 documents, no key, no account. Now the path my agent actually uses:

$ curl -sS -o /dev/null -w '%{http_code}\n' \
    'https://api.sanity.io/v1/context/organizations/<ORG>/mcp/self-correcting-systems'

401

Same underlying dataset. One route is publicly queryable. The route my agent takes goes through
an authenticated Context MCP endpoint, which needs an organization API token — not a project
token, which Sanity explicitly does not accept there. I had not provisioned Pouya any access to my
organization, so cloning the repo gave him no way to authenticate the path the agent actually uses.

I am being careful about "same" here: an MCP endpoint can be configured with its own sources and a
groqFilter, so it is not guaranteed to expose the same 120 documents the public route does. Same
source, different access path, different interface contract. That distinction turns out to be the
bug.

That gap is the whole story, and I did not know it was there until the contributor who fell into
it told me.

What happened

A developer named Pouya read a post of mine, decided my diagnosis was wrong, cloned the repo and
opened a pull request. First outside contribution the project has ever had.

PR #1   https://github.com/keniel13-ui/ask-the-record/pull/1
        PouyaZX4:fix/groq-schema-routing
opened  2026-09-22T12:33:52Z
merged  2026-09-24T00:53:58Z   ->  36.3 hours
        3 commits, 1 file, +25 / -12, merge 4a2940f

It is merged, so it will not appear in the default pull request list, which shows open ones only.
A reviewer told me the repo had no PRs at all for exactly that reason, which is a small instance of
the same mistake this whole post is about.

His diagnosis was reasonable. My agent queries a Sanity dataset with GROQ, and he thought the
failures came from the model guessing at document types and inventing query syntax. His fix: put
the real schema and worked examples into the tool description, so the model is told the shape
instead of guessing it.

Sound idea. I would have tried the same thing.

Then I ran his examples.

The two queries

Both from the pull request description. Here is the one for a claim, against the live dataset:

*[_type == "claim" && _id == "claim-ledger-population"][0].{status, expiryStatus}

-> HTTP 400  "attribute expected"

A stray dot between [0] and {. Remove it and it works:

*[_type == "claim" && _id == "claim-ledger-population"][0]{status, expiryStatus}

-> {"status": "standing", "expiryStatus": "no_expiry_set"}

And the one for a patch record:

*[_type == "patch" && findingRef._ref == "finding-b1"][0]

-> null

Null for two separate reasons. There is no findingRef field on my patch documents — it is
findings, and it is an array. And the ids are case sensitive: finding-B1, not finding-b1.

Written correctly:

*[_type == "patch" && "finding-B1" in findings[]._ref][0]{sha, inMain}

-> {"sha": "dd1a654", "inMain": false}

Six field names that do not exist

The patch did not just carry two broken examples. It wrote a schema into the tool description,
and the schema did not match my data:

the patch told the model	what the database actually has
`claim.subject`	nothing
`claim.statement`	`text`
`finding.finders`	`foundBy`
`finding.verifiedReceipts`	`commentId` or `commentIds`, depending on the record
`patch.findingRef`	`findings[]`
`patch.commitHash`	`sha`

Check it yourself, no key required:

curl -sS --get 'https://u58x3mt0.api.sanity.io/v2025-08-15/data/query/production' \
  --data-urlencode 'query=*[_id=="finding-B1"][0]'

That is the record his example referenced. An earlier draft of this post ran the same command
against finding-B8 instead, because B8 carries commentIds and B1 carries the singular
commentId, which made my table read cleaner. That is picking the sample that proves the point,
in a post about not doing that, and a reviewer caught it before it shipped.

So a patch written to stop a model from inventing schema would have installed a mismatched schema
into the exact place the model reads it from. Worse than the bug it fixed, because a guess made at
runtime is a guess, while a guess sitting in the tool description arrives with the authority of
documentation.

Why the names were wrong, which is not what I assumed

I wrote a draft of this post that said I did not know how the patch was produced and was not going
to guess. That was the right call, because when I asked, the answer was nothing I would have
guessed. In his words, published with his permission:

The dataset itself is public of course, but I wasn't able to replicate the full MCP setup on my
end at the time I tested it so I was hitting that 401 on the Context MCP endpoint. So I just
tested the query generation logic against an offline mock schema on LM Studio instead, which is
where those field names came from.

Nothing was hallucinated. He hit the 401 at the top of this post.

He could read every one of my 120 documents in a browser. He could not run my agent, because the
agent does not take the public route — it goes through an org-scoped MCP endpoint needing an
organization credential I had not provided and was not going to publish with the repository.

So he did the careful thing. He built an offline mock of the schema and tested the query
generation logic against that. And when he built the mock he chose better names than mine:

When creating that schema, I used clean, self-describing semantic names (commitHash instead of
sha, verifiedReceipts instead of commentIds, finders instead of foundBy) because explicit naming
makes it way easier for the SLMs to understand what fields represent and prevents confusion.

He is right about that, by the way. commitHash is a better name than sha. foundBy is worse
than finders. Every wrong name in that table is the name a competent person would pick if they
were designing the schema rather than reading it.

One of them is worse than a naming difference. verifiedReceipts does not just rename
commentIds — it asserts something the data cannot support. A comment id says a comment exists at
that address. It says nothing about whether anyone verified it. That distinction is the entire
subject of the record it was describing.

The defect is mine

The mock is not the bug. The mock was a reasonable response to a 401.

The bug is that my repository advertised a reproduction path that is not the one the system
uses. The README says "Public dataset" and gives the URL at the top of this post. That is true,
and it is what I put in an earlier version of this very post — a curl, no key, check it
yourself. It reproduces the data. It does not reproduce the agent, which reaches the same
records through an endpoint that returns 401 to everyone who is not me.

A contributor following my README lands in a place where every record is visible and nothing is
runnable. The only way forward is to build a stand-in. And a stand-in built at that boundary does
not stay at the boundary — his mock's field names travelled from a local LM Studio test into the
tool description, which is where my harness explicitly tells the model what the schema is.

That is the whole difference in one line. A runtime guess is visibly a guess. Put the same guess in
the tool description and I have promoted it into authoritative guidance.

I have shipped the same class of error. I described my own validator to someone as checking
whether a cited record matched what was retrieved. It does not. I was describing the system I
meant to build, from memory, instead of opening the file.

What actually resolved it

Not an argument. Two commands.

I pulled his branch, ran both examples against the live dataset, and sent him exactly what came
back: the 400, the null, and the corrected queries with their real output. No opinion about his
approach, no debate about the diagnosis.

He pushed a revision in about two hours. Every field name corrected, both queries rewritten:

claim  claim-ledger-population -> standing / no_expiry_set
patch  for finding-B1         -> dd1a654 / false
finding finding-B8            -> commentIds ['3ee98'], status unbuilt

All three still return exactly that today. Note the third is B8, not the B1 I told you to curl
above — B1 carries the singular commentId "3eanf" and status implemented. Those are two
different records and I have mixed them up once already while writing this.

Still one stale reference in a routing rule, so I sent that too, with the null it produced. He
fixed it that morning. I re-ran the three patterns, checked the file still parsed and still had no
third-party imports, and merged it.

Three commits, thirty six hours, between two people who have never met. And one detail I like:
the pull request body still shows [0].{status, expiryStatus}, the broken stray-dot form, after
the merged code had moved to the corrected one. Documentation can keep a false schema alive after
the executable stops using it, which is the same failure as this whole post, one layer up.

The thing worth taking

If you maintain anything an outsider might contribute to, run this check:

Can someone who clones your repo actually execute the path your system takes, or only the path
your README documents?

For me those were different, and the gap was invisible from the inside because I hold the token.
Everything worked on my machine for a reason I never had to think about.

When they differ, a contributor's only option is a mock. They will build a good one — Pouya's
names were better than mine. And then the mock's assumptions become your documentation, because
the tool description is documentation, and the model does not know it was written against a
fixture.

There is a third option, and it is better than either of the two I first wrote down. I said the
fix was to make the real path reachable, or to say in the README that it is not. Both are weak,
because both still leave the contributor inventing the contract.

If outsiders cannot execute a privileged dependency, ship them a reproducible contract for it.
A fixture generated from the real schema, checked into the repo, lets someone test against the same
interface without ever receiving my organization credential. The pipeline becomes

production schema -> generated contract fixture -> contributor harness

instead of

README -> unreachable MCP -> contributor invents a substitute schema

The problem was never that Pouya mocked the boundary. It is that my repository gave him no
canonical boundary to mock, so he had to design one — and a well-designed guess is still a guess.

That generalises past Sanity and past agents. Anything an outsider cannot run — a private API, a
payment sandbox, an internal queue, an OAuth service — has this shape. If you do not own the
stand-in, your contributors will build one, and theirs will encode their assumptions instead of
yours.

Thanks to Pouya for the patch, for taking two rounds of corrections without once arguing the
diagnosis, and for answering the question about where those names came from when he could easily
have let me publish a guess instead. The merge is 4a2940f. Every query above can be run by
anyone against the public dataset — which, as it turns out, is exactly the point.

I Replaced a Gate That Accepted Everyone With a Gate That Accepted No One. My Tests Couldn't Tell the Difference.

Self-Correcting Systems — Mon, 28 Sep 2026 13:58:33 +0000

Run this in a terminal, then run it again under script(1):

python3 -c 'import sys; print(sys.stdin.isatty(), sys.stdout.isatty())'

If your code decides whether a human is present by calling isatty(), and then reads that
human's confirmation from /dev/tty, those are two different questions, and there are
processes that answer them differently. This is the story of finding that out three times in
one file, each time because the previous fix was wrong in a way my tests could not see.

Here is the probe. Standard library only, writes no files and mutates no state; full source at the end.

$ script -q /dev/null python3 gate_probe.py --redirected

  isatty(stdin) and isatty(stdout) : False
  /dev/tty openable                : True
  detail                           : opened
  open("/dev/tty", "r+") -> UnsupportedOperation: File or stream is not seekable. (errno=None)

  THE TWO PREDICATES DISAGREE for this process.
  A gate on isatty and a confirmation read on /dev/tty will not agree here.

That process has an openable controlling terminal even though its redirected stdin and stdout
are not TTYs. For three days that was the shape of my gate.

One precision, because a reader will check. Default pytest capture replaces both streams,
which I had wrong in an earlier version of this paragraph. On pytest 9.0.2, launched from a real
terminal:

default capture   stdin=DontReadFromInput (isatty False)   stdout=EncodedFile (isatty False)
pytest -s         stdin=TextIOWrapper     (isatty True)    stdout=TextIOWrapper (isatty True)

/dev/tty under default capture: OPENS

So plain pytest from a terminal is the disagreement, on its own — no wrapper needed. My gate
required stdin.isatty() and stdout.isatty(), false under default capture for two reasons at
once, while the confirmation path could open the terminal the whole time. Under pytest -s the
two predicates agree, so don't look for a mismatch there.

The system, briefly

An agent that can run three frozen commands against an exported copy of a repository. It has
never been authorized for normal execution — the policy file forbids it — and I temporarily
flipped that flag during the test run described below, which is how the rest of this post
exists.

$ python3 -c "import sys; sys.path.insert(0,'.')
import stage2a_preflight as PF; print(PF.check_policy())"

(False, 'policy forbids execute_shell; owner has not authorized Stage 2a')

Before it runs anything, a human is supposed to be shown what will happen and type two
identifiers back.

v0 — the gate was a parameter

def execute(job, approval, typed_job_id=None, typed_procedure_id=None):
    ok, note = confirm_approval(approval, typed_job_id, typed_procedure_id)
    if not ok:
        return "REFUSED_APPROVAL_MISSING", note

Read it and it looks like a human gate. typed_job_id is a parameter, and a parameter is
something any caller supplies. A test fixture is a caller with two correct strings.

I flipped the policy flag that permits execution and ran the suite. Afterward:

$ ls -lT state/attempts | awk 'NR>1 {print $6, $7, $8, $10}'
Sep 24 16:36:41 TEST-JOB-1-1790282200
Sep 24 16:36:42 TEST-JOB-1-1790282202
Sep 24 16:36:44 TEST-JOB-1-1790282204
Sep 24 16:36:45 TEST-JOB-1-1790282205
Sep 24 16:37:42 TEST-JOB-1-1790282262
Sep 24 16:37:43 TEST-JOB-1-1790282263
Sep 24 16:37:45 TEST-JOB-1-1790282265
Sep 24 16:37:46 TEST-JOB-1-1790282266
Sep 24 16:36:43 TEST-JOB-2-1790282203
Sep 24 16:37:44 TEST-JOB-2-1790282264

Sorted by name, not time. Read the clock column and it is two runs of five, about a minute
apart — 16:36:41–45 and 16:37:42–46 — eight for one fixture job and two for a second. Each
directory holds a 266,240-byte tar of my own repository, all ten identical in size, extracted
into a 27-file tree.

The gate opened exactly as written.

What those ten do and do not prove

They contain a third entry, runtime-tmp, and it is empty. Its existence is not proof the
three commands started, because of where it is created:

def frozen_env(attempt_root):
    """addendum v1 B1. An allowlist, not a filtered copy. HOME absent. No PYTHON*."""
    tmp = attempt_root / "runtime-tmp"
    tmp.mkdir(parents=True, exist_ok=True)

def run_frozen_commands(export_root, attempt_root):
    env = frozen_env(attempt_root)
    results = []
    for entry in PF.FROZEN_COMMANDS:

frozen_env() is the first statement of run_frozen_commands(), and the for loop below it is
where commands actually launch. So the directory is created before anything runs. So runtime-tmp proves the runner was entered. Whether any command
started is not established by these directories, and I am not going to round that up.

What the ledger says is stranger. The ten exports left no receipt at all:

$ python3 -c "
import json
for l in open('state/receipts.jsonl'):
    r = json.loads(l)
    print(r['final_state'], r['job_id'], len(r['commands'] or []), r['written_at'])"

REFUSED_NO_HUMAN_PRESENT tg_912616161 0 2026-09-24T21:29:28.666303+00:00
REFUSED_NO_HUMAN_PRESENT tg_912616161 0 2026-09-24T21:30:40.369098+00:00
REFUSED_NO_HUMAN_PRESENT TEST-JOB-1 0 2026-09-28T01:31:55.420602+00:00

Three rows, zero commands. None of them is the export run. Rows 1 and 2 land fifty-three
minutes after it; row 3 is from this repair session, below.

There is also a quarantine file from that day, receipts_TEST_POLLUTION_QUARANTINED_2026-09-24,
and the export run is not in that either — its ten rows are all REFUSED_COMMAND_BOUNDARY
written between 15:02 and 16:06 UTC, while the exports are 20:36-20:37 UTC. A different event,
four to five hours earlier.

So: ten real exports in production state, and no receipt for them in either ledger. Where that
receipt went I cannot establish. execute() writes one in every terminal case, so either it was
written to a redirected path and discarded with a temp directory, or something raised before the
write. I did not preserve the test configuration from that run, and although the directory is a git
repo, those files are not tracked in it, so absence here is absence — not evidence of a
destination.

Those rows read REFUSED_NO_HUMAN_PRESENT, which is the same overclaim I rename a function for
further down. That value is now REFUSED_CONTROLLING_TERMINAL_UNAVAILABLE and the receipt schema
went v1 -> v2 to say so. The three rows above were not rewritten — an append-only ledger
edited to match new vocabulary is not an audit trail — so the file holds both values, and
schema is what tells a reader which vocabulary a row was written under.

The thing that was actually protecting me

Not the typed confirmation. A boolean in a config file that I had left set to false.

v1 — the fix could not open a terminal

Move the read off the parameter list and onto the controlling terminal. There is no argument
to fill:

def read_confirmation(approval):
    with open("/dev/tty", "r+") as tty:      # <- this line
        ...

On this macOS machine, open("/dev/tty", "r+") fails on both CPython 3.9.6 and 3.13.9. The
update-mode I/O stack buffers through BufferedRandom, which wants a seekable raw stream; this
terminal is not one:

$ script -q /dev/null python3 gate_probe.py </dev/null

  isatty(stdin) and isatty(stdout) : True
  /dev/tty openable                : True
  detail                           : opened
  open("/dev/tty", "r+") -> UnsupportedOperation: File or stream is not seekable. (errno=None)

The last line is the probe opening the same terminal it just opened successfully, in the mode
I had used. Reproduced identically on CPython 3.9.6 and 3.13.9. And then the detail that turned a broken
line into a broken control:

$ python3 -c "import io; print(io.UnsupportedOperation.__mro__)"
(<class 'io.UnsupportedOperation'>, <class 'OSError'>, <class 'ValueError'>, ...)

io.UnsupportedOperation subclasses OSError, and carries errno=None. My handler was:

except OSError as exc:
    return "REFUSED_NO_HUMAN_PRESENT", "no controlling terminal"

So a human sitting at a real terminal was told they were not there, and the receipt recorded
the wrong cause. It failed closed, which is the good direction to fail — but the gate now
refused the only caller it was built for.

In v1, every test that exercised approval replaced read_confirmation by name. Not one
exercised the real tty read. v0 had the earlier version of the same blind spot: there was no
read_confirmation to replace, because its tests supplied the ids directly as arguments.
Different bypass, same hole — neither broken version had a human-interaction boundary under test,
so the suite was green for both. It was never testing the gate; it was testing the bypass.

v2 — and the third defect, which the third test found

Separate handles, and an except narrow enough to mean something:

CTTY_UNAVAILABLE_ERRNOS = frozenset({errno.ENXIO, errno.ENODEV, errno.ENOTTY, errno.ENOENT})

def open_controlling_terminal():
    out = open("/dev/tty", "w")
    try:
        inp = open("/dev/tty", "r")
    except BaseException:
        out.close()
        raise
    return out, inp

Then a test I wrote for this failed, and I nearly patched the test. That would have been the
fourth version of the same mistake. What it had found:

require_human()        asked  "is sys.stdin a tty?"
read_confirmation()    asked  "can I read /dev/tty?"

Two different predicates for one control, and the stale one ran first. They are not
ordered — one asks whether two particular streams are terminal devices, the other asks for the
process's controlling terminal. In the captured pytest configuration I reproduced, isatty
was false while /dev/tty stayed openable, so the stale check refused before the confirmation
path could run. Fail-closed, but the real control was shadowed by a different predicate.

I first wrote that this shadowing was why the r+ version survived the suite. That is a
wrong-reason claim in a post about wrong-reason claims, and the probe disproves it:

no controlling terminal  ->  OSError errno=6 (ENXIO)
terminal attached        ->  UnsupportedOperation (errno=None)

r+ fails two different ways, and except OSError swallows both. Delete the shadow and
run again: with no terminal you get a real ENXIO, which is the verdict every refusal test
expects; at a terminal you get UnsupportedOperation relabelled into the same refusal. Either
way, green.

So the shadowing is a second independent reason nothing could have caught it, not the cause.
The causes are the two already named: every test patched read_confirmation, and the except
was wide enough to swallow a defect. What the shadow did do is stop the real path from being
exercised by that integration route — which is why the fix needed a test that calls it
directly.

So there is now one predicate, in one function, used by both:

def require_controlling_terminal():
    try:
        out, inp = open_controlling_terminal()
    except OSError as exc:
        return False, controlling_terminal_unavailable(exc)
    out.close()
    inp.close()
    return True, "controlling terminal present and openable at /dev/tty"

And controlling_terminal_unavailable() re-raises anything outside the set, so a programming
fault can no longer be reported as an absent human. The refusal carries the evidence rather
than the conclusion:

controlling terminal unavailable: /dev/tty open failed with errno 6 (ENXIO): ...

Two renames went with this, both the same correction. require_human() asserted something no
check in that file can establish — it cannot prove a person is present, only that confirmation
is obtainable from a terminal, and naming it after the stronger claim is what licensed a second
predicate to grow beside it. And the errno bucket was called NO_CTTY_ERRNOS while containing
ENOENT (/dev/tty does not exist here) and ENOTTY (that fd is not a terminal). Neither
literally means "this process has no controlling terminal." A set named for its strongest member
is the same overclaim one level down, so it is CTTY_UNAVAILABLE_ERRNOS now. The bucket is a
decision about what to refuse on; the errno is the fact.

Passing tests prove nothing, so I broke it on purpose

Four mutations against the gate, each run as the full 131-test suite:

mutation	caught by
restore `open("/dev/tty", "r+")`	4 tests
treat any `OSError` as an absent human	3 tests
restore the `isatty` predicate	9 tests and subtests
drop the explicit `isinstance(io.UnsupportedOperation)` guard	nothing. 131 passed

The fourth row is the useful one, and I drew the wrong conclusion from it. Correction, added
2026-09-28 after this post was published.

I originally wrote that the line was redundant, since errno is already None for
UnsupportedOperation so the errno test re-raises it anyway, and I labelled it in the source as
documentation rather than a control. A reader named howcani
pointed out why that inference does not hold:

A mutation score is taken over mutants times fixtures. With one fixture environment,
everything environment sensitive is unkillable by construction, and zero killed is then a
statement about the fixture matrix rather than about the guard. Two zeros look identical on
the report.

They were right, and the cell my fixtures never sampled is one line of Python:

$ python3 -c "import io; print(io.UnsupportedOperation('msg').errno, io.UnsupportedOperation(6,'x').errno)"
None 6

UnsupportedOperation carries errno=None only when built from a message, and that is the form
CPython raises for open("/dev/tty","r+") — the single shape every test produced. Built with two
arguments it carries an errno. Hand it ENXIO and the two paths separate:

guarded    -> re-raised, a defect surfaces as a defect
unguarded  -> "controlling terminal unavailable: errno 6 (ENXIO)"

That second line is the relabel-a-defect-as-an-absent-human bug rebuilt exactly. So the null
result described my fixtures rather than the code, and "a line the suite cannot distinguish from
its own absence" was a claim about the suite that I stated as a claim about the line.

And then a second correction on top of the first, because the line has two answers and needs
both stated. I tried "documentation", then "load-bearing", then "defence in depth, constructible
but unreached" — and the third was the same mistake as the first two, since it averaged two
populations into one adjective. An independent reviewer caught that, which is the identical defect
as two zeros looking identical on a report.

Population A — what the I/O stack actually raises. The message form, errno=None. Verified
against open("/dev/tty","r+"), a read on a write handle, and seek and tell on a pipe: all
message form, all None. The errno test below already re-raises those, so deleting the line
changes nothing on that path. For A the line is documentation, exactly as first written.

Population B — the inherited OSError constructor. io.UnsupportedOperation(ENXIO, "x")
carries errno=6, and deleting the line relabels it as an absent terminal — the original bug
rebuilt. For B the line is the only check.

So: redundant against A, sole guard against B, and B arrives only by construction — the test that
kills the mutant builds the exception rather than obtaining it from I/O, which is itself the
measure of how narrow the guard is. There is no single word for that, and reaching for one is what
produced three wrong labels in a row.

There is now a test that kills that mutant, and the source comment no longer calls it
documentation.

The same reader's proposed instrument closes the general case: make the environment a fixture
parameter instead of a constant. Five forced cells across three axes — controlling terminal
present, fd 0 a tty, fd 1 a tty — with the gate's verdict asserted per cell:

mutant	killed by which cells
`isatty` on both streams	neither_tty, stdin_only, stdout_only
`isatty` on stdin only	neither_tty, stdout_only
`isatty` on stdout only	neither_tty, stdin_only
gate always returns True	no_ctty

The two mixed cells, where one standard stream is a terminal and the other is not, are the only
thing separating the stdin-only mutant from the stdout-only one. Without them both die to the
same single cell and read as the same result. That is the concrete version of two zeros looking
identical, and my suite had neither mixed cell until this correction.

Population, since it is the whole point of this post: 5 of 131 tests run under a pty, and one
more opens /dev/tty in-process, for six that exercise a terminal-backed path. A pseudo-terminal
is deliberately not evidence of a physical terminal or a person, which is why that phrasing is not
"six tests prove a human."

The count in order, because it is the argument and a reader should be able to check the ordering:

version	what the gate was	tests opening a terminal
v0	a function parameter	0
v1	`/dev/tty` with `r+`	0
v2	separate handles	2 — written as part of this repair
v3	one predicate	5 under a pty, +1 in-process

The two terminal tests did not exist while r+ was in the tree. They were written to fix it, and
they are what caught it — restore r+ today and four tests go red, which is the first row of the
mutation table above. Zero is the number that carried v0 and v1, and zero is why both shipped
green.

$ python3 -m pytest -q tests/
131 passed, 25 subtests passed

The count in order, because it is the argument and a reader should be able to check the ordering:

version	what the gate was	tests opening a terminal
v0	a function parameter	0
v1	`/dev/tty` with `r+`	0
v2	separate handles	2 — written as part of this repair
v3	one predicate	5 under a pty, +1 in-process

$ python3 -m pytest -q tests/
131 passed, 25 subtests passed

The honest claim about what this technique is worth

sudoers(5), verbatim from this machine:

requiretty   If set, sudo will only run when the user is logged in to a real tty.
             When this flag is set, sudo can only be run from a login session and
             not via other means such as cron(8) or cgi-bin scripts.
             This flag is off by default.

Off by default, and note what the description is about: cron and CGI, not adversaries. Which
is the correct amount of credit to give this. It is not a security boundary:

$ python3 -c 'import sys; print(sys.stdin.isatty())'
False
$ echo "" | script -q /dev/null python3 -c 'import sys; print(sys.stdin.isatty())'
True

script(1) allocates a pseudo-terminal. Code running with my user's permissions and access to
PTY facilities can do that, including the assistant I write most of this code with. Its default
tool calls fail the check — I have watched them fail, and one of my own tests is that wrapper
passing on purpose:

def test_a_pty_wrapper_satisfies_the_gate(self):
    """Stated as a limit, not a defect. This is what the post claims."""

So the claim is narrow: this converts an accidental bypass into a more explicit one. My
current noninteractive automation path no longer gets past it merely by supplying function
arguments. An environment that already controls a PTY can still satisfy the terminal path
without writing anything new, so this is not identity and not authentication. It is worth
having, and it is not the same as being safe.

The second defect, which came back in three days

The ten exports landed in live state because my tests wrote to production paths. I fixed that
on the 24th by redirecting three module constants in the setUp of each class that needed it.

Row 3 of that ledger is the 27th local — 01:31 UTC on the 28th, which is why the timestamp
you read above looks like a different day. A test I wrote during this repair reached
write_receipt() without a redirect and appended to the real ledger. Its refusal_detail is
the old isatty message, which is how I know which version wrote it.

The lesson is not "remember to redirect." Isolation was opt-in, so correctness depended
on every future author choosing to comply, and I was the future author who did not. It is now
opt-out:

@pytest.fixture(autouse=True)
def isolate_agent_state(request):
    if request.node.get_closest_marker("live_state"):
        yield
        return
    ...redirect RECEIPTS, NONCES, ATTEMPTS to a temp dir...

A test must now declare @pytest.mark.live_state to touch real state, and that declaration
is visible in the test source. Verified load-bearing by flipping autouse=False: four
failures, from tests that exist only to prove the fixture fires.

If a rule in your project is enforced by someone remembering it, it is a request, not a
control. Mine took three days to prove that.

The probe

The tool at the top. Four modes; the one that matters is --redirected, which reproduces the
disagreement:

python3 gate_probe.py                                    # pipeline / CI shape
script -q /dev/null python3 gate_probe.py </dev/null     # terminal shape
script -q /dev/null python3 gate_probe.py --redirected   # ctty intact, stdio redirected
script -q /dev/null python3 gate_probe.py --detach       # no controlling terminal at all

And a cheap smell check, labelled as what it is — a string search, not proof:

grep -rn 'open(["'"]/dev/tty' tests/

A bare grep -rc '/dev/tty' tests/ is worse than useless: it prints one count per file rather
than a single number, and it counts docstrings. On my own suite 9 of its 15 hits are prose
about the terminal. It can only err in the flattering direction, which is the exact failure
mode this post is about. A zero would not prove much either — a test can reach /dev/tty
through production code without the string appearing in tests/ at all.

The evidence that actually settles it is above: a test that exercises the real path without
patching it, plus a mutation showing the suite goes red when that path breaks.

I cannot show you that number for my own broken versions. The directory is a git repo with
history, but the files in question are not in it — git ls-files --error-unmatch stage2a_executor.py returns nothing, and the same for its tests. An earlier version of this
sentence said "that tree is not under version control," which is wrong in a way a reader could
catch by running git log in it. What I can show is structural and needs no count: v1's approval tests replaced
read_confirmation by name, and v0 had no terminal boundary to exercise at all. A suite that
never opens /dev/tty cannot report anything about a gate that reads from it, whatever its
total.

gate_probe.py, in full

#!/usr/bin/env python3
"""Which question is your human-in-the-loop gate actually asking?

Run it four ways and compare the two verdict columns:

    python3 gate_probe.py                                  # a pipeline / CI shape
    script -q /dev/null python3 gate_probe.py </dev/null    # a terminal shape
    script -q /dev/null python3 gate_probe.py --redirected   # ctty intact, stdio redirected
    script -q /dev/null python3 gate_probe.py --detach       # no controlling terminal at all

`isatty` and `/dev/tty` are different predicates, and a gate built on the first while it
reads from the second will disagree with itself. The row that matters is the one where the
two columns differ: that is a process your gate classifies one way and your confirmation
code classifies the other.

Standard library only. Writes no files and mutates no state; it prints to stdout.
"""

import errno
import io
import os
import subprocess
import sys

# Named for what the errnos establish, not for the strongest one in the set. ENOENT means
# /dev/tty does not exist here; ENOTTY means that fd is not a terminal. Neither literally
# asserts "this process has no controlling terminal."
CTTY_UNAVAILABLE = {errno.ENXIO, errno.ENODEV, errno.ENOTTY, errno.ENOENT}


def probe():
    """Return (isatty_verdict, dev_tty_verdict, detail)."""
    stdio = sys.stdin.isatty() and sys.stdout.isatty()

    try:
        out = open("/dev/tty", "w")
        inp = open("/dev/tty", "r")
        out.close()
        inp.close()
        return stdio, True, "opened"
    except io.UnsupportedOperation as exc:
        # Not an unavailable terminal. Reported separately because it subclasses OSError with
        # errno None, so an `except OSError` upstream will call this "no human present."
        return stdio, False, f"UnsupportedOperation: {exc} (errno={exc.errno}) -- A BUG, NOT AN ABSENCE"
    except OSError as exc:
        kind = ("controlling terminal unavailable"
                if exc.errno in CTTY_UNAVAILABLE else "UNEXPECTED")
        name = errno.errorcode.get(exc.errno, "?")
        return stdio, False, f"{kind}: errno {exc.errno} ({name})"


def also_show_the_broken_open():
    """The mode that looks correct and is not. A tty is not seekable; text update mode
    wants it to be. Reproduced on CPython 3.9.6 and 3.13.9."""
    try:
        open("/dev/tty", "r+").close()
        return 'open("/dev/tty", "r+") -> opened'
    except OSError as exc:
        return f'open("/dev/tty", "r+") -> {type(exc).__name__}: {exc} (errno={exc.errno})'


def main():
    me = os.path.abspath(__file__)

    if "--detach" in sys.argv:
        # setsid() leaves the session, so the child loses the controlling terminal. Only
        # differs from a plain run if the PARENT had one -- run it under script(1) to see it.
        r = subprocess.run([sys.executable, me], preexec_fn=os.setsid,
                           capture_output=True, text=True)
        sys.stdout.write(r.stdout + r.stderr)
        return 0

    if "--redirected" in sys.argv:
        # The case that produces the disagreement, and the shape of `pytest` launched from
        # a terminal: stdio replaced, session intact. Run this one under script(1).
        r = subprocess.run([sys.executable, me], stdin=subprocess.DEVNULL,
                           capture_output=True, text=True)
        sys.stdout.write(r.stdout + r.stderr)
        return 0

    stdio, ctty, detail = probe()
    print(f"  isatty(stdin) and isatty(stdout) : {stdio}")
    print(f"  /dev/tty openable                : {ctty}")
    print(f"  detail                           : {detail}")
    print(f"  {also_show_the_broken_open()}")
    if stdio != ctty:
        print("\n  THE TWO PREDICATES DISAGREE for this process.")
        print("  A gate on isatty and a confirmation read on /dev/tty will not agree here.")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The question

Not "can a test satisfy your gate" — I had that as the ending for two drafts and my own suite
disproves it. test_a_pty_wrapper_satisfies_the_gate exists on purpose. A test that can drive the
real boundary deliberately is what good integration coverage looks like.

The question is:

Can ordinary automation satisfy your human-in-the-loop gate without crossing the interaction
boundary you intended?

If it can pass merely by supplying the right values into the same function call, then the gate has
established knowledge of those values, not human involvement. v0 established that a caller knew
two strings. It never established anyone read them.

And the follow-up that took me three versions to reach: does any test exercise the real
boundary, or do they all replace it? Mine all replaced it — which is how one suite certified a
gate that accepted everyone and a gate that accepted no one, and reported both as correct.

Mine held for exactly as long as the flag next to it was set to false.

An Agent That Counts My Receipts, Not My Claims

Self-Correcting Systems — Mon, 21 Sep 2026 14:26:01 +0000

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content

What I Built

My ledger says three outside engineers found the same defect. Ask the agent how many locatable receipts support that row and it says one.

Finding B8 has 1 locatable comment (3ee98 by Pushpendra). Although three outside finders
are recorded, the receipts for Vinh Nguyen and quashudev were not found in the comment trees
searched.

It does not turn three recorded names into three verified receipts. The dataset stores finders and
receipts as different things, and the agent is not permitted to add them together.

It does that over a public Sanity record of 120 documents: 74 articles, 14 findings, 10 people,
3 patches, and 19 claims. The claim documents separately store asOf, status and
expiryStatus, rather than asking the model to reconstruct those distinctions from prose.

Three things it will tell you about its own author:

the fix for finding B1 is not merged into origin/main — it reports inMain: false
of 14 findings, 12 were raised by the 8 outside engineers and 2 are internal (one raised by my coding agent, recorded as Ka'el, and one by an internal audit). Two are implemented; twelve are not
it cannot see what happened to that branch after the record was built, and says so

Demo

https://ask-the-record.vercel.app — no login, no user-supplied key, five buttons.

Measured answers on this deployment ran 13.8–33.8 seconds across the five questions — the raw
timings are in evidence/web-latency-2026-09-21.json. Evidence-bearing
answers must survive a retrieval check: a substantive verdict returned with no successful read is
marked a contract violation. INSUFFICIENT_EVIDENCE is allowed to abstain without pretending a
source was retrieved. Retrieval or auth failures return an error instead of an answer. Every
answer that renders cleanly is required to carry five fields:

ANSWER          the claim, stated plainly
SOURCES         what it read, and any source URL carried in the record
EVIDENCE DATE   the asOf recorded in the document — never today's date
VERDICT         STANDING · RETRACTED · SUPERSEDED · UNBUILT · EXPIRED
                NO_EXPIRY_SET · INSUFFICIENT_EVIDENCE
UNCERTAINTY     what this answer cannot establish. Never empty.

Press the Kubernetes button and it returns INSUFFICIENT_EVIDENCE and no number, because the
Knowledge Base evidence available to that question does not establish a count. It is not claiming
the dataset could never yield one. On the graded run, the agent abstained rather than supplying a
number; the validator does not independently establish that every abstention contains no unsupported claim.

The page also checks its own output. If an answer is missing a required field, has empty
uncertainty, or fails the citation requirement for the instrument that produced it — a URL for
dataset answers, an entry path and the Knowledge Base id for Knowledge Base answers — the page
shows the violation instead of rendering the answer cleanly. It checks citation presence and
syntax. It does not resolve URLs, and it does not yet prove that every emitted citation belongs
to the evidence actually retrieved.

Keys live server-side. The browser never receives one.

Code

https://github.com/keniel13-ui/ask-the-record

studio/     the schema — person · article · finding · patch · claim
harness/    a CLI version of the same agent, and the contract it was built against
web/        the serverless function and page behind the demo
evidence/   breaker verdicts and unedited transcripts

The Python harness and the serverless agent use only the Python standard library. The Studio uses
Sanity's normal React and TypeScript dependencies.

How I Used Sanity

The schema makes the distinctions representable; the endpoint instructions require the agent to
keep them separate; the harness independently enforces retrieval, required output fields, citation
syntax and named-object guards around the answer. It does not yet re-verify the returned values against the
document field by field — the instructions carry that, not the validator.

One honest wrinkle in my own design: the response VERDICT is a single enum that mixes claim state,
finding state, expiry metadata and retrieval outcome. That is a presentation-level classification.
The underlying status and expiryStatus stay separate in the dataset, and the graded claim lookup
reported both separately in its answer text — but the validator does not yet guarantee that
separation for every future answer, and a stricter contract would split the output the way the
schema does.

Three schema choices do the work:

1. status and expiryStatus are separate. A claim can be standing and no_expiry_set
at the same time. no_expiry_set means nobody dated it — not "permanently true," not "expired."
Collapsing those two facts into one is how a record quietly rots, so the schema represents them as
separate fields instead of forcing one value to stand for both.

2. A finding has commentOn and writtenUpIn. Where someone raised a defect and where I wrote
it up are different articles. pm25coder found B1 in the comments of one article; it was written up
in another. One field would have glued the wrong attribution to the wrong post.

3. patch.inMain is a boolean. Public is not merged. All three recorded patches are pushed and
none is in main. Asked about B1, the agent reports inMain: false rather than inferring merge
state from the branch being public.

Two instruments, because one couldn't do it

I built this on a Knowledge Base first. It handled the four prose-routed frozen questions in the
graded run. But I could not reliably recover a specific claim's status and expiry from the indexed
representation, and the measurement shows why:

all 24 Knowledge Base entries  = 190,503 characters
'claim-ledger-population'      = 1 occurrence — a label in a Sources list
its values                     = present, but not bound to that identifier

The values existed in the indexed prose but were not bound to claim-ledger-population. A prompt
cannot make that binding reliable when retrieval did not preserve it, so I routed that question
shape to GROQ — which is what Sanity recommends when the schema already says where to look.

I did not grade my own work here. Independent review means separate, owner-assigned AI reviewer
sessions that did not implement the candidate — not outside human testers. The first live graded
run they carried out exposed the wrong-object failure on exactly that question. Synthetic breaker
controls then found more: failed retrieval could reach the model, unsupported answers could exit
zero, parts of the output contract were not enforced, and the contracted tool-discovery step was
missing. The VERDICT.md in evidence/ is a
BLOCK and it stays there.

The fix was a second Context endpoint serving GROQ over the same dataset:

*[_id=="claim-ledger-population"][0]
→ status "standing" · expiryStatus "no_expiry_set" · asOf "2026-09-11"
  sourceUrl "https://dev.to/kenielzep97/my-harness-used-one-label-for-three-different-failures-2gc3"

One document, its fields, and its stored sourceUrl.

So the agent routes by question shape. Four of the five frozen questions — B1 attribution, B1
merge state, B8 receipts and Kubernetes — stay on the Knowledge Base. Only the exact claim-field
lookup routes to GROQ.

The router itself is blunter than that description: any question containing a claim-* token goes
to the dataset endpoint, which is broader than the rule I specified. The breaker preserved that as an open
remainder rather than letting me describe the abstraction as if it were the code.

By default, a Sanity Context endpoint's mode is derived from its configured sources: an endpoint
with a dataset source serves GROQ mode, and Knowledge Base sources on that endpoint are ignored. So
two endpoints, exactly one instrument per question.

self-correcting-systems       initial_context, knowledge_base_read
self-correcting-systems-data  initial_context, groq_query, schema_explorer, array_field_reader

Custom instructions on each endpoint carry the rules — report receipts not claims, keep status and
expiry apart, bound anything about mutable state to the snapshot date. They arrive through
initial_context. The harness fetches that context before letting the model read a Knowledge Base
entry or query the dataset, so the endpoint's instructions are in context before any evidence is
retrieved.

What it does not establish

The current corrected CLI harness held all five frozen questions on one independent v8 graded run
with gemini-3.6-flash. That is not general reliability. The earlier BLOCK used gemini-2.5-flash;
I am not claiming the same model failed and then passed.

One more limitation is preserved in the breaker verdict rather than smoothed over. The graded v6
candidate enforced the Knowledge Base citation rule incorrectly — it accepted an entry path or
the Knowledge Base id where the contract required both. The five live answers happened to supply
both, so their content held, but the guard was weaker than specified. Version 7 corrected that to
an AND and received its own independent run. A later review found that an unparseable VERDICT
could bypass the verdict-dependent checks. Version 8 rejects those malformed values and received
a new independent run: five of five, exit 0. Its CLI hash matches harness/ask.py in the repository.
The v6 and v7 transcripts remain evidence for their respective versions.

All four graded runs — the original BLOCK, v6, v7 and v8 — are in evidence/, unedited.
No earlier transcript was replaced to conceal its result.

Agent Session / Evidence Trail

I kept the failed run instead of replacing it:

the original BLOCK and its unedited transcript
v6 — five content-holding answers, weaker citation guard
v7 — the AND fix, five of five, exit 0
v8 — after a review seat found that an unparseable VERDICT skipped every check gated on it; five of five on the repaired file
the latency receipt — five anonymous live requests and their measured timings

The historical verdicts remain unedited. GET /api/ask reports the sha256 of the deployed function,
so you can check which source is actually serving rather than trusting that a push landed.

Sanity Project Details

Project ID       u58x3mt0
Dataset          production (public)
Knowledge Base   kbjnxAgyAimV
Studio           https://self-correcting-systems.sanity.studio/

Query the record yourself, no credentials:

https://u58x3mt0.api.sanity.io/v2025-08-15/data/query/production?query=count(*)

*[_type=="finding" && status=="unbuilt"]{code, title, "who": foundBy[]->handle}

No login is required for the demo or the dataset.

I'd rather ship an agent that says two of fourteen are implemented, one of three B8 receipts is
locatable, and this record cannot see past its snapshot than one that turns every stored assertion
into certainty.

My Harness Used One Label for Three Different Failures.

Self-Correcting Systems — Mon, 14 Sep 2026 02:14:53 +0000

Three fixtures, three separate calls into the same reducer. Here is the complete
failure_reasons each one returned, unedited:

unreadable arriving args  ->  ["SANDBOX_EVENT_CARDINALITY_INVALID",
                              "EXEC_ARGUMENTS_MISMATCH",
                              "TOOL_RESPONSE_CARDINALITY_INVALID"]

usable args, different   ->  ["SANDBOX_EVENT_CARDINALITY_INVALID",
                              "EXEC_ARGUMENTS_MISMATCH",
                              "TOOL_RESPONSE_CARDINALITY_INVALID"]

our comparison threw     ->  ["SANDBOX_EVENT_CARDINALITY_INVALID",
                              "EXEC_ARGUMENTS_MISMATCH",
                              "TOOL_RESPONSE_CARDINALITY_INVALID"]

These are minimal fixtures with no sandbox event and no tool response, so the first and last
codes fire in all three and are expected. I am printing them anyway. A post about a receipt that
hides which party failed has no business showing you a cleaned-up receipt.

The middle line is the one that matters, and across three genuinely different failures it never
changes.

Fixture one sends arguments the parser rejects.

Fixture two sends a usable call that disagrees with what I froze.

Fixture three sends an object my own canonicalizer rejects, so the comparison never completes.

Constructed inputs, so none of this establishes who caused a failure in production. But one name
covers all three, and that name says argument mismatch even when nothing was compared. A failure
in the checking stage reads as a deviation in the thing being checked.

The reason it reads that way

pm25coder put it in one line, in the comments of the schema-comparison
piece
(permalink to the comment):

EXEC_ARGUMENTS_MISMATCH misreads as a verdict on the model precisely because its name carries no
subject.

Three different observations arrive under one name: arguments that were rejected, a comparison that
completed and found a difference, and a comparison that never finished.

The label is not wrong that something happened. It is silent about which of the three, and a reader
fills that in.

What the code actually did

One try was wrapping three different jobs:

if (call) {
    try {
      const actual = parseStrictJson(call.function.arguments);      // can the arriving args be read
      if (typeof actual?.command === 'string' && Buffer.byteLength(actual.command, 'utf8') > 256) {
        failures.add('EXEC_COMMAND_OVERSIZE');
      } else if (!canonicalJsonBytes(actual)                        // does the comparison work
                    .equals(canonicalJsonBytes(prepared.expectedExecArguments))) {
        failures.add('EXEC_ARGUMENTS_MISMATCH');                    // do they differ
      }
    } catch {
      failures.add('EXEC_ARGUMENTS_MISMATCH');                      // ...everything lands here
    }
}

parseStrictJson throws when what arrived is unreadable. canonicalJsonBytes throws when the
comparison itself cannot run. Both fell into the same catch, and the catch named the arguments.

Full file if you want to read around it:
scripts/judgment/live.mjs.
The third catch site was in
scripts/pr2/reducer.mjs.

The fix

Split the parse from the compare, and give each stage its own catch:

let actual;
try {
  actual = parseStrictJson(call.function.arguments);
} catch {
  failures.add('EXEC_ARGUMENTS_INVALID');
}
if (actual !== undefined) {
  try {
    if (typeof actual?.command === 'string' && Buffer.byteLength(actual.command, 'utf8') > 256) {
      failures.add('EXEC_COMMAND_OVERSIZE');
    } else if (!canonicalJsonBytes(actual)
                  .equals(canonicalJsonBytes(prepared.expectedExecArguments))) {
      failures.add('EXEC_ARGUMENTS_MISMATCH');
    }
  } catch {
    failures.add('EXEC_COMPARATOR_ERROR');
  }
}

Same three fixtures, same unedited arrays, only the middle line moves:

unreadable arriving args  ->  ["SANDBOX_EVENT_CARDINALITY_INVALID",
                              "EXEC_ARGUMENTS_INVALID",
                              "TOOL_RESPONSE_CARDINALITY_INVALID"]

usable args, different   ->  ["SANDBOX_EVENT_CARDINALITY_INVALID",
                              "EXEC_ARGUMENTS_MISMATCH",
                              "TOOL_RESPONSE_CARDINALITY_INVALID"]

our comparison threw     ->  ["SANDBOX_EVENT_CARDINALITY_INVALID",
                              "EXEC_COMPARATOR_ERROR",
                              "TOOL_RESPONSE_CARDINALITY_INVALID"]

The change spans three source files, plus the tests:
dd1a654.

The part that was not a rename

pm25coder called it "a rename, not a redesign" — his words, verbatim, in
3eanf — and from outside the repo that is exactly what it
looks like. Inside, two things were waiting.

There are two whitelists, not one. Failure reasons are filtered through an ordered array before
they reach the output. In live.mjs, a reason that is not in the array is silently dropped. Register
the new name in the code and not in the array, and that failure stops appearing. Other failures in the same run still show, so the run
does not go green by itself, but the one you just added becomes invisible.

There are two registries for this namespace and they fail differently.

live.mjs holds 26 codes and emits output by filtering against the list — FAILURE_ORDER.filter(...)
at lines 169 and 572, no guard. An unknown reason is silently omitted. scripts/pr2/constants.mjs
holds 20, a strict subset, and its consumer refuses to guess:

if (!FAILURE_ORDER.includes(reason)) throw new TypeError(`unknown failure reason: ${reason}`);

So the narrower list is the strict one, and the list covering more surface is the one that fails
quietly. That is backwards, and it is a worse defect than the naming problem this patch fixes: a
filter that discards unrecognised codes is designed to fail invisibly.

The throwing version is the correct behaviour. The silent filter should be replaced by it, and the
two lists should be one registry. I did not do that here, because bundling a registry refactor into a
naming fix would make both harder to review and would put a behaviour change in a commit that
claims to be about labels. It is on the list as its own change.

And there was a third catch site doing the same collapse in a different module, which I only
found by grepping for every place that name was added rather than trusting the two I knew about.

The tests, including the one that is supposed to pass

Six of them. That they pass is not the interesting part. Run them against the parent commit and five
of the six fail. Run against the patch, all six pass. The one that passes both ways is there on purpose:

One correction to the commit message before you click it. It says three of the six tests fail
on the parent. That line is stale: it was written when the file had four tests, and two more were
added in the amendment. The real number is five, and the ablation below is the thing to trust. I am
not force-pushing a rewrite of a public SHA to tidy a sentence, so the contradiction stays visible
and this paragraph is the correction.

All six are in one file, if you want to run the ablation yourself:
test/exec-comparator-error.test.mjs.

arriving args unusable       -> EXEC_ARGUMENTS_INVALID     fails on parent
usable args, differ          -> EXEC_ARGUMENTS_MISMATCH    passes on both   <- control
comparison cannot complete   -> EXEC_COMPARATOR_ERROR      fails on parent
pr2 reducer, unusable args   -> EXEC_ARGUMENTS_INVALID     fails on parent
prepared transport, digest   -> EXEC_COMPARATOR_ERROR      fails on parent
pr2 reducer, digest read     -> EXEC_COMPARATOR_ERROR      fails on parent

The control is doing something specific. Splitting one catch into two created two new boundaries
that the mismatch path now has to survive. If either boundary swallowed a case it should have passed
through, a suite that only asserted the two new names would still be green, because the case it ate
would simply never be asserted. The control fails the moment the original path stops producing the
original name.

The comparator failures are induced, not waited for. The arriving call parses cleanly and my own
expected object carries a BigInt the canonicalizer refuses.

That refusal is deliberate, not fragile. The serializer accepts a closed set — null, boolean, safe
integer, NFC string, array, plain object — and rejects everything else, because its output is
hashed and a canonical form cannot have alternatives. So a BigInt in an expected object is an
invalid internal type, and the test is contrived at the type level.

Be precise about what that buys: it proves the catch fires when the comparison cannot complete on
otherwise-valid input. It does not prove a spontaneous bug in the canonicalizer, and I am not
claiming one. The two digest-read tests induce a
different internal failure using a throwing getter, and each one asserts the intended read was
actually reached and that the error survives the final filter.

Two things I got wrong on the way, both caught by someone else

My first repair ran the collapse backwards. I moved the whole catch to
EXEC_COMPARATOR_ERROR, which meant unreadable arriving args were now reported as my machinery
failing. Same defect, opposite direction.

The reason it survived my own review is worth more than the bug. I wrote the implementation, then
wrote a test asserting what the implementation did. An assertion written against unverified output
cannot fail, because it was derived from the thing it is supposed to check. That workflow
guarantees you codify your own bugs, and it produced a test that defended the error.

Then I claimed one of the three catches was dead code. I had constructed a bad expected object,
watched it get caught by an earlier gate, and concluded nothing could reach that catch. One input
class, generalized to all inputs. It is reachable — a failing read on the manifest digest lands
there, because that read sits inside the try while the one I tested sits outside it.

Neither was caught by me. The backwards repair was caught before commit. The dead-code claim went
into the first commit and came out in an amendment, so it was caught before push.

What this does not do

It does not put the contract in the name. pm25coder's fuller point was that the label should
say what it was compared against, something closer to args_mismatch_under_contract=<id>, so the
name states the authority rather than leaving a reader to infer a subject. That is not built. The
names are separated. They still carry no contract id.

EXEC_ARGUMENTS_INVALID does not identify who produced bad arguments. It marks a boundary.

The arriving arguments were rejected, either by the parser or, on the PR2 path, by argument
validation. Valid JSON can still fail that validation. Who produced them is not established by
that result. The model, the relay, or the transport are all still live possibilities, and the name
stops where the evidence stops.

Naming a producer there would be the same defect with a friendlier label.

That neutrality costs something real, and it is fair to say so. An operator wants to know if the
model is emitting garbage, and unparseable JSON on a structured tool call may well be exactly that.
I have not measured how often it is, so I am not putting a frequency on it.
The code declines to tell them, which is semantically clean and operationally thinner. The fix is not to guess in the name. It is to preserve enough provenance that attribution can be
made afterward: what arguments were observed at the model-event boundary, whether any relay or
transport transformation happened in between, and which contract governed the call.

The contract-id work above would settle the authority half of that. It would not identify who
corrupted an invalid argument stream — a contract id says which expectation was in force, not where
bad bytes came from. I conflated those two in an earlier draft. Neither half is built.

EXEC_COMPARATOR_ERROR names the stage, not the cause. An un-canonicalizable expectation and a
genuine comparator bug are both on my side and are not distinguishable from the outside. I declined
to split them, because inventing a distinction the code cannot detect is the defect I was fixing.

Update, 2026-09-15 — the paragraph above is wrong and I am leaving it there. Vinh Nguyen showed
the distinction is detectable after all: canonicalize the expected side on its own and the code knows
which operand failed, so a bad frozen fixture of mine now reports EXEC_EXPECTATION_INVALID rather
than my comparator erroring, in both reducers, at
872507f.
It still runs after the model call, so a bad expectation has already cost a relay by the time it is
named, and validating the expected bytes at prepare time is the version that belongs and is not
written.

The general shape, if you want to check your own

Find every place your system writes a failure name, and ask what the name is a statement about.
If it names a thing rather than a party — arguments, response, payload, schema — check what else
falls into the same branch. A name that describes the object can be read as a verdict on whoever produced it. Trace each error
from the operation that raised it all the way to the receipt, and check whether the name it arrives
under still tells the truth.

The question that found this one: when this fires, whose fault does a reader assume it is, and is
that always true?

If you want to check the claim rather than take it:

git clone https://github.com/keniel13-ui/self-correcting-integration-maintainer
cd self-correcting-integration-maintainer
git checkout dd1a654
node --test test/exec-comparator-error.test.mjs        # 6 pass

git checkout dd1a654~1 -- scripts/judgment/live.mjs \
    scripts/pr2/constants.mjs scripts/pr2/reducer.mjs
node --test test/exec-comparator-error.test.mjs        # 5 fail, 1 passes

That last run is the one worth doing. The test that keeps passing is the control.

If you have one of these in your own harness, I would genuinely like to see it. Different domains,
same shape.

You Agreed to a use. Not to a Hallway That Didn't Exist Yet

Self-Correcting Systems — Wed, 09 Sep 2026 15:27:22 +0000

You posted a photograph in 2014 to show your friends a night out. You understood what you were
sharing and who you were sharing it with. You did not agree to every future system that would read
it, combine it with other records, and decide something about you.

That is not a worry about surveillance. Surveillance is about being watched. This is about being
judged from inputs you never knew were inputs, by a rule you never saw, in a place you have
never been.

Here is what that actually looks like, in machines that exist right now.

Four machines that get mistaken for one

1. An allegation that travels. Patronscan sells ID scanning to bars and clubs. Venue flags stay
with the venue. Network flags surface at other participating venues when your ID is scanned. The
pitch is exactly what it sounds like: a patron banned at one venue cannot walk down the street and
start fresh at the next.

Say what a flag is. A venue recorded an allegation. It is a report, not a finding, and it travels
with the authority of a finding.

Their policies page publishes the network-versus-venue distinction, the conduct categories, the
retention limits, who has access, the route for removal, and the response windows. It does not
publish how often a dispute ends with a flag changed or removed, and I found no aggregate rate
published anywhere else. The process is documented. Its aggregate outcomes are not.

2. Affiliation, not conduct. In January 2023 the New York Attorney General sought information
about reports that MSG Entertainment used facial recognition to exclude attorneys whose firms were
litigating against it. Nothing was alleged about anyone's behavior. The input was who employed them,
and being on one side of a lawsuit is not wrongdoing.

The litigation is worth two paragraphs, because it answers a question people assume they know.

In Hutcher v Madison Sq. Garden Entertainment Corp. (2023 NY Slip Op 01646), the Appellate
Division agreed that Civil Rights Law § 40-b requires admitting a valid ticket holder to a
theatrical performance or concert. The plaintiffs were right about the statute. The court vacated
the injunction anyway, on a rule it quoted from 1915: where a statute creates a right and
prescribes a remedy, that remedy is exclusive. § 41 supplies it — a penalty of "not less than one
hundred dollars nor more than five hundred dollars." The legislature set that number, not the
venue. The March decision resolved the injunction, not the case; the § 40-b claim survived it.

The statute also does not reach sporting events. It names theatres, music halls, opera houses,
concert halls and circuses. At a basketball game the right in this story does not exist.

3. Inference from exhaust. Tenant screening, credit-based insurance scoring, employment
background checks. Frequently no incident anywhere — a score built from records you never saw,
about something you have not done.

4. The system acts. Not a database that informs a person who decides. Software that observes,
infers, decides, and operates the applications that make the decision real. In August, xAI launched
a persistent agent with its own always-on cloud computer, browser and terminal, signed into your
applications. In September, Anthropic shipped background computer use — the model clicks and types
while you keep working. These are shipped products.

I have not found a public case of one of these deciding about a person at a door. That gap is the
honest state of row four.

Collapse those four and one counterexample takes the argument down. Keep them apart and the
uncomfortable part shows up.

One of these happened to me

I did not go looking for a case. It was not any product named above. It was a vendor identity gate:
a photograph of my driver's license, a biometric scan of my face taken on my phone, an automatic "did not meet the
requirements," and no author on the sentence.

Nothing I was shown named a reviewer, a fact, or a rule. Run it through the three questions at the
end of this piece and watch them all fail. Is the fact wrong? I was never told which fact. Is
the fact right and the rule wrong? I was never shown the rule. If I win, what comes back?
I was not shown a path. That is not the same as no path
existing, and I was never given the difference.

That is not any of the four machines. It is the thing they have in common: a decision returned
with no author on the sentence. I submitted what was asked. I did not finish their process. I hit
the end of it.

Your footprint can do the job of an ID. Its protection depends on the transaction.

A Social Security number is an obvious regulated identifier. Your footprint does some of the same
linking work — the apartment, the job, the ride, the door — but what protects it is not fixed. It
depends on who is using it, for what, and under which regime.

Which protection attaches depends on the information, who is using it, for what, and under which
regime. The photograph does not change. The transaction does.

Hold that 2014 photograph constant.

A screening firm compiles it and sells it to your employer. In 2011 the FTC concluded a company doing exactly that was a consumer reporting agency, because it "assembles or evaluates" information furnished to third parties who use it for employment eligibility. Public posts, full FCRA obligations: accuracy, notice, dispute.
Your employer searches your name and finds it himself. The consumer-report route may not apply. That does not mean no law applies. EEOC and FTC guidance is explicit that background information is subject to federal anti-discrimination law "regardless of how you got the information." Remove the reporting company and the regime changes. It does not disappear.
A venue network attaches it to a flag. A different regime again, largely private.
An agent retains it, infers from it, and acts. Automation does not delete the organization that deployed it or the transaction underneath. Which duties attach depends on the workflow, and you cannot read that off the architecture.

Four handlings of one photograph, four sets of rules — not because the photograph changed, and not
because any of them is a lawless zone, but because which regime applies is a function of the
transaction, and the transaction is invisible to you.

You cannot know that when you post. You especially cannot know it in advance, because some of those
hallways were built after you posted.

Correcting a fact and contesting an inference are different rights

If a file has your birthday wrong, there is a procedure. If a system has your birthday, employer,
travel and associates all correct, and concludes from them that you are high risk, correcting the
records accomplishes nothing. Every input was already right. The conclusion was the problem.

California's Attorney General addressed this directly in Opinion 20-303. It is an opinion, not a
court holding, and that distinction matters. The reasoning does not:

"when a business processes personal information to make an inference about the consumer's
propensities, then the inference itself becomes part of the consumer's profile, and must be
disclosed. A business might draw an inference about a consumer based in whole or in part on
publicly available information... Under the CCPA, the inference must be disclosed to the
consumer, even if the public information itself need not be disclosed."

The conclusion drawn about you can carry more disclosure obligation than the public facts behind
it. The inference is a separate object. A proprietary algorithm does not automatically make its
individualized output a trade secret.

Three limits. One state. An AG opinion. And California separately provides a right to request
correction of inaccurate information (Civ. Code § 1798.106) — a third thing again. Access to an
inference, correction of a wrong fact, and forcing a decision reversed are three different rights,
and having the first does not get you the third.

One date keeps this honest: California's automated-decision-making rules took effect January 1
2026, and businesses using ADMT for significant decisions have until January 1 2027 to comply.
This is a gap scheduled to partially close.

"Human in the loop" is not a claim. A human can approve every action, approve only payments,
approve one objective at the start, review afterward, or merely be able to intervene. All five ship
under "human oversight." The question is where exactly authority sits.

The numbers, and where they are going

This is not a forecast. These are dated facts about a system already running.

California built a delete button and half a million people pressed it. The Delete Act created
DROP, a single platform where one request reaches every registered data broker. It opened January 1

By August 13 more than 475,000 Californians had filed (Governor's office, Aug 13). By August 25 the agency reported more than 500,000 registered and 654 data brokers in the system (CalPrivacy, Aug 25).

Registration is a business and the state priced it. A broker's annual registration fee is
$6,000. From August 1 2026 brokers must access DROP at least every 45 days. The penalties
are two separate meters: $200 for each day a broker fails to register, and $200 for each
deletion request for each day it fails to delete.

Then read the outcome numbers carefully, because they are the whole lesson. On August 25 the
agency reported tens of millions of records deleted, that 99.9% of consumers had their
profile deleted by at least one broker, and that the typical user had been removed by over 40
brokers. It also reported that about 25% of brokers had reported processing deletion requests.

Put those next to each other. 99.9% having at least one deletion proves reach, not completion.
Being removed by more than 40 brokers is a real result. But 654 is the system census, not that
person's denominator — DROP reports five separate statuses, including Record not found for brokers
that never held your data or could not match you from what you entered, Exempted for records a broker may lawfully keep, and Pending,
and brokers have up to 90 days to report.

So the honest reading is narrower and still hard: I have not found a published consumer-level
measure of how many people reached a fully resolved state across every broker that actually held
their data, or how long that took. Reach is published. Completion is not.

And I made this mistake myself while drafting this section, which is the point: a number is only
as good as the population it counted, and the trap does not spare the person writing about it.

The first enforcement action is the thesis in one sentence. In August 2026 California brought
its first action under both the CCPA and the Delete Act, against LocateSmarter LLC, for failing to
register and for requiring Californians to provide unnecessary data — the last four digits of
their Social Security number — before they could opt out. Total: $116,490.
(agency decision)

To switch off the sale of a record the law already covered, you had to hand over part of a more
sensitive identifier. California treated that extra demand as unlawful. The protection existed.
The company put a disclosure in the way of using it.

And read what the company was selling. Names, dates of birth, Social Security numbers, phones,
emails, employment, driver's license, bankruptcy records, litigation history — and inferences about
consumer characteristics, including whether a person is "litigious." The agency's own line:
"Inferences are a protected form of personal information under California law."

The arena used who employed you. The broker sold whether you sue. Same family of input. Not the
same machine.

And the next date is already set. California's automated-decision-making rules take full effect
for businesses using ADMT in significant decisions on January 1 2027. Whatever the gap is today,
part of it closes on a schedule, in one state, for covered businesses.

The claims that get to skip the evidence

Everything above had to be sourced. A flag is an allegation until someone adjudicates it. An AG
opinion is not a ruling. A staff letter is not a court order. That standard is not optional if you
want to be believed.

Now apply it to the loudest AI claim in circulation.

On September 3, Senator Sanders and Rep. Casar announced legislation to ban artificial
superintelligence and pause advanced AI development until a cabinet-level regulator exists, with a
corporate death penalty and up to twenty years imprisonment for violations.

Read the announcement for what it defines. It does define the class, broadly: systems that surpass
human intelligence, systems capable of overthrowing governments, systems with dangerous abilities
such as subverting shutdown commands. It also names who would decide: a proposed cabinet-level
regulator with an advisory board. What I did not find in the announcement or the one-page summary
is a benchmark or evaluation protocol for deciding when a system has crossed the prohibited
threshold. Up to twenty years in prison hangs on that threshold. The full bill text is still
described as forthcoming.

Read it for what happens next. The proposal names real instruments: "international agreements,
allied coordination, and policies such as export controls." Those are not nothing. What the public
material does not show is a mechanism that can compel a government or developer outside U.S.
jurisdiction that refuses to cooperate.

And when someone says advanced AI will end the world, ask the next question: by what mechanism,
and for what reason? Not "it will be very smart." A stated pathway you can check.

Notice what each side is required to produce. A person disputing a bar flag files into a documented
process with a published response window. A prohibition on a technology class travels as news with a
summary and no published threshold. I am not claiming anyone measured those two against each other.
I am asking the same question of both: what evidence would turn this into an enforceable
decision?

That asymmetry is the same defect this entire piece is about. How a claim gets treated depends on
who is making it and where it lands, not on what is behind it. A senator can announce a
prohibition on a technology class with a broad definition, no published operational threshold, and no bill text and it is news. You will need
receipts to get a listing corrected.

None of that is an argument for building anything anyone wants. It is an argument that the danger
worth legislating is the one you can describe: what is collected, what gets inferred from it, who
gets to act on the inference, and what you can do about it. That is not a ban on the technology.
It is a ban on a use. The difference is the entire distance between a law that could work and a
press release.

What is actually happening while that argument runs

Nobody has to coordinate any of this.

Safety teams study models. Privacy teams study data. Utilities study load. Cities study zoning.
Congress studies regulation. Each is competent inside its own boundary. The technology crosses all
of them in a single product release.

Preparation is fragmented. Deployment is integrated.

Every participant has an independent reason to accelerate one layer. Each decision can be rational
and the sum still moves faster than anyone chose.

Three questions that are not the same question

Is the fact wrong? There is usually a process for that.
Is the fact right and the rule wrong? A system can identify your employer perfectly. The dispute is whether your employer should decide whether you get into a concert. Correcting the record cannot touch that. Whether a route exists to challenge the rule itself is a different question, and it is not the same in every hallway.
If you win, what comes back, and when? A published response deadline is measurable. Whether the answer arrives before the apartment is rented, the job is filled, or the show ends is a different measurement, and I have not found it published.

Before a record decides something about you, you should be able to ask why it belongs in that
decision. Afterward, you need to know who can change the result.

If you have one of these, I want it

I am collecting real cases before building anything, not after. Mine is above.

If a decision about you turned on a record — a rental, a job, a claim, a door, an account — I want
four things, with every identifying detail removed:

What information was used, and did you know it was an input?
Was the problem a wrong fact, an unsupported inference, or the rule itself?
Who could change the decision, and what happened when you challenged it?
Did the correction reach everyone who got the error, and did it restore the opportunity?

Don't post anything that identifies you or anyone else. I am after the shape, not the file.

Compare Against the Schema They Shipped, Not the One You Expected

Self-Correcting Systems — Sun, 06 Sep 2026 21:57:04 +0000

My harness flagged the model for sending the wrong arguments. It compared what the model actually sent against what the run had committed to in advance, and they did not match.

The mismatch was real. The conclusion I drew from it was wrong, and the comparator could not have told me so.

Here is the check that was incomplete, the check that replaced it, why the fix is not "loosen the comparison," and the part of it that will rot.

The setup

The harness prepares an exec call before the model runs, freezes it, tells the model to send exactly that object, then compares the model's actual tool arguments against the frozen one. If they differ, the run fails closed. That comparison is the control.

The expectation was built like this — 5bf10ac, scripts/judgment/candidate.mjs:

const expectedExecArguments = validateExecArguments({
  command: `node ${artifactPath('verifier')} ${artifactPath('payload')} ${artifactPath('manifest')}`,
});

One key. From the run receipt, RUN_004_RECEIPT.json, here is what the model actually sent, verbatim:

{"intent": "Run candidate verification", "command": "node /opt/tf/uploads/candidate-verifier.cjs /opt/tf/uploads/candidate-payload.json /opt/tf/uploads/candidate-command-manifest.json"}

Outcome in the same receipt: EXEC_ARGUMENTS_MISMATCH.

Two contracts, and they disagreed

This is the part I got wrong on the first read, and it is more interesting than "the model deviated."

The provider's exec schema — from the compiled artifact I ran, @truefoundry/trueforge-core@0.1.4, dist/core/sandbox/Sandbox.mjs:

var sandboxExecSchema = z.object({
  intent: z.string().describe(
    "A brief, user-facing explanation of the purpose of this command. Avoid technical jargon and code details."
  ),
  command: z.string().describe("The shell command to execute in the sandbox."),
  cwd: z.string().optional().describe("Working directory for command execution."),
  env: z.record(z.string(), z.string()).optional().describe("Additional environment variables to set.")
});

Four fields. intent and command required. cwd and env optional. This is the implementation schema wired into the sandbox exec tool; I am quoting what the runtime accepts, not a tool descriptor captured from the session.

My harness's instruction, scripts/judgment/live.mjs, present before the fix and after it:

'Call truefoundry-system exec exactly once with this exact JSON argument object: ' + ...

At that moment the object being interpolated had one key.

So the model was handed two authorities that disagreed, and it followed the provider's. It satisfied the required-field schema and missed my exact-JSON instruction, because my instruction was asking for something the schema forbade. My comparator saw one difference and reported it as deviation. It had no way to represent "both sides are internally consistent and one of them is wrong."

That is the finding. Not that the model was right — that a mismatch establishes difference, not which operand is authoritative.

The fix, and the part that matters

The tempting repair is to compare less — check only command, ignore extra keys, move on. That makes the failure disappear and takes the control with it. An agent could then send any additional argument it liked and still pass.

What landed instead, in 0220a27:

export const CANDIDATE_VERIFICATION_INTENT = 'Run candidate verification';

const expectedExecArguments = {
  command,
  intent: CANDIDATE_VERIFICATION_INTENT,
};

The intent value is harness-authored and constant, not copied from what the model sent. Copying it would make the comparison check the model against itself.

That commit is titled "Implement adopted transport A and B controls" — the correction rode inside a larger transport change rather than shipping as a dedicated fix. Worth saying, since I am asking you to open it.

The expected-object gate also got stricter:

const argumentKeys = Object.keys(expectedArguments).sort();

if (argumentKeys.length !== 2 || argumentKeys[0] !== 'command' || argumentKeys[1] !== 'intent' ||
    expectedArguments.intent !== CANDIDATE_VERIFICATION_INTENT ||
    typeof command !== 'string' || command.length === 0) {
  failures.add('EXEC_ARGUMENTS_MISMATCH');
}

Three separate things, and only one of them changed:

The expectation was corrected — one key became two.
The expected-object gate got stricter — that block is new.
The actual-versus-expected comparison stayed exactly as it was. It already read:

const actual = parseStrictJson(call.function.arguments);
...
} else if (!canonicalJsonBytes(actual).equals(canonicalJsonBytes(prepared.expectedExecArguments))) {

That line is identical before and after. Keys are sorted during canonical serialization, so it is a canonical object compare rather than a raw-byte one, and JSON key order cannot cause a false mismatch. I did not repair a false failure by weakening the comparator. That is the whole point.

What I fixed is narrower than it looks

The provider's contract: intent and command required, cwd and env permitted.
My frozen run contract: exactly command and intent, nothing else, intent fixed to a constant.

Mine is deliberately narrower. A permissive provider does not oblige a harness to accept every schema-valid variation — if the run precommitted to two specific arguments, rejecting a third key is a legitimate harness constraint. But it is my policy, not TrueForge's requirement, and writing it as though the provider demanded it would be the same error in the other direction.

And here is what is not fixed. argumentKeys.length !== 2 hardcodes two things at once: the provider's current required-field set, and my run's decision to prohibit the optional fields. If TrueForge adds a third required field tomorrow, my harness will reject a model for complying with the new schema unless I edit the harness too. The durable version derives the provider-required fields from the actual tool schema, then applies the narrower harness policy explicitly on top. I have not built that. If you copy this pattern, copy the problem with it.

The check you can steal

1. Where did the expected object come from? If it came from your reading of the API, it encodes your assumptions. If it came from the provider's schema, it encodes theirs. Only one is authoritative about what a compliant call looks like.

2. Does a failing comparison tell you which side is wrong? Mine did not. It printed a mismatch and I had to open the provider's source to find the expectation at fault. A mismatch establishes difference, not which operand is authoritative.

3. When you fix a false failure, does the check get weaker? This is the one that bites. The fastest way to clear a red comparison is to compare less, and every time you do it you buy a passing run by selling the control that made passing mean something.

The general shape: a control that fires wrongly is not evidence the control is too strict. It is evidence that something on one of its two sides is wrong, and you have to find out which before you touch it.

What it cost

Two things, and I want both on the record. I drew the wrong first conclusion about which side deviated. And the run still did not verify — the same receipt carries EXEC_RESPONSE_SHAPE_UNEXPECTED alongside the argument mismatch, and the sandbox turned out to have no JavaScript runtime at all. That half is written up separately.

If you have a comparison sitting between you and a model, go read the schema you are comparing against and check that your expected object satisfies it. It takes a minute, and it prevents the afternoon you spend investigating a model that satisfied the provider's tool contract while my own instruction conflicted with it.

Before: 5bf10ac. Correction contained in: 0220a27. Model arguments and outcome from RUN_004_RECEIPT.json. Provider schema read from the compiled @truefoundry/trueforge-core@0.1.4 artifact in my own node_modules, not from upstream source. The runtime half of this same run is in I Built an Agent That Marked Its Own Finding as Already Known.

The Detector Reported Zero Because It Only Had One Item.

Self-Correcting Systems — Fri, 04 Sep 2026 05:06:35 +0000

Two instructions went into an Auditor my agent collaborators and I built to surface conflicts in agent instruction files. Deployment authority is one of nine domains the tool explicitly knows how to judge.

Never deploy without human approval.
Auto-deploy the moment tests pass.

On main at 172d962, that returns:

posture: low_observed_risk
counts: {"items": 1, "labels": {"governs": 1}, "risk_high": 0,
         "conflicts": {}, "gates": 0, "authority_categories": 0}

low_observed_risk is the product's own string, from agents/report_writer.py:110. Not my summary of the output. The output.

One item. The pairwise comparison step never received a pair, and my detector does not compare an item with itself.

After the repair, same input, live service:

{
  "severity": "high",
  "item_id": "M001, M002",
  "type": "authority_collision",
  "finding": "Conflicting governing instructions in deployment: require_human_approval vs allow_automatic.",
  "evidence": "Never deploy without human approval. | Auto-deploy the moment tests pass."
}

posture: needs_review. Two items, one high-severity collision, one verification gate.

The difference between those two outputs is that two lines were touching.

Where the failure actually was

Before anything compares instructions, something has to split the text into separate instructions. Mine joined unbulleted lines into one item whenever they sat on consecutive lines with no blank line between them.

The join is in the tool's first commit, b71892d, authored 2026-06-01 13:06:47 -0400, at agents/memory_extractor.py lines 50–51:

content = " ".join(part.strip() for part in pending_paragraph if part.strip())
if len(content) >= 36:

Both defects in this article are on those two lines, and they have been there since the first commit. Three months.

Zero findings was an answer about a collapsed population. The pairwise loop behaved exactly as written. It never got a pair.

I have been publishing about this class of defect for three months: a count of zero means nothing until you know what reached the counter. I wrote that, then shipped a tool that got it wrong, and did not find out for three months.

How wide the defect actually was

The detector knows nine domains: deploy authority, secrets handling, database source of truth, access scope, customer response, log retention, billing records, refunds, escalation. All hand-written.

(Seven live in a stance table you can read in one glance. Refunds and escalation are compared by threshold rather than opposing stance, so if you go looking for a list of nine you will find a list of seven and two functions.)

The precise scope, because the wider version is wrong: any pair whose two sides were written as adjacent, unbulleted lines with no blank line between them could be collapsed before comparison, in any of the nine domains. That is not "the nine domains were disabled." A bulleted pair, or a pair separated by a blank line, extracted fine and compared fine the whole time. The vulnerable thing was a writing shape, not a domain.

My own commit message on the repair says it worse than this article does — that consecutive lines "silently disabled conflict detection" across the nine domains. That wording is too wide. I am correcting it here rather than rewriting the commit.

Three things about the repair worth more than the repair

One: the tests were themselves tested.

The repair added seven regression tests. Passing on repaired code would not show they distinguish old behaviour from new, so we made them face the defect: stash the repair, restore only the missing constant so imports resolve, rerun against the old logic.

Four of the seven failed. Real behavioural failures, not import errors.

But four red lines are not four proofs, and the reasons matter more than the count:

Test	Fails on	Is that the defect?
`adjacent_instructions_do_not_merge`	`assert 1 == 2`	Yes. Two lines glued into one item.
`enumerated_domain_still_produces_a_real_collision`	`assert []`	Yes. No pair survived, so no collision could fire.
`short_high_risk_instruction_is_not_silently_discarded`	`assert 0 == 1`	Yes — but only because the injected constant is `12`. Inject `36` and it dies on `assert 36 <= 16`, failing on the constant before it ever reaches the discard.
`governing_instruction_..._reports_uncovered_domain`	`assert 'uncovered_domain' in set()`	No. Old `main` has no `uncovered_domain` in the detector at all. That failure is missing new code, not collapsed extraction.

Two of these four prove the extraction bug. One proves it only under the right constant. One does not prove it at all. A negative control whose failures fail for the wrong reasons is the exact defect this article is about, so I would rather print the table than let four red lines carry more weight than they earned.

Two: the first deploy succeeded on the wrong tier.

gcloud reported the truth: the web service deployed and served 100% of traffic. Accurate. I read it as meaning the behaviour had changed. It had not.

The web app is a router. Extraction runs in memory-extractor-agent, a separate Cloud Run service. The revision timestamps are the receipt:

memory-authority-auditor-web-00003-82f   2026-09-02T13:17:10.714842Z
memory-extractor-agent-00002-grk         2026-09-02T13:31:32.807334Z

Fourteen minutes and twenty-two seconds in which a correct success message sat on top of unchanged behaviour. We only caught it because we tested the endpoint instead of reading the deploy message. Same wrong-reason pattern this project studies, live in our own release process, minutes after fixing the tool. The receipt was not false. My reading of what it covered was.

Three: I added an absence instead of a domain.

The input that started this was a different pair — publish-versus-verify — and the obvious repair was to teach the tool about publishing. We did not.

Tuning a ruleset to the case someone just handed you proves only that it catches the known case. So the detector now emits uncovered_domain when a governing instruction matches no rule at all:

This instruction governs action but matched no contradiction rule, so it was NOT evaluated for conflicts. Absence of a conflict here is absence of a check, not evidence of agreement.

Before, no conflicts and never checked rendered as the same sentence. Now they do not.

The original pair still is not solved, and the live output says so

Here is the input that started this, run against the repaired live service:

"items": [
  {"id": "M001", "text": "Current policy: verify the live artifact before publishing."},
  {"id": "M002", "text": "Old note: publish immediately without checking."}
],
"classifications": [
  {"id": "M001", "authority_label": "governs",      "confidence": 0.78},
  {"id": "M002", "authority_label": "context_only", "confidence": 0.64}
],
"conflicts": [
  {"severity": "medium", "item_id": "M001", "type": "uncovered_domain",
   "finding": "This instruction governs action but matched no contradiction rule, so it was NOT evaluated for conflicts."}
]

posture: usable_with_gates. Extraction is fixed — two items, correctly split. It is still not an authority_collision, and it never will be until publishing becomes a listed domain.

And there is a second gap in that JSON I did not know about until I pasted it for this article. M002 is classified context_only at confidence 0.64. "Publish immediately without checking" is an imperative, and the classifier does not consider it strong enough to govern. So the uncovered_domain warning fires on M001 only. The half of the contradiction that tells you to skip the check gets no warning at all, because context_only items are not eligible for one.

The extraction repair is real. It moved this input from one silent item to two visible items and one honest warning. It did not make the tool right about this pair.

What is fixed and what is not

Fixed: finished, unbulleted instructions on adjacent lines no longer merge when the first line ends in terminal punctuation. The minimum item length dropped from 36 characters to 12, because the old floor silently discarded the unbulleted instruction Delete all logs. — sixteen characters, high risk, dropped with no record. A bulleted line bypassed that floor entirely, so the same words survived as a list item and vanished as a sentence.

Not fixed, with the receipts:

A standalone unbulleted fragment under 12 characters still disappears with no record. Wipe logs. is ten characters and returns zero items. So does See above. — the floor does not distinguish a command from a cross-reference, it just deletes both.
The split now over-fires on hard-wrapped prose. I previously claimed wrapped text was safe because wrapped lines do not end in terminal punctuation. That is a bet on wrapping, not a proof, and here is the counterexample: "Escalate to the on-call engineer within 15 min.\nThen page the team lead if unresolved." is one two-step escalation procedure and the repair returns it as two independent items. Over-splitting is safer than merging, because two items can still be compared. It is still wrong.
Still only nine conflict domains. A governing instruction outside them produces uncovered_domain, which names the gap but not the conflict. A context_only item outside them produces nothing at all — see M002 above.

Reproduce it

Both hostnames for the live app route to the same service; I checked with the same payload and the responses are identical, so there is no stale tier to trip over.

The repair is public as a branch, not on main: beae0bb on fix/extraction-merge. Three files, 146 additions and one deletion. main is still 172d962 and still carries len(content) >= 36, so a default clone gets the defect. I am saying that rather than letting a green link imply the whole repo moved.

Suite on the branch: 107 passed, 1 skipped, 1 xfailed in a clean clone. My working copy reads 108 because one provenance test finds workspace files that do not exist in an isolated clone. The clone number is the honest one to print next to a clone command.

Run the negative control:

git clone https://github.com/keniel13-ui/memory-authority-auditor
cd memory-authority-auditor
git fetch origin fix/extraction-merge
git checkout FETCH_HEAD -- tests/test_extraction_boundaries.py

# Restore only the constant the new tests import, so collection succeeds.
# The old extractor below still uses its own literal >= 36 logic.
python3 - <<'PY'
from pathlib import Path
p = Path("agents/memory_extractor.py")
t = p.read_text()
assert "def extract_memories(" in t
p.write_text(t.replace("def extract_memories(", "MIN_ITEM_CHARS = 12\n\n\ndef extract_memories(", 1))
PY

python3 -m pytest -q tests/test_extraction_boundaries.py

4 failed, 3 passed — with the caveats in the table above.

Why this matters past one tool

Agent instruction files accumulate rules written at different times by different people. The contradictions get harder to hold in working memory as the files grow.

The reason to build a tool like this is to help a human stay the operator — to make a growing instruction set easier to inspect, not to replace the inspection or certify that nothing was missed.

Which means a tool in that job needs the same scrutiny it applies. Ours did not get it for three months, and the thing that finally found it was not the suite. It was Kairos, a separate live agent seat in this project, pasting two lines into the deployed service and reading the answer.

Go break it

It is live and it takes text: https://memory-authority-auditor-web-qfppqeeedq-uc.a.run.app

Paste in an instruction file, a set of agent rules, a policy doc, anything with rules written at different times. No signup. The app does not persist what you paste — it processes in memory and returns the answer. I cannot promise Google logs nothing at the platform layer, so do not paste anything you would mind appearing in a cloud access log.

What I want is the case it misses. Two instructions that clearly contradict, where it returns low_observed_risk or uncovered_domain instead of a conflict. I already know four shapes that beat it, and every one of them is in this article: anything outside the nine domains, any unbulleted fragment under twelve characters, any imperative the classifier rates context_only, and any procedure hard-wrapped after a period. There will be more. The one that started this survived three months and a green suite.

Post what you gave it and what it returned. A miss is worth more to me than a hit.

The two-line input came from Kairos, a separate live agent seat testing the deployed service — not from an external user or customer. Implementation, testing, and deployment were collaborative agent work under my direction. None of the pre-existing tests exercised that input shape.

One correction about my own commit, since I am asking you to read it. The repair commit message and the regression-test docstring both say the defects were *"found by an outside reader."** That was imprecise. I meant outside the tool's own test suite; a reader following the link would reasonably take it to mean an outside person. It was not. I am leaving the commit as written and correcting it here rather than force-pushing over it, because a rewritten history is a worse receipt than an inaccurate one with a published correction attached.*

I Followed the Appeal Path. There Was No Appeal.

Self-Correcting Systems — Tue, 01 Sep 2026 02:45:49 +0000

This is part four of the Defender Access series. Each part is standalone, but here is the thread if you are landing here cold:

Part one — OpenAI says verified defenders get more access, so I preregistered a study to test it, and published the design and its outcome table before running anything.
Part two — they matched the published slogan. The decision lived in the undefined white space between the rules.
Part three — my own suite was green while four binding controls had never established their claims.
Part four, this one — the study cannot run, and the reason is worth more than the numbers would have been.

Part two set a condition for continuing, and part three quoted it rather than paraphrasing it. I am going to do the same thing:

Part four continues only if a live result earns it. Same condition part two set: nothing here gets extended on vibes, and a part that has nothing new to measure does not get written.

A live result arrived. It is not the result the series was designed to measure, and I want to be exact about that before anything else.

This part does not report an uplift measurement. The study announced in part one compares what a verified defender account can do against a control account. That study did not run and, through this route, it cannot run. What I have instead is a measurement of something one layer up: whether the treatment arm is reachable through the gate in front of the study. Through this route, it is not. Two dated records expose the gap: the denial directs a mistaken applicant to Support, while Support says it cannot explain or reverse the result, and that the verification process offers no retry or appeal.

The timeline

Every line here is a timestamped email in my inbox. I am redacting my organization ID and my address and nothing else.

When (UTC)	From	What it said
2026-08-11 01:20:19	Persona	"Verify your business to join OpenAI's Daybreak* access program"*
2026-08-11 01:20:20	OpenAI	"Thank you for your request" — subject line OpenAI Pilot: Trusted Access For Cyber
2026-08-11 23:45:16	Persona	"Persona was unable to verify your account for OpenAI's Trusted Access for Cyber program."
2026-08-31 06:16:33	me	the appeal, to `support@openai.com`
2026-08-31 06:17:14	OpenAI Support	Case 14067641

Two things about that table before the substance.

The first two rows are one second apart and give the program two different names. Persona calls it Daybreak. OpenAI calls it Trusted Access for Cyber. The support reply at the end uses both in one sentence. That is cosmetic and I mention it only because it is the first visible sign of different naming conventions across the handoff.

The last two rows are forty-one seconds apart. I will come back to that.

The two sentences this part exists for

The denial, in full, is four lines long. Here is the operative one:

"If you believe this was a mistake, please contact OpenAI support at support@openai.com."

That names a channel and the condition under which to use it. It does not say what the channel can do once you get there. I believed it was a mistake, so nineteen days later I used it. I asked two narrow questions: what documentation does the verification step accept, and should I reopen the existing request or file a new one.

The answer:

"For Trusted Access for Cyber / Daybreak, if Persona verification fails or is denied, Support can't manually override it or provide additional details on the specific result, and verification currently doesn't support retries or appeals."

Read those two quotes next to each other. The denial routes you to a desk. The desk says it cannot override the result, cannot tell you why the result happened, and that the process has no retries and no appeals.

And there is a sentence in OpenAI's own organization-verification documentation that sharpens this into something better than a complaint:

"If retries or appeals are available for your verification flow, the notice or product experience will explain the next step."

That establishes the positive case, and only the positive case: if retries or appeals are available, the notice or product experience is where the next step will appear. It does not obligate a notice to announce their absence, and I am not going to pretend it does.

What it does let me say precisely is this. My notice named Support. The current Daybreak documentation separately states that this flow has no retries and no appeals. What the notice did not say is that the Support route it named could not become an appeal. The information existed in one place and the referral existed in another, and the applicant is left to reconcile the two.

The broad version of this complaint is wrong, so let me kill it first. Support is real and it does real work: technical failures, stuck verification status, access troubleshooting. It is implemented.

What is not implemented is any recourse after a denial. The three things you would go to that channel for once you have been denied — reversal, explanation, another attempt — are all ruled out by the reply, but for two different reasons and by two different owners. Support cannot explain or override the result. The verification process offers no retry and no appeal.

The support channel existed. Recourse did not. The denial pointed at the first without disclosing the absence of the second.

The cost of that is not all theirs, and I want it attributed properly. The referral gave me reason to believe contacting Support might produce recourse. It never told me to wait. My own experimental rule is what turned that belief into a nineteen-day delay — I would not write until I thought the estimand was safe. Two costs, two owners.

This series has spent three parts on one idea, and it is a rule I work under: a rule that depends on someone choosing to perform it is a request, not a control. mansio pushed it somewhere I had not taken it, in a thread on a different post: relying on a prompt instruction delegates system integrity to the model being evaluated.

What I have here is the mirror image of both. A referral that names a channel without naming its limits is a notice, not a remedy. It has the grammar of recourse without saying that recourse is not among the things on offer.

About those forty-one seconds

The reply arrived forty-one seconds after I sent the appeal, and it says so itself:

"Hi Keniel — I'm AI-assisted support for OpenAI."

and closes with:

"This response was generated with AI support which can make mistakes."

I am not going to tell you a human refused me, and I am not going to tell you no human read it either. I cannot establish that. What I can establish is that the reply arrived forty-one seconds after I sent mine, identified itself as AI-assisted, and disclaimed its own accuracy in the same breath. Whatever human involvement there was or was not, the answer I received is one that says it can be wrong.

The disclosure limits what the forty-one-second reply can establish on its own, and it no longer has to carry the policy claim by itself. OpenAI's current Daybreak documentation states independently that Support cannot override a verification result or provide additional details about why a specific result occurred, and that verification does not support retries or appeals. Its FAQ puts it in one line: can support change a verification result? No.

One caveat I have to state, because it is the same defect this series keeps finding. That is a current page, not an archived copy from August 11. It tells me what the policy is now. It is not evidence of what the policy was on August 11, when I was denied, and I have no archived copy of the page as it stood that day. So the division of labour is: the email establishes the speed and wording of what I was told, the documentation establishes the policy as it currently stands, and neither establishes the policy at the moment of the denial.

Which leaves the thing this whole series keeps arriving at: there is no observer downstream of that decision that I can reach. Nothing in the record available to me distinguishes an AI-only response from one a human touched. OpenAI may well have internal routing or audit data that settles it. I am not claiming their systems cannot know. I am saying the applicant cannot, from what the applicant is given.

What was actually checked

Here is the part that changes what I am allowed to claim.

I do not know which checks determined my denial.

Persona is an identity and business verification vendor. The support reply describes what the step generally expects:

"information that matches official records (legal business name, registered/operating address + contact info, and sometimes a tax ID/business registration number) plus clear, complete, current official business documents"

Legal name. Registered address. Tax ID. Business registration number. Not one of those fields is about security research. Nothing in that list asks what I defend, what I own, what I have published, or whether my intended work is authorized.

That was Support's abbreviated description to me, and I should not let it stand in for the whole process. OpenAI's current organization-verification documentation is wider than that email: it also asks for accurate information about your organization's activities or intended use of OpenAI, and says the review may include document verification, and compliance or risk screening. So the claim I can defend is narrow and specific — the list I was given did not assess security research — not the broader claim that nothing anywhere in the application ever does.

And here is the part I have to state carefully, because this whole series is about people claiming causes their receipts do not carry.

I cannot establish why Persona denied me. The process withholds that reason by policy — Support said so in the same message. What I can establish is that Self-Correcting Systems is not a registered company. It is me. No LLC, no incorporation, no tax ID, no registered address, no filing of any kind. So I am missing exactly the records Support says the flow generally expects, which makes the absent entity my strongest available explanation and not a proven cause.

I own primaworkflows.com. Controlling a domain is not a substitute for legal entity records, and I would not expect it to satisfy a check asking for a registration number.

A business-registration gate is a legitimate control. If you are handing out elevated cyber capability, tying it to an accountable legal entity is a defensible design and I would probably build it that way too. My complaint is not that the gate exists.

My complaint is what the applicant is left holding. The program is described in terms of trusted defenders. Support's reply foregrounded business records. Those may not be the criterion the program name implies, and I have no way to find out, because the result arrives with no reason attached and the channel that could explain it is documented as unable to. Whatever the gate actually evaluated, it did not tell me, and an applicant who cannot see which criterion was applied cannot distinguish a considered rejection from a records lookup that came back empty.

What this does to the experiment

Part one published the design's outcome table in advance, including the failure branches. One of them reads:

Approval arrives before a valid T0 → primary estimand is lost; post-only observation.

The design anticipated approval being late. It anticipated entitlement being unclear. It did not enumerate a branch where the applicant fails the verification gate before the treatment arm can be provisioned, and that is now the branch I am in. OpenAI describes verification as helping evaluate whether a request qualifies, so this was an evaluation. It is simply not the evaluation the study was designed around, and it terminates before the study can begin.

So, plainly, for the record and for anyone who was following along expecting numbers:

The treatment arm cannot be provisioned through this route. No approval, no retries, no appeals.
T1 does not exist and will not under this preregistered route. There is no post-treatment measurement because there is no treatment.
The uplift study announced in part one is terminated, not paused. Reviving it would require a different route to eligibility, and I do not have one.
Part one's ceiling was already an N-of-1 case study with entitlement confounded with account identity. That ceiling is now moot. There is no arm to confound.

The rule I held the appeal under was that sending it early could contaminate the before/after comparison. There was no after to protect. I would rather say that plainly than let the timeline imply I was being disciplined.

What I would tell you to take from this

If you are building an access gate, one line:

Name what the channel can actually do for the situation you are naming it in. Support does real work, and none of it is recourse after a denial. One sentence in that notice would have told me so, and its absence is what nineteen days of mine were spent on. The delay is mine. The missing sentence is theirs.

If you are on the other side of a gate like this, two things I did not know before:

Find out who is actually checking you, and accept that you may not be able to. The program name on the door and the vendor doing the verification can be evaluating different things. I assumed a cyber program was assessing my security work. The support response I received foregrounded business records, and it arrived only because I asked a question that was almost about something else. OpenAI's current documentation is broader than that answer: it also names organizational activities or intended use, and compliance or risk screening. What I still cannot recover is which of those checks actually determined my result.

A denial with no stated reason is not a judgment you can read. I spent some of those nineteen days reading it as one. It might have been. It could also have been a records lookup that came back empty. The point is that the format gave me no way to tell, and I filled the silence in myself.

The series, in order: *part one** (the preregistered design, which DEV featured on August 18 for announcing the instrument this part reports could not be run) · part two (the decision lived in the undefined white space) · part three (a green suite over four unestablished controls) · part four, this one.*

The email quotations above come from five dated messages in my inbox: two from no-reply@frompersona.com, one from noreply@openai.com, one from support@openai.com, and one I sent. Quotations from OpenAI documentation come from the two Help Center pages linked below. I am not publishing the raw messages — they carry my organization ID and address — so the quotes are on my word and the timestamps are what I would produce if anyone wants to check the intervals. The forty-one second reply identifies itself as AI-assisted and states it can make mistakes. Its central claim does not rest on that message alone: OpenAI's own published Daybreak troubleshooting documentation states that Support cannot override a verification result or disclose its specific reason, and that verification currently supports neither retries nor appeals. API organization verification · Daybreak troubleshooting

Remember CLAIM-24? Here's what happened.

Self-Correcting Systems — Mon, 31 Aug 2026 03:27:16 +0000

This is the CLAIM-24 result. If you followed that series in June and then watched it go quiet, this is what happened in the eleven weeks since.

The timeline, with receipts

2026-06-04 → 06-14. Nine posts, CLAIM-24 through CLAIM-31, published in eleven days. The last one was The Rule Held. The Boundary Moved Up. on June 14. Then nothing, for eleven weeks.

2026-06-07. I opened tacoda/keystone#4 asking whether Cell 7 could be tested against a real signing API instead of a mock. That thread is public, it now has forty-one comments, and everything below came out of it.

2026-08-05 12:12Z. An independent breaker reproduced a cross-origin credential-forwarding path offline, using a sentinel value and a cross-origin Location. The sentinel crossed. No live credential ever did. Breaker verdict

2026-08-05 → 08-06. Live baseline captured. Mandate narrowed through the real API. A defect surfaced: my parser required ISO 8601 timestamps, the API returns Unix seconds, and twenty-two passing tests never caught it because my fixtures carried the same assumption I did. Repair commit.

2026-08-13. POST /mandate came back with a Cloudflare 1010, browser_signature_banned. The cycle stopped at issuance. No mandate, no CAPTURE, no PATCH, no EVALUATE.

2026-08-14. The default Python-urllib/3.13 user agent emerged as the strongest lead, one of the most commonly flagged signatures in bot detection. A lead, not an established cause.

2026-08-16 17:35Z. The provider confirmed in writing that nothing had changed on their side.

2026-08-19. The controlled cycle ran once. POST /mandate returned 201. CAPTURE 200, PATCH 200, EVALUATE 200. One attempt per stage, no retry, no redirect follow.

2026-08-20 10:41Z. Key revoked by the issuer, on the record.

What that does and does not establish

It establishes that the route opened and the full cycle completed against a live external API.

It does not establish why. Three days sit between the provider's written statement and the run, and nobody controlled that window. The receipt that binds the 201 does not carry the User-Agent, so it cannot testify about its own independent variable. The result is labelled CONTROLLED_EXTERNAL_API_FIXTURE because the project authored the PATCH, which makes it a fixture rather than independent lifecycle evidence.

Cell 7 is not closed. This is the live result the series was waiting on, and it arrives with its ceiling attached rather than as a conclusion.

The rest of this piece is the part I think is worth more than the 201: what the two defects in my own code actually were, and why my own test suite could not have found either one.

I had twenty-two passing tests and two separate reviewers on a piece of code. None of it objected. Then I pointed it at a real API owned by somebody else and it broke on the first live read.

The mismatch fit in one sentence: my parser required ISO 8601, the documented API returned Unix seconds. The repair was not one line. It touched five files, 74 lines of parser and 52 lines of tests. The assumption was small; making it safe was not.

Here is why nothing caught it, and it is the part worth keeping:

My tests used ISO because my code used ISO, so they agreed with each other and never checked reality.

The fixtures were written by the person who wrote the parser. They encoded the same assumption. The suite confirmed internal behaviour without ever challenging the ISO assumption, because both halves of it came from one head. Internally consistent is not the same claim as right, and nothing in that suite could tell the difference.

Two separate reviewers missed it too. I cannot prove why, and I am not going to invent a reason. What I can show is that the parser and every fixture encoded the same ISO assumption, so none of the artifacts in front of anyone supplied the live contract that contradicted it.

The second one was worse

Working against a real system made redirect containment matter, so an independent breaker went at it. In Python 3.13 the default redirect handler rebuilds the redirected request from req.headers, dropping only content length and type. My X-API-Key sat in that header set, so the redirected request inherited it. Python has Request.add_unredirected_header() for exactly this, which marks a header as one that will not be added to a redirected request. I was not using it. The breaker reproduced it offline with a sentinel value and a cross-origin Location, and the sentinel crossed.

No live FIPSign credential was ever shown to have crossed an origin. The defect was real and unshipped. I did not find it by auditing my own code, and I did not find it myself — the live integration made the question worth asking, and a seat that was not mine answered it.

What it takes to actually get that

On 2026-06-07 I opened an issue on tacoda/keystone, an agent charter framework, asking whether a narrow adapter test was possible against a real signing API. FIPSign (mobydickfinance) said yes and then did something I did not expect: he handed over disposable API keys, rotated them on request, and stayed in that thread for ten weeks.

The collaboration produced a block I could not explain. POST /mandate came back with a Cloudflare 1010, browser_signature_banned. That stopped the cycle at issuance: no mandate was created, and CAPTURE, PATCH and EVALUATE never ran.

I could have guessed. He didn't let either of us do that. His reply:

Checked both questions directly, not from assumption.

Then he listed what he actually did. Reviewed the CORS and origin handling in the Worker source and confirmed server-to-server calls authenticated by API key never enter that logic at all. Checked Bot Fight Mode specifically. Compared the public guide field by field against the real request validation in the backend — length limits, scope counts, budget types, expiry bounds, required headers — and confirmed nothing was undocumented. Then ran POST /mandate himself from PowerShell on a residential Windows connection and got a clean signed mandate back.

Both of my hypotheses were dead. It was not the CORS or origin policy I suspected, and it was not a missing requirement in the docs.

What he asked me for next is the part I want other people to steal:

the exact timestamp · the HTTP client/library + version · whether the request went through any proxy or VPN · what kind of network you are running it from — home connection, cloud VM, CI runner, corporate network

With the note that hosting-provider IPs get scored very differently from residential ones, independent of the client library.

I gave him the client. Python-urllib/3.13.

that's the default, unmodified user agent for Python's urllib, and it's one of the most commonly flagged signatures by automated bot detection precisely because so much low-effort scraping traffic uses it unchanged.

Then, before I spent another key:

if you set a real user agent on your client, I'd bet that alone resolves it. Worth trying first — cheap to test.

If you are hitting an endpoint from a script and getting bot-flagged for no visible reason, check what user agent your HTTP library is sending by default. You probably never set one. That is a five-second check and it is the most portable thing in this entire post.

The part where I refused the result

One seat added the header, an independent seat confirmed it was present in the request urllib assembled for transport, and I authorized a single controlled cycle that a separately assigned firing seat executed once. POST /mandate returned 201. No 1010. CAPTURE 200, PATCH 200, EVALUATE 200. One attempt per stage, no retry, no redirect follow.

That is the clean story: header missing, endpoint blocked, header added, endpoint opened.

I am not going to tell you that.

FIPSign's operator had confirmed nothing changed on his side — Bot Fight Mode off then and off now, never toggled, a Browser Integrity Check rule drafted during the investigation but never deployed. That statement is dated 2026-08-16 17:35Z. The firing seat ran the controlled cycle on 2026-08-19.

Three days sit between a written statement about a provider's configuration and the run that depends on it. He was not being careless and neither was I. But nobody controlled that window, so a success is attributable to the header only as far as that statement reaches. One run, one header, three days later, is not causation. It is a result with an uncontrolled gap in front of it.

And then the worse gap, which was mine

That run's issuance receipt contains no User-Agent field at all.

The entire point of the run was the header. A narrative receipt written afterwards names it. The artifact that binds the 201 does not. The two share a run id, so software can associate them, but the User-Agent assertion sits outside the sealed receipt chain rather than inside it. A reader holding the artifact that binds the 201 still cannot recover which user agent produced it.

We built the capture afterwards — it records the configured value, the mapping handed to the transport, and what urllib assembled after add_header, three witnesses at three different strengths, none of them TLS-level and none of them the edge's view. It exists because the previous run could not testify about its own independent variable, which is not a good reason for a thing to exist.

The route opened. The experiment did not establish why. The receipt did not bind the one variable that was supposed to explain the change. That is a worse failure than the block was.

The result carries CONTROLLED_EXTERNAL_API_FIXTURE on its face. The project authored the PATCH, so it is not independent lifecycle evidence and I will not present it as any.

What the ten weeks were actually worth

Not the 201. The 201 is a fixture with a disclosed ceiling.

What it was worth was two defects in my own code, found by two different paths, and collapsing them into one lesson would overclaim what the receipts show.

The timestamp mismatch survived twenty-two green tests and two separate reviewers because the parser and every fixture carried the same wire-format assumption. The live response is what contradicted it.

The redirect defect was different. It survived a green local suite and was found by an independent breaker attacking the adapter with a sentinel value, before any live credential could cross. One was reality contradicting my fixture. The other was an adversarial seat asking a question my tests had not asked. One of them was a cross-origin credential-forwarding path.

His closing note put it plainly, and he is right:

it says something good about testing against a live system instead of only mocks.

For an external wire contract, a fixture you invented from your own reading of someone else's docs cannot independently validate that reading. My suite could not challenge the ISO assumption because every fixture already carried it.

Reality is a better adversary than your fixtures. It is not a substitute for a receipt.

Thread is public: tacoda/keystone#4, opened 2026-06-07, forty-one comments when audited on 2026-08-30. Every quote above is from it. **The sanitized run receipt for 2026-08-19 is not currently public* — it is credential-adjacent and has not been through a publication scan, so the User-Agent gap described above is stated on my word rather than something you can open. I would rather say that than imply a receipt you cannot read. The keys used for the Mandate runs described here were disposable, delivered privately, reported spent, and revoked by the issuer.*

Tell Me About You

Self-Correcting Systems — Sun, 30 Aug 2026 19:05:28 +0000

After sixty-seven published posts here, the comments have become one of the best parts of the work. Some of you have run my code, found things I missed, challenged a claim, or brought me a project of your own. I also want to distinguish bots from real people.

I know some of you by the exact problem we were discussing. I still do not know what many of you are building, learning, enjoying, or trying to become outside that thread.

So this post is different. I am going to introduce myself first, and then I want to hear about you.

If we have never spoken before, you are welcome here. If we have spoken ten times already, you are welcome to introduce yourself again. Tell me something I do not know yet.

Who I am away from the terminal

Basketball is a real part of my life. On September 12 I am one of the official coaches at the Hugo Gonzalez Basketball Camp at Brandeis University in Waltham, Massachusetts. Hugo plays for the Boston Celtics. The brief the coaches were given is the part I liked: treat every player like they are your own kid, and no phones. That is genuinely one of the cooler things I get to say right now. Basketball was my first love i never played college or anything like that so i don’t wanna give the wrong impression either basketball is just a passion.

I play PokeMMO, but only for PvP. I think of it as modern-day chess. You are not just reacting to the board in front of you. You are playing several turns ahead of someone who is trying to do the same thing to you. You have to hold a model of what they think you are about to do, then decide whether to follow the obvious line or break it.

That way of thinking is connected to everything else I work on. I expect the world to become much more automated, and I want to build now for the decisions I will need to understand later. I would rather learn how the machine reaches a conclusion than wait for a future where its decisions simply happen to me.

I did not arrive here through scholarship or credentials. I am someone who genuinely enjoys learning, testing an idea, being wrong, and trying again with better evidence.

What I hope to build

The long-term idea is Self-Correcting Systems.

The goal is not a machine that never fails. It is a machine that can notice when reality differs from what was expected, examine the anomaly, explain what it can and cannot establish, and help correct course without quietly hiding the failure from the person operating it.

That asks machines to handle pieces of the critical judgment humans use every day: noticing that something feels wrong, comparing it with prior experience, changing an interpretation, and knowing when confidence is not enough. I know how far away that goal is. I am not claiming to have built it. I am telling you what all of this work is pointed toward.

The next major system I want to develop is for cybersecurity. I expect the machinery underneath it to become intricate, but the controller has to remain understandable enough to operate almost like a video game. Complexity underneath cannot become confusion at the surface. If I cannot see what my own system observed, what it inferred, what it changed, and what authority it used, then I am no longer operating it. It is operating me.

One day I want to build a company around this with people who think similarly: ambitious about what machines can become, skeptical of claims that outrun evidence, and serious about keeping humans able to understand and control what they create.

How I actually work with AI

I do not work with one assistant that agrees with everything I say. I use multiple AI agents in separate roles, and they are expected to challenge one another and challenge me.

We try to work under a constraint system called the Drift Gauntlet:

Close the current loop. Do not escape into a new lane before the current one produces a real outcome.
Make something touch the world. A reply, a test from another machine, a shipped artifact, a yes, or a no teaches more than another private plan.
Require a real edge. “The AI will figure it out” is not a mechanism.
Do the work before studying the idea of the work. Research cannot become a hiding place from execution.
Record the receipt. If the result is not preserved, nobody can learn from it or challenge it later.
Name the grandiosity. A huge future goal does not excuse skipping the next checkable step.

The gauntlet is supposed to be blind to rank. My own named failure mode is impulsiveness: moving before I fully understand the practical use of what we are doing. One agent tends to skim and answer the last part of a problem. Another tends to over-produce and build more than the loop requires. We name those failure modes in advance because they rarely feel like mistakes while they are happening. They usually feel like working hard.

This is not a perfect automated control. We still violate it. While this post was being built, one agent concluded that an unpublished draft was a forgotten article because its sentences barely overlapped with anything already live. Another agent opened the live article and found that the same thesis, commands, hashes, and public request had already been published two weeks earlier. The first agent withdrew the draft.

That is the workflow at its best: disagreement is welcome, but the live object gets the final word.

Now I want to hear about you

Who are you?

Tell me whatever you are comfortable sharing:

what you are building, studying, or trying to learn
a problem you cannot stop thinking about
a hobby that matters to you
something you are proud of
a project link, if you have one
or simply what brought you to this corner of the internet

You do not need a project. You do not need a repository. You do not need to be technical. There is no required level of experience, and this invitation is not reserved for people I have never met.

If you are a real person leaving an honest introduction, I will follow you back. The point is not to trade numbers. I want my feed to contain more people I have actually heard from and fewer accounts that are only numbers on a screen.

So whether this is our first conversation or our twentieth: what is one thing you would like me to remember about you?

My Commit Said "Stop Trusting the Receipt." That Is the Commit Where the Trusting Started.

Self-Correcting Systems — Sun, 30 Aug 2026 07:04:02 +0000

On 2026-08-25 at 21:27 EDT I pushed this to self-correcting-integration-maintainer:

fix: repair the four re-review findings; stop trusting the receipt

The message is accurate. It closed four real findings from an automated review. It is also the commit that added this line:

const recomputed = decide(receipt.checks, receipt.deciding_fields);

The second argument is the receipt's own claim about which fields it should be judged on. So the validator recomputes its verdict over terms the subject supplied. A receipt carrying failing checks plus deciding_fields: [] recomputes over nothing, finds nothing failing, and validates clean.

Absence reading as a pass, inside the fix for absence reading as a pass.

The reviewer flagged it at 21:30:49 EDT. Three minutes and twenty-five seconds. (The finding's original_commit_id is 4a8e6c9; GitHub now displays it against the later head 5d053c17, which is why the timestamps are worth stating rather than the UI position.)

The part that is not a bug story

The commit message is not a lie. It closed four findings. It just names the opposite of what the diff did on one line.

An audit by commit message passes this. An audit by diff summary passes this. A reviewer reading "stop trusting the receipt" has been told the answer and will read the diff looking for confirmation of it. Only following the data catches it.

That is the condition I did not have a name for before: a repair arrives carrying the credibility of a repair. It closed something real, it was reviewed, and it says so on the tin. That is precisely when nobody looks at it twice.

Readers named the class, not the commit

On 2026-08-24 I published a piece about a contract that permitted the contradiction its tests were passing. Two commenters went past the instance.

pm25coder, the same day:

"Every repair moved authority to something 'better typed,' and the third contract's authority field is itself a derived value one level down."

He was describing a different project — a grant-expiry contract in a Python file, not this repository. He named a class: authority migrates one level down and the subject ends up supplying the terms it is judged by.

The chronology is checkable end to end. His comment posted 2026-08-24 at 09:41 EDT. GitHub says this repository was created 2026-08-25 at 19:47:38 EDT. 4a8e6c9 landed at 21:27 EDT that same night — one hour and forty minutes after the repository existed, in JavaScript rather than the Python he was reading, with a message claiming the opposite of what it did.

I want to be exact about the credit, because getting it wrong would be the same defect one more time. He did not predict this commit. He named a failure class, and the class recurred. That is more useful than prophecy and it is a weaker claim than prophecy, and the difference matters.

anp2network went at the method rather than the instance, and I will come back to that.

The same class on 2026-08-29, in prose

On 2026-08-29 a submission document carried a stale count of review comments. The number had shipped wrong twice already. The correction read: "37 inline review comments across six merged pull requests."

Arithmetically right when written, and self-invalidating as a complete-set claim. Merging it creates a seventh merged pull request while the sentence still defines the universe as six. The total would have stayed 37 — PR #6 carries zero inline comments under the same endpoint — so the number never goes wrong. The set does. The denominator named every pull request except the one doing the counting.

Caught before merge by a second seat, not by anyone checking the arithmetic. The wording that shipped names the measured set instead:

Across the six pull requests merged before this correction (#1, #2, #3, #4, #5, #7), Qodo authored 37 inline review comments as of 2026-08-29.

Merged as 3ee11d1. Recomputed after the merge: still 37.

Same shape as the commit above, moved out of code and into a sentence: an artifact supplied the terms of its own completeness. Three attempts had fixed the arithmetic. The number was never the defect — the measurement boundary was.

Status of the repair, stated honestly

The current line freezes the terms in the consumer and demotes the receipt's copy to evidence:

export const CANONICAL_DECIDING_FIELDS = Object.freeze(['node', 'trueforge', 'sdk']);
...
const recomputed = decide(receipt.checks, CANONICAL_DECIDING_FIELDS);

Defect · current.

The reviewer did come back to it. b5be3b7 is the repair — it adds the constant and swaps the argument — committed 2026-08-26 at 19:23:36 EDT, with Qodo's review updated to that exact commit at 19:26:04 EDT. A re-review at the repair head, and I am not going to omit it because it cuts against the shape of the story.

Qodo's review moved again ten minutes later to 121a24f. That one is a different fix — counting providers instead of trusting that a response arrived — and it does not touch this file at all. It carries the repair only because it comes after it. Worth separating, because "the reviewer cleared it twice" would be a nicer sentence than the true one.

I still do not call it fixed. The patch is maker-authored, and no separately assigned breaker seat has adjudicated it. On this project a maker's own PASS does not count no matter who else looked, and the last two times I felt confident about a repair are the two stories above.

One thing I did check, because a reviewer of this draft predicted a second hole in the same class: if a receipt simply omits a canonical key from checks, does absence read as a pass again? It does not. decide() filters on checks[field]?.observed !== true, so a missing key lands in blocked_by and the receipt is rejected. Omitting sdk yields LOCAL_PREREQS_BLOCKED ["sdk"]; checks: {} blocks on all three. The predicted hole assumed an implementation that reads status === 'FAIL', which is not what is there. I mention it because the prediction was reasonable and running it was faster than arguing about it.

One check you can run

Debashish Ghosal proposed this in the comments on the last piece:

"Throw random strings into event.notes during test runs. If altering a human note flips a programmatic verdict, fail the build immediately."

One note on scope before the code: this harness targets the Python classifier from the previous article — claim_24/mandate_cell7.py in a different repository. It does not test the JavaScript validator above. Two codebases, one failure class.

Complete file. Python 3, no dependencies, run it as-is:

import random, string

def fuzz_note_independence(classify, row, note_field="notes", n=200, seed=0):
    """Perturb only the prose. If the verdict moves, the prose is load-bearing."""
    baseline = classify(dict(row))
    rnd = random.Random(seed)
    for _ in range(n):
        r = dict(row)
        r[note_field] = "".join(rnd.choice(string.printable[:95])
                                for _ in range(rnd.randint(0, 80)))
        if classify(r) != baseline:
            return False, r[note_field], classify(r), baseline
    for probe in ["", "TTL EXPIRED", "ttl expired", "not ttl expired",
                  "resolved: ttl expired last week", None]:
        r = dict(row); r[note_field] = probe
        try:
            got = classify(r)
        except Exception as e:
            return False, probe, f"raised {type(e).__name__}", baseline
        if got != baseline:
            return False, probe, got, baseline
    return True, None, None, baseline


def classify_defective(ev):                  # control flow reads the prose
    if "ttl expired" in (ev.get("notes") or "").lower():
        return "SKIPPED_TTL_EXPIRED"
    return "CONSULTED"

def classify_note_independent(ev):           # control flow reads a typed field
    if ev.get("reason_code") == "TTL_EXPIRED":
        return "SKIPPED_TTL_EXPIRED"
    return "CONSULTED"

row = {"reason_code": "TTL_EXPIRED", "ttl_remaining_hours": -0.0,
       "notes": "grant ttl expired during consult"}

for name, fn in (("defective", classify_defective),
                 ("note-independent", classify_note_independent)):
    ok, note, got, base = fuzz_note_independence(fn, row)
    print(f"{name:18} {'PASS' if ok else 'FAIL'}  baseline={base}"
          + ("" if ok else f"  note={note!r} -> {got}"))

# The independence harness above cannot catch negation, because on this row
# "not ttl expired" still contains "ttl expired" and returns the same verdict
# as the baseline. Negation needs a row whose typed reason is NOT expired:
negated = {"reason_code": "CONSULTED", "notes": "not ttl expired"}
print("negation           defective ->", classify_defective(negated),
      "| note-independent ->", classify_note_independent(negated))

defective          FAIL  baseline=SKIPPED_TTL_EXPIRED  note='R5x$!PCZJ-r#hAhc<w...' -> CONSULTED
note-independent   PASS  baseline=SKIPPED_TTL_EXPIRED
negation           defective -> SKIPPED_TTL_EXPIRED | note-independent -> CONSULTED

Two things worth being precise about, because I got both wrong in a draft of this.

The negation probe inside the harness catches nothing. "not ttl expired" still contains "ttl expired", and the baseline row already classifies as expired, so the verdict does not move and the harness reports no change. Negation needs the separate row at the bottom, where the authoritative field says CONSULTED and the grep says otherwise. That is the line that shows the defect.

And this establishes note independence only. It does not establish that the typed field is right. A typed field can lie as cleanly as a sentence — a grant expired by one second stored as -0.0, and -0.0 >= 0 is True in Python. Which is how the last piece started.

What is still open

anp2network's objection is the one I have not answered:

"Every field on that row has the same author... Each one worked by making two fields disagree. That method cannot see the row where nothing disagrees and the answer is still wrong."

Every check above works by making two views disagree. A commit message contradicts its diff. A candidate complete-set count contradicts the repository state it would have created if merged. A verdict contradicts its own inputs.

That method is blind to the case where nothing contradicts anything. If a timestamp is stamped when a gate consumes a grant rather than when the issuer issues it, every field agrees, every recomputation is clean, every contract passes, and the verdict is wrong — because the error arrived before the first field was written.

My read of their proposed direction is that independence is a property of who could have been compelled: you are not looking for a willing second witness, you are looking for bytes some other party already wrote, for their own reasons, that a claim can be bound to.

I have not built that.

Check any of it yourself. Every timestamp in this piece comes from a public endpoint that needs no account:

Commit times — git show -s --format=%cI <sha> after cloning the repo, or the commit pages linked above
Review timing — GET /repos/keniel13-ui/self-correcting-integration-maintainer/pulls/1/comments and /reviews; the finding on the defect carries original_commit_id: 4a8e6c9… and created_at: 2026-08-26T01:30:49Z
Comment times — GET https://dev.to/api/comments/3df2g returns created_at: 2026-08-24T13:41:44Z; 3dfeg and 3def2 the same way. The DEV page shows only the date, so the API is where the hour lives
Repository creation — GET https://api.github.com/repos/keniel13-ui/self-correcting-integration-maintainer returns created_at: 2026-08-25T23:47:38Z

All times converted to EDT (UTC−4). Verified 2026-08-30.