<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Rain S</title>
    <description>The latest articles on DEV Community by Rain S (@rain6fish).</description>
    <link>https://dev.to/rain6fish</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4130044%2F46f9d682-ed36-45af-abd6-6b85ea30d06f.jpg</url>
      <title>DEV Community: Rain S</title>
      <link>https://dev.to/rain6fish</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/rain6fish"/>
    <language>en</language>
    <item>
      <title>AI Execution Governance, Part 13: The Key Protects the Call, Not the Money</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Thu, 19 Nov 2026 00:34:24 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-execution-governance-part-13-the-key-protects-the-call-not-the-money-124a</link>
      <guid>https://dev.to/rain6fish/ai-execution-governance-part-13-the-key-protects-the-call-not-the-money-124a</guid>
      <description>&lt;p&gt;We build a runtime that lets AI agents take real write actions inside enterprise systems: create a record, send a payment. Every action carries permissions, approvals and a record someone can review afterwards. Before anything goes out, we take a local hold whose name is the idempotency key — the same call arriving twice is refused by it.&lt;/p&gt;

&lt;p&gt;This piece is about what that hold actually protects, and about the place a reader found where it protects nothing.&lt;/p&gt;

&lt;p&gt;A $100 payment times out. The request may have reached the bank, or it may not have. The claim sits at &lt;code&gt;claimed&lt;/code&gt;, outcome unknown. An operator watches it for a while, decides to stop handling it, and closes the case.&lt;/p&gt;

&lt;p&gt;What happens next is less than most people expect: the disposition changes, the status stays unknown, the key stays occupied, and nothing moves at the target.&lt;/p&gt;

&lt;p&gt;The reader's question: if closing the case frees the $100 reservation, a second payment with a different key could spend it before the first one settles.&lt;/p&gt;

&lt;p&gt;The premise does not hold — we have nothing to free, and no notion of a reservation. But the second half of what he wrote is right, and I had never written it down.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Three spaces, and the key names the innermost
&lt;/h2&gt;

&lt;p&gt;"Don't pay twice" can be named three ways here:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;One call&lt;/strong&gt;: this set of arguments, this once, inside one conversation. That is the idempotency key, and the conversation is part of it (&lt;code&gt;sha256(user:conversation:tool:args)&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One operation&lt;/strong&gt;: the same user, the same tool, the same canonicalised arguments, from any conversation. We added this later — the conversation sits inside the key, so a different conversation walks around the first layer.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The resource&lt;/strong&gt;: $100. There is one of it, however many names exist for the call.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The layers run from inside out, and the key names only the innermost. Most of the time the three coincide, which is why the difference is invisible: one call, one operation, one payment. They come apart exactly where money gets lost.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1eb6jwfdxg7kfkg2893q.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1eb6jwfdxg7kfkg2893q.png" alt=" " width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  2. The half we do block
&lt;/h2&gt;

&lt;p&gt;The second layer works, and that should be said first.&lt;/p&gt;

&lt;p&gt;After canonicalisation, another call with the same user, tool and arguments is refused by a partial unique index in the database: the reading is &lt;code&gt;operation_in_flight&lt;/code&gt;, and the refusal names the row that holds it and the conversation it came from. The arbitration is in the schema, not in application logic, so two concurrent requests cannot both pass.&lt;/p&gt;

&lt;p&gt;Closing the case does not release that lease. The lease only looks at whether the status is &lt;code&gt;claimed&lt;/code&gt;, and closing does not change the status. The exposure the reader feared — "closing opens it up" — does not happen on the call side. It cannot be opened from there.&lt;/p&gt;

&lt;p&gt;The lease is valid only inside the unknown window: once anything settles, the same arguments can go out again. That is deliberate — identical arguments may genuinely be a second, legitimate payment. But it also says what we protect: the period of not knowing, not the money.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. The half that opens: one character changes the operation
&lt;/h2&gt;

&lt;p&gt;The second layer is a function of the arguments.&lt;/p&gt;

&lt;p&gt;Change one character — a different reference, a note that says "retry", &lt;code&gt;100.00&lt;/code&gt; written as &lt;code&gt;100&lt;/code&gt; — and canonicalisation produces a different identity. The lease does not recognise it, the second payment goes out, and the bank sees the same $100.&lt;/p&gt;

&lt;p&gt;A second case is more complete: a person paying directly in the business system. That path never passes through us — no key, no hold, no record.&lt;/p&gt;

&lt;p&gt;A third is the amount itself. If the target froze the funds as part of the call, our ledger never sees the freeze; when it expires under the target's own rules, we never learn. The row says the outcome is unknown. It does not say that up to $100 may have moved.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. What a key would have to be, to protect the money
&lt;/h2&gt;

&lt;p&gt;The question is not whether to record an amount. The key points at the wrong thing. To protect the resource, one key would have to satisfy four things at once: it is minted by &lt;strong&gt;the party holding the balance&lt;/strong&gt; (or at least accepted by it as the only identity); there is &lt;strong&gt;one key per resource&lt;/strong&gt; (ours is a function of the arguments — one character and it is a different key); it &lt;strong&gt;outlives the resource&lt;/strong&gt; (ours is valid only inside the unknown window); and it is &lt;strong&gt;carried by every path that can touch the money&lt;/strong&gt; (a person paying in the UI carries nothing).&lt;/p&gt;

&lt;p&gt;Miss one and the thing protected is not the resource. So the move is not to make the call key stricter but to change what it names: send the resource identity as an argument, let the party holding the balance mint and accept the key, and shorten our own promise to one sentence — &lt;strong&gt;the same resource always carries the same key&lt;/strong&gt;. Only we can do that part, because only we see both the call and its arguments.&lt;/p&gt;

&lt;p&gt;The cost is knowing what the resource is. &lt;code&gt;customer_id&lt;/code&gt; is a resource, &lt;code&gt;amount&lt;/code&gt; is a quantity, &lt;code&gt;note&lt;/code&gt; is a description, and that division belongs to the business, not to the tool. Declare it from the tool, or guess from the arguments; a wrong guess is worse than no record.&lt;/p&gt;

&lt;p&gt;One boundary survives all of this: even with the right resource, any path that does not carry the key walks around it. &lt;strong&gt;A key protects a path, not the money.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;So who keeps the money from being spent twice? Three parties, each holding one segment: we hold "the same call is not replayed"; the target holds "the same key is not accepted twice" — an assumption; the business side holds "the same balance is not spent twice" — through its own limits and reconciliation. Together they are one guarantee; separately nobody holds it, and "keep the exposure on the books" needs the whole of it: which resource, how much, when it can be closed.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. The real question: which promise is it making?
&lt;/h2&gt;

&lt;p&gt;So the question is not whether to add an amount column:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When a system calls itself idempotent, which of the three is it promising?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;What we can say is the negative: treating "the same call is not replayed" as the same thing as "the money cannot be spent twice" passes off the first promise as the third. On paper, the two sentences look almost alike.&lt;/p&gt;

&lt;p&gt;The implementation described here comes from the open-source project KeelBase:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>architecture</category>
      <category>backend</category>
      <category>security</category>
    </item>
    <item>
      <title>AI Execution Governance, Part 12: Closing an Unknown Is Not a Verdict</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Mon, 16 Nov 2026 00:44:27 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-execution-governance-part-12-closing-an-unknown-is-not-a-verdict-1mb7</link>
      <guid>https://dev.to/rain6fish/ai-execution-governance-part-12-closing-an-unknown-is-not-a-verdict-1mb7</guid>
      <description>&lt;p&gt;Context, since this assumes none: I build a runtime that lets AI agents take real actions in business systems — create a record, send a request to another service — with permissions, approvals and an audit trail around each action.&lt;/p&gt;

&lt;p&gt;This post is about what happens when such an action's outcome is unknown, and why the record must not be allowed to settle as "nothing happened". That failure — manufacturing certainty about a side effect you cannot observe — is the same class of error as a monitor reporting green because it stopped looking.&lt;br&gt;
An outbound write times out. The claim sits at &lt;code&gt;claimed&lt;/code&gt;, and nobody knows the outcome. Three weeks later a hundred of them have piled up, and someone clicks release.&lt;/p&gt;

&lt;p&gt;What comes back is not what they asked for: the disposition changes (&lt;code&gt;closed_unknown&lt;/code&gt;), the status does not (still &lt;code&gt;claimed&lt;/code&gt;), and the idempotency key stays occupied. They wanted the row gone and the key reusable. We did not do that.&lt;/p&gt;

&lt;p&gt;This is not a defence of our choice. It is the disagreement laid out: what each path costs, which one we took, and what is still unsolved.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw4opjrsibwo357wvfh6l.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw4opjrsibwo357wvfh6l.png" alt=" " width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  1. The disagreement is about what the key prevents
&lt;/h2&gt;

&lt;p&gt;After a timeout there are three ways forward: a late completion settles the row; a human asks the target one read-only question (reconcile); or a human decides to stop handling it (release). The first two remove the unknown, the third only stops working on it. Nobody argues about that part. The argument is whether the idempotency key goes back.&lt;/p&gt;

&lt;p&gt;It is easy to read reconciliation as "ask once and that is it". Precisely: &lt;strong&gt;one query per call&lt;/strong&gt;. As long as the claim is still &lt;code&gt;claimed&lt;/code&gt; and not released, it can be asked again, and every attempt is stored with its time and conclusion. Nobody has to click for it either: once a hold is in the pending-decision queue, the system asks again every hour, with the gaps growing to one, six, twenty-four and seventy-two hours, then stops. Each pass asks at most twenty-five rows; the rest wait for the next hour. Whether it has stopped is written on the row.&lt;/p&gt;

&lt;p&gt;The cleanup camp is direct: a claim is a resource, the outcome is not coming, so close it out, return the key and let the same call become a clean new call; the target's own idempotency handles duplicates. Queue TTLs and close-on-timeout are what most systems do.&lt;/p&gt;

&lt;p&gt;We did not. This key does not prevent &lt;em&gt;duplicate execution&lt;/em&gt;; it prevents the system from &lt;strong&gt;quietly treating an old call as a new one&lt;/strong&gt;. Whether the target deduplicates is something we do not know — that is a bet across a trust boundary. The key is the one thing we can assert on our own side.&lt;/p&gt;

&lt;p&gt;Where the two sides differ:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;What the key prevents.&lt;/strong&gt; Cleanup: a duplicate execution, handled by the target. Ours: the system passing off an old call as a new one.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Returning the key on release.&lt;/strong&gt; Cleanup: yes, or it is occupied forever. Ours: no, returning it is the same as permitting a resend.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What the queue should end up as.&lt;/strong&gt; Cleanup: shorter. Ours: not shorter.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Main cost.&lt;/strong&gt; Cleanup: one real double write when the target is not idempotent. Ours: the key is never returned and the pending queue only grows.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  2. The cost: the key is not returned, and the queue does not get shorter
&lt;/h2&gt;

&lt;p&gt;After a release the row is still there. It moves from "in flight" to "released", it is still selectable by &lt;code&gt;decisionState&lt;/code&gt;, and someone still has to look at it. The one thing the operator wanted — a shorter queue — is not delivered at all.&lt;/p&gt;

&lt;p&gt;And the key really does not come back: no path deletes the row and none returns it to a state where it can be reoccupied. Every release permanently spends one idempotency key. We accept that cost; we do not have a solution for it.&lt;/p&gt;

&lt;p&gt;That is not a side effect. It is the net price. &lt;strong&gt;We are selling "stop handling it"; they were buying "make it stop appearing".&lt;/strong&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  3. What breaks if we follow the cleanup camp
&lt;/h2&gt;

&lt;p&gt;Both counterexamples are one step down from the code that exists.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A: release returns the key.&lt;/strong&gt; The next retry looks like a brand-new call and goes out. If the first one did reach the target, that is a real double write — and no record can tie it back to the first, because the key is gone. The release created the thing it exists to prevent.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;B: the expiry sweep marks rows released.&lt;/strong&gt; A slow call may still be running; once it is marked, its late completion has no row to record itself into, and the claim stays at "nobody knows" forever, even after the evidence arrives. So the sweep decides nothing itself: every hour it moves overdue &lt;code&gt;open&lt;/code&gt; rows to &lt;code&gt;awaiting_decision&lt;/code&gt;, without changing the status, writing a conclusion or releasing anything. The asking is a separate path — the scheduled re-asks — and "released" still comes either from the target's answer or from a person.&lt;/p&gt;

&lt;p&gt;The endpoint's shape follows from the same reasoning:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;POST /ai/tool-effects/claims/:id/release
-&amp;gt; { outcome: "released_unknown", status: "claimed", decisionState: "closed_unknown" }
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;status&lt;/code&gt; staying &lt;code&gt;claimed&lt;/code&gt; is not a typo. The status answers "did this write produce a result", and the answer is still no. Only the disposition changed.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. The requirements we impose on a release are weak
&lt;/h2&gt;

&lt;p&gt;A release needs a note of 8 to 500 characters and may carry an external reference; the act writes one chained audit row with no outcome. The requirement came from a reader: &lt;strong&gt;"the queue is full" is not a reason.&lt;/strong&gt; A release must point at something someone looked at, or it is indistinguishable from clicking the queue away.&lt;/p&gt;

&lt;p&gt;But we only check the length. "queue pressure" is 14 characters and it passes. The system does not know whether the note points at evidence, or whether that evidence exists.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. The real problem: what the operator saw was never frozen into the record
&lt;/h2&gt;

&lt;p&gt;The system does not keep what was in front of the person when they pressed the button. The reference is free text they typed; the claim's state, whether the target could be queried, how often, what it answered — none of it is frozen with the decision.&lt;/p&gt;

&lt;p&gt;So the auditability of a release reaches "someone decided". It does not reach "the decision held up at the time". And the whole justification for this path rests on "it must point at evidence" — if the evidence cannot be reviewed afterwards either, that sentence is closer to a formality.&lt;/p&gt;

&lt;p&gt;One more thing is not closed: reconciliation presupposes that the target offers a status query. Targets that cannot answer (&lt;code&gt;target-status-unavailable&lt;/code&gt;) cannot even be asked, and the scheduled re-asks do not fire at them at all; those rows never converge and the only way out is a release. &lt;strong&gt;The volume of releases is set by a gap somewhere else.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;So the question worth arguing is not whether release should change the status. It is this:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;If a release cannot be reviewed afterwards for whether it was justified, is it a decision or a formality?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;We do not have an answer. What we are sure of is the other direction: quietly ruling the row "executed" or "not executed" is worse — then not even the formality is left.&lt;/p&gt;

&lt;h2&gt;
  
  
  Closing
&lt;/h2&gt;

&lt;p&gt;The implementation described here comes from the open-source project KeelBase:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>discuss</category>
      <category>agents</category>
    </item>
    <item>
      <title>After the AI Writes, Part 1: Don't Carry Someone Else's Work Into Your Commit</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Wed, 11 Nov 2026 08:23:53 +0000</pubDate>
      <link>https://dev.to/rain6fish/after-the-ai-writes-part-1-dont-carry-someone-elses-work-into-your-commit-17di</link>
      <guid>https://dev.to/rain6fish/after-the-ai-writes-part-1-dont-carry-someone-elses-work-into-your-commit-17di</guid>
      <description>&lt;p&gt;Three AI sessions are working in the same checkout. One of them has just finished; you glance at the diff, it looks right, and you run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git add -A &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; git commit -m &lt;span class="s2"&gt;"feat: ..."&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Ten minutes later you open the commit, and three changes are in it that the assistant never wrote and you never read.&lt;/p&gt;

&lt;p&gt;One sentence: &lt;strong&gt;a commit is made of the index, not of the files you edited.&lt;/strong&gt; On a shared checkout, that difference decides whether someone else's work ends up in your commit.&lt;/p&gt;

&lt;p&gt;If that is enough, the list and the three readings can be skipped.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. How one sweep happened
&lt;/h2&gt;

&lt;p&gt;I added a section to a roadmap on a shared checkout. Before committing I ran:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git status --short &lt;span class="o"&gt;|&lt;/span&gt; head -3
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three lines, and it looked clean. So I committed.&lt;/p&gt;

&lt;p&gt;Later I found the commit also carried three cherry-pick markers from another session and two status lines from a third workstream. They were sitting in the working tree when I committed; &lt;code&gt;head -3&lt;/code&gt; cut them off.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;head&lt;/code&gt; is the wrong tool for "is anyone else's change here": it truncates exactly the part you needed to see.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Four spellings, four sweeps
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;git add -A&lt;/code&gt; stages whole files by path, not your hunks. Another session's uncommitted lines in the same file go into the index with yours.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;git commit&lt;/code&gt; with no pathspec commits the whole index. Work another session had already staged rides along.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;git checkout &amp;lt;rev&amp;gt; -- &amp;lt;file&amp;gt;&lt;/code&gt; does not just fetch an old version; it writes the working tree and the index. The next commit can publish the old version.&lt;/li&gt;
&lt;li&gt;Your new untracked file is not safe either: another session's &lt;code&gt;git add -A&lt;/code&gt; picks it up, and it lands in their commit.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The third is the quietest. It is not merely misattribution; it is a content regression. The index holds the old version, so committing it erases whatever the other session changed afterwards. The working tree looks perfectly fine, and the tests read the working tree.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Four rules you can follow
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Rule 1: Name the paths.&lt;/strong&gt; &lt;code&gt;git commit -F msg.txt -- &amp;lt;your files&amp;gt;&lt;/code&gt;. &lt;code&gt;-F&lt;/code&gt; goes before &lt;code&gt;--&lt;/code&gt;; after &lt;code&gt;--&lt;/code&gt;, only this change's files.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Rule 2: Check the staged list first.&lt;/strong&gt; Before any bare commit, run &lt;code&gt;git diff --cached --name-only&lt;/code&gt;. One file that does not belong to this task means stop. I missed this once: the list plainly showed two files, I did not stop, and another session's execution-log edits came along. I split it afterwards with &lt;code&gt;git reset --soft HEAD^&lt;/code&gt;; nothing was lost.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Rule 3: Rebuild hot files from HEAD.&lt;/strong&gt; For roadmaps, claim tables and execution logs, the ones several sessions rewrite whole: back up the working tree, run &lt;code&gt;git checkout HEAD -- &amp;lt;file&amp;gt;&lt;/code&gt;, insert only your line, and check with &lt;code&gt;git diff -- &amp;lt;file&amp;gt;&lt;/code&gt; that only that line is staged before committing; after the commit, put the backup back so the others' in-flight lines stay in the working tree. I broke this rule once: I appended my line directly to the shared working tree and staged the file. The message said "one line at the end of the table"; the commit carried seven added and three deleted lines from three other sessions. By the time I noticed, they had committed twice on top of mine, so the only honest option was to record it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Rule 4: Read old versions read-only.&lt;/strong&gt; Use &lt;code&gt;git show &amp;lt;sha&amp;gt;:&amp;lt;path&amp;gt;&lt;/code&gt;, never &lt;code&gt;git checkout &amp;lt;rev&amp;gt; -- &amp;lt;path&amp;gt;&lt;/code&gt;; the latter writes the index.&lt;/p&gt;

&lt;p&gt;The smallest self-check is three lines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git diff --cached --name-only    &lt;span class="c"&gt;# only your files&lt;/span&gt;
git commit -F msg.txt -- a.ts b.ts
git show --stat HEAD             &lt;span class="c"&gt;# only your files&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  4. Three readings that mislead
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A truncated &lt;code&gt;git status&lt;/code&gt;.&lt;/strong&gt; &lt;code&gt;head -3&lt;/code&gt; or &lt;code&gt;head -20&lt;/code&gt; can cut off exactly the line. Read all of it, or read &lt;code&gt;git diff&lt;/code&gt; for the file you care about.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A clean &lt;code&gt;git status&lt;/code&gt; after committing.&lt;/strong&gt; It says the working tree and the index agree, not that the commit contains what you meant. Check scope with &lt;code&gt;git show --stat HEAD&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;git log -1 &amp;lt;sha&amp;gt;&lt;/code&gt; printing the commit.&lt;/strong&gt; That proves the object still exists, not that it is on your main line. Ask &lt;code&gt;git merge-base --is-ancestor &amp;lt;sha&amp;gt; HEAD&lt;/code&gt;; after a history rewrite the old object is still printable.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  5. Two places the rules do not reach
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Hooks. A &lt;code&gt;post-commit&lt;/code&gt; hook that runs &lt;code&gt;git commit --amend&lt;/code&gt; without pathspec re-commits the whole index, and your &lt;code&gt;-- &amp;lt;paths&amp;gt;&lt;/code&gt; cannot stop it. That is a toolchain problem, not a git one: fix the hook, or make sure the index is clean before you commit.&lt;/li&gt;
&lt;li&gt;Work that has already been swept in. Do not rewrite history by default. The content is usually still there; what moved is attribution, and a rewrite moves work other people have already built on. First find which commit took it: &lt;code&gt;git log -1 --format=%h -- &amp;lt;path&amp;gt;&lt;/code&gt;; then see what else that commit carried: &lt;code&gt;git show --stat &amp;lt;sha&amp;gt;&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7rm4v1xbic5vy2iybzx0.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7rm4v1xbic5vy2iybzx0.png" alt=" " width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Closing
&lt;/h2&gt;

&lt;p&gt;After the AI writes, the commit is the first action a human takes over. It is the easiest one to treat as a command rather than a decision about scope.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A commit is made of the index.&lt;/strong&gt; It contains what you decided it contains — once you decide.&lt;/p&gt;

&lt;p&gt;These rules come from day-to-day work on the open-source project KeelBase, where several AI sessions commit to the same checkout:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>programming</category>
      <category>codequality</category>
    </item>
    <item>
      <title>AI Execution Governance, Part 11: Revocation Hands Over Three Records, Not One "Stopped"</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Wed, 11 Nov 2026 00:40:31 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-execution-governance-part-11-revocation-hands-over-three-records-not-one-stopped-hfp</link>
      <guid>https://dev.to/rain6fish/ai-execution-governance-part-11-revocation-hands-over-three-records-not-one-stopped-hfp</guid>
      <description>&lt;p&gt;An outbound write is in flight. In another window, an administrator revokes the role of the user who started it.&lt;/p&gt;

&lt;p&gt;Minutes later someone asks: did that call stop?&lt;/p&gt;

&lt;p&gt;If the system answers only "stopped", that answer covers three questions at once: what allowed the call in the first place, whether it actually stopped, and whether what landed on the target has been undone.&lt;/p&gt;

&lt;p&gt;One sentence: after a revocation the system can hand over three layers of evidence — the request, the confirmation, and the reconciliation. "Stopped" is the conclusion of the middle layer only; it cannot stand in for the other two.&lt;/p&gt;

&lt;p&gt;If that is enough, the list and the four boundaries can be skipped.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2okcolb2m1q628nr5vyo.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2okcolb2m1q628nr5vyo.png" alt=" " width="800" height="400"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;One word covering three things&lt;br&gt;
Revoking the grant answers "may this run next time". It takes effect before the next call: that call is refused at the execution point, and the record carries the role and policy revision the decision was made under.&lt;br&gt;
Cancelling answers "did this run stop". It happens during execution, and the record is a three-state process — requested, confirmed, unknown — resolving to aborted or abandoned.&lt;br&gt;
Revoking the effect answers "can what already happened be taken back". It happens after execution, and the record is a revoke window (unacknowledged / awaiting_target) plus a reconciliation conclusion (unknown / executed / not_executed).&lt;br&gt;
The three can be delivered separately, and they can fail separately. Folding them into "stopped" treats three different kinds of evidence as one.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Layer one: the request&lt;br&gt;
The first thing an execution record can prove is what allowed it at the time.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In KeelBase the decision basis is frozen onto the write claim: the role and the governance policy revision are read at decision time and stored with the claim. A revocation happens later, and the row can still answer "under which role and which policy revision was this allowed".&lt;/p&gt;

&lt;p&gt;The point is "frozen". When a reader cannot resolve either value it reports unknown, and never backfills it. Looking up today's role after a revocation returns today's fact, not the one in force then.&lt;/p&gt;

&lt;p&gt;This layer answers the request, not the outcome. It proves the call was permitted. It does not prove it ran.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Layer two: the confirmation
A cancellation is a process, not a click.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;When we ask, the state is requested. Only when the tool actually returns is it confirmed. If we cannot tell, it is unknown. The conclusion has one rule:&lt;/p&gt;

&lt;p&gt;reading = abortable === 'yes' &amp;amp;&amp;amp; state === 'confirmed' ? 'aborted' : 'abandoned'&lt;br&gt;
Only a tool that declares it can stop, plus a confirmed cancellation, earns aborted. Every other combination is abandoned, with the outcome left unknown.&lt;/p&gt;

&lt;p&gt;That line guards a specific misreading: a tool classified no or unknown may return too. What returned is our wait, not the target's work. Recording that as "aborted" would dress a local return up as a remote stop.&lt;/p&gt;

&lt;p&gt;There is a boundary that has to be stated: revocation does not cancel an in-flight call. After the execution point and before the tool returns, authorization is not re-checked. Revocation affects the next call. For the one in flight, the only control is cancellation, and for an external tool that cannot be stopped, cancellation only ends our wait.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Layer three: the reconciliation
The first two layers are about what we know. The third asks the target.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;After an outbound write times out, the ledger sits at unknown: the request may have arrived, or it may not. The only way to resolve it is one read-only question to the target, constructed so it cannot become a second write — GET only, once, no retry.&lt;/p&gt;

&lt;p&gt;Only two answers converge: the target says executed, and the claim settles; the target says not_executed, and the claim is released. A timeout, an unreachable host, a non-2xx, an unrecognised body — all unknown, and the ledger stays where it is.&lt;/p&gt;

&lt;p&gt;Silence is not an answer. This is where the layer is easiest to get wrong: reading "we could not find out" as "it did not run" manufactures evidence out of the absence of evidence.&lt;/p&gt;

&lt;p&gt;One more record in this layer is easy to misread. An administrator can release a stuck claim with a written note. The disposition becomes closed_unknown while status stays claimed. That is not a verdict about the write; it is a decision about the handling: nobody knows whether it landed, and nobody will quietly resend it.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;A point in time: the "all three are zero" in two earlier pieces
When the two earlier pieces were written, these layers really were zero in the code: no cancellation contract, no reconciliation, no in-flight listing, no grant version on the claim row. Writing "all three are zero" was accurate at the time.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That changed on 2026-11-07 with v1.3.1. It carries the cancellation contract and its three states, the in-flight claim listing with evidence-carrying release, read-only reconciliation with a query history, operation identity, role and policy revisions on the claim, tool capability declarations, and record-level review receipts.&lt;/p&gt;

&lt;p&gt;This does not overturn the earlier conclusion, but it changes what can be delivered today. The sentence that still holds is narrower: there is no kill switch. Cancellation is a contract, not a guarantee. A tool classified no or unknown may still run to completion, and the ledger says abandoned. What was added is the ability to reconcile and revoke afterwards, not to stop the call.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Four boundaries still open
Release validates the length of the note (8–500 characters), not whether the reason is real. "Queue pressure" passes. The gap is still there.
Only the WebSocket entry has a cancellation signal. An SSE client disconnect and the R4 retry path have none.
Reconciliation needs the target to offer a read-only query. A target that does not offer one is recorded as a target-status-unavailable gap in the coverage statement; "never asked" cannot be read as "did not run".
Revocation does not re-check mid-flight. That window is kept on purpose: adding a re-check means defining the points and the failure policy, and an unanswerable check is unknown, never "stopped".
Closing
What a revocation hands over should not be a sentence. It should be three records that can each be verified: what allowed the call, whether it stopped, and what actually happened on the other side.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;"Stopped" is one of them. It can be questioned; it cannot answer for the other two.&lt;/p&gt;

&lt;p&gt;The implementation described here comes from the open-source project KeelBase:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;br&gt;
Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
    </item>
    <item>
      <title>AI Audit Evidence, Part 6: A Record Nobody Read Is Only Stored</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Mon, 09 Nov 2026 00:41:19 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-audit-evidence-part-6-a-record-nobody-read-is-only-stored-22nj</link>
      <guid>https://dev.to/rain6fish/ai-audit-evidence-part-6-a-record-nobody-read-is-only-stored-22nj</guid>
      <description>&lt;p&gt;A record sat in the database for a year before anyone opened it.&lt;/p&gt;

&lt;p&gt;That is not negligence. There is no place in the system that says who should read it, or how often.&lt;/p&gt;

&lt;p&gt;One sentence: a record was written is not the same as a record was read. A record nobody read is only stored.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fumdgy7whbli6xppasuij.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fumdgy7whbli6xppasuij.png" alt=" " width="800" height="400"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If that is enough, the list and the three criteria can be skipped.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Three acts that look like review, and are not&lt;br&gt;
Write confirmation happens before execution. It proves someone approved this action. It does not prove anyone looked at the record afterwards.&lt;br&gt;
Alert acknowledgement happens after an anomaly. It proves someone handled the alert. It does not prove the routine records were read.&lt;br&gt;
Dispute acknowledgement happens when a revocation does not match. It proves someone decided that dispute. It does not prove the rest were read.&lt;br&gt;
All three are a person doing something to an event. None answers "who read this record, and how often".&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Three criteria&lt;br&gt;
A named reviewer and a cadence, in configuration. Not an oral agreement, and not "everyone looks".&lt;br&gt;
The receipt is a row on the chain. Each review is itself a chain row, verifiable offline, so "reviewed" cannot be backfilled without breaking the signature.&lt;br&gt;
Unreviewed intervals are computed from the chain and reported as unknown in the coverage statement. They are never green.&lt;br&gt;
Miss one and the capability does not hold: criterion 1 alone is a rota, criterion 2 alone is a receipt, all three are a verifiable review.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The receipt has to say four things: who read it, when, which interval it covers, and what the conclusion was. The conclusion has three values only, and "roughly looked at" is not one of them.&lt;/p&gt;

&lt;p&gt;One more premise: the reviewer cannot be the person who executed the action. Reading your own record is not review, it is a look back. High-risk records get a tighter cadence; routine records can be looser. The cadence lives in configuration, not in a document.&lt;/p&gt;

&lt;p&gt;The reading has three values, no fourth:&lt;/p&gt;

&lt;p&gt;receipt present and chain verifies  -&amp;gt; reviewed&lt;br&gt;
no receipt                          -&amp;gt; unknown&lt;br&gt;
receipt present but chain fails     -&amp;gt; invalid (never reviewed)&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What can fail
Delete a review receipt: the interval returns from reviewed to unknown.
Forge a receipt: the chain fails.
These two are what make the capability testable. Without them, "reviewed" is a field, and it makes the same mistake as "revoked" being a status bit.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The easiest mistake is not skipping the review, but storing the receipt in the wrong place: a mutable table where anyone can add a row, while the chain gains nothing. Then "reviewed" appears, and nothing can prove it happened at the time.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Where we are
AiAuditLog has no reviewedAt or reviewedBy, and the codebase has no notion of a reviewer or a cadence.
The three human-action mechanisms above do not cover record review.
So this item is zero from fields to criteria. This is a design, not an implementation.&lt;/li&gt;
&lt;li&gt;The reader made the question sharper
A plate-search log existed for a year before it was first reviewed.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We first read the problem as "nobody has a reason to read it". The reader corrected that: what is missing is ownership, not motivation. It was not that nobody wanted to look; nobody was assigned to look.&lt;/p&gt;

&lt;p&gt;The same reader then upgraded the criteria to three: a named reviewer and cadence, the receipt on the same chain, and unreviewed intervals reported as unknown.&lt;/p&gt;

&lt;p&gt;The three criteria in this article are the version after that correction.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Three limits
All three criteria are zero. This is a design, not an implementation.
A receipt on the chain changes the shape of the chain and needs a contract change. The cost is not estimated.
"How often" has no universal answer; it has to be set per record type.
One question this article does not answer: what happens after a review finds a problem? It decides whether the receipt needs a follow-up field, and it is its own piece.
Closing
The value of audit evidence is not that it is stored. It is that someone read it, and that the reading itself can be verified.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A log nobody read only proves that we recorded diligently.&lt;/p&gt;

&lt;p&gt;The implementation described here comes from the open-source project KeelBase:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;br&gt;
Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>llm</category>
      <category>agents</category>
    </item>
    <item>
      <title>AI Execution Governance, Part 10: Can Revocation Stop a Call That Is Already Running?</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Wed, 04 Nov 2026 00:49:17 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-execution-governance-part-10-can-revocation-stop-a-call-that-is-already-running-393n</link>
      <guid>https://dev.to/rain6fish/ai-execution-governance-part-10-can-revocation-stop-a-call-that-is-already-running-393n</guid>
      <description>&lt;p&gt;A tool call has passed the gate and is running. Now the role is revoked.&lt;/p&gt;

&lt;p&gt;Does it stop, or does it finish?&lt;/p&gt;

&lt;p&gt;Ours finishes. Revocation governs the next call, not this one.&lt;/p&gt;

&lt;p&gt;That is not the intent of the design; it is what the current implementation does. Three things follow: what each of the three "undo" acts actually stops, when the gate takes effect, and what would have to be added for a call to be stoppable. If the first sentence is enough, the rest can be skipped.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Three "undo" acts are not one&lt;br&gt;
Revocation acts on a permission, before a call starts. Evidence it leaves: the next call is refused.&lt;br&gt;
Cancellation acts on the call itself, while it runs. Evidence it leaves today: none, because there is no cancellation channel.&lt;br&gt;
Revocation of effects acts after execution. Evidence it leaves: a compensation record that can be checked later.&lt;br&gt;
Collapse them into one word and the usual false sentence appears: "it is revoked, so it stopped." What stopped may have been only the next call.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;When the gate takes effect&lt;br&gt;
The write path runs four checks at the execution entry:&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;whether the tool is enabled and the role is allowed (irreversible actions are blocked outright);&lt;br&gt;
whether the arguments stay inside the declared writable field domain and destination allowlist;&lt;br&gt;
whether the destination bound at confirmation time is still the destination now;&lt;br&gt;
whether the approval requirement still holds (a single confirmation is not a second person's approval).&lt;br&gt;
The third applies only to writes that carry a confirmation artifact. An automatic write with no artifact is not subject to it.&lt;/p&gt;

&lt;p&gt;All four run before the tool's own code starts. The role is read from the database on every call, so a revocation takes effect on the next call immediately. For the call that has already entered, there is no second look.&lt;/p&gt;

&lt;p&gt;It also means revocation during the confirmation or approval wait is effective: the four checks run again before the tool actually executes. Revocation stops being able to stop the call only once execute has begun.&lt;/p&gt;

&lt;p&gt;await gate.assertToolAllowed(toolName, userId);&lt;br&gt;
await gate.assertWithinDeclaredScope(toolName, args, userId);&lt;br&gt;
// destination binding, approval requirement...&lt;br&gt;
const result = await tool.execute(args, userId); // from here, no path back to the gate&lt;br&gt;
In the same system, the model HTTP path does have a ceiling: the proxy aborts the request with AbortController when it times out. The tool execution path has nothing equivalent. The two paths do not share a boundary, and that is not a typo.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Three gaps
The tool signature is execute(args, userId). No signal, no abort.
The executor has no AbortController, no Promise.race, no timeout.
The abort on the realtime channel stops the stream, not a tool call already sent.
Together: there is no kill switch today.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;One boundary worth stating: the module does contain other timeouts — one around evaluation batches, one around probing an external tool's status — but neither sits on the tool execution path. They cannot stand in for the missing one.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What it would take
Two options, or both.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A cancellation contract. The tool signature takes a signal; a long call that is aborted returns "aborted", and the side-effect ledger records "aborted" rather than "executed". Passing the signal is the easy part. The hard part is that after an abort the ledger has to say what the target actually did.&lt;/p&gt;

&lt;p&gt;A bounded window. Local tools take milliseconds; calls that leave the system get a ceiling, and anything past it is unknown, not success. This one does not require changing every tool signature, but it does require every outbound call to share the same ceiling.&lt;/p&gt;

&lt;p&gt;Either option can be written as an acceptance check that can fail: revoke a role during a long call and the call is aborted, with the ledger recording an abort; exceed the window and the conclusion is unknown, never success.&lt;/p&gt;

&lt;p&gt;Three questions for your own system:&lt;/p&gt;

&lt;p&gt;Does your tool signature carry a cancellation signal?&lt;br&gt;
Does the executor put a ceiling on a single call?&lt;br&gt;
When a role is revoked, does the system ever read a state called "in flight"?&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Three limits
Cancellation contract, bounded window and an aborted state are all absent today.
The verification level here is design. This comes from reading the code, not from an end-to-end test.
The cost of changing the tool interface everywhere is not estimated. There are 61 tools; this is not a one-parameter change.
Closing
Revocation is an act on a permission. Cancellation is an act on an execution. Treating the first as the second is how you discover, at the moment you most need to stop, that nothing stops.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;What we can honestly deliver is the ledger: a call that finishes is still recorded, and can be revoked or compensated afterwards. That is weaker than stopping it, but it is not the sentence "already revoked".&lt;/p&gt;

&lt;p&gt;The implementation described here comes from the open-source project KeelBase:&lt;br&gt;
&lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/p&gt;




&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
    </item>
    <item>
      <title>AI Execution Governance, Part 9: Revocation Complete Is a Conclusion, Not a Status Field</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Mon, 02 Nov 2026 00:46:02 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-execution-governance-part-9-revocation-complete-is-a-conclusion-not-a-status-field-3l76</link>
      <guid>https://dev.to/rain6fish/ai-execution-governance-part-9-revocation-complete-is-a-conclusion-not-a-status-field-3l76</guid>
      <description>&lt;p&gt;A side-effect row reads revoke_status = revoked. On the screen it becomes a green badge that says revoked.&lt;/p&gt;

&lt;p&gt;Those three words are not a field value. They are a conclusion, and a conclusion is supposed to be argued: did the compensation actually run, did the target actually return to its previous state, did the other system actually answer.&lt;/p&gt;

&lt;p&gt;One sentence: "revocation complete" is at least three claims, and folding them into one boolean replaces three unperformed checks with a badge.&lt;/p&gt;

&lt;p&gt;If that is enough, the rest is a table and three limits.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Completion is three things, not one&lt;br&gt;
The compensation happened. The ledger says it ran. This is our own action and the easiest to prove, which is exactly why it gets mistaken for the conclusion.&lt;br&gt;
The target returned. The business row is no longer in effect. The ledger and the business table are different places and they can disagree.&lt;br&gt;
The outside answered. When the target lives in someone else's system, only that system can say it is gone. On our side the best we record is "the request arrived" or "it arrived and nothing came back".&lt;br&gt;
The three carry different evidence from different sources. Folding them into one boolean folds away which one is still missing.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;What we return&lt;br&gt;
Our restore check returns three values:&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;compensating                    → unknown&lt;br&gt;
revoked, target not soft-deleted → restored&lt;br&gt;
anything else                    → not_restored&lt;br&gt;
Two details matter.&lt;/p&gt;

&lt;p&gt;First, compensating is unknown, not "not revoked". It covers both "compensation is still running" and "it never started". Neither state knows the outcome, and writing either one down would be a claim.&lt;/p&gt;

&lt;p&gt;Second, the boolean view's false means unproven. We also expose a boolean, and it returns false for unknown as well. The comment on it says plainly that false means not proven restored, not known not revoked. Callers have to go back and read the three-valued state rather than treat the boolean as the answer.&lt;/p&gt;

&lt;p&gt;On the outside we have two words as well: possibly not delivered, and delivered, no answer. That second one is the absence of a conclusion, and we do not relabel it as completion.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Why two values are not enough
Both ways of collapsing are wrong.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Collapse to "revoked" turns "I do not know" and "not revoked" into completion. This is what happens whenever the outside is unreachable.&lt;br&gt;
Collapse to "not revoked" turns "in progress" into failure, so the revocation is retried or performed a second time.&lt;br&gt;
The middle value is not cautious wording. It is the only value that does not lie. A system with only two states will eventually assert something it does not know, on the first external timeout it meets.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;A conclusion has to be refutable
A badge that says revoked counts as evidence only if some observation could overturn it. We give the conclusion a refutation path.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Declared against held. What this call says it will revoke is compared against what the ledger actually holds, in both directions. A difference marks the group disputed and refuses to report the revocation complete.&lt;br&gt;
The conclusion is blocked, not the write. When they disagree the write still completes. What is stopped is the sentence "it is done". That keeps non-deterministic tools from stalling on writes whose declarations legitimately differ from what is held.&lt;br&gt;
The test runs backwards. Instead of asking whether two paths are independent, ask whether some fault can move one side and not the other. A break that moves both together means the agreement was never a check.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Four checks you can run
Which claims is this "complete" made of? If you cannot say, it is a status field.
Does the state machine have a middle value? Without unknown, it will lie on some timeout.
What does the boolean's false mean, refuted or unproven? Mixing the two is the dangerous part.
What observation would overturn this conclusion? If none, it was never verified.
Closing
Back to the badge. It should stand for three claims holding at once, not for three fields happening to line up.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;"Revocation complete" is not a status field. It is a conclusion that has to be argued, and when the argument cannot be made the right answer is unknown, not green.&lt;/p&gt;

&lt;p&gt;The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The three-valued check is a way of reading here, not a shipped feature; KeelBase is its implementation and evidence carrier.&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Three things stated plainly. The outside's "delivered, no answer" is a design value we have not tested end to end, so we cannot claim it was verified against a real external system. The declared-versus-held difference triggers only on the retry path that hits the idempotency key, because a first write has nothing to compare against. And the three-valued check is a reading of the code, without a bypass experiment.&lt;/p&gt;

&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
    </item>
    <item>
      <title>AI Audit Evidence, Part 5: Five Sources in One File Do Not Check Each Other</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Fri, 30 Oct 2026 00:49:49 +0000</pubDate>
      <link>https://dev.to/rain6fish/ai-audit-evidence-part-5-five-sources-in-one-file-do-not-check-each-other-37ok</link>
      <guid>https://dev.to/rain6fish/ai-audit-evidence-part-5-five-sources-in-one-file-do-not-check-each-other-37ok</guid>
      <description>&lt;p&gt;Has the same conclusion been answered by two paths that share nothing?&lt;/p&gt;

&lt;p&gt;Our evidence package puts five sources into one file: the authorization snapshot, the decision evidence, the audit row, the side-effect row and the operation audit row. A root digest pins them together. The package verifies offline and the signature matches.&lt;/p&gt;

&lt;p&gt;What that proves is that the five were signed together. It does not prove that they said anything to each other. A digest only answers whether the group was swapped out.&lt;/p&gt;

&lt;p&gt;One sentence: cross verification is not "find more sources". It is one conclusion answered independently by two non-sharing paths, with the system itself able to notice when they disagree.&lt;/p&gt;

&lt;p&gt;If that is enough, the rest is a table, a code fragment and three admissions.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What counts, and what is only packaging
Three criteria. Miss one and it does not count.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The two paths do not share a source. Not two views of the same data, and not the same code run twice.&lt;br&gt;
A disagreement is detectable. Code compares them and returns a distinguishable conclusion. Two values placed side by side, waiting for the reader to spot the conflict, does not count.&lt;br&gt;
It happens at runtime. A comparison that only exists in a test does not count, and neither does one a reader performs by hand.&lt;br&gt;
By those criteria, bundling five sources and hashing the bundle is packaging. Its value is real: the bundle cannot be altered without breaking the hash. But "cannot be altered" answers "was it changed", not "do the parts corroborate each other". Those two questions get conflated because both travel under the word integrity.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Our own account: two and a half
This write was authorized. The second path recomputes from the policy version recorded in the row. What it compares is a single item, whether the tool was enabled. Role, writable field domain, destination and approval requirement are not recomputed.
This write actually happened. The ledger, the business table and the audit row each carry a view. They are displayed side by side and never judged against each other.
The revocation actually completed. The second path compares what this call declared against what the ledger holds, in both directions. This is the only place where a disagreement is detectable.
The record was not altered. No second path in this deployment. The external anchor exists as a mechanism; this deployment has no receiver for it, so it is off rather than absent.
The coverage statement holds. No second path. The conclusion is derived from one set of facts.
Out of five conclusions, one meets all three criteria.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The authorization row is half. A recomputation does happen, but for one of five checks. A reader sees "authorization was reviewed" and what was reviewed is a single item.&lt;/p&gt;

&lt;p&gt;Read that list as three grades: complete, meaning a second source that is actually compared; half, meaning a second source with one item compared; and none, meaning no second source at all.&lt;/p&gt;

&lt;p&gt;There is a fourth grade that is not on the list, and it costs the most: both sources are present and nobody compares them.&lt;/p&gt;

&lt;p&gt;We run two chains, an AI audit chain and an operation audit chain, and the same request lands in both. The material is there, the write points do not overlap, and it looks exactly like two independent paths. But no line of code pairs them by action. They are juxtaposed rather than corroborating.&lt;/p&gt;

&lt;p&gt;It is not on the list of five because we never treated "the two chains describe the same event" as a conclusion the package has to answer. The five are what the package claims. This one nobody claimed. The expensive kind of false cross is not missing material. It is material that nobody remembers to compare.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Why the one that counts, counts
declared targets   vs   targets the ledger holds
only declared → onlyDeclared        only held → onlyStored
either non-empty ⇒ mark the group disputed, refuse to report the revocation complete
Independence: one side is what this call said it would write, the other is what the ledger already recorded. They are produced in different places, so an error in one does not automatically appear in the other.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Detectability: a mismatch becomes a state, disputed, with the difference kept as evidence. Not two fields waiting to be noticed.&lt;/p&gt;

&lt;p&gt;The cost, stated plainly: it triggers in one situation, when a retry with the same arguments hits the idempotency key.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Something a reader asked out of us
This section is about us.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We published a five-layer evidence model: that a record exists, its source and integrity, its link to execution, coverage, and the verification of the result. A reader replied that a layer was missing. Some claims have no record at all. "This system complies with X", with nothing attached.&lt;/p&gt;

&lt;p&gt;He was right, and it was worse than he put it. Our first layer says that a record exists is not the same as the event happened, and it presupposes that a record exists. A claim with nothing attached sits outside the model: it neither satisfies nor violates it. It cannot enter.&lt;/p&gt;

&lt;p&gt;We added the layer, numbered L0. To be explicit: it was added after publication, and the published version does not contain it. If you go back to the article, you will not find it there.&lt;/p&gt;

&lt;p&gt;That exchange was itself a cross check. Our own conclusion was asked again by someone who does not share our view, in a different form, and the different form found the gap. The same question in the same wording returns the same blind spot.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Four checks you can run
What is the second path for this conclusion? If you cannot name one, it is single source. Label it that way instead of calling it corroborated.
Are the two paths independent, or is it the same data dressed differently? Bundling, merging and digesting all fall on the second side.
When they disagree, does the system return a state, or does it place two values next to each other?
Does that comparison run in production? If only in a test, it proves that test rather than the steady state.
Closing
Back to the file. It can prove it was not altered. It cannot prove the parts corroborate each other.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;To prove that, each conclusion has to name its second source, and the ones that cannot get labelled single source. The list will look bad. It will at least be accurate.&lt;/p&gt;

&lt;p&gt;The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. Cross verification is a way of reading here, not a shipped feature; KeelBase is its implementation and evidence carrier.&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Three things stated plainly. One of five conclusions is a complete cross check; the rest are half or nothing. The authorization row reviews a single item out of five. And L0 was added after publication, so anyone citing it should carry that sentence along.&lt;/p&gt;

&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
    </item>
    <item>
      <title>AI Execution Governance, Part 8: Actions the Tool Registry Does Not Know About</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Wed, 28 Oct 2026 00:53:05 +0000</pubDate>
      <link>https://dev.to/rain6fish/actions-the-tool-registry-does-not-know-about-who-owns-them-4i9i</link>
      <guid>https://dev.to/rain6fish/actions-the-tool-registry-does-not-know-about-who-owns-them-4i9i</guid>
      <description>&lt;p&gt;One application, one events table, two rows.&lt;/p&gt;

&lt;p&gt;One row comes from your AI assistant: it calls create_event, a registered tool marked R3 that waits for your confirmation. The other comes from POST /api/v1/events: an ordinary HTTP route that touches no tool and writes nothing to the AI side-effect ledger.&lt;/p&gt;

&lt;p&gt;The two rows look the same. What differs is the entry, not the action.&lt;/p&gt;

&lt;p&gt;One sentence: what a governance system can vouch for is not what the registry contains, but which execution paths were verified and which were not — and the deliverable should be a list of unverified entries, not a coverage percentage.&lt;/p&gt;

&lt;p&gt;If you only want the conclusion, you can stop here. What follows is the four things that have to be separated, one entry-by-entry list, and a failure of our own; you can skim it.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;A registry is a set; an entry is a path
At boot this runtime registers 61 tools, all unconditional, into one Map. GET /ai/tools lists them with risk level and gate mode.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That set answers "what do we intend to govern". It does not answer "where does the action enter".&lt;/p&gt;

&lt;p&gt;Three concrete places show this is not a matter of wording.&lt;/p&gt;

&lt;p&gt;First, there are two catalogues, and an auditor can only see one. GET /ai/tools maps the local registry only and requires read ToolInventory. External MCP tools are discovered at runtime, merged when tools are handed to the model, but absent from that list — they appear on /admin/mcp/tools, which requires manage all. An audit reader holds read only, so on the question "what can the agent call", the auditor can answer half of it; the other half needs an administrator, or is missed.&lt;/p&gt;

&lt;p&gt;Second, the gate does not treat "unregistered" as its own denial condition. When assertToolAllowed cannot find a tool, it catches the error and carries on; the comment says it plainly: let the later execute throw "not found". The executor does refuse: toolRegistry.execute calls getTool first and throws Tool "..." not found when the name is absent. So nothing here licenses a bypass — an unregistered name cannot reach the business layer. What it does license is a narrower claim: passing the gate is not the same as being on the registry. The gate checks a tool it found against its declaration; it does not reject what it did not find. Unregistered actions are held by the executor layer, and the two layers each mind their own business. Reading "the gate let it through" as "this action is registered" is exactly the merge this article is trying to take apart.&lt;/p&gt;

&lt;p&gt;Third, our own table drifted. The tool metadata table once had four labels pointing at tools that did not exist, while twelve real tools had none. The fix was not another hand-written list; it was to run the check against the live registry. That is our own incident, not a hypothetical: a hand-maintained table drifts silently the moment someone adds a tool and forgets it, or renames a tool and leaves the old entry behind.&lt;/p&gt;

&lt;p&gt;Raise the same class of error one level and you get this article's question: we have a gate that checks metadata against the registry, and no gate that checks the registry against the runtime's entry set. Nothing turns red when a new write path appears.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Four things that have to be separated
Action catalogue: what we declare we govern. Today: 61 registrations, one metadata table, plus externally discovered tools at runtime.
Execution entries: where an action actually starts — an HTTP route, an SSE stream, a WebSocket event, an MCP endpoint, a scheduled job, a workflow node, a queue consumer, a plugin route.
Governance coverage: which entries' writes really pass the same gate and the same executor.
Verification evidence: what proves the previous line — a source call site, a unit test, an end-to-end test, or "I read it once".
Fold these four into a percentage and you get the usual "90% coverage". The problem is the denominator: the set of execution entries, which is exactly what we do not know. A percentage with an unknown denominator is a claim about a set you cannot enumerate. A list is different: it names each unverified entry, can be argued with, and can be closed.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Put another way: coverage answers "how much did we do"; a list answers "what is left". Auditing needs the second.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;How to audit the entries
Start from the executor, not the registry. Find the few lines where a write actually happens, then walk backwards.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In this repository the write executor is executeWrite, and the whole repository has exactly three call sites: two in the streaming chat loop (the trusted direct write, and the write after a confirmation is approved), and the R4 approval executor. Walking up from those three lines yields the SSE stream, WebSocket ai:chat, the admin streaming assistant, out-of-band decisions, and the governance-console callback.&lt;/p&gt;

&lt;p&gt;Then go the other way: for every remaining entry that can carry AI authorship, ask how its write happens. This step is reading source, not reading the registry. The thing most easily missed is not "is there a gate" but whether the gate is an optional argument:&lt;/p&gt;

&lt;p&gt;const execute = readOnlyExecutor ?? ((tool: string, args: Record, uid: string) =&amp;gt; toolRegistry.execute(tool, args, uid));&lt;br&gt;
Plan and sub-agent steps are supposed to be read-only: with the read-only executor injected, write tools are refused. But the executor is optional; omit it and execution falls back to toolRegistry.execute — no gate, no confirmation, no side-effect ledger. The sub-agent has the same shape.&lt;/p&gt;

&lt;p&gt;We ran one call with no executor injected: the planner returned a write-tool step, and the tool name landed directly on registry.execute. Every call site in the current assembly injects the executor, so this is a path no one walks today — not a production bypass. But it is reproducible: a unit test even pins "no executor means direct registry call" as expected behaviour. One optional parameter turns "the gate" into "if the caller remembered".&lt;/p&gt;

&lt;p&gt;This method has a flaw it cannot fix itself: it is a snapshot. Add a controller tomorrow that calls toolRegistry.execute directly and nothing goes red. The closest thing to a runtime self-check today is /app/provenance, but it returns counts — total, read/write split, risk distribution. A count cannot name an entry.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Status, evidence level, reachability
The status answers one question only: what is the conclusion.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;verified — within the scope declared for this path, evidence shows it satisfies the stated governance condition: its write passes the gate and the executor, or an explicit refusal stops it.&lt;br&gt;
failed — it is proven that at least one path violates that condition.&lt;br&gt;
unknown — the evidence is insufficient to decide. Not counted as covered.&lt;br&gt;
not_applicable — it is proven this path is outside the declared governance target, and the scope it belongs to is stated.&lt;br&gt;
A second question is not answered by those four values: on what ground. Folding "I read the source" and "I ran an end-to-end test" into one verified repeats exactly the merge this article criticises. So the evidence level is a separate field, independent of the status:&lt;/p&gt;

&lt;p&gt;E1 — source read: the call chain reaches the gate and the executor at the execution point.&lt;br&gt;
E2 — a unit or integration test exercises the path.&lt;br&gt;
E3 — an end-to-end test, or a real run, verifies it.&lt;br&gt;
E4 — a third party reproduces it from the steps in this article in an independent environment.&lt;br&gt;
(E0 means a claim with no evidence and is not used here.) verified / E1 and verified / E3 are not the same sentence: source proves "this call site calls executeWrite", not the runtime assembly, dependency injection, configuration switches or error branches; a passing test proves only the path it ran. The highest level in this piece is E3; nothing reaches E4.&lt;/p&gt;

&lt;p&gt;The entries, one by one — status, then evidence level, then whether the current assembly can reach it:&lt;/p&gt;

&lt;p&gt;POST /ai/chat (non-streaming), /headless/chat, admin non-streaming. Write tools are refused, not executed; the gate is consulted at the start. No write occurs. Status verified (the refusal holds). Evidence E2. Reachable in the current assembly.&lt;br&gt;
POST /ai/chat/stream, WS ai:chat, admin streaming. Writes go through executeWrite; the execution point re-checks the policy switch, field domain, destination and approval requirement, then claims the call. Audit row + claim row + side-effect ledger. Status verified. Evidence E1 (the entry-to-executor chain) plus E2 for the executor itself. Reachable.&lt;br&gt;
POST /mcp (MCP exit). Write tools return "requires confirmation" and are not executed; executeToolForExternal applies the gate. A tool_call audit row is written with outcome=pending. Status verified (the refusal holds). Evidence E2. Reachable.&lt;br&gt;
POST /admin/mcp/call (external MCP gateway). Calls the third-party server directly after the policy switch, R5 and confirmation checks. Writes an audit row; no local side-effect ledger. Status verified (gate yes, ledger no). Evidence E2. Reachable.&lt;br&gt;
Confirmation and approval: in-conversation decision, out-of-band decision, governance-console callback. On approval, the write goes through executeWrite — the same execution point as streaming. Status verified. Evidence E2. Reachable.&lt;br&gt;
Plan / sub-agent with the read-only executor injected. Write tools are refused; executeAgentRead enforces read-only. Audit is written by the caller. Status verified. Evidence E2. Reachable.&lt;br&gt;
Plan / sub-agent with the executor omitted (the fallback). Calls toolRegistry.execute directly. No gate, no confirmation, no ledger. Status failed on the code structure. Evidence E1 plus the out-of-tree probe. Unreachable in the current assembly (every known call site injects the executor).&lt;br&gt;
FLOW ai_task / human_task. One text generation / a flow_task row plus a notification. No tools involved. Status not_applicable (outside the AI tool surface). Evidence E1.&lt;br&gt;
Scheduled AI (daily digest, 0 8 * * &lt;em&gt;). LLM output becomes a notification row. No tools, no tool_call row. Status not_applicable (writes a notification, not a business record). Evidence E1.&lt;br&gt;
Queues (knowledge / push / reminder). Transcription, push, reminders. No tools. Status not_applicable (outside the AI tool surface). Evidence E1.&lt;br&gt;
Direct business REST writes (e.g. POST /events). Calls the business service directly; no AI gate; operation audit, not the AI ledger. Status not_applicable (outside the AI entry surface). Evidence E1.&lt;br&gt;
Plugin routes (POST /plugins/&lt;/em&gt;). A plugin can fetch any host service and register a route. No gate, no ledger. Status unknown (not examined). Evidence E1 (the interface only). Reachability not examined.&lt;br&gt;
Two things about that list need saying.&lt;/p&gt;

&lt;p&gt;A note on the one failed, which has to be read as two sentences. Code structure: failed. With no executor injected, a write-tool name lands on registry.execute (whose real body is tool.execute), and gate, confirmation and ledger are all absent; this is reproducible, not inferred. Current assembly: no evidence that anything walks it. Every call site injects the executor — but that covers only the assembly points checked today, not a call site added tomorrow, and not runtime branches, so it does not amount to "safe". The two sentences must not be merged: writing "the code allows it" as "it happens in production" is one kind of dishonesty; writing "nobody walks it today" as "the code is fine" is the other. The reachability column exists for exactly this.&lt;/p&gt;

&lt;p&gt;not_applicable only says the path is outside the declared scope; it does not mean the business write is safe, and it does not mean KeelBase governs it. A direct business REST write and the AI tool write call the same service function (the create_event tool and POST /events both end at eventsService.create) and leave rows in the same table. It is not an AI entry, so the only promise this table makes about it is "we do not claim to govern it" — and if an AI path ever called it, the gate would not stand in the way. Reading not_applicable as "this write is already covered" turns out-of-scope into a pass.&lt;/p&gt;

&lt;p&gt;unknown cannot be read as verified. This runtime has a gap registry; each gap declares whether it makes the conclusion partial or unknown, and unknown means "we cannot even say what is missing", which is stronger than partial. A test pins that semantic: only a high-severity gap may produce unknown.&lt;/p&gt;

&lt;p&gt;The problem: the registry contains exactly two entries that describe this article's subject — risk-level-not-expressible-for-non-discrete-actions (an action surface that is not a discrete named tool cannot carry a risk level) and host-ai-path-not-governed (the host runs an AI path of its own that does not pass through the engine) — but this deployment's fact set pins both conditions to false, so a statement never declares them.&lt;/p&gt;

&lt;p&gt;We can name the gap; we cannot make it appear in the statement. That is slightly worse than not having the gap at all: it looks checked. The file gives a written reason for two of the pinned facts; these two have none. That is what an honest unknown looks like — not a blank, but "there is something here, and we have no input that can decide it".&lt;/p&gt;

&lt;p&gt;But this one is no longer an ordinary unknown; it is a defect in the statement mechanism itself. The two cases have to be recorded apart. An ordinary unknown is a missing evidence: not examined, or examined and undecidable, and another round of checking may clear it. This one is a missing mechanism: the gap is in the registry, its name can be said out loud, and yet the fact set pins the trigger to false, so the statement necessarily omits it and no amount of re-checking changes that. A statement that only distinguishes "implemented" from "not implemented" leaves this class permanently in the seam between the two.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Seven checks you can run
Which entry did this action come through? A client tap, an HTTP route, a WebSocket event, a scheduled job, a queue, a workflow node — write it down; do not write just "the AI".
Does that entry's write pass the same executor? Ask "is there a gate", not "is there an audit". An audit says it happened; a gate says it was allowed.
Does the same action produce the same row from two entries? If it does, the row alone cannot say who wrote it — unless you join the audit row and the effect row.
How many catalogues are there? Local registrations, the metadata table, externally discovered tools — and who reads each.
Where does unknown live? If the response has no value for unknown, absence will be read as a pass.
What goes red when a new entry appears? If the answer is "nothing", the coverage is hand-maintained.
What level of evidence backs this verified? Source reading and an end-to-end run both get written as "verified", and they vouch for very different amounts.&lt;/li&gt;
&lt;li&gt;What we have, and what we do not
What exists: 61 unconditionally registered tools with an auditor-readable inventory; a write executor that re-checks policy switch, field domain, destination, approval requirement and claim at the execution point; refusal of write tools on the MCP exit and the non-streaming entries; read-only enforcement for plan and sub-agent steps; a gap registry with unknown semantics and a verdict computed from facts; a completeness gate that checks tool metadata against the live registry.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Boundaries: there are two catalogues (the local inventory omits externally discovered tools); the external MCP gateway has a gate and an audit but no local side-effect ledger (the target is someone else's); the plan/sub-agent fallback makes the gate optional; the gate does not refuse unregistered names; two gaps that should say "there is a path we cannot see" are pinned to false and never appear; a direct business write and an AI write share one service function and cannot be told apart from the row alone.&lt;/p&gt;

&lt;p&gt;What is not done: there is no runtime enumeration or assertion of entries — "the action surface is enumerable" remains a bet here, not a proven fact; no test proves "every AI entry is closed"; this table is a manual audit dated 2026-10-26, not a product feature; plugin routes and host-side AI paths have not been assessed.&lt;/p&gt;

&lt;p&gt;Keep three surfaces apart, or one of them will end up vouching for another.&lt;/p&gt;

&lt;p&gt;The action surface: the 61 registered tools, the metadata table, the external MCP tools discovered at runtime — "what we intend to govern".&lt;br&gt;
The entry surface: HTTP, SSE, WebSocket, MCP, scheduled jobs, queues, workflows, plugins that can trigger execution — "where the action enters".&lt;br&gt;
The business side-effect surface: the write points that actually change business state — "what changed".&lt;br&gt;
Each can be incomplete on its own. A complete entry list does not imply a complete side-effect surface: a plugin can fetch any host service, and a general business service can be reached by a path nobody audited. So the conclusion should not stop at "enumerate the entries" — every business write inside the governed scope must complete only through a verified execution path, and every write outside that scope must have its boundary stated. The entry list is only an approximation of that sentence against today's code.&lt;/p&gt;

&lt;p&gt;Closing&lt;br&gt;
Back to the two rows: whether a write is governed depends on which entry it came through, not on which table it landed in.&lt;/p&gt;

&lt;p&gt;The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The entry audit is a way of reading, not a shipped feature; KeelBase is its implementation and evidence carrier.&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Four things stated plainly. Six entries carry E2 (one of them only at the executor layer), five carry only E1 (source), and the fallback row is E1 plus one out-of-tree run; nothing reaches E4. The fallback bypass was reproduced with a script that injects no executor; the production assembly does not walk it. The not_applicable calls for scheduled jobs, queues and workflows rest on "no tools are involved", not on a test that writes are impossible. And we do not write "all entries are covered", because nothing proves it.&lt;/p&gt;

&lt;p&gt;Responsibility is not in one place. The runtime must prove its own execution paths are actually governed — gate, confirmation, ledger, reproducible by test; the host application must identify and disclose the business entries that bypass the runtime, because only it knows where its writes come from; the integrator must confirm which business writes fall inside the governed scope, because scope is set at integration time. And the governance system must keep verified, failed, unknown and out-of-scope apart, rather than letting one tool catalogue vouch for the whole application.&lt;/p&gt;

&lt;p&gt;The registry says what we intend to govern; the entry list says what we actually reach. The difference between them is the governance boundary.&lt;/p&gt;

&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
      <category>mcp</category>
    </item>
    <item>
      <title>AI Audit Evidence, Part 4: What Can One AI Audit Record Actually Prove?</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Mon, 26 Oct 2026 01:06:39 +0000</pubDate>
      <link>https://dev.to/rain6fish/what-can-one-ai-audit-record-actually-prove-445d</link>
      <guid>https://dev.to/rain6fish/what-can-one-ai-audit-record-actually-prove-445d</guid>
      <description>&lt;p&gt;An audit record is on the table: one confirmation, one execution, one side effect. The hash chain verifies.&lt;/p&gt;

&lt;p&gt;What does that prove?&lt;/p&gt;

&lt;p&gt;One sentence: one record does not prove all five layers: a valid signature reaches only the second, and coverage and the external result are separate checks.&lt;/p&gt;

&lt;p&gt;Most people read it as two things: the event happened, and the record is true. There are five different questions inside that sentence, and one record usually answers only one or two of them.&lt;/p&gt;

&lt;p&gt;This piece gives a five-layer evidence model. If you only want the conclusion, you can stop here. The rest is the failure modes and the five claims behind that sentence, and you can skip it. The bold sentence in each section carries the point.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Five questions, not five fields
A record can answer five questions, and they are not the same thing:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Event record: what did the system record?&lt;br&gt;
Source and integrity: who produced the record, and was it altered?&lt;br&gt;
Execution linkage: does this record correspond to the path that actually ran?&lt;br&gt;
Coverage: which execution paths were verified?&lt;br&gt;
Result verification: what actually happened to the business side effect?&lt;br&gt;
The layers build on each other, but they do not upgrade automatically. A higher layer cannot be inferred from a lower layer's fields. A signature does not add coverage. A coverage statement does not verify an external result.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What one real record looks like
This is a record from a local run on 2026-10-15. The tool is create_followup_task, risk level R3, approved from the pending list:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;confirmation #3   create_followup_task  R3  approved&lt;br&gt;
  decided 05:54:37.327   claimed .332   executed .365   error null&lt;br&gt;
audit #27         action=tool_call   tool=null   outcome=null   is_error=0&lt;br&gt;
  detail="create_followup_task({\"customerId\":1,\"title\":\"Followup Task\"})"&lt;br&gt;
  hash=b0d6bf…   prev_hash=565b3f…&lt;br&gt;
effect #1         crm_task:10&lt;br&gt;
  before_snapshot=null   after_snapshot={"id":10,...}   revoke_class=local_compensate&lt;br&gt;
The three rows together are "one operation".&lt;/p&gt;

&lt;p&gt;But there is a non-obvious detail: audit row #27 has tool and outcome empty. The tool name exists only inside the detail string, and the result can only be inferred from is_error=0. A later row in the same database (#37, later that day) has tool=create_followup_task and outcome=executed. The same kind of record can look different at different times.&lt;/p&gt;

&lt;p&gt;The record is not false. It is authentic, on the chain, and the hashes are continuous. It simply does not explain itself. A reader who does not know the detail convention cannot say what ran or what the result was.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Reading the record layer by layer&lt;br&gt;
Layer 1, event record: three rows exist, a confirmation, an audit row and a side-effect row. It does not prove the event happened as recorded; the record itself can be wrong.&lt;br&gt;
Layer 2, source and integrity: the audit row is on the chain, with continuous hash and prev_hash. It does not prove the tail was not deleted or that nothing was omitted. The confirmation and side-effect rows are not on this chain.&lt;br&gt;
Layer 3, execution linkage: the confirmation's execution axis is executed, and the effect is crm_task:10. It does not prove this effect was produced by this confirmation; the link is by tool, argument hash and time, not one foreign key.&lt;br&gt;
Layer 4, coverage: the package's coverage statement can be consulted separately. One record does not say other execution paths were covered.&lt;br&gt;
Layer 5, result verification: after_snapshot is present. It does not make the change comparable, because before_snapshot is null, and it does not prove the external target's state.&lt;br&gt;
The strongest layer in this record is layer 2. The weakest is layer 5. It is a real record, not a complete piece of evidence.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Two counterexamples&lt;br&gt;
Counterexample A: a valid signature over an incomplete record.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Audit row #27 is on the chain, with continuous hash and prev_hash. But its tool and outcome are empty, and the side-effect row has no before_snapshot. So:&lt;/p&gt;

&lt;p&gt;you can prove the audit row was not altered&lt;br&gt;
you cannot use this record alone to say which tool ran, what the result was, or what changed before it&lt;br&gt;
A signature authenticates bytes, not meaning. A real, signed record can still fail to answer the question it appears to answer.&lt;/p&gt;

&lt;p&gt;Counterexample B: a complete record with an unknown business result.&lt;/p&gt;

&lt;p&gt;This is a design-level scenario, not a test of the record above. A write is sent to an external system. The local side-effect row says executed, but the external system is unreachable and has no query endpoint. The correct conclusion is unknown, not success.&lt;/p&gt;

&lt;p&gt;A complete local record only says the local side is complete. An unverified external state cannot be written as success.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Not a field checklist
The five layers are not a field checklist. They are five refutable claims:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;L1: a record exists is not the same as the event happened.&lt;br&gt;
L2: a valid chain or signature is not the same as no omissions and no tail deletion.&lt;br&gt;
L3: a record is linked to an execution is not the same as the execution matched its authorization.&lt;br&gt;
L4: one action was covered is not the same as every path was covered.&lt;br&gt;
L5: a local result was recorded is not the same as the external business state was verified.&lt;br&gt;
These claims do not try to prove that AI never makes mistakes. They ask which guarantees a system can offer when AI misreads, overreaches, retries, times out or hits an exception, and how far each guarantee can be verified.&lt;/p&gt;

&lt;p&gt;They also explain why adding fields does not add guarantees: a signature at L2 does not make coverage at L4 true, and coverage at L4 does not make an external result at L5 verified. Each layer needs its own evidence.&lt;/p&gt;

&lt;p&gt;The test of these claims is not more terminology. The test is whether they can point at a counterexample, narrow a guarantee, or turn a vague promise into a verifiable criterion. All five are candidates, not proven theorems.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What we have, and what we do not
What exists: the confirmation's execution axis (decided_at, execution_claimed_at, executed_at, execution_error); the audit hash chain; side-effect rows with before and after snapshots; the coverage statement; the audit reader role.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Boundaries: the record above already shows three. Structured fields can be empty. A side-effect row can lack a before snapshot. The confirmation and side-effect rows are not on the audit chain. Beyond that: without an external anchor, tail completeness can only be unknown; an unreachable external result is unknown; the operation audit row scope is not yet narrowed by organisation.&lt;/p&gt;

&lt;p&gt;What is not done: the five-layer model is not an API or a product feature. It is a way to read evidence, which you can apply to any agent runtime without waiting for us to ship five endpoints.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Five checks you can run
Which layer does this record answer? "Recorded" or "verified"?
What proves integrity? A chain, a signature, a coverage statement, or only a sentence?
What links the record to the execution? One id, or tool plus argument hash plus time? Can someone else rebuild the link?
Where is coverage written? How are uncovered paths expressed: unknown, absent, or pretended complete?
How far does result verification go? When the external state is unknown, is the conclusion unknown?
Closing
Back to the opening: what a record proves depends on which layer you are asking about.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;What we do: split evidence into five layers and say what each cannot prove; show a real record where "authentic but incomplete" is visible; keep external results and tail completeness as unknown instead of writing them as success.&lt;/p&gt;

&lt;p&gt;The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The five-layer model is a way to read evidence, not a shipped feature; KeelBase is its implementation and evidence carrier.&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Four things stated plainly. The record above comes from a local run, not production data. The five-layer model is a method, not a shipped feature. External results and tail completeness remain unknown, and the operation audit row scope is still an open item. The five layers are candidates, and counterexamples are welcome.&lt;/p&gt;

&lt;p&gt;An honest record does not claim all five layers are full. It says where each layer stops.&lt;/p&gt;

&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
      <category>llm</category>
    </item>
    <item>
      <title>AI Audit Evidence, Part 3: An Auditor Is Not an Admin</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Fri, 23 Oct 2026 01:04:15 +0000</pubDate>
      <link>https://dev.to/rain6fish/an-auditor-is-not-an-admin-giving-someone-the-evidence-is-not-giving-them-the-system-41be</link>
      <guid>https://dev.to/rain6fish/an-auditor-is-not-an-admin-giving-someone-the-evidence-is-not-giving-them-the-system-41be</guid>
      <description>&lt;p&gt;An auditor has to review what an AI did.&lt;/p&gt;

&lt;p&gt;The cheapest way to let them do that is to hand them an admin account. But then the person they are supposed to review becomes themselves.&lt;/p&gt;

&lt;p&gt;This piece is about building a read-only auditor role: its reach is an explicit list, every field is rebuilt from a whitelist, row scope comes from the account rather than the request, and the signed evidence stays whole and unprojected.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Admin is the cheapest answer, and the wrong one
An audit reader's job is to review what someone else did. They need evidence, not operational authority.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;But in most systems read access follows write access: whoever can manage everything can obviously read everything. So the shortest path to "let them see the audit" is to give them admin. The cost is that they also gain the ability to change policy, approve requests and execute tools, which is exactly what they came to review.&lt;/p&gt;

&lt;p&gt;What is needed is not a weaker admin. It is a role whose reach can be enumerated item by item. If you cannot enumerate it, it is still an admin.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The role's reach is a list
Our auditor role gets five rules, and every one grants read only:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;{ roleCode: UserRole.AUDITOR, subject: 'AuditLog',         actions: ['read'], ownerField: null },&lt;br&gt;
{ roleCode: UserRole.AUDITOR, subject: 'OperationAudit',   actions: ['read'], ownerField: null },&lt;br&gt;
{ roleCode: UserRole.AUDITOR, subject: 'GovernancePolicy', actions: ['read'], ownerField: null },&lt;br&gt;
{ roleCode: UserRole.AUDITOR, subject: 'ToolInventory',    actions: ['read'], ownerField: null },&lt;br&gt;
{ roleCode: UserRole.AUDITOR, subject: 'ApprovalQueue',    actions: ['read'], ownerField: null },&lt;br&gt;
That list is everything the role can touch. Everything else is denied by the policy guard by default.&lt;/p&gt;

&lt;p&gt;It deliberately gets none of the USER ownership rules. This role is a reader of the governance record, not a user of the application. It should have no write action either. The day one of these rules grows a write, the role stops being an auditor.&lt;/p&gt;

&lt;p&gt;The shape of the rule is worth one sentence too. The underlying role seeds originally had only a subject and an owner field, and every seed implied manage. We added an optional actions; omitting it still means manage, so no existing role changed behaviour. Writing it is how a role narrows itself. Read access is written down, not inferred from management.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Fields: rebuilt from a whitelist, not deleted from a full row
The role decides which surface you can reach. The field list decides what you can see on that surface.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We keep one whitelist and rebuild the projection from it:&lt;/p&gt;

&lt;p&gt;only keys named on the list are written&lt;br&gt;
a column added tomorrow stays invisible to an audit reader until someone deliberately adds it to the list&lt;br&gt;
deleting forbidden fields from a full row would fail open: the next column leaks by default&lt;br&gt;
That is not a style choice. It is a choice about defaults. Deleting from a full row defaults to visible; rebuilding from a whitelist defaults to invisible.&lt;/p&gt;

&lt;p&gt;The fields left off the list have reasons. A conversation id is a pointer into a person's conversation. IP and device ids bind activity to a device or a person. A user's feedback about the assistant is not the audit record. Delegation context is free text that may quote a prompt. The bounded business intent and the agent and caller ids carry the attribution a reader actually needs.&lt;/p&gt;

&lt;p&gt;One test injects a key that is not on the list at runtime and asserts it does not appear in the projection. It is not testing deletion. It is testing the default.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Rows: organisation from the account, period from the request
Beyond the role and the fields there is the question of which rows. The same surface, with the same fields, must still show different rows to different readers.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The organisation comes from the account, not the request. An organisation parameter on the request can only narrow further, ANDed with the account scope, never replace it.&lt;br&gt;
An auditor who belongs to no organisation falls back to their own rows. A failed organisation lookup also falls back to their own rows. A failed lookup is not a reason to widen.&lt;br&gt;
The period comes from the request window. There is no per-reader period attribute today and no read-plane policy. Inventing one here would invent a dimension, so we use the window the caller asks for.&lt;br&gt;
Fetching a single row by id returns 404 when it is out of scope, the same answer as "no such row". The gate already passed; telling the reader that another organisation has that row is itself a disclosure.&lt;br&gt;
The by-id path and the list call the same scope function. Two implementations would eventually disagree.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Aggregates need the scope too, and so does the cache key
Narrowing the list while leaving the counts narrow-free is the same as not doing it.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The auditor's stats, cost, report and coverage statement all read through the same organisation scope. One thing is easy to miss: the aggregates are cached for 60 seconds, and the cache key has to carry the scope.&lt;/p&gt;

&lt;p&gt;We hit that class of defect. With no scope in the key, the first reader's organisation answer was served to the next reader for a minute. The fix is not to clear the cache. The fix is to put the scope in the key: all, org: or own:. It is the same class as a verification verdict computed for one database being served for another. A cache does not know who is asking, so the key has to.&lt;/p&gt;

&lt;p&gt;One thing we do not slice by organisation: the hash chain inside a signed evidence package. Slicing a chain and verifying the slice produces a false conclusion, so the chain keeps its full scope. Rows can be narrowed. A chain cannot.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What stays closed
Listing what the auditor can reach matters as much as listing what they still cannot:&lt;/li&gt;
&lt;/ol&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Audit log, single-event explanation, chain, verify, stats, cost, report&lt;/strong&gt;: the auditor reads projected rows within their organisation scope; an admin reads full rows.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Operation audit&lt;/strong&gt;: the auditor reads, but row scope is not yet narrowed by organisation; an admin reads.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Governance policy, its history, tool inventory, approval queue&lt;/strong&gt;: the auditor reads; an admin reads, writes and decides.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Signed evidence package and verify chain payload&lt;/strong&gt;: the auditor reads them unprojected so they stay verifiable; an admin reads them too.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Users, organisation, sessions, platform analytics, writes, approvals, execution&lt;/strong&gt;: unreachable for the auditor; reachable for an admin.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Three points deserve their own paragraph:&lt;/p&gt;

&lt;p&gt;The signed evidence package and the verify chain payload are not projected. They are verification artifacts whose value is the complete canonical payload. Projecting them makes them unverifiable. Privacy projection covers row-shaped responses, not the evidence itself.&lt;br&gt;
The auditor can read the approval queue but cannot decide it. Approving, rejecting and retrying execution stay admin-only. Reading the queue is not handling it. The same applies to flow approvers: the role list deliberately excludes the auditor. An auditor must not become the kind of person they are reviewing.&lt;br&gt;
Operation audit's read access is open, but its row scope is not narrowed yet. The service filters only by the request's userId; it does not apply the reader's account scope. This is an open item from the role's widening, and saying so here keeps the five rules from implying that every surface is already scoped.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The scene: widening one surface at a time
Our first cut changed only the audit controller to read AuditLog. Operation audit still judged manage all, so the auditor reached that surface and got a 403.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That was not an omission. It was deliberate: open one surface, see whether it is enough, then widen one surface at a time. Later, operation audit, governance policy, tool inventory and the approval queue each got their own read subject. Writes, approvals and execution never entered the widening.&lt;/p&gt;

&lt;p&gt;The order has a second benefit. Each surface can be verified on its own: what should be visible is visible, what should be blocked is blocked. Opening everything at once makes a hole impossible to attribute.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Five checks you can run
Which surfaces can your read-only role reach? Can you enumerate them, or is it "roughly like admin"?
Are new fields visible by default? Deleting from a full row fails open. Rebuilding from a whitelist fails closed.
Does row scope come from the account or the request? Can the request only narrow it? Where does a missing organisation fall back to?
Do the aggregates and the cache carry the same scope? A narrowed list with unscoped counts is not narrowed.
Is the signed evidence projected? If it is, can it still be verified?
Closing
Back to the opening: giving someone the evidence is not giving them the system.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;What we do: five read-only rules define the role's entire reach; fields are rebuilt from a whitelist and new columns are invisible by default; organisation comes from the account and the period from the request; an out-of-scope row returns 404; aggregates and cache keys carry the same scope; signed evidence stays whole.&lt;/p&gt;

&lt;p&gt;The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline.&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Four things stated plainly. The role is widened one surface at a time: everything outside the list is still unreachable, and that is default deny rather than unfinished work. Signed evidence cannot be projected, so privacy minimisation has an explicit boundary here. An auditor cannot decide approvals or serve as a flow approver. If they could, they would become the person they are reviewing. And operation audit's row scope is not narrowed by organisation yet: the read access is open, the scope is not, and that is an open item rather than a finished surface.&lt;/p&gt;

&lt;p&gt;Read-only is not "a little less permission". It is a reach you can enumerate.&lt;/p&gt;

&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
    </item>
    <item>
      <title>AI Audit Evidence, Part 2: Silence Reads as Completeness</title>
      <dc:creator>Rain S</dc:creator>
      <pubDate>Wed, 21 Oct 2026 01:29:45 +0000</pubDate>
      <link>https://dev.to/rain6fish/silence-reads-as-completeness-an-ai-audit-package-must-carry-a-coverage-statement-4pp4</link>
      <guid>https://dev.to/rain6fish/silence-reads-as-completeness-an-ai-audit-package-must-carry-a-coverage-statement-4pp4</guid>
      <description>&lt;p&gt;An auditor receives an evidence package. The hash chain verifies.&lt;/p&gt;

&lt;p&gt;What will they read that as?&lt;/p&gt;

&lt;p&gt;Most people read it as two things: the rows were not altered, and they are all here.&lt;/p&gt;

&lt;p&gt;The chain proves only the first. Delete the last few rows and the remaining chain is still internally consistent.&lt;/p&gt;

&lt;p&gt;This piece proposes one thing: every evidence package should carry a coverage statement, in data, saying what it read and what it cannot show, signed together with the package.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What the chain proves, and what it does not
A hash chain means each row carries the digest of the one before it. Change a row and the rest stop matching. Remove a row in the middle and the chain breaks. Reorder them and it breaks.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;It does not stop deletion from the tail. Remove the last few rows and what remains is a shorter chain that is entirely self-consistent.&lt;/p&gt;

&lt;p&gt;So "the chain verifies" answers internal consistency, not completeness. Completeness needs something outside the chain, such as periodically writing the tail digest somewhere else.&lt;/p&gt;

&lt;p&gt;This is not an implementation detail. It is the inference a reader will make. A record that does not state its boundary gets read as complete.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The scene: a gap we deliberately left out
We maintain a registry of gaps. Each entry says one thing this kind of package cannot show.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;As of this writing there are eleven entries.&lt;/p&gt;

&lt;p&gt;One candidate is deliberately not among them: the claim that a side-effect row cannot show who did it. We did not add it, because whether it still holds has not been settled. Our side-effect entity carries an owner. That reading may only mean the entity has the field while the endpoint does not expose it.&lt;/p&gt;

&lt;p&gt;An unverified gap is worse than no gap. It attaches a false caveat to every package, and the reader has no way to tell which caveat is real.&lt;/p&gt;

&lt;p&gt;So one of our tests is dedicated to the absence. It asserts that the registry contains no origin for that claim.&lt;/p&gt;

&lt;p&gt;This may be the least intuitive line in the design: this test does not check that something is there. It checks that something is not.&lt;/p&gt;

&lt;p&gt;It also says the coverage statement needs a coverage boundary of its own. We can only declare the gaps we have verified.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Gaps are data, not prose
If every report carried a hand-written paragraph saying "this report may not include...", that paragraph would drift away from the system. Someone changes collection and forgets the caveat, or changes the caveat and forgets the system.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;So the gaps live in one place. Each entry has:&lt;/p&gt;

&lt;p&gt;a stable id that is never reused&lt;br&gt;
a severity&lt;br&gt;
whether it limits this package or the whole deployment&lt;br&gt;
a predicate over a closed set of facts, all of which must hold for the gap to apply&lt;br&gt;
one English statement, the field that carries the caveat on the wire&lt;br&gt;
an internal origin that does not&lt;br&gt;
The closed set is deliberate. A predicate that can read anything is a predicate nobody can evaluate, and an entry whose condition cannot be evaluated can only ever apply. That is the silence-is-completeness failure again.&lt;/p&gt;

&lt;p&gt;So each package declares the gaps that actually apply to it, not a copied list of caveats that drifts.&lt;/p&gt;

&lt;p&gt;Adding a gap changes the registry only, as long as it reads the facts we already collect. The statement gains an entry and the service code does not change. This is structural: change the registry and the registry revision in the statement moves with it. A search for the eleven gap ids in the service code returns nothing.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;What a coverage statement contains
First make the statement an object rather than a paragraph. In our package it currently has these fields:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;registryRevision: which version of the gap registry the statement was computed against&lt;br&gt;
scope: whether it covers one action, a period or an entity, with its start, end and subject&lt;br&gt;
sources: the sources actually read, each with a state and a row count&lt;br&gt;
verdict: complete, partial or unknown&lt;br&gt;
gaps: the gaps that apply, each with an id, severity, scope and one English statement&lt;br&gt;
That is the minimum machine-readable shape. It answers three questions: which rules were used, what was read, and how the verdict reads.&lt;/p&gt;

&lt;p&gt;If you want human review in the statement, one sentence is not enough. You need three fields: review status, who confirmed it, and when; plus an exception reason, meaning why something is not covered and who accepted that exception. Those fields are not in our statement today. They live on the authorization and confirmation rows. Putting them in means deciding on a cross-table read and making sure they are signed too. Otherwise you have added another paragraph that drifts.&lt;/p&gt;

&lt;p&gt;One more thing about the signature: the statement is not signed on its own. It is signed as part of the package's canonical string. A separate hash for the statement would be a different design.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The verdict is computed
The registry also has to say how a verdict reads. Our rule has three lines:&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;no applicable gap means complete&lt;br&gt;
any gap that says "we cannot even tell what is missing" means unknown&lt;br&gt;
everything else means partial&lt;br&gt;
The important boundary is that unknown is not a stronger partial. partial means I know what is missing. unknown means I cannot even tell.&lt;/p&gt;

&lt;p&gt;Reporting unknown as partial would claim we know what we are missing, which is exactly what that gap says we do not.&lt;/p&gt;

&lt;p&gt;The verdict is not written by hand. A hand-written verdict is just another value that drifts.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The statement is signed with the package
If the coverage statement were a note beside the package, it could be rewritten after the fact into a nicer version and the signature would still verify. The section would be decoration.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;So in the new evidence-package format the coverage statement is a required section, and it is part of the canonical string that gets signed. The offline verifier checks for it first; a package of this version without it fails.&lt;/p&gt;

&lt;p&gt;The same statement has to say what was consulted. It lists only the sources this package actually read, not the sources the system happens to have. Listing a source nobody queried is the same silence this whole thing exists to break.&lt;/p&gt;

&lt;p&gt;The signature covers the statement's content, and two values in that content are not computed. They are pinned, and neither is optimistic.&lt;/p&gt;

&lt;p&gt;The first is whether one cache serves more than one deployment. The code cannot see a second deployment. false is the correct answer for the deployment shape this repository supports, one deployment with one set of databases. The gap stays in the registry for shapes it does not support. There is no input for that today, so we record it rather than paper over it.&lt;/p&gt;

&lt;p&gt;The second is whether the package has an external anchor. The anchor has no endpoint, which we measured. So a package cannot carry one, and the statement says unknown instead of pretending the chain is enough.&lt;/p&gt;

&lt;p&gt;Both are worth stating precisely, because they show what the statement is for. It does not promise completeness. It points at where completeness is missing.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;Three common "only" cases&lt;br&gt;
Only a trace, no statement: the reader takes a missing trace as a missing action. Absence of a trace and absence of the action are two different facts.&lt;br&gt;
Only an approval, no approved scope: approving one call is not approving every argument and destination of that run. If the scope is not written, the next person has to guess.&lt;br&gt;
Only a log, no integrity check: a log can be edited. A chain proves internal consistency, but the tail can still be removed. Without an external anchor, completeness is still an inference.&lt;br&gt;
All three point at one thing: the presence of a record is not the same as coverage.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;A boundary this design does not cross&lt;br&gt;
A coverage statement makes incompleteness readable. It does not make an audit complete.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;It can only cover facts we can compute. What we cannot compute goes to unknown, or, if it is a guess we have not verified, it stays out of the registry entirely.&lt;/p&gt;

&lt;p&gt;The fact set is closed, and that is both its shape and its ceiling. A gap that needs a new fact does not appear by adding a row to the registry.&lt;/p&gt;

&lt;p&gt;So it is not a compliance certificate. It is a usage note: this is what the package can answer, and this is what it cannot.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Four checks you can run
Does your evidence package say what it did not read? If all it says is "verified", the reader will supply "probably complete".
Is that sentence hand-written, or computed from a registry? The hand-written one drifts.
Is the verdict written or computed? In particular, can unknown be downgraded to partial?
Is the statement inside the signature? If not, it can be rewritten after the fact without breaking the seal.
If you run high-risk agent sessions, the cheapest implementation is not a new report page. Make package generation an automatic step at the end of the session, and make the coverage statement a required section of the package. The system computes it; the person writing the report does not fill it in. That is the difference between a coverage statement and a disclaimer.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Closing&lt;br&gt;
Back to the opening: silence gets read as completeness.&lt;/p&gt;

&lt;p&gt;What we do: the gaps live in one place, eleven entries with an origin for each; predicates read a closed set of facts; the verdict is computed and unknown is not downgraded; the coverage statement is signed with the package; and only the sources actually read are listed.&lt;/p&gt;

&lt;p&gt;The mechanism above is taken from KeelBase, an open-source runtime that sits between an AI agent and a business system, re-checks identity, authorization, scope, risk and confirmation state before a tool runs, and writes every decision into a record that can be verified offline. The coverage statement sits at the seam between confirmation and audit in that governance loop: confirmation answers who approved this action, and the statement answers how far the record can prove.&lt;/p&gt;

&lt;p&gt;Repository (a Gitee mirror is the faster route from inside China):&lt;/p&gt;

&lt;p&gt;GitHub: &lt;a href="https://github.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://github.com/rain6fish/KeelBase&lt;/a&gt;&lt;br&gt;
Gitee: &lt;a href="https://gitee.com/rain6fish/KeelBase" rel="noopener noreferrer"&gt;https://gitee.com/rain6fish/KeelBase&lt;/a&gt;&lt;br&gt;
Two things stated plainly. The external anchor has no endpoint today, so any conclusion about tail completeness is unknown. That is not caution, it is the fact. And the deliberate absence: we would rather declare one gap fewer than attach an unverified caveat to every package.&lt;/p&gt;

&lt;p&gt;An honest audit package does not say it has no gaps. It says which ones it has.&lt;/p&gt;

&lt;p&gt;Drafted with an AI assistant. The system, the positions and the mistakes are mine.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>security</category>
      <category>agents</category>
    </item>
  </channel>
</rss>
