<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: asagiman</title>
    <description>The latest articles on DEV Community by asagiman (@suirindo).</description>
    <link>https://dev.to/suirindo</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4194144%2Faf20c549-b1cf-431d-bd71-d950bc601519.jpg</url>
      <title>DEV Community: asagiman</title>
      <link>https://dev.to/suirindo</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/suirindo"/>
    <language>en</language>
    <item>
      <title>Tests pass for both patches; one edits an out-of-scope file</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Tue, 17 Nov 2026 02:45:06 +0000</pubDate>
      <link>https://dev.to/netsujo/tests-pass-for-both-patches-one-edits-an-out-of-scope-file-4l5h</link>
      <guid>https://dev.to/netsujo/tests-pass-for-both-patches-one-edits-an-out-of-scope-file-4l5h</guid>
      <description>&lt;p&gt;Two patches pass the same unit tests, and one of them also edits &lt;code&gt;config/release.json&lt;/code&gt;, a file outside the write authority declared for the task. Agent Role Contracts (ARC) reports that path when it is given the list of changed paths. It does not stop the write: in this example the file really changes.&lt;/p&gt;

&lt;p&gt;You can reproduce the result with one clone and one command. No AI account, API key or running agent is involved. The two patches are synthetic fixtures that stand in for an agent's output.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/suirindo/agent-role-contracts.git
git -C agent-role-contracts checkout --detach b42a308d5be59132310cfb782ed828745d7fd3bf
node agent-role-contracts/experiments/git-diff-demo/run.mjs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Requirements: Git 2.32 or newer and Node.js 22.5 or newer. There is no &lt;code&gt;npm install&lt;/code&gt;. Only cloning uses the network.&lt;/p&gt;

&lt;p&gt;Patch A changes only &lt;code&gt;src/greet.mjs&lt;/code&gt;. Patch B makes the same change and also edits &lt;code&gt;config/release.json&lt;/code&gt;.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;Unit tests&lt;/th&gt;
&lt;th&gt;ARC&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;A&lt;/td&gt;
&lt;td&gt;PASS&lt;/td&gt;
&lt;td&gt;PASS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;B&lt;/td&gt;
&lt;td&gt;PASS&lt;/td&gt;
&lt;td&gt;FAIL (exit 1)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;In case B, ARC flags &lt;code&gt;config/release.json&lt;/code&gt; and reports &lt;code&gt;TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What ARC does not do here
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;It does not read Git history or file contents. The demo lists the changed paths and passes each one to ARC. A path that is not listed is not checked.&lt;/li&gt;
&lt;li&gt;It does not stop writes. In case B the out-of-scope file changed, and the tests passed.&lt;/li&gt;
&lt;li&gt;It does not authenticate who or what made the change.&lt;/li&gt;
&lt;li&gt;It does not show that a reviewer is independent of the implementer.&lt;/li&gt;
&lt;li&gt;It does not show that the code is correct or safe. A PASS is a statement about one path and one declaration.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Full output
&lt;/h2&gt;

&lt;p&gt;This is the complete standard output at commit &lt;code&gt;b42a308d5be59132310cfb782ed828745d7fd3bf&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ARC #130 demo: unit tests pass, the scope check still flags the extra file.
Declared write authority: src/greet.mjs

Case A: allowed patch
  Changed paths (from Git): src/greet.mjs
  Unit tests (node --test): PASS (exit 0)
  ARC explain src/greet.mjs: PASS (exit 0)

Case B: same patch + one out-of-scope edit
  Changed paths (from Git): config/release.json, src/greet.mjs
  Unit tests (node --test): PASS (exit 0)
  ARC explain config/release.json: FAIL (exit 1) TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY
  ARC explain src/greet.mjs: PASS (exit 0)

Result: tests passed in both cases; ARC flagged 1 path in case B: config/release.json
Demo complete: every observed result matched the expectation (demo exit 0).
Checked by ARC 1.0.1: each path this demo listed from Git, against the declared authority.
Not checked: code correctness, who made the change, or whether any write was blocked.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The demo exits &lt;strong&gt;0&lt;/strong&gt; when both expected outcomes are reproduced. The &lt;code&gt;FAIL (exit 1)&lt;/code&gt; line is the ARC result for one path. It is not the exit code of the demo.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the demo works
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;In a temporary Git repository, the demo applies patch A, and then patch B in a fresh copy, with &lt;code&gt;git apply --index&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;For each case it runs the same unit test file with &lt;code&gt;node --test&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;The demo lists the changed paths from Git, then runs the &lt;code&gt;explain&lt;/code&gt; command of the ARC command-line tool once per path. Each run uses a copy of the task whose scope is that single path.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The declared write authority is one file, &lt;code&gt;src/greet.mjs&lt;/code&gt;. Patch B contains patch A byte for byte, plus one edit to &lt;code&gt;config/release.json&lt;/code&gt;. The demo writes nothing to your clone and removes its temporary files.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the tests do not see it
&lt;/h2&gt;

&lt;p&gt;The unit tests answer one question: does the function still return what the tests expect? Both patches make the same change to &lt;code&gt;src/greet.mjs&lt;/code&gt;, so both answers are yes. No test reads &lt;code&gt;config/release.json&lt;/code&gt;, so the extra edit has no effect on the result.&lt;/p&gt;

&lt;p&gt;The declaration check answers a different question: does each changed path fit the write authority declared for this task? That answer does not depend on whether the code works.&lt;/p&gt;

&lt;p&gt;Neither check replaces the other. Keep running your tests.&lt;/p&gt;

&lt;h2&gt;
  
  
  How long it takes
&lt;/h2&gt;

&lt;p&gt;One measurement, taken on 2026-11-17 on macOS 26.6.2 with Git 2.51.2 and Node.js 24.22.1: the three commands above, including the clone, finished in 1.2 seconds of wall-clock time. That number does not include installing Git or Node.js or reading the output, and it will vary with your connection. This run was not repeated on Linux or Windows.&lt;/p&gt;

&lt;p&gt;The issue that opened this lab records one maintainer rerun from a clean public clone (2026-11-15, macOS). That is an internal reproduction, not third-party validation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Using the check in your own repository
&lt;/h2&gt;

&lt;p&gt;The demo's README has the steps, including how to list changed paths and which results mean you should not continue: &lt;a href="https://github.com/suirindo/agent-role-contracts/blob/b42a308d5be59132310cfb782ed828745d7fd3bf/experiments/git-diff-demo/README.md#use-it-in-your-own-repository" rel="noopener noreferrer"&gt;Use it in your own repository&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Two points decide how much the check can tell you. The wider the declared scope, the fewer paths ARC can flag. And ARC checks only the paths you pass to it, so the list of changed paths has to be complete and has to come from a method you trust.&lt;/p&gt;

&lt;h2&gt;
  
  
  Add a failure case
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/suirindo/agent-role-contracts/issues/134" rel="noopener noreferrer"&gt;ARC Failure Lab #1&lt;/a&gt; is open for one synthetic, self-contained example from a workflow you can describe publicly. A reply should include:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Boundary: what the agent was permitted to do.&lt;/li&gt;
&lt;li&gt;Deviation: one precise extra action, stale receipt, missing field or policy mismatch.&lt;/li&gt;
&lt;li&gt;Reproduce: minimal code, configuration, steps and versions, ideally runnable offline.&lt;/li&gt;
&lt;li&gt;Expected and observed: exact output and exit code where possible. Mark anything you have not run as &lt;code&gt;NOT_RUN&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;What ARC could check, and what needs an independent host or gate.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Reports where ARC cannot detect the failure are welcome, because they show where the boundary is. Review and adoption as a fixture are not guaranteed.&lt;/p&gt;

&lt;p&gt;Do not post production credentials, secrets, private logs, confidential files or details about identifiable people. Report suspected vulnerabilities through &lt;a href="https://github.com/suirindo/agent-role-contracts/security/advisories/new" rel="noopener noreferrer"&gt;private security reporting&lt;/a&gt;, not in a public comment.&lt;/p&gt;

&lt;p&gt;Source, fixture and tests: &lt;a href="https://github.com/suirindo/agent-role-contracts/tree/b42a308d5be59132310cfb782ed828745d7fd3bf/experiments/git-diff-demo" rel="noopener noreferrer"&gt;experiments/git-diff-demo&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>agents</category>
      <category>devops</category>
    </item>
    <item>
      <title>I Run Coding Agents on Three Macs. Here's What I Refuse to Delete.</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Mon, 16 Nov 2026 11:23:28 +0000</pubDate>
      <link>https://dev.to/suirindo/i-run-coding-agents-on-three-macs-heres-what-i-refuse-to-delete-22i3</link>
      <guid>https://dev.to/suirindo/i-run-coding-agents-on-three-macs-heres-what-i-refuse-to-delete-22i3</guid>
      <description>&lt;p&gt;&lt;em&gt;What managing three AI coding-agent environments taught me about disk recovery, reproducibility, and knowing when not to clean up.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;On November 16, 2026, one of my development Macs had about 28 GiB of free disk space.&lt;/p&gt;

&lt;p&gt;I had a target of keeping at least 100 GiB available on each of my three machines.&lt;/p&gt;

&lt;p&gt;There was a tempting opportunity to reclaim roughly 10 GiB. The files looked like candidates for cleanup, but they still had open references.&lt;/p&gt;

&lt;p&gt;So I deleted nothing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;That particular cleanup recovered exactly 0 GiB. And I consider that the correct decision.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I run AI-assisted development across three Macs: a MacBook Pro and two other machines I refer to as M1 and M3.&lt;/p&gt;

&lt;p&gt;The agents create code, run tests, build applications, inspect failures, and prepare changes. Over time, their environments accumulate dependencies, build artifacts, temporary workspaces, models, and evidence from previous tasks.&lt;/p&gt;

&lt;p&gt;Disk space eventually becomes a constraint on how much work can run.&lt;/p&gt;

&lt;p&gt;The challenge is deciding what can safely disappear without destroying something another agent still needs.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem wasn't just large files
&lt;/h2&gt;

&lt;p&gt;When I first investigated our disk usage, I found a familiar pattern.&lt;/p&gt;

&lt;p&gt;Each coding task could create its own working directory. Some tasks installed dependencies; others generated Next.js build output. Completed tasks were moved into archive directories.&lt;/p&gt;

&lt;p&gt;But archiving a workspace didn't make its generated files disappear.&lt;/p&gt;

&lt;p&gt;On October 23, an archived area on our Pro machine had grown to approximately 273 GiB. Within it, 262 directories of &lt;code&gt;node_modules&lt;/code&gt; and &lt;code&gt;.next&lt;/code&gt; artifacts accounted for about 120.8 GiB of measured target-side storage.&lt;/p&gt;

&lt;p&gt;A cleanup limited to artifacts classified as regenerable was followed by a whole-volume free-space increase from 56.2 GiB to 165.5 GiB.&lt;/p&gt;

&lt;p&gt;That was a useful recovery. It wasn't proof that every future cleanup would be safe.&lt;/p&gt;

&lt;p&gt;The target-side size and the 109.3 GiB change in free space are different measurements. APFS sharing and concurrent disk activity mean they cannot be treated as identical.&lt;/p&gt;

&lt;p&gt;And the detailed safety evidence available for an earlier single-directory deletion wasn't independently reconciled for all 262 later targets.&lt;/p&gt;

&lt;p&gt;I learned to keep that limitation visible instead of letting a successful cleanup become a blanket rule.&lt;/p&gt;

&lt;h2&gt;
  
  
  What happened across the three Macs
&lt;/h2&gt;

&lt;p&gt;Here are selected observations from November 16, 2026, in Japan time.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Machine&lt;/th&gt;
&lt;th&gt;Observation&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;M1&lt;/td&gt;
&lt;td&gt;Approximately 55 GiB to 57.1 GiB free&lt;/td&gt;
&lt;td&gt;1.69 GiB recovered in a bounded operation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;M3&lt;/td&gt;
&lt;td&gt;Approximately 25 GiB to 28.0 GiB free&lt;/td&gt;
&lt;td&gt;3.17 GiB recovered&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;M3, later operation&lt;/td&gt;
&lt;td&gt;27.9 GiB to 29.1 GiB free&lt;/td&gt;
&lt;td&gt;Approximately 1.19 GiB recovered&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;M3, subsequent inspection&lt;/td&gt;
&lt;td&gt;About 28 GiB free&lt;/td&gt;
&lt;td&gt;0 GiB recovered; protected candidates retained&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Pro&lt;/td&gt;
&lt;td&gt;Approximately 101.3 GiB free at inspection&lt;/td&gt;
&lt;td&gt;No further deletion because active work was protected&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These are separate observation windows, not a continuous benchmark. The measurements do not establish sustained capacity or improved execution speed.&lt;/p&gt;

&lt;p&gt;The successful bounded removals targeted regenerable dependency and build output.&lt;/p&gt;

&lt;p&gt;The zero-recovery inspection was different.&lt;/p&gt;

&lt;p&gt;On M3, a large data area still had open-file references. Other potential candidates lacked sufficient proof of safe restoration or deletion authority.&lt;/p&gt;

&lt;p&gt;I couldn't establish that removing them would preserve ongoing work.&lt;/p&gt;

&lt;p&gt;The correct action was to leave them untouched.&lt;/p&gt;

&lt;p&gt;The overall three-machine capacity objective remained incomplete.&lt;/p&gt;

&lt;h2&gt;
  
  
  A directory name is not a deletion policy
&lt;/h2&gt;

&lt;p&gt;It's tempting to treat certain names as automatically disposable:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;node_modules&lt;/code&gt; — potentially rebuildable dependencies, provided the exact environment and installation inputs are retained&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;.next&lt;/code&gt; — generated Next.js output, provided the build inputs and configuration are available&lt;/li&gt;
&lt;li&gt;Archived worktrees — potentially inactive, but still capable of holding unique source or unmerged commits&lt;/li&gt;
&lt;li&gt;Logs and execution receipts — possibly the only retained evidence explaining a failed or partially completed task&lt;/li&gt;
&lt;li&gt;Downloaded models and datasets — potentially expensive, difficult, or impossible to reconstruct identically&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The category tells me what to investigate. It doesn't authorize deletion.&lt;/p&gt;

&lt;p&gt;For example, &lt;code&gt;node_modules&lt;/code&gt; is usually regenerable. But I still need to know whether the workspace is active, whether its dependency lockfile exists, and whether the required package versions remain available.&lt;/p&gt;

&lt;p&gt;A model file presents a different question. Even if it can technically be downloaded again, another running process may be using it. Its precise version might also matter to reproducibility.&lt;/p&gt;

&lt;p&gt;I treat active work, unshared changes, evidence, secrets, and files without verified restoration paths as protected.&lt;/p&gt;

&lt;h2&gt;
  
  
  My pre-deletion inspection
&lt;/h2&gt;

&lt;p&gt;I prefer a bounded, read-only inspection before considering any destructive operation.&lt;/p&gt;

&lt;p&gt;For a Git-backed workspace, the starting point is simple:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Set this to the exact repository under inspection.&lt;/span&gt;
&lt;span class="nv"&gt;repo&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"/path/to/repository"&lt;/span&gt;

git -C &lt;span class="s2"&gt;"$repo"&lt;/span&gt; rev-parse HEAD
git -C &lt;span class="s2"&gt;"$repo"&lt;/span&gt; status --porcelain=v1
git -C &lt;span class="s2"&gt;"$repo"&lt;/span&gt; worktree list --porcelain

&lt;span class="c"&gt;# Inspect local branch commits absent from remote refs.&lt;/span&gt;
git -C &lt;span class="s2"&gt;"$repo"&lt;/span&gt; log --branches \
  --not --remotes --oneline

&lt;span class="c"&gt;# Observe current volume capacity.&lt;/span&gt;
df -k &lt;span class="s2"&gt;"$repo"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These commands don't delete anything.&lt;/p&gt;

&lt;p&gt;They establish part of the workspace's current state, but they aren't enough by themselves.&lt;/p&gt;

&lt;p&gt;I also inspect the exact candidate directory, relevant processes and open files, active task ownership, backup or regeneration evidence, and the scope of any existing authorization.&lt;/p&gt;

&lt;p&gt;For a narrowly identified directory, &lt;code&gt;du -sk&lt;/code&gt; can estimate allocated size, while a scoped &lt;code&gt;lsof +D&lt;/code&gt; inspection may reveal open-file references. The latter can be expensive on large trees, and a lack of visible results is not conclusive proof of safety.&lt;/p&gt;

&lt;p&gt;I don't execute a deletion just because a shell command returned successfully or produced no output.&lt;/p&gt;

&lt;h2&gt;
  
  
  The difference between removing bytes and recovering space
&lt;/h2&gt;

&lt;p&gt;Another lesson came from comparing directory sizes with actual available capacity.&lt;/p&gt;

&lt;p&gt;Suppose a cleanup candidate is measured at 20 GiB.&lt;/p&gt;

&lt;p&gt;After removal, the filesystem reports only 12 GiB more free space.&lt;/p&gt;

&lt;p&gt;I do not report that the machine recovered 20 GiB.&lt;/p&gt;

&lt;p&gt;Those two measurements answer different questions.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;du&lt;/code&gt; helps describe the selected files. &lt;code&gt;df&lt;/code&gt; measures available space on the filesystem. Shared blocks, snapshots, and concurrent activity can affect the observed change.&lt;/p&gt;

&lt;p&gt;For a meaningful recovery record, I retain the target, its measured size, the volume's free capacity immediately before and after the operation, and the state of protected work.&lt;/p&gt;

&lt;p&gt;I also distinguish immediate recovery from sustained headroom.&lt;/p&gt;

&lt;p&gt;On our Pro, the October 23 cleanup was followed by approximately 167 GiB free the next morning, 159 GiB later on October 24, and 158 GiB on October 25.&lt;/p&gt;

&lt;p&gt;Capacity recovery was accepted on October 25 after additional checks. That did not establish improved CPU performance, reduced task latency, or permanent resolution of resource pressure.&lt;/p&gt;

&lt;p&gt;And by November, maintaining free space remained an active operational problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  I turned the decision process into a Skill
&lt;/h2&gt;

&lt;p&gt;Repeating this investigation manually was becoming part of the problem.&lt;/p&gt;

&lt;p&gt;Every time disk pressure returned, I had to reconstruct the same questions: which workspace owns this directory, whether it is active, what created the files, and how they can be restored.&lt;/p&gt;

&lt;p&gt;I began encoding that process in a local skill named &lt;code&gt;mac-capacity-recovery&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Its purpose is to standardize the investigation and preserve successful recovery patterns.&lt;/p&gt;

&lt;p&gt;The workflow follows a small sequence:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Measure current headroom and identify exact candidates.&lt;/li&gt;
&lt;li&gt;Check active ownership, references, Git state, and restoration evidence.&lt;/li&gt;
&lt;li&gt;Perform only specifically authorized, bounded cleanup operations.&lt;/li&gt;
&lt;li&gt;Measure the resulting free space and verify that protected work remains intact.&lt;/li&gt;
&lt;li&gt;Record the result, including an explicit stop when safety cannot be established.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A Skill makes the procedure easier to reuse. It doesn't grant additional filesystem permissions or make uncertain deletions safe.&lt;/p&gt;

&lt;p&gt;The local rollout and its acceptance also need to be distinguished. Having a Skill file on one machine doesn't prove that every host has the same active, verified procedure.&lt;/p&gt;

&lt;p&gt;For me, the useful improvement is that the next inspection can start from a known method instead of rediscovering the same precautions.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sometimes the most valuable result is zero
&lt;/h2&gt;

&lt;p&gt;When disk space is low, an agent naturally has an incentive to find something to remove.&lt;/p&gt;

&lt;p&gt;That's where a cleanup workflow needs a boundary.&lt;/p&gt;

&lt;p&gt;A process that always finds files to delete may appear productive. But it could be making the environment less reproducible with every successful run.&lt;/p&gt;

&lt;p&gt;On November 16, M3 still had far less free space than my operating target.&lt;/p&gt;

&lt;p&gt;The large candidate with open references remained protected. Other candidates lacked the evidence required for safe removal.&lt;/p&gt;

&lt;p&gt;The cleanup stopped without reclaiming additional space.&lt;/p&gt;

&lt;p&gt;I still had a capacity problem to solve.&lt;/p&gt;

&lt;p&gt;But I hadn't converted a storage problem into a source-integrity or recovery problem.&lt;/p&gt;

&lt;p&gt;That distinction matters when you operate AI coding agents over weeks or months instead of running a single coding session.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I would change from the beginning
&lt;/h2&gt;

&lt;p&gt;If I were setting up these machines again, I would design workspace retention alongside workspace creation.&lt;/p&gt;

&lt;p&gt;Every temporary workspace would have an identifiable owner, a source revision, a retention reason, and a clear point at which generated artifacts could become cleanup candidates.&lt;/p&gt;

&lt;p&gt;I would keep restoration instructions close to the files they describe.&lt;/p&gt;

&lt;p&gt;And I would measure physical headroom after cleanup rather than relying on the size of selected directories.&lt;/p&gt;

&lt;p&gt;Not every development environment needs this level of tracking. A disposable local experiment is different from an agent workspace containing unmerged changes or evidence required for a release.&lt;/p&gt;

&lt;p&gt;The distinction is the point.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;An AI coding agent should be able to explain why a file is safe to remove before it removes it. When it cannot, keeping the file is a valid outcome.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I previously wrote about the gap between CI success and actual macOS installation in &lt;a href="https://dev.to/suirindo/ci-can-be-green-while-the-macos-installer-fails-stage-by-stage-acceptance-from-a-real-controller-1jbo"&gt;CI Can Be Green While the macOS Installer Fails&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;This is another part of operating coding agents beyond the code-generation step: keeping the machines usable while preserving the work they produce.&lt;/p&gt;

&lt;p&gt;I'm curious how other developers handle this.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Do your AI coding agents clean up their own workspaces? If they do, what prevents them from deleting something another task still needs?&lt;/strong&gt;&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm the founder of &lt;a href="https://netsujo.jp/" rel="noopener noreferrer"&gt;Netsujo&lt;/a&gt;, based in Kyoto, Japan. I work on AI agent systems, software engineering, and reliable execution workflows. The observations in this article come from our internal development operations; they are not presented as an independently reproduced public benchmark.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>ai</category>
      <category>productivity</category>
      <category>discuss</category>
    </item>
    <item>
      <title>ARC Failure Lab #1: Two Green Tests. One Unauthorized File.</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Mon, 16 Nov 2026 10:19:12 +0000</pubDate>
      <link>https://dev.to/netsujo/arc-failure-lab-1-two-green-tests-one-unauthorized-file-50h5</link>
      <guid>https://dev.to/netsujo/arc-failure-lab-1-two-green-tests-one-unauthorized-file-50h5</guid>
      <description>&lt;p&gt;A coding agent finishes a task. Its unit tests pass. The pull request looks ordinary.&lt;/p&gt;

&lt;p&gt;There is one more question to ask: &lt;strong&gt;Did it change only the files it was authorized to change?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Welcome to &lt;strong&gt;ARC Failure Lab&lt;/strong&gt;. Each challenge starts with a small, reproducible boundary failure. We publish the expected results, show which part of the problem &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt; can check, and invite developers to propose a better counterexample.&lt;/p&gt;

&lt;p&gt;For Challenge #1, no AI account or live agent is needed. The failure is small enough to run offline in a few minutes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The experiment
&lt;/h2&gt;

&lt;p&gt;Imagine an agent assigned to update one file:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Allowed:&lt;/strong&gt; &lt;code&gt;src/greet.mjs&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Not allowed:&lt;/strong&gt; &lt;code&gt;config/release.json&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;We have two Git patches:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A:&lt;/strong&gt; changes only &lt;code&gt;src/greet.mjs&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;B:&lt;/strong&gt; makes the exact same allowed change &lt;strong&gt;and&lt;/strong&gt; changes &lt;code&gt;config/release.json&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both patches pass the same unit tests. But only one fits the declared write authority.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Patch&lt;/th&gt;
&lt;th&gt;Unit tests&lt;/th&gt;
&lt;th&gt;ARC declaration check&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;A: allowed edit&lt;/td&gt;
&lt;td&gt;PASS&lt;/td&gt;
&lt;td&gt;PASS&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;B: allowed edit + extra config change&lt;/td&gt;
&lt;td&gt;PASS&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;config/release.json&lt;/code&gt;: FAIL, &lt;code&gt;TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;There is no trick in the unit tests. A unit test can be perfectly correct while being &lt;strong&gt;silent&lt;/strong&gt; about changes outside the functionality it covers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reproduce it
&lt;/h2&gt;

&lt;p&gt;You'll need Git 2.32+ and Node.js 22.5+. The repository uses a pinned demonstration and does not require an npm install, credentials, API keys, or running an agent.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/suirindo/agent-role-contracts.git
git -C agent-role-contracts checkout --detach b42a308d5be59132310cfb782ed828745d7fd3bf
node agent-role-contracts/experiments/git-diff-demo/run.mjs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The script creates temporary Git repositories and applies the two patches separately. It then runs the same tests and passes each changed path to the actual ARC CLI using a task declaration.&lt;/p&gt;

&lt;p&gt;The important lines from the expected output are:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Case A: allowed patch
  Unit tests (node --test): PASS (exit 0)
  ARC explain src/greet.mjs: PASS (exit 0)

Case B: same patch + one out-of-scope edit
  Unit tests (node --test): PASS (exit 0)
  ARC explain config/release.json: FAIL (exit 1) TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY
  ARC explain src/greet.mjs: PASS (exit 0)

Demo complete: every observed result matched the expectation (demo exit 0).
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;strong&gt;overall demo exit code is 0&lt;/strong&gt; because the negative case failed &lt;em&gt;as expected&lt;/em&gt;. The &lt;code&gt;exit 1&lt;/code&gt; line reports ARC's rejection of an out-of-scope path, not a failed execution of the demonstration.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Rerun evidence:&lt;/strong&gt; The Netsujo maintainer reran the experiment on a clean macOS/M3 clone at the pinned commit on November 15, 2026. Both unit tests passed, and the declaration check rejected the extra file as expected. This is an internal reproduction, not a claim of independent third-party validation.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/suirindo/agent-role-contracts/blob/b42a308d5be59132310cfb782ed828745d7fd3bf/experiments/git-diff-demo/README.md" rel="noopener noreferrer"&gt;Full source, expected output and limitations&lt;/a&gt; · &lt;a href="https://github.com/suirindo/agent-role-contracts/blob/b42a308d5be59132310cfb782ed828745d7fd3bf/experiments/git-diff-demo/README.ja.md" rel="noopener noreferrer"&gt;Japanese instructions&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What this proves—and what it does not
&lt;/h2&gt;

&lt;p&gt;Here is the important division of responsibility:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The demo's Git wrapper&lt;/strong&gt; lists the changed paths. &lt;strong&gt;ARC&lt;/strong&gt; checks each supplied path against the task's declared write authority. &lt;strong&gt;The unit tests&lt;/strong&gt; check the behavior they were written to cover.&lt;/p&gt;

&lt;p&gt;ARC does &lt;strong&gt;not&lt;/strong&gt; independently inspect Git history, stop a write, authenticate which agent produced a change, establish reviewer independence, or prove the code is correct or safe. If the wrapper fails to supply a path, ARC will not check it. A real integration needs a trusted, complete change inventory and separate host-side enforcement where required.&lt;/p&gt;

&lt;p&gt;This distinction matters because an automated reviewer can truthfully say "PASS" about the evidence it was asked to inspect while saying nothing about an &lt;strong&gt;unexamined boundary&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Your turn: bring one reproducible failure
&lt;/h2&gt;

&lt;p&gt;What can go wrong in your own AI-agent workflow even when the ordinary checks are green?&lt;/p&gt;

&lt;p&gt;Open the &lt;strong&gt;&lt;a href="https://github.com/suirindo/agent-role-contracts/issues/134" rel="noopener noreferrer"&gt;ARC Failure Lab #1 participation thread&lt;/a&gt;&lt;/strong&gt; and share a &lt;em&gt;small, public, synthetic&lt;/em&gt; case using this template:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Boundary:&lt;/strong&gt; What was the agent allowed to do?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deviation:&lt;/strong&gt; What extra action, stale review receipt, missing provenance, or policy mismatch happened?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reproduction:&lt;/strong&gt; What exact steps, code, versions and inputs are needed?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Expected vs observed:&lt;/strong&gt; What output or exit code did you actually see? Mark unrun ideas &lt;code&gt;NOT_RUN&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Responsibility:&lt;/strong&gt; Which part could ARC check, and which part requires your host or acceptance gate?&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Failures that ARC &lt;strong&gt;cannot&lt;/strong&gt; detect are also useful. They help us explain exactly where another control is necessary.&lt;/p&gt;

&lt;p&gt;Please do not publish customer data, credentials, production logs, private paths, or suspected exploitable vulnerabilities. Use the repository's &lt;a href="https://github.com/suirindo/agent-role-contracts/security/advisories/new" rel="noopener noreferrer"&gt;private security reporting flow&lt;/a&gt; for vulnerabilities.&lt;/p&gt;

&lt;h2&gt;
  
  
  Join the lab
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Run the fixture:&lt;/strong&gt; &lt;a href="https://github.com/suirindo/agent-role-contracts/tree/b42a308d5be59132310cfb782ed828745d7fd3bf/experiments/git-diff-demo" rel="noopener noreferrer"&gt;ARC Failure Lab #1 demo&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Submit a failure case or challenge the test:&lt;/strong&gt; &lt;a href="https://github.com/suirindo/agent-role-contracts/issues/134" rel="noopener noreferrer"&gt;GitHub Issue #134&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Read or contribute to ARC:&lt;/strong&gt; &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;GitHub repository&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;ARC is an open-source project developed by &lt;a href="https://netsujo.jp/" rel="noopener noreferrer"&gt;Netsujo&lt;/a&gt;. The goal of Failure Lab is to turn precise technical disagreements into small, repeatable checks—not to treat a declaration-level PASS as proof of complete runtime safety.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>opensource</category>
      <category>testing</category>
      <category>javascript</category>
    </item>
    <item>
      <title>Two AI Agents Passed Review. Why Couldn't Their Code Be Merged?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Fri, 13 Nov 2026 15:02:50 +0000</pubDate>
      <link>https://dev.to/suirindo/two-ai-agents-passed-review-why-couldnt-their-code-be-merged-4dca</link>
      <guid>https://dev.to/suirindo/two-ai-agents-passed-review-why-couldnt-their-code-be-merged-4dca</guid>
      <description>&lt;p&gt;Two AI coding agents finish their tasks.&lt;/p&gt;

&lt;p&gt;Both pull requests pass their own tests. Both receive independent reviews. Neither agent changes the same file.&lt;/p&gt;

&lt;p&gt;Then the team integrates the branches—and the product breaks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What exactly passed?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In an earlier discussion, &lt;a href="https://dev.to/suirindo/two-ai-agents-reviewed-the-same-pr-was-that-actually-independent-5h22"&gt;I asked whether two AI agents reviewing the same pull request were actually independent&lt;/a&gt;. A conversation with &lt;a href="https://dev.to/jes_p_d4f61f7aa2ec405a418/comment/3l3p0"&gt;Dev_PJS&lt;/a&gt; sharpened a different issue: a task may be independently verifiable yet have no realistic path to integration.&lt;/p&gt;

&lt;p&gt;That distinction matters as teams increase the number of coding agents they run in parallel.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two green branches, one broken product
&lt;/h2&gt;

&lt;p&gt;Consider this &lt;strong&gt;hypothetical example&lt;/strong&gt;. Both agents start from the same commit, &lt;code&gt;M0&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Agent A&lt;/strong&gt; updates an order API. The old response is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"total"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;49.95&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The new response uses integer cents:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;"totalCents"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;4995&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Agent A updates the existing consumers it knows about. Its tests pass. An independent reviewer approves the exact commit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Agent B&lt;/strong&gt; adds a CSV export feature on its own branch. It uses the old API shape:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="n"&gt;csvAmount&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;order&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;total&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toFixed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Agent B's tests use fixtures based on &lt;code&gt;M0&lt;/code&gt;. They pass too. A separate reviewer approves the commit.&lt;/p&gt;

&lt;p&gt;No files overlap. Git can merge both changes without a textual conflict.&lt;/p&gt;

&lt;p&gt;But once both changes are combined, the exporter receives an order with &lt;code&gt;totalCents&lt;/code&gt;, not &lt;code&gt;total&lt;/code&gt;. Its assumption is now false. A real producer-consumer integration test would catch it; isolated tests with old fixtures may not.&lt;/p&gt;

&lt;p&gt;The review receipts can all be authentic. The reviews can all be competent. &lt;strong&gt;They still describe two different candidate states.&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;               M0 (shared base)
               /             \
       M0 + A: PASS       M0 + B: PASS
               \             /
                A + B: UNTESTED
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two isolated PASS results do not imply a PASS for the combined change.&lt;/p&gt;

&lt;h2&gt;
  
  
  There are two admission questions
&lt;/h2&gt;

&lt;p&gt;Before an agent starts an implementation run, I want to ask:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Can this be checked?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Is there a defined acceptance condition, an independent evidence path, and enough verification capacity to evaluate the result?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Can this land?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Is there a plausible route from the proposed change to an integrated candidate that can be tested and accepted? What shared interfaces or dependencies might move? Who will resolve conflicts? Is integration and acceptance capacity actually available?&lt;/p&gt;

&lt;p&gt;The second question is not simply whether Git currently reports a branch as mergeable. Git detects certain textual conflicts. It does not establish that two independently developed features remain semantically compatible.&lt;/p&gt;

&lt;p&gt;A queue of agent runs is useful when it contains work &lt;strong&gt;waiting to start&lt;/strong&gt;. A queue of completed-but-unverified or completed-but-unintegratable changes ties up review time, forces repeated rebases, and makes activity look like delivery.&lt;/p&gt;

&lt;p&gt;Sometimes the cheapest decision is to defer starting a new write-heavy run until the team can realistically validate and land it. Read-only investigation or other independent work can still proceed.&lt;/p&gt;

&lt;h2&gt;
  
  
  A provisional PASS needs a fresh integration check
&lt;/h2&gt;

&lt;p&gt;Even a good admission decision can become stale while an agent works.&lt;/p&gt;

&lt;p&gt;A base commit moves. A shared API changes. Another writer modifies an overlapping contract. A review covers commit &lt;code&gt;A1&lt;/code&gt;, but the author pushes &lt;code&gt;A2&lt;/code&gt; after the review.&lt;/p&gt;

&lt;p&gt;The historical review of &lt;code&gt;A1&lt;/code&gt; remains useful evidence about &lt;code&gt;A1&lt;/code&gt;. It is not automatically authorization to merge &lt;code&gt;A2&lt;/code&gt;, nor evidence that &lt;code&gt;A1 + B1&lt;/code&gt; works.&lt;/p&gt;

&lt;p&gt;A practical workflow could keep these decisions separate:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Stage&lt;/th&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;Evidence&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Start admission&lt;/td&gt;
&lt;td&gt;Can the task be verified and integrated?&lt;/td&gt;
&lt;td&gt;Base SHA, scope, dependencies, review and integration plan&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Isolated verification&lt;/td&gt;
&lt;td&gt;Did this exact candidate meet its checks?&lt;/td&gt;
&lt;td&gt;Candidate SHA, test outputs, independent review&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Integration verification&lt;/td&gt;
&lt;td&gt;Does the actual combined candidate work?&lt;/td&gt;
&lt;td&gt;Integration SHA, contract and system tests, dependency checks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Final acceptance&lt;/td&gt;
&lt;td&gt;Does this deliver the agreed result?&lt;/td&gt;
&lt;td&gt;Authorized acceptance against the actual integrated artifact&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;If the task includes deployment, production verification is a &lt;strong&gt;further stage&lt;/strong&gt;. Merge success is not proof of a successful release.&lt;/p&gt;

&lt;p&gt;I would preserve old observations while marking stale decisions as stale. That keeps the audit trail intact without recycling an outdated PASS.&lt;/p&gt;

&lt;h2&gt;
  
  
  The failure-injection test I'd run
&lt;/h2&gt;

&lt;p&gt;This is a small experiment a team can reproduce without a fleet of AI agents:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Pin a shared base commit &lt;code&gt;M0&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;On branch A, change an API field and update existing consumers.&lt;/li&gt;
&lt;li&gt;On branch B, add a consumer of the old field, using old fixtures. Make the file changes non-overlapping.&lt;/li&gt;
&lt;li&gt;Run and record the isolated tests and reviews for A and B.&lt;/li&gt;
&lt;li&gt;Construct the exact proposed integration candidate, then run a contract test using the &lt;strong&gt;new producer and new consumer together&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;Confirm that the integration gate refuses acceptance when the contract test fails, even though both isolated branches passed.&lt;/li&gt;
&lt;li&gt;Repair the consumer, rebuild the combined candidate, and verify that a valid integration is allowed to proceed.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I'd add one more negative case: approve a candidate, change its head commit, and verify that an approval bound to the old revision cannot authorize the new one without appropriate fresh checks.&lt;/p&gt;

&lt;p&gt;The positive case matters too. A system that refuses every merge is safe only in a trivial sense. It must also allow a properly checked, compatible combination to land.&lt;/p&gt;

&lt;h2&gt;
  
  
  What should we measure?
&lt;/h2&gt;

&lt;p&gt;More agent runs are not necessarily more delivery.&lt;/p&gt;

&lt;p&gt;The metrics I would watch are &lt;strong&gt;accepted integrated changes&lt;/strong&gt;, time from starting work to acceptance, integration rework, and how much finished-but-unaccepted work accumulates.&lt;/p&gt;

&lt;p&gt;This changes the optimization target. Running five agents simultaneously may increase generated code. If only one change can be independently checked and integrated at a time, the other four may create a larger queue rather than faster delivery.&lt;/p&gt;

&lt;p&gt;Parallelism still has a place. Independent write sets and stable interfaces can support useful concurrency. The constraint is the team's ability to validate the resulting system, not the number of terminals it can open.&lt;/p&gt;

&lt;h2&gt;
  
  
  A note on ARC
&lt;/h2&gt;

&lt;p&gt;I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt;, an open-source checker for declared AI-agent roles, responsibilities, task scope, and related bindings. Its current declaration checks are &lt;strong&gt;not&lt;/strong&gt; an integrated merge queue, runtime enforcement system, or proof that independently reviewed branches compose safely.&lt;/p&gt;

&lt;p&gt;The integration-admission and combined-candidate tests above are &lt;strong&gt;proposed workflow requirements&lt;/strong&gt;, not a claim that ARC already enforces them.&lt;/p&gt;

&lt;p&gt;The question I keep coming back to is simple:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;A verified change is not yet an integrated result.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;How does your team handle this today? Do you limit agent runs by available review and integration capacity, serialize interface-changing work, or allow broad parallelism and resolve conflicts at the merge queue?&lt;/p&gt;

&lt;p&gt;I'd be particularly interested in a case where &lt;strong&gt;two genuinely green, independently reviewed PRs produced a broken combined candidate&lt;/strong&gt;—and the smallest test that would have caught it.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>devops</category>
      <category>discuss</category>
      <category>testing</category>
    </item>
    <item>
      <title>You Revoked Approval. Why Did the AI Agent Still Execute?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Fri, 13 Nov 2026 14:11:53 +0000</pubDate>
      <link>https://dev.to/suirindo/you-revoked-approval-why-did-the-ai-agent-still-execute-kdi</link>
      <guid>https://dev.to/suirindo/you-revoked-approval-why-did-the-ai-agent-still-execute-kdi</guid>
      <description>&lt;p&gt;You pressed &lt;strong&gt;Revoke&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The dashboard said &lt;strong&gt;Revoked&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;A few moments later, your AI agent still changed a record in the external system.&lt;/p&gt;

&lt;p&gt;Did the revocation fail? Did the agent ignore it? Or had the operation already crossed a point where revocation no longer meant cancellation?&lt;/p&gt;

&lt;p&gt;Those are three different failure investigations. A green preflight check cannot answer any of them.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The example below is fictional. It describes a failure mode to test, not an incident we have observed in a production ARC deployment.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  One operation, two competing events
&lt;/h2&gt;

&lt;p&gt;Consider an agent authorized to change a purchase order in an ERP system.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The operator approves an exact task and action.&lt;/li&gt;
&lt;li&gt;The agent passes its role/scope/approval preflight checks.&lt;/li&gt;
&lt;li&gt;The gateway checks the approval and prepares an outbound request.&lt;/li&gt;
&lt;li&gt;The operator revokes the approval.&lt;/li&gt;
&lt;li&gt;The gateway forwards the request anyway.&lt;/li&gt;
&lt;li&gt;The ERP accepts the change.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The important question is not simply &lt;strong&gt;“Was there approval?”&lt;/strong&gt; There was.&lt;/p&gt;

&lt;p&gt;It is: &lt;strong&gt;“At what boundary did that approval become authority to dispatch this exact operation?”&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Here is the vulnerable shape in pseudocode:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;approvalIsCurrent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;operation&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;prepareRequest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;operation&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;forwardToERP&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;operation&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The approval may be current at the check and revoked before &lt;code&gt;forwardToERP&lt;/code&gt; executes. This is a time-of-check/time-of-use race (TOCTOU), a class of weakness documented as &lt;a href="https://cwe.mitre.org/data/definitions/367.html" rel="noopener noreferrer"&gt;CWE-367&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The gap is not unique to AI. Agents make it harder to notice because they can queue, delegate, resume, retry, and hand off operations long after an initial approval.&lt;/p&gt;

&lt;h2&gt;
  
  
  “Revoked before execution” is not yet a specification
&lt;/h2&gt;

&lt;p&gt;Before calling this a security failure, define the revocation contract.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Case A — revocation committed before dispatch admission.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If revocation and admission share an authoritative ordering, the revoked grant must not admit a new protected dispatch. A fail-closed decision and credible no-forward evidence are the expected outcomes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Case B — revocation committed after dispatch admission.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;An operation may already be legitimately admitted or in flight. Revocation does not automatically undo an external write, recall a network packet, or cancel a remote transaction. If the product promises cancellation of not-yet-sent requests, that is an additional contract requiring its own dispatch/cancellation synchronization.&lt;/p&gt;

&lt;p&gt;The UI's “Revoked” label is not enough to decide which case occurred. Neither are timestamps from two hosts whose clocks and authority propagation may differ. The evidence must show the relevant ordering inside the authority domain.&lt;/p&gt;

&lt;p&gt;This distinction also appears in ordinary token systems. &lt;a href="https://www.rfc-editor.org/rfc/rfc7009" rel="noopener noreferrer"&gt;OAuth 2.0 Token Revocation (RFC 7009)&lt;/a&gt; explicitly discusses propagation delay between servers. A successful revocation response is not, by itself, a complete proof of every downstream executor's state.&lt;/p&gt;

&lt;h2&gt;
  
  
  A final recheck helps. It does not make the gap atomic.
&lt;/h2&gt;

&lt;p&gt;One inexpensive improvement is to revalidate immediately before forwarding, against the &lt;strong&gt;current&lt;/strong&gt; authorization state and the &lt;strong&gt;exact&lt;/strong&gt; operation:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Authenticated actor and task/run identity&lt;/li&gt;
&lt;li&gt;Approved action and immutable request digest&lt;/li&gt;
&lt;li&gt;Destination, effective tool/adapter identity, and scope&lt;/li&gt;
&lt;li&gt;Authorization epoch or revision, expiry, and one-use state&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If any required fact is stale, mismatched, or unavailable, refuse the dispatch.&lt;/p&gt;

&lt;p&gt;That catches revocations occurring between the original preflight and the last check.&lt;/p&gt;

&lt;p&gt;But imagine this interleaving:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Gateway                         Authority / operator
-------                         --------------------
Final authorization check: ALLOW
                                Revocation commits
Forward request to ERP
ERP changes record
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Moving a check closer to &lt;code&gt;forward&lt;/code&gt; narrows this window; it does &lt;strong&gt;not&lt;/strong&gt; eliminate the window. Reporting this sequence as a safety PASS would conceal the part of the system that was never protected.&lt;/p&gt;

&lt;p&gt;The stronger design target is a &lt;strong&gt;serialized admission/consumption boundary&lt;/strong&gt;. The authoritative gate orders revocation and admission, checks the current policy/epoch, consumes or reserves the grant, and binds an immutable operation before the local executor can start.&lt;/p&gt;

&lt;p&gt;That gives a precise, limited claim: &lt;strong&gt;revocations ordered before admission deny admission&lt;/strong&gt;. It is not the same as atomic execution at a remote ERP. Dispatch can fail after local consumption; an HTTP error can arrive after a remote effect has committed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Four tests I would require before trusting the claim
&lt;/h2&gt;

&lt;p&gt;You do not need a production ERP to expose the first race. Start with a deterministic scheduler and an instrumented test receiver. Pause execution at named boundaries, then insert a revocation deliberately.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Fixture&lt;/th&gt;
&lt;th&gt;Forced ordering&lt;/th&gt;
&lt;th&gt;Expected finding&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;R0 — live-grant control&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Current grant → admission → send&lt;/td&gt;
&lt;td&gt;One admitted attempt, one observed forward; validates that the test does not pass merely by refusing everything&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;R1 — revoke before admission&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Preflight → revocation committed → authoritative admission&lt;/td&gt;
&lt;td&gt;Admission denied, zero gateway forwards; independent observation required&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;R2 — revoke after final check&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Preflight → last non-atomic check → revocation committed → send&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;Gap exposed&lt;/strong&gt;, not a security PASS, if the request can still be forwarded&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;R3 — revoke after admission&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Serialized admission → revocation committed → local forward&lt;/td&gt;
&lt;td&gt;Record an already-admitted/in-flight attempt; do not claim the revocation cancelled it&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For a system that promises &lt;em&gt;cancellation up to physical send&lt;/em&gt;, add an explicit fifth test for a queued admitted operation revoked before egress. The admission-only contract does not satisfy that stronger promise.&lt;/p&gt;

&lt;p&gt;A small deterministic Node.js fixture can illustrate this ordering in memory. A real acceptance test needs the protected gateway, its authoritative store, and independently observed egress. Simulation is not production verification.&lt;/p&gt;

&lt;h3&gt;
  
  
  The negative proof needs a real observer
&lt;/h3&gt;

&lt;p&gt;A gateway returning &lt;code&gt;DENY&lt;/code&gt; is an assertion about its decision, not proof that no request escaped.&lt;/p&gt;

&lt;p&gt;For the no-forward claim, verify all relevant egress paths with an independent test receiver or an equivalently complete, authenticated observer. Correlate observations with the exact logical operation and attempt. An empty ERP row alone is weaker: a request may have been sent and rejected, delayed, or rolled back.&lt;/p&gt;

&lt;p&gt;Likewise, &lt;code&gt;HTTP 409&lt;/code&gt;, timeout, or a failed client response does not prove that an ERP write did not occur. If the side effect is uncertain, preserve &lt;strong&gt;UNKNOWN&lt;/strong&gt; until reliable reconciliation establishes the outcome. Do not retry solely because the client saw an error.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I want a receipt to prove
&lt;/h2&gt;

&lt;p&gt;A useful receipt separates the facts that tend to get collapsed into one green status:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Evidence&lt;/th&gt;
&lt;th&gt;Question it answers&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Approved subject and request digest&lt;/td&gt;
&lt;td&gt;&lt;em&gt;Which exact operation was approved?&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Authority revision and ordered admission/revocation record&lt;/td&gt;
&lt;td&gt;&lt;em&gt;Was the grant current at the authoritative gate?&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;One-use consumption and dispatch attempt identity&lt;/td&gt;
&lt;td&gt;&lt;em&gt;Which attempt was admitted, and could it be replayed?&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Independent gateway egress observation&lt;/td&gt;
&lt;td&gt;&lt;em&gt;Did a request actually cross the forwarding boundary?&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Authoritative downstream effect/readback&lt;/td&gt;
&lt;td&gt;&lt;em&gt;What changed at the destination, and can it be attributed to this operation?&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Verdict with policy/fixture revision&lt;/td&gt;
&lt;td&gt;&lt;em&gt;Which limited claim do these observations support?&lt;/em&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;An illustrative verdict might read:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;fixture: R2
observation: final_check=ALLOW; revocation_before_forward=true
observation: gateway_forward_count=1 (instrumented receiver)
verdict: KNOWN_GAP — no atomic check-to-forward guarantee
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That is a successful &lt;em&gt;detection test&lt;/em&gt;. It is not a successful &lt;em&gt;security control&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;A different fixture may legitimately return &lt;code&gt;PASS&lt;/code&gt; for “revocation before serialized admission prevents a local start.” It must not silently upgrade that finding into “all in-flight external effects can be cancelled.”&lt;/p&gt;

&lt;h2&gt;
  
  
  Where Agent Role Contracts fits — and where it does not
&lt;/h2&gt;

&lt;p&gt;I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt;, an open-source checker for supplied role, task, scope, review, and action declarations.&lt;/p&gt;

&lt;p&gt;The published ARC &lt;strong&gt;1.0.1&lt;/strong&gt; preflight checks do &lt;strong&gt;not&lt;/strong&gt; authenticate a live operator's revocation, mediate remote network forwarding, or prove ERP side effects. A declaration PASS is not runtime permission. Our &lt;a href="https://github.com/suirindo/agent-role-contracts/blob/main/docs/INTEGRATION.md" rel="noopener noreferrer"&gt;integration documentation&lt;/a&gt; makes that boundary explicit.&lt;/p&gt;

&lt;p&gt;We have &lt;a href="https://github.com/suirindo/agent-role-contracts/blob/main/docs/ASSURANCE.md" rel="noopener noreferrer"&gt;proposed an assurance design&lt;/a&gt; with an ordered dispatch admission point and one-use local consumption, and we track related &lt;a href="https://github.com/suirindo/agent-role-contracts/issues/117" rel="noopener noreferrer"&gt;research questions around gateway and independently observed ERP effects&lt;/a&gt;. These are design and research boundaries, &lt;strong&gt;not&lt;/strong&gt; an end-to-end production revocation guarantee or a completed ERP validation.&lt;/p&gt;

&lt;p&gt;The practical goal is to distinguish a valid declaration, a current runtime admission, an observed forward, and an independently verified external effect — without pretending one receipt proves all four.&lt;/p&gt;

&lt;h2&gt;
  
  
  The question for engineers shipping agents
&lt;/h2&gt;

&lt;p&gt;Suppose the operator revokes an approval after your gateway's final authorization check but before the outbound network call.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What actually stops the send in your system?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A serialized gate? A cancellable dispatch queue? A downstream authorization check? Or nothing, with the race documented as a known gap?&lt;/p&gt;

&lt;p&gt;I'd be interested in concrete answers using this format:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;authority store | admission point | revocation ordering | no-forward observer | in-flight policy | remaining gap&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;A system that shows the gap clearly is easier to improve than one that calls an untested window &lt;strong&gt;PASS&lt;/strong&gt;.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Related discussion: &lt;a href="https://dev.to/suirindo/your-ai-agent-passed-preflight-which-receipt-proves-the-change-landed-3fl8"&gt;Your AI Agent Passed Preflight. Which Receipt Proves the Change Landed?&lt;/a&gt;. Technical background: &lt;a href="https://cwe.mitre.org/data/definitions/367.html" rel="noopener noreferrer"&gt;MITRE CWE-367&lt;/a&gt;, &lt;a href="https://www.rfc-editor.org/rfc/rfc7009" rel="noopener noreferrer"&gt;RFC 7009&lt;/a&gt;, and the &lt;a href="https://github.com/suirindo/agent-role-contracts/issues/117" rel="noopener noreferrer"&gt;ARC research issue&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>agents</category>
      <category>security</category>
      <category>devops</category>
    </item>
    <item>
      <title>How Many Times Did You Step In Before Your AI Agent Finished?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Fri, 13 Nov 2026 03:57:16 +0000</pubDate>
      <link>https://dev.to/suirindo/how-many-times-did-you-step-in-before-your-ai-agent-finished-16do</link>
      <guid>https://dev.to/suirindo/how-many-times-did-you-step-in-before-your-ai-agent-finished-16do</guid>
      <description>&lt;p&gt;The code is written. Before the task is accepted, someone still has to recover a failed run, find missing evidence, or explain the same constraint again.&lt;/p&gt;

&lt;p&gt;How much human work is left?&lt;/p&gt;

&lt;p&gt;Our &lt;a href="https://dev.to/suirindo/your-coding-agent-hit-its-usage-limit-mid-task-who-owns-the-next-move-3o7p"&gt;earlier discussion about agents reaching their usage limits&lt;/a&gt; focused on safe handoffs. I want to extend that discussion to the human effort across an entire task.&lt;/p&gt;

&lt;p&gt;Here, &lt;strong&gt;finished means the result has met the task's agreed acceptance criteria&lt;/strong&gt;, including a check in the intended environment where relevant.&lt;/p&gt;

&lt;p&gt;The following is a proposed measurement method. I am not presenting a measured productivity improvement.&lt;/p&gt;

&lt;h2&gt;
  
  
  Follow one task through acceptance
&lt;/h2&gt;

&lt;p&gt;Imagine an agent updating a small feature. It prepares the code quickly. During the remaining work, a person clarifies a requirement, retrieves a review result, investigates an interrupted operation, and approves the release.&lt;/p&gt;

&lt;p&gt;Those interventions have different purposes. I would record them separately:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Intervention&lt;/th&gt;
&lt;th&gt;Example&lt;/th&gt;
&lt;th&gt;Useful record&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Necessary judgment&lt;/td&gt;
&lt;td&gt;Choose between two valid behaviors or authorize publication.&lt;/td&gt;
&lt;td&gt;The decision, decision holder, and active time.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Missing information&lt;/td&gt;
&lt;td&gt;Repeat a constraint or locate evidence that the successor needs.&lt;/td&gt;
&lt;td&gt;What was missing, where it should live, and active time.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Recovery&lt;/td&gt;
&lt;td&gt;Establish what happened before an interruption and prepare a safe restart.&lt;/td&gt;
&lt;td&gt;The uncertainty, recovery action, and active time.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The goal is to preserve necessary decisions while reducing avoidable recovery and information-search work.&lt;/p&gt;

&lt;p&gt;A lower intervention count needs context. One long investigation may cost more than several quick decisions. A task that silently skips an approval would also make the count misleading.&lt;/p&gt;

&lt;h2&gt;
  
  
  A log small enough to use
&lt;/h2&gt;

&lt;p&gt;For each intervention, record:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Task identifier and the agreed outcome.&lt;/li&gt;
&lt;li&gt;Intervention reason.&lt;/li&gt;
&lt;li&gt;Human active minutes.&lt;/li&gt;
&lt;li&gt;Information or evidence needed to continue.&lt;/li&gt;
&lt;li&gt;The next permitted action and who owns it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;At the task level, record elapsed time, waiting time, final status, and acceptance evidence. Keep waiting time separate from active work. When several people participate, record their active time separately and add it for the task total.&lt;/p&gt;

&lt;p&gt;Statuses should include accepted, still in progress, blocked, and abandoned. Keeping those entries makes unfinished work visible when reviewing the workflow.&lt;/p&gt;

&lt;p&gt;The first improvement might be quite small: retain the exact reviewed revision, preserve the unresolved operation after a timeout, or put the current task constraints somewhere a successor can find them.&lt;/p&gt;

&lt;p&gt;The log should show which change would remove a recurring intervention.&lt;/p&gt;

&lt;h2&gt;
  
  
  Add cost without hiding the unfinished work
&lt;/h2&gt;

&lt;p&gt;For a defined group of comparable tasks and an observation window, I would track three measures together:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Human active time through the end of the observation window.&lt;/li&gt;
&lt;li&gt;Time to acceptance and the number still unresolved.&lt;/li&gt;
&lt;li&gt;Actual cash spending, including retries and recovery.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Record subscription usage separately from additional charges. If a subscription fee is allocated across tasks, state the allocation rule. Keep human time in minutes unless a labor-cost assumption is explicitly supplied.&lt;/p&gt;

&lt;p&gt;For a cost-per-accepted-task calculation, retain the spending on failed and unfinished tasks in the same group. Report the acceptance count alongside the ratio. If nothing has been accepted, report spending and zero acceptances directly.&lt;/p&gt;

&lt;p&gt;This creates a basis for comparing similar work before and after a workflow change. The comparison still needs matching scope and acceptance requirements.&lt;/p&gt;

&lt;h2&gt;
  
  
  Start with the intervention you keep repeating
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;For your last accepted agent-assisted task, how many times did you step in, and which intervention would you remove first?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A rough breakdown of active time and intervention reasons is enough to start. I am particularly interested in repeated explanations, evidence searches, and recovery steps that a better handoff could address.&lt;/p&gt;




&lt;p&gt;Disclosure: I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt; at Netsujo. ARC checks declared responsibilities, scope, reviews, and handoffs. The intervention log proposed here measures the surrounding workflow; any claim of reduced effort would need actual observations.&lt;/p&gt;

</description>
      <category>discuss</category>
      <category>ai</category>
      <category>productivity</category>
      <category>programming</category>
    </item>
    <item>
      <title>All Tests Passed. The Customer Said No. What Happens Next?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Fri, 13 Nov 2026 03:52:29 +0000</pubDate>
      <link>https://dev.to/suirindo/all-tests-passed-the-customer-said-no-what-happens-next-29oi</link>
      <guid>https://dev.to/suirindo/all-tests-passed-the-customer-said-no-what-happens-next-29oi</guid>
      <description>&lt;p&gt;A website passes every technical check. The person authorized to approve publication explicitly rejects the release. A scheduled publishing job is still waiting to run.&lt;/p&gt;

&lt;p&gt;The useful question is what the system does with that refusal.&lt;/p&gt;

&lt;p&gt;In our &lt;a href="https://dev.to/suirindo/two-ai-agents-reviewed-the-same-pr-was-that-actually-independent-5h22"&gt;discussion about independent review&lt;/a&gt;, Suki described a deliverable that passed its checks and was still refused. Suki also raised the decision window: whether the refusal can still stop the workflow when the checks finish.&lt;/p&gt;

&lt;p&gt;I want to make that boundary testable using a fictional website release. &lt;strong&gt;The scenario below is a proposed experiment, with no results claimed.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Fix the agreement before testing
&lt;/h2&gt;

&lt;p&gt;For this exercise, assume the following rules are already agreed:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A named customer representative can accept or refuse publication of this deliverable.&lt;/li&gt;
&lt;li&gt;Each decision identifies the exact release revision.&lt;/li&gt;
&lt;li&gt;An explicit refusal remains effective until the designated authority changes it.&lt;/li&gt;
&lt;li&gt;A scheduled job has no separate authority to override that refusal.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These assumptions define this particular workflow. Other agreements may assign authority differently.&lt;/p&gt;

&lt;p&gt;Now queue a release, record a refusal while it is waiting, and let the scheduled publication time arrive.&lt;/p&gt;

&lt;p&gt;The expected result is specific: the publishing operation is withheld because the current, applicable decision refuses it. The system should preserve the reason and identify what authorized change would make reconsideration possible.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three states worth testing separately
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;State&lt;/th&gt;
&lt;th&gt;Expected handling under this exercise's agreement&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;An explicit refusal is still effective.&lt;/td&gt;
&lt;td&gt;Block publication of the identified revision, including publication from a queued job.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Nobody responds before a decision deadline.&lt;/td&gt;
&lt;td&gt;Apply the separately agreed deadline rule. If that rule is unspecified, keep acceptance unresolved.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;The deliverable changes after refusal or approval.&lt;/td&gt;
&lt;td&gt;Determine which decision applies to the new revision; obtain a current decision where required.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A silence-based automatic approval rule needs its own definition. Its existence alone does not specify how an already recorded refusal ends. The relationship between those rules belongs in the agreement and the test inputs.&lt;/p&gt;

&lt;p&gt;This also gives the experiment a useful normal case: a current approval, for the matching revision, permits the simulated publication when every other required condition is satisfied.&lt;/p&gt;

&lt;h2&gt;
  
  
  A small acceptance record
&lt;/h2&gt;

&lt;p&gt;I would start with six fields:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Decision holder:&lt;/strong&gt; who is authorized to decide, and how that authority is established.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Subject:&lt;/strong&gt; the exact deliverable or revision.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Decision and conditions:&lt;/strong&gt; accepted, refused, or unresolved, including any constraints on the content.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Validity:&lt;/strong&gt; when the decision applies and when it expires, if applicable.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Change authority:&lt;/strong&gt; who can revise the decision or extend the window.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deadline behavior:&lt;/strong&gt; what the workflow does when no applicable decision exists.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For a content constraint, the check must address the actual deliverable. For example, a refusal to publish customer-identifying material needs evaluation against the material being published.&lt;/p&gt;

&lt;p&gt;Recording these fields in a ticket is one part of the design. Enforcement belongs at the point where publication is authorized.&lt;/p&gt;

&lt;h2&gt;
  
  
  Evidence that the refusal took effect
&lt;/h2&gt;

&lt;p&gt;For the queued-job test, I would want a decision record tied to the candidate, evidence that the publishing path withheld the operation for that reason, and independent observation of the test destination.&lt;/p&gt;

&lt;p&gt;A crash or unavailable destination would leave the outcome unverified. The intended observation is that a valid refusal controls an otherwise executable workflow.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;If an authorized person rejects a deliverable, which exact step stops—and what happens when the decision window expires?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A concrete example from your acceptance or publishing process would be useful.&lt;/p&gt;




&lt;p&gt;Disclosure: I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt; at Netsujo. ARC validates supplied declarations. Demonstrating that an authorized refusal actually stops publication requires a separately implemented and tested acceptance mechanism. This article proposes such a test.&lt;/p&gt;

</description>
      <category>discuss</category>
      <category>ai</category>
      <category>agents</category>
      <category>security</category>
    </item>
    <item>
      <title>Would Your AI Reviewer Catch a Bug You Planted?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Thu, 12 Nov 2026 13:40:59 +0000</pubDate>
      <link>https://dev.to/suirindo/would-your-ai-reviewer-catch-a-bug-you-planted-3ge</link>
      <guid>https://dev.to/suirindo/would-your-ai-reviewer-catch-a-bug-you-planted-3ge</guid>
      <description>&lt;p&gt;An AI reviewer approves a change. I want to know what would make that same review process reject it—and whether the rejection would actually stop the merge.&lt;/p&gt;

&lt;p&gt;In our &lt;a href="https://dev.to/suirindo/two-ai-agents-reviewed-the-same-pr-was-that-actually-independent-5h22"&gt;discussion about independent AI review&lt;/a&gt;, &lt;a href="https://dev.to/glenallen/comment/3l081"&gt;Glen Allen suggested seeding defects&lt;/a&gt; to understand which kinds of problems different reviewers detect.&lt;/p&gt;

&lt;p&gt;That suggests a useful next step: give a review workflow a small, controlled opportunity to be wrong.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;This is a proposed test design. I have not run this comparison yet.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Start with a known-good candidate
&lt;/h2&gt;

&lt;p&gt;Consider a fictional task: update a function under &lt;code&gt;src/**&lt;/code&gt;, preserve its documented support for empty input, and review the exact candidate before acceptance.&lt;/p&gt;

&lt;p&gt;The test owner first establishes a baseline that satisfies those requirements. Record its commit or content digest, the task specification, and the evidence available to each reviewer.&lt;/p&gt;

&lt;p&gt;Then create isolated variants, changing one condition at a time:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;Deliberate change&lt;/th&gt;
&lt;th&gt;Expected outcome in this exercise&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Control&lt;/td&gt;
&lt;td&gt;Keep the valid baseline and current evidence.&lt;/td&gt;
&lt;td&gt;Pass the exercise's checks.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Behavior defect&lt;/td&gt;
&lt;td&gt;Reject empty input even though the task requires accepting it.&lt;/td&gt;
&lt;td&gt;Identify the broken requirement and prevent acceptance.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scope defect&lt;/td&gt;
&lt;td&gt;Add a change outside the authorized &lt;code&gt;src/**&lt;/code&gt; write set.&lt;/td&gt;
&lt;td&gt;Identify the scope violation and prevent acceptance.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Evidence defect&lt;/td&gt;
&lt;td&gt;Attach a review for an earlier commit to the current candidate.&lt;/td&gt;
&lt;td&gt;Reject that review as evidence for this candidate; require a current review.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The test owner records expected outcomes before the run. Reviewers receive the requirements and their normal evidence access, with the injected locations and case labels withheld.&lt;/p&gt;

&lt;p&gt;The control matters: it exposes a workflow that rejects every candidate. Passing this exercise establishes only the specified checks; release authorization remains a separate decision.&lt;/p&gt;

&lt;h2&gt;
  
  
  Record the detector and the consequence
&lt;/h2&gt;

&lt;p&gt;For each run, I would record:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The exact candidate and the defect introduced.&lt;/li&gt;
&lt;li&gt;Which component detected it: an AI reviewer, a test, a scope check, or an artifact-binding check.&lt;/li&gt;
&lt;li&gt;The finding and the evidence supporting it.&lt;/li&gt;
&lt;li&gt;The acceptance decision and which subsequent action was blocked.&lt;/li&gt;
&lt;li&gt;Anything the observer could not establish.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This makes the result interpretable. A scope guard catching an unauthorized path is evidence about that guard. A reviewer reproducing the empty-input failure is evidence about that review. An infrastructure timeout leaves detection unestablished.&lt;/p&gt;

&lt;p&gt;Detection and enforcement also need separate observations. A precise warning is useful, but the exercise should show whether the workflow actually withheld acceptance because of that warning.&lt;/p&gt;

&lt;h2&gt;
  
  
  Keep the experiment separate from the candidate
&lt;/h2&gt;

&lt;p&gt;In the &lt;a href="https://dev.to/suirindo/your-coding-agent-hit-its-usage-limit-mid-task-who-owns-the-next-move-3o7p"&gt;earlier discussion about interrupted agents&lt;/a&gt;, Jackson Ly described a reviewer leaving an intentional test mutation in the working tree after reaching a usage limit. That is his reported experience; I have not independently reproduced it.&lt;/p&gt;

&lt;p&gt;I would use disposable copies, keep the baseline unchanged, and verify its identity after the experiment. Any attempted merge or publication would target a test destination.&lt;/p&gt;

&lt;p&gt;Each result supports a claim about that particular case and setup. Broader conclusions about independence or reliability would require additional cases and evidence about shared failure sources.&lt;/p&gt;

&lt;h2&gt;
  
  
  One case to bring to the discussion
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What is one deliberately broken change your review process must catch, and what evidence would show it was caught?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A short example with the expected detector and stopping point would help make the next comparison concrete.&lt;/p&gt;




&lt;p&gt;Disclosure: I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt; at Netsujo. ARC checks the consistency of supplied role, scope, review, and handoff declarations. Actual permissions, reviewer behavior, and acceptance enforcement require controls in the surrounding system. This proposed experiment concerns those controls.&lt;/p&gt;

</description>
      <category>discuss</category>
      <category>ai</category>
      <category>agents</category>
      <category>testing</category>
    </item>
    <item>
      <title>An AI Reviewer Said PASS. Could the Implementer Have Written the Receipt?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Wed, 11 Nov 2026 09:46:47 +0000</pubDate>
      <link>https://dev.to/suirindo/an-ai-reviewer-said-pass-could-the-implementer-have-written-the-receipt-2fm3</link>
      <guid>https://dev.to/suirindo/an-ai-reviewer-said-pass-could-the-implementer-have-written-the-receipt-2fm3</guid>
      <description>&lt;p&gt;&lt;em&gt;An executable synthetic test for a subtle failure in AI-agent review evidence.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Two AI agents can approve the same pull request. They can even attach two different names to their reports. That still leaves a question: &lt;strong&gt;who actually had the authority to produce the review evidence?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;In discussions following my posts on &lt;a href="https://dev.to/suirindo/when-an-ai-agent-says-done-what-exactly-has-finished-5h3p"&gt;what an AI agent means by “done”&lt;/a&gt; and &lt;a href="https://dev.to/suirindo/two-ai-agents-reviewed-the-same-pr-was-that-actually-independent-5h22"&gt;whether two AI reviewers are truly independent&lt;/a&gt;, TokenLat made a useful distinction: establish that the required evidence exists and is bound to the right candidate first; then test whether an unauthorized producer could make that evidence look independent.&lt;/p&gt;

&lt;p&gt;I turned that question into a small, &lt;strong&gt;fully synthetic Node.js fixture&lt;/strong&gt;. It uses no AI model, real repository, or production credentials.&lt;/p&gt;

&lt;h2&gt;
  
  
  The one-field problem
&lt;/h2&gt;

&lt;p&gt;Imagine an evaluation event:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"e-1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"evaluation"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"candidate"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"sha256:synthetic-pr-head-a"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"claimedProducer"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"independent-reviewer"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A naive checker might look for an evaluation event, compare the candidate, see &lt;code&gt;independent-reviewer&lt;/code&gt;, and return &lt;code&gt;PASS&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;But what if the implementer emitted that exact JSON?&lt;/p&gt;

&lt;p&gt;Changing a producer label is not a change of authority. If the same component can write both the alleged independent review and the receipt naming its author, the evidence is self-attestation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Four cases, not just one happy path
&lt;/h2&gt;

&lt;p&gt;I used one fixed candidate identity and three event types: declaration, evaluation, and acceptance. The fixture tests:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;What changes?&lt;/th&gt;
&lt;th&gt;Required verdict&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1. Complete&lt;/td&gt;
&lt;td&gt;All three events exist and have matching, authorized provenance&lt;/td&gt;
&lt;td&gt;&lt;code&gt;PASS&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2. Missing evaluation&lt;/td&gt;
&lt;td&gt;Remove the evaluation event&lt;/td&gt;
&lt;td&gt;&lt;code&gt;PENDING&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3. Wrong candidate&lt;/td&gt;
&lt;td&gt;Bind the evaluation to a different candidate&lt;/td&gt;
&lt;td&gt;&lt;code&gt;DENY&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4. Spoofed reviewer&lt;/td&gt;
&lt;td&gt;The executor emits an evaluation while claiming to be the independent reviewer&lt;/td&gt;
&lt;td&gt;&lt;code&gt;DENY&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The fourth test only makes sense after the first three: otherwise a rejection could simply be caused by a missing or mismatched record, not a failure to verify producer independence.&lt;/p&gt;

&lt;h2&gt;
  
  
  A claimed name is not an authenticated identity
&lt;/h2&gt;

&lt;p&gt;The essential check is not:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="n"&gt;receipt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;claimedProducer&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="s1"&gt;'independent-reviewer'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It is closer to:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="n"&gt;hostRecord&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;authenticatedRole&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="s1"&gt;'independent-reviewer'&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt;
&lt;span class="n"&gt;hostRecord&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;candidate&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="n"&gt;receipt&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;candidate&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt;
&lt;span class="n"&gt;hostRecord&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;eventDigest&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="nf"&gt;digest&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;receipt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The crucial distinction is &lt;strong&gt;where &lt;code&gt;hostRecord&lt;/code&gt; comes from&lt;/strong&gt;. It must come through a separately controlled, authenticated evidence channel. A producer-supplied field named &lt;code&gt;authenticatedRole&lt;/code&gt; would accomplish nothing.&lt;/p&gt;

&lt;p&gt;In my toy fixture, a mock host journal supplies that trusted provenance. The check is deliberately narrow: it demonstrates how a string-only verifier can wrongly accept a forged reviewer label, while a verifier using independent provenance rejects it. &lt;strong&gt;The mock does not provide real authentication, signature verification, filesystem isolation, or runtime enforcement.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I ran the four cases and one additional regression test against the naive checker. All five Node.js tests passed. In the spoof case, the string-only verifier returned &lt;code&gt;PASS&lt;/code&gt; and the provenance-aware toy verifier returned &lt;code&gt;DENY&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this does—and does not—say about ARC
&lt;/h2&gt;

&lt;p&gt;I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt;, which checks the consistency of declared agent roles, task scopes, and review relationships. That is a declaration-level check.&lt;/p&gt;

&lt;p&gt;An ARC declaration by itself does not authenticate the reviewer who produced an event, prevent an executor from writing to a review channel, or prove that the reviewed revision was the one merged. Those responsibilities belong to the host, authorization system, CI, and final acceptance process.&lt;/p&gt;

&lt;p&gt;The same discipline applies to exemptions. “Independent review not required for this task” should be an explicit, authorized scope decision. Silence or a missing event is not an exemption.&lt;/p&gt;

&lt;h2&gt;
  
  
  The question I would ask your agent system
&lt;/h2&gt;

&lt;p&gt;If the implementer changed just the &lt;code&gt;producer&lt;/code&gt; field in its own evaluation receipt to the expected reviewer name, &lt;strong&gt;what independent control would prevent that receipt from authorizing the next step?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I am particularly interested in minimal examples that prove the boundary without adding an entire second orchestration platform.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Disclosure: This is a synthetic fixture and a proposed evidence boundary, not a claim that ARC currently enforces runtime producer authentication. The runnable &lt;code&gt;.mjs&lt;/code&gt; file accompanies the draft and should be published at a stable source URL before referring to it as publicly reproducible.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>testing</category>
      <category>security</category>
      <category>discuss</category>
    </item>
    <item>
      <title>Two AI Agents Reviewed the Same PR. Was That Actually Independent?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Tue, 10 Nov 2026 07:49:20 +0000</pubDate>
      <link>https://dev.to/suirindo/two-ai-agents-reviewed-the-same-pr-was-that-actually-independent-5h22</link>
      <guid>https://dev.to/suirindo/two-ai-agents-reviewed-the-same-pr-was-that-actually-independent-5h22</guid>
      <description>&lt;p&gt;Imagine this workflow:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Agent A writes a patch and opens a pull request.&lt;/li&gt;
&lt;li&gt;Agent B reviews it and says "LGTM."&lt;/li&gt;
&lt;li&gt;CI turns green.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Two agents, one approved PR. Sounds like independent review.&lt;/p&gt;

&lt;p&gt;Now change a few details.&lt;/p&gt;

&lt;p&gt;Both agents can modify the same workspace. The reviewer relies on the author's generated summary. The tested commit might not be the commit that ultimately gets merged.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Do we still have independent verification, or just two agreeing opinions?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I'm increasingly interested in the boundaries that make a review independent, rather than simply the number of models involved.&lt;/p&gt;

&lt;p&gt;For example, a reviewer could use a read-only checkout pinned to an exact commit, inspect the raw diff, and verify which tests actually executed. That gives us evidence the implementing agent cannot simply rewrite.&lt;/p&gt;

&lt;p&gt;But how much separation is enough?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;I'd love to hear how others handle this:&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Is a second AI model with a separate prompt sufficient for low-risk changes?&lt;/li&gt;
&lt;li&gt;Do you require separate workspaces, credentials, and immutable artifacts?&lt;/li&gt;
&lt;li&gt;What evidence makes you trust an AI-generated review?&lt;/li&gt;
&lt;li&gt;Where do you still require human approval?&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Share your actual setup, a failure case, or even a simpler approach that works.&lt;/p&gt;

&lt;p&gt;I'm especially interested in small, reproducible tests that could challenge our assumptions.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Disclosure: I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt;, an open-source checker for declared agent responsibilities and task boundaries. Runtime enforcement and independent review require additional controls. This discussion is about what those controls should actually prove.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>discuss</category>
      <category>testing</category>
      <category>security</category>
    </item>
    <item>
      <title>Your AI Agent Passed Preflight. Which Receipt Proves the Change Landed?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Sun, 08 Nov 2026 03:34:52 +0000</pubDate>
      <link>https://dev.to/suirindo/your-ai-agent-passed-preflight-which-receipt-proves-the-change-landed-3fl8</link>
      <guid>https://dev.to/suirindo/your-ai-agent-passed-preflight-which-receipt-proves-the-change-landed-3fl8</guid>
      <description>&lt;p&gt;A question for engineers building AI agents that actually change things.&lt;/p&gt;

&lt;p&gt;Imagine this fictional workflow:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;An agent is approved to update an ERP record.&lt;/li&gt;
&lt;li&gt;The role, task scope, and review binding pass preflight.&lt;/li&gt;
&lt;li&gt;The tool definition changes before dispatch.&lt;/li&gt;
&lt;li&gt;The gateway forwards the operation, but the client times out.&lt;/li&gt;
&lt;li&gt;Another agent takes over and considers retrying.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;What evidence would you need before trusting the result?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I'm exploring four possible ingredients for a minimal evidence chain:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The exact approved task/action digest&lt;/li&gt;
&lt;li&gt;The authenticated executor and effective tool version&lt;/li&gt;
&lt;li&gt;The logical operation ID and retry identity&lt;/li&gt;
&lt;li&gt;A fresh, authoritative readback of the actual result&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;But every extra receipt has a cost: storage, implementation, maintenance, and operational complexity.&lt;/p&gt;

&lt;p&gt;I maintain &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt;, an open-source project that checks supplied role, scope, review, and task/action declarations offline. Runtime enforcement and independent verification of external effects are separate responsibilities.&lt;/p&gt;

&lt;p&gt;We're investigating these boundaries through a research pilot, with the real-world validation stages explicitly tracked.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My question: What's the smallest evidence chain you would trust in production?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Have you solved this with database logs, idempotency keys, event sourcing, workflow engines, or something simpler?&lt;/p&gt;

&lt;p&gt;A real example would be especially useful:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;system | side effect | failure | trusted evidence | retry rule | cost&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;I'm collecting concrete engineering perspectives in &lt;a href="https://github.com/suirindo/agent-role-contracts/issues/120" rel="noopener noreferrer"&gt;this GitHub discussion&lt;/a&gt; as well.&lt;/p&gt;

</description>
      <category>discuss</category>
      <category>ai</category>
      <category>agents</category>
      <category>devops</category>
    </item>
    <item>
      <title>Your Coding Agent Hit Its Usage Limit Mid-Task. Who Owns the Next Move?</title>
      <dc:creator>asagiman</dc:creator>
      <pubDate>Sat, 07 Nov 2026 01:37:38 +0000</pubDate>
      <link>https://dev.to/suirindo/your-coding-agent-hit-its-usage-limit-mid-task-who-owns-the-next-move-3o7p</link>
      <guid>https://dev.to/suirindo/your-coding-agent-hit-its-usage-limit-mid-task-who-owns-the-next-move-3o7p</guid>
      <description>&lt;p&gt;The coding agent was making progress. Then it hit a usage limit.&lt;/p&gt;

&lt;p&gt;You switch to another agent so the task can continue. Sensible enough.&lt;/p&gt;

&lt;p&gt;But consider the last operation before the interruption:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Did the file write finish? Was the review still current? Does the next agent have the same authority?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;"Continue where it left off" sounds like one instruction. In practice, it can hide several independent decisions.&lt;/p&gt;

&lt;h2&gt;
  
  
  A plausible failure sequence
&lt;/h2&gt;

&lt;p&gt;Consider a fictional coding task:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Agent A is assigned a scoped fix.&lt;/li&gt;
&lt;li&gt;Agent A prepares a change and starts a write.&lt;/li&gt;
&lt;li&gt;Its session ends before a reliable success receipt is recorded.&lt;/li&gt;
&lt;li&gt;Agent B starts with a summary: "The change is probably done. Please continue."&lt;/li&gt;
&lt;li&gt;Agent B retries or moves to review.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;We now have at least three unknowns: whether the original write persisted, whether Agent B can act within the current scope, and whether the review covers the current subject.&lt;/p&gt;

&lt;p&gt;Depending on the tool, blindly retrying an uncertain action can duplicate an effect. Blindly declaring success can leave the work incomplete.&lt;/p&gt;

&lt;p&gt;Neither choice deserves to be made from a conversational summary alone.&lt;/p&gt;

&lt;h2&gt;
  
  
  Five questions at the handoff
&lt;/h2&gt;

&lt;p&gt;I would want the handoff record to answer these:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;Minimum useful record&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;What is the task now?&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Current task ID, exact objective, allowed scope, and acceptance criteria&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Who takes over?&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Declared role, plus authenticated runtime identity and effective permissions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;What was last confirmed?&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Exact commit/artifact or observable tool receipt&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;What remains unknown?&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;A named unresolved item, for example "whether the last write persisted"&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;What is permitted next?&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Fresh authority check, current review/approval binding, and retry decision if needed&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That last row is easy to skip. Recording uncertainty and authorizing a retry are different operations.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a declaration checker can catch
&lt;/h2&gt;

&lt;p&gt;In &lt;a href="https://github.com/suirindo/agent-role-contracts" rel="noopener noreferrer"&gt;Agent Role Contracts (ARC)&lt;/a&gt;, we published a &lt;a href="https://github.com/suirindo/agent-role-contracts/blob/main/docs/AGENT-SWITCH-REVALIDATION.md" rel="noopener noreferrer"&gt;guide to rechecking tasks when agents switch&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;It distinguishes five cases with concrete checks:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Same declared role, new runtime instance:&lt;/strong&gt; recheck the current task and scope. A change of instance alone does not establish a new declared role.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Different declared role:&lt;/strong&gt; check whether the handoff follows the allowed route.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Task scope changed:&lt;/strong&gt; recheck it against the current role authority; an out-of-scope request produces &lt;code&gt;TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Task or action changed after review:&lt;/strong&gt; the old binding can fail with &lt;code&gt;G1_SUBJECT_MISMATCH&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Previous outcome uncertain:&lt;/strong&gt; record a blocked handoff with the unknown explicitly unresolved; claiming completion with unresolved work is rejected as &lt;code&gt;HANDOFF_COMPLETE_UNRESOLVED&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The guide is backed by fictional fixture tests. Those tests establish declaration behavior under the tested cases. They do not observe a real filesystem write or provider delivery.&lt;/p&gt;

&lt;p&gt;For a repository that already has ARC declarations, the sequence looks like this (paths are examples; use your own task files):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Record and validate the declared handoff.&lt;/span&gt;
node bin/agent-role-contracts.mjs handoff &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bundle&lt;/span&gt; .arc/bundle.json &lt;span class="nt"&gt;--task&lt;/span&gt; .arc/task.json &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--handoff&lt;/span&gt; .arc/handoff.json &lt;span class="nt"&gt;--format&lt;/span&gt; text

&lt;span class="c"&gt;# Recheck the task before another agent works on it.&lt;/span&gt;
node bin/agent-role-contracts.mjs explain &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--bundle&lt;/span&gt; .arc/bundle.json &lt;span class="nt"&gt;--task&lt;/span&gt; .arc/task.json &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--format&lt;/span&gt; text
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If the task, policy, or action changed, recompute its subject and bind any required new review and approval. The runtime must separately authenticate Agent B, verify actual permissions, inspect real effects, and decide whether a retry is safe.&lt;/p&gt;

&lt;h2&gt;
  
  
  The rule I'd like teams to debate
&lt;/h2&gt;

&lt;p&gt;My candidate rule is:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Every interrupted agent task should preserve the last confirmed effect and a list of unknowns. A successor should recheck current authority before any consequential action. Unknown outcomes should remain explicit until evidence resolves them.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This adds some bookkeeping. It may also prevent the much more expensive work of investigating an accidental duplicate or a falsely completed task.&lt;/p&gt;

&lt;p&gt;Some runtimes already offer durable, idempotent operations and strong recovery records. For them, this may be unnecessary duplication. That comparison matters.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When a coding agent hits a usage limit or crashes halfway through a task, what do you let the next agent do automatically?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Resume reading? Edit code? Re-run a tool call? Retry an external side effect? Where do you require human review or an external receipt?&lt;/p&gt;

&lt;p&gt;I'd like to hear a concrete rule that has survived a real interruption.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Disclosure: I maintain ARC at Netsujo. The handoff cases discussed here are declaration-level test scenarios; actual execution effects and runtime identity require separate verification. &lt;a href="https://github.com/suirindo/agent-role-contracts/blob/main/docs/AGENT-SWITCH-REVALIDATION.md" rel="noopener noreferrer"&gt;Agent-switch guide&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>productivity</category>
      <category>discuss</category>
    </item>
  </channel>
</rss>
