<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Paul Redman</title>
    <description>The latest articles on DEV Community by Paul Redman (@paulredman).</description>
    <link>https://dev.to/paulredman</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4201936%2Fb334bc4e-a151-4005-82d8-2d6299af7a92.jpg</url>
      <title>DEV Community: Paul Redman</title>
      <link>https://dev.to/paulredman</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/paulredman"/>
    <language>en</language>
    <item>
      <title>Slipmark for Codex: bind every verdict to the instructions it was given</title>
      <dc:creator>Paul Redman</dc:creator>
      <pubDate>Fri, 30 Oct 2026 08:59:56 +0000</pubDate>
      <link>https://dev.to/paulredman/slipmark-for-codex-bind-every-verdict-to-the-instructions-it-was-given-2mc6</link>
      <guid>https://dev.to/paulredman/slipmark-for-codex-bind-every-verdict-to-the-instructions-it-was-given-2mc6</guid>
      <description>&lt;p&gt;Codex reads AGENTS.md from the repo it works in. So does every other coding agent now. That makes AGENTS.md part of the input to every review, and the 2026 attacks on coding agents went for exactly that: Ghostcommit, Deadbugz, poisoned skills.&lt;/p&gt;

&lt;p&gt;Slipmark 0.6 adds an MCP server. Three lines in &lt;code&gt;~/.codex/config.toml&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="k"&gt;[&lt;/span&gt;&lt;span class="n"&gt;mcp_servers&lt;/span&gt;&lt;span class="k"&gt;.&lt;/span&gt;&lt;span class="n"&gt;slipmark&lt;/span&gt;&lt;span class="k"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;command&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s"&gt;"npx"&lt;/span&gt;
&lt;span class="n"&gt;args&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"-y"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"slipmark@0.6.1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"mcp"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Codex then has two tools. &lt;code&gt;slipmark_attest&lt;/code&gt; records its verdict with the SHA-256 of each file it read, and of AGENTS.md, skills and MCP config. &lt;code&gt;slipmark_verify&lt;/code&gt; checks every verdict against the files as they are now.&lt;/p&gt;

&lt;p&gt;In CI, the Slipmark Action refuses a pull request when:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;a file changed after the verdict that covered it&lt;/li&gt;
&lt;li&gt;the agent's instructions changed after the verdict&lt;/li&gt;
&lt;li&gt;AGENTS.md changed without a person approving it&lt;/li&gt;
&lt;li&gt;changed instructions hide text from reviewers&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The MCP server is deliberately narrow. An agent can't record a human approval through it, can't run commands, and can't overwrite another verdict.&lt;/p&gt;

&lt;p&gt;Demo, each attack as an open pull request, plus one verdict recorded through the MCP server: &lt;a href="https://github.com/founder-ship-it/slipmark-context-demo" rel="noopener noreferrer"&gt;https://github.com/founder-ship-it/slipmark-context-demo&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/founder-ship-it/Slipmark" rel="noopener noreferrer"&gt;https://github.com/founder-ship-it/Slipmark&lt;/a&gt;&lt;/p&gt;

</description>
      <category>openai</category>
      <category>security</category>
      <category>ai</category>
      <category>mcp</category>
    </item>
    <item>
      <title>Your AI reviewer passed the code. Who wrote its instructions?</title>
      <dc:creator>Paul Redman</dc:creator>
      <pubDate>Thu, 29 Oct 2026 18:11:12 +0000</pubDate>
      <link>https://dev.to/paulredman/your-ai-reviewer-passed-the-code-who-wrote-its-instructions-1ofn</link>
      <guid>https://dev.to/paulredman/your-ai-reviewer-passed-the-code-who-wrote-its-instructions-1ofn</guid>
      <description>&lt;p&gt;This week GitHub launched a deeper Copilot code review tier. Since July it has read your AGENTS.md, skills and MCP servers. Every coding agent does the same now.&lt;/p&gt;

&lt;p&gt;So here's the question I couldn't shake.&lt;/p&gt;

&lt;p&gt;If a pull request changes AGENTS.md, and an agent reviews that pull request, whose instructions is it following?&lt;/p&gt;

&lt;p&gt;The PR's.&lt;/p&gt;

&lt;h2&gt;
  
  
  The attacks this year went for the instructions, not the code
&lt;/h2&gt;

&lt;p&gt;Look at what actually happened in 2026.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Ghostcommit.&lt;/strong&gt; A PR added an AGENTS.md line pointing at an image. The image had text in it a human wouldn't notice. A later, unrelated task leaked &lt;code&gt;.env&lt;/code&gt;. The code reviewers saw nothing, because there was nothing in the code to see.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Deadbugz.&lt;/strong&gt; One account opened 23 PRs in 74 minutes adding MCP server configs. The tool definitions turned malicious after three calls. The config looked fine on the day it was approved.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hidden skills.&lt;/strong&gt; VirusTotal looked at 35,878 agent skills and flagged 6,637. Most had no malicious code at all. Just instructions, hidden in HTML comments, invisible Unicode, or hundreds of blank lines.&lt;/p&gt;

&lt;p&gt;None of these touched the code under review. They changed what the reviewer was told.&lt;/p&gt;

&lt;h2&gt;
  
  
  A verdict is only as good as what it was conditioned on
&lt;/h2&gt;

&lt;p&gt;I build Slipmark, an open-source gate that binds an agent's verdict to a SHA-256 of the exact files it read. If the files change after the verdict, the gate refuses.&lt;/p&gt;

&lt;p&gt;That covers the code. It didn't cover the reviewer.&lt;/p&gt;

&lt;p&gt;v0.5.0 does. Add one line to &lt;code&gt;.slipmark/policy.json&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"schema"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"slipmark/policy@1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"context"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Every agent verdict records the hashes of AGENTS.md, CLAUDE.md, skills, hooks, Cursor rules and MCP configs.&lt;/li&gt;
&lt;li&gt;Change any of them and earlier verdicts stop counting.&lt;/li&gt;
&lt;li&gt;A change to them needs a person's approval.&lt;/li&gt;
&lt;li&gt;Changed instructions must be plain Markdown that GitHub shows in full. Images, HTML, comments and invisible characters are refused. In Actions, every word in the file has to appear on GitHub's own rendered page.&lt;/li&gt;
&lt;li&gt;The policy is read from the branch you merge into, so a PR can't switch any of this off for itself.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  See it refuse them
&lt;/h2&gt;

&lt;p&gt;I put each attack in a public repo as an open pull request. Every attacking PR also carries a passing verdict from an agent, the way a real attacker's would.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/founder-ship-it/slipmark-context-demo" rel="noopener noreferrer"&gt;https://github.com/founder-ship-it/slipmark-context-demo&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Five attacks, five red crosses. A normal code change and an approved AGENTS.md change both go green.&lt;/p&gt;

&lt;p&gt;The Ghostcommit PR gets caught three ways. The image is refused. The change isn't approved. And GitHub's renderer check notices that words in the file never make it onto the page.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it doesn't do
&lt;/h2&gt;

&lt;p&gt;It hashes MCP configs and pinned versions, not the tool definitions a server returns at run time. Files your instructions point to only count if you add them to the policy. An approval marked "human" is a claim unless you pair it with CODEOWNERS or signed approvals. All of this is in the docs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx slipmark@0.5.0 init
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then add &lt;code&gt;"context": {}&lt;/code&gt; to the policy. Apache-2.0.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/founder-ship-it/Slipmark" rel="noopener noreferrer"&gt;https://github.com/founder-ship-it/Slipmark&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The base-branch policy fix came from a comment on my last post here. Kent Bodrov asked what stops a PR loosening the rules it's judged by. Nothing did. Now something does. More of that please.&lt;/p&gt;

</description>
      <category>security</category>
      <category>ai</category>
      <category>github</category>
      <category>devops</category>
    </item>
    <item>
      <title>Our AI verification job passed for months. It was checking the wrong branch.</title>
      <dc:creator>Paul Redman</dc:creator>
      <pubDate>Fri, 23 Oct 2026 07:35:19 +0000</pubDate>
      <link>https://dev.to/paulredman/our-ai-verification-job-passed-for-months-it-was-checking-the-wrong-branch-l6p</link>
      <guid>https://dev.to/paulredman/our-ai-verification-job-passed-for-months-it-was-checking-the-wrong-branch-l6p</guid>
      <description>&lt;p&gt;I run a nightly pipeline of more than 20 AI agents: research, architecture, security audit, build, QA. Every morning there's a report of what passed.&lt;/p&gt;

&lt;p&gt;For months, everything passed.&lt;/p&gt;

&lt;p&gt;Then I found out why. The final verification job was checking out the wrong branch. It re-tested a tree that had already passed, so it could never fail. Nothing crashed. Nothing errored. It just quietly approved the wrong thing, every night.&lt;/p&gt;

&lt;h2&gt;
  
  
  The failure nobody alerts on
&lt;/h2&gt;

&lt;p&gt;Monitoring is built for things that break. A job times out, a test goes red, a deploy falls over, and someone gets paged.&lt;/p&gt;

&lt;p&gt;This was different. The job ran, finished and reported green. Its only problem was that the green tick was about different code from the code we were shipping.&lt;/p&gt;

&lt;p&gt;Agents make this kind of failure much more common:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;An agent reviews a file and says "pass". Ten minutes later the file changes. The pass is still sitting there.&lt;/li&gt;
&lt;li&gt;An agent says it checked the code against the spec. Which version of the spec?&lt;/li&gt;
&lt;li&gt;An agent never produces a verdict at all, and nothing notices the gap.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In every case the verdict looks fine. It's just about the wrong bytes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Bind the verdict to what was read
&lt;/h2&gt;

&lt;p&gt;The fix I ended up with is boring on purpose. When an agent gives a verdict, it records the SHA-256 of every file it based that verdict on:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"schema"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"slipmark/attestation@1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"by"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"security-audit"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"verdict"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"cleared"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"at"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"2026-10-17T09:00:00.000Z"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"subjects"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"src/auth/session.ts"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"sha256"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"9f3c…"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Before anything merges, a gate re-hashes those files and compares:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Slipmark verify: 1 verified · 1 refused

  ✓ verified        review-agent verified examples/attestation/brief.md
  ✗ hash_mismatch   security-audit-agent gave "cleared" for a different version of
                    examples/attestation/spec.md. The file changed after the verdict.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If a file changed after the verdict, the verdict stops counting. If a file you require was never attested, the gate refuses. If the agent's own verdict wasn't a pass, it refuses. An empty gate is a refusal, never a silent pass.&lt;/p&gt;

&lt;p&gt;No model calls. Nothing to tune. Same inputs, same answer, every time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Check content, not names
&lt;/h2&gt;

&lt;p&gt;This is the part that would have caught the original bug. The gate checks content, not names. Branch names, job names and timestamps can all be wrong without anyone noticing. A hash can't. Run the check on the exact tree you're about to merge, and a verdict produced anywhere else simply won't match.&lt;/p&gt;

&lt;p&gt;When I pointed it at our 23 open agent pull requests, it refused all 23.&lt;/p&gt;

&lt;p&gt;That was a bad morning. It was also the most useful thing the pipeline had told me in months.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it doesn't do
&lt;/h2&gt;

&lt;p&gt;It doesn't judge whether the agent was right. A bad review of the right file still passes. All it proves is that the verdict is about the code you're actually shipping. You still need good reviewers, human or otherwise.&lt;/p&gt;

&lt;p&gt;It's also strict. Run a formatter after the review and the verdict stops counting, the same as any other change. For now, run formatters before the agent attests (first in your pre-commit hooks) and most of that goes away. Formatting-tolerant hashing is next, built on one rule: a false mismatch is annoying, but a false match would make the whole thing worthless.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;I've open-sourced it as Slipmark, under Apache-2.0.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx slipmark init
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That adds an attestation folder, a GitHub workflow that starts in report-only mode, and a short block in &lt;code&gt;AGENTS.md&lt;/code&gt; (and &lt;code&gt;CLAUDE.md&lt;/code&gt;, Cursor rules or Copilot instructions if you have them) telling your coding agents how to attest. There's also a Claude Code plugin that asks Claude to re-check a file it signed off when that file changes.&lt;/p&gt;

&lt;p&gt;As a GitHub Action:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/checkout@v4&lt;/span&gt;
&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;founder-ship-it/Slipmark@v0.4.0&lt;/span&gt;
  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;require&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;|&lt;/span&gt;
      &lt;span class="s"&gt;specs/your-spec.md&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The gate is free and needs no account. There's an optional hosted ledger if you need the record as well as the build result.&lt;/p&gt;

&lt;p&gt;Code and docs: &lt;a href="https://github.com/founder-ship-it/Slipmark" rel="noopener noreferrer"&gt;github.com/founder-ship-it/Slipmark&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If you're letting agents sign off work in CI, I'd like to know how you check their verdicts are about the right code, and where this breaks for you.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>devops</category>
      <category>githubactions</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
