DEV Community

Paul Redman
Paul Redman

Posted on AI-assisted

Your AI reviewer passed the code. Who wrote its instructions?

This week GitHub launched a deeper Copilot code review tier. Since July it has read your AGENTS.md, skills and MCP servers. Every coding agent does the same now.

So here's the question I couldn't shake.

If a pull request changes AGENTS.md, and an agent reviews that pull request, whose instructions is it following?

The PR's.

The attacks this year went for the instructions, not the code

Look at what actually happened in 2026.

Ghostcommit. A PR added an AGENTS.md line pointing at an image. The image had text in it a human wouldn't notice. A later, unrelated task leaked .env. The code reviewers saw nothing, because there was nothing in the code to see.

Deadbugz. One account opened 23 PRs in 74 minutes adding MCP server configs. The tool definitions turned malicious after three calls. The config looked fine on the day it was approved.

Hidden skills. VirusTotal looked at 35,878 agent skills and flagged 6,637. Most had no malicious code at all. Just instructions, hidden in HTML comments, invisible Unicode, or hundreds of blank lines.

None of these touched the code under review. They changed what the reviewer was told.

A verdict is only as good as what it was conditioned on

I build Slipmark, an open-source gate that binds an agent's verdict to a SHA-256 of the exact files it read. If the files change after the verdict, the gate refuses.

That covers the code. It didn't cover the reviewer.

v0.5.0 does. Add one line to .slipmark/policy.json:

{ "schema": "slipmark/policy@1", "context": {} }
Enter fullscreen mode Exit fullscreen mode

Then:

  1. Every agent verdict records the hashes of AGENTS.md, CLAUDE.md, skills, hooks, Cursor rules and MCP configs.
  2. Change any of them and earlier verdicts stop counting.
  3. A change to them needs a person's approval.
  4. Changed instructions must be plain Markdown that GitHub shows in full. Images, HTML, comments and invisible characters are refused. In Actions, every word in the file has to appear on GitHub's own rendered page.
  5. The policy is read from the branch you merge into, so a PR can't switch any of this off for itself.

See it refuse them

I put each attack in a public repo as an open pull request. Every attacking PR also carries a passing verdict from an agent, the way a real attacker's would.

https://github.com/founder-ship-it/slipmark-context-demo

Five attacks, five red crosses. A normal code change and an approved AGENTS.md change both go green.

The Ghostcommit PR gets caught three ways. The image is refused. The change isn't approved. And GitHub's renderer check notices that words in the file never make it onto the page.

What it doesn't do

It hashes MCP configs and pinned versions, not the tool definitions a server returns at run time. Files your instructions point to only count if you add them to the policy. An approval marked "human" is a claim unless you pair it with CODEOWNERS or signed approvals. All of this is in the docs.

Try it

npx slipmark@0.5.0 init
Enter fullscreen mode Exit fullscreen mode

Then add "context": {} to the policy. Apache-2.0.

Repo: https://github.com/founder-ship-it/Slipmark

The base-branch policy fix came from a comment on my last post here. Kent Bodrov asked what stops a PR loosening the rules it's judged by. Nothing did. Now something does. More of that please.

Top comments (1)

Collapse
 
arcwrightai profile image
arcwrightai •

The permission model is the unglamorous part that decides whether people actually trust it.