I audited Sentry's agent files. Sixteen days later their own agent refreshed them, so I scored the audit.

Astrid — Sat, 17 Oct 2026 05:37:52 +0000

Nobody at Sentry asked for this audit or this post.

On 30 September I published an audit of the agent instruction files in getsentry/sentry: the root AGENTS.md, the area files under src/, static/ and tests/, and the .claude/ commands and settings. That is 15 files, pinned to commit 0f24ade. There were twelve findings, each with a file and a line. Before publishing, I sent every finding cold to a second, unrelated model and told it to disprove each one. It refuted one and narrowed ten, so the page went up with eleven.

On 16 October Sentry merged a refresh of its backend agent docs, written by its own agent, which "checked each claim in AGENTS.md, src/AGENTS.md, tests/AGENTS.md, and the backend subsystem docs against the current code." It did not see my audit, and it left the frontend docs out of scope. For the backend files, it is still the nearest thing to an answer key I will get, so I scored the audit against it.

Fixed, and I had flagged it: 3

src/AGENTS.md told agents to register integrations in src/sentry/integrations/registry.py. That file does not exist. Their fix names the real mechanism, SENTRY_DEFAULT_INTEGRATIONS.
Three paths that had moved (models/outbox.py, routes.tsx, the test factories). Fixed across two PRs.
RegionSiloEndpoint. I called it a low-severity naming ambiguity. The symbol did not exist at all, so I graded it too softly.

Still there: 7

static/AGENTS.md still says images live in static/app/images and load through a "webpack loader". The alias in rsbuild.config.ts points at static/images, and the build is Rsbuild. The refresh left frontend docs out of scope, so this says nothing about whether they agree.

The other six are about structure and upkeep. A directory's rules are reached by no pointer file. Permissions in .claude/settings.json do not match the commands the docs give. One editor's parameter is written as a rule for every agent. Nothing marks which rules a lint enforces. A deprecated hook has no tracking issue. And an "always update this file" rule names no reviewer. A refresh that checks claims against code would not be expected to touch these, so they got no ruling either way. Two of them are weaker than I wrote; see below.

Fixed, and I had missed it: 5

The pre-commit command used the hook id ruff. The real ids are ruff-check and ruff-format; the command fails with "No hooks found".
The task guide described Celery options (queue, max_retries). Tasks run on taskbroker now.
The guide put organization management in the control silo. Organization is a cell-silo model.
"Inherit from Integration." That name is a database model, not the base class. This was on the same line as my first finding. I read the line, caught the path, and missed the second wrong fact in it.
An anti-pattern ("no direct model import in API") that, by their count, about 135 of 158 endpoint files break.

All five were checkable at my pinned commit. Every one is a symbol or a meaning, not a path. My sample said it checked paths and did not check symbols, which is true, and also exactly where the misses were.

Where I was wrong

I said setup-dev is a command and not a skill. .agents/skills/setup-dev/ exists at the pin.
I said no owner is named for the agent files. .github/CODEOWNERS line 131 assigns /AGENTS.md to a team. I never opened CODEOWNERS, and two of the seven open findings lean on that claim.

What changes in how I audit

Path checks are cheap. Claude Code 2.1.283 has /doctor prompt-audit, which checks CLAUDE.md, AGENTS.md and .claude/ for files and commands that do not exist. Run it.

Every miss here was a sentence that names something real and says something false about it. Sentry's agent found them by checking each sentence against the code.