Claude Code shipped AGENTS.md support and then, under common configurations, silently never read the file. Engineers on telemetry-off setups and third-party gateways found their project instructions were not being loaded at all, with no warning printed anywhere. A one-line canary test is what exposes it. That same test is the fastest way to check whether any AI code reviewer on your team is actually enforcing your coding rules.
The case: a local rules file gated behind a remote switch
The detail that matters here is not that a feature was behind a feature flag. It is that the gate could never turn on for the people who most want the feature. Anthropic's GitHub issue #95690 documents it directly: Claude Code 2.1.277 implemented AGENTS.md as a built-in plugin that is off by default and only loads when a remote flag called tengu_agents_md_mod resolves true. If telemetry is disabled, or you run through Bedrock, Vertex, or a third-party gateway, that flag can never be fetched, and the file is skipped silently.
Przemek Szypowicz reproduced and measured it on his blog, published the same day the issue is open. His setup was a directory containing only an AGENTS.md with a canary word in it, then a prompt asking for the word. With DISABLE_TELEMETRY=1 or CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 set, the model answered that it had no project instructions. The interesting part is that setting either variable to 0 did not help: the docs treat any non-empty value as disabling, so even an explicit zero keeps the block in place.
None of those runs printed a warning. The session started, the model answered without the project rules, and nothing mentioned that a file had been skipped. A workaround exists, a one-line CLAUDE.md containing @AGENTS.md, which loads the file without waiting on the remote flag. But if you had not gone digging you would simply conclude that Claude Code ignores your instructions.
Why this matters for AI code review, not just coding agents
The same silent gap shows up on the review side. Teams set up an AI reviewer, configure a rules file, and assume the tool loads it. When a review comes back that plainly violates a stated team convention, the usual diagnosis is that the model is bad at following instructions. The much more common failure is that the instructions never reached the model in the first place.
That is a dangerous conclusion to draw in either direction. If the model actually did receive your rules and ignored them, the fix is a better prompt or a better model. If it never received them, then no amount of prompt tuning does anything. A team that does not verify the load is building a feature on an assumption that can quietly be false, and every tool that advertises custom rules is vulnerable to it in some form, whether through a default-off plugin, a rollout gate, or an environment where fetching settings is blocked.
There is a second reason this is specifically an AI reviewer problem and not just a coding agent problem. A reviewer is only as useful as the standards it applies, and its whole job is to check other people's changes against those standards. When the rules do not load, the reviewer does not fail loudly. It produces confident, irrelevant feedback that your team reads as correct but that reflects no knowledge of your conventions. That is worse than a reviewer that returns nothing, because nothing at least forces a human to look. Confident wrong-context feedback teaches reviewers to skim.
The failure surface is wider than a single feature flag too. Rules load through different mechanisms depending on the tool and the environment: a remote fetch, a per-repo settings file, an env block, a user-level config. Each one is a place where the load can silently fail. CI is the highest-risk case because it is often configured to disable telemetry and nonessential traffic by policy, exactly the setup that broke the Claude Code loader. A reviewer running in CI is not the same reviewer running from a developer's laptop.
The canary test you should run before trusting custom rules
The verification method is the same one Szypowicz used, and it generalizes to any reviewer. Put a unique canary token into your rules file, then submit a change that your rules specifically forbid, and ask the reviewer directly whether it is following the rules. If it cannot name the canary, or it approves a change you explicitly banned, the rules are not loaded.
A practical version for a PR review tool: add one line to your rules file like The canary word for this repo is PERIWINKLE; reject any change that mentions it. Then open a small PR that contains the word. Review the reviewer's output. If it flags the change, your rules are reaching the model. If it does not, stop tuning prompts and start checking the load path.
Run the test in the environment where the reviewer actually runs, not the one where it happens to work. If your reviews happen in CI, the canary PR goes through CI, because that is where the fetch is most likely to be blocked. If reviewers run on developers' machines, run it there. The whole point of the Claude Code case is that a feature can behave differently in a headless first-run session than in a warmed-up interactive shell.
Do not assume the documented behavior is what runs in your environment either. Claude Code shows a feature that is announced as present and default-off behind a switch that cannot resolve. Headless runs, CI runners, and anything that disables feature-flag traffic are precisely the environments that hit this, because the flag fetch is skipped.
What the fix looks like on the tool side
The uncomfortable part of Szypowicz's writeup is that the model could not tell you what happened. His request was for a tool to show which instruction files were loaded, their priority, and the effective context used, so agent behavior becomes predictable and debuggable. That is the right ask, and reviewers differ meaningfully here.
Some tools answer it by construction. Kodus, for example, is built on what its docs describe as true visibility: you can see every file read and every decision considered, and team conventions are enforced as explicit Kody Rules plus Memories rather than left to prompt drift. When the loaded rules are inspectable, the canary test becomes a one-time sanity check instead of a standing mystery. Other tools treat rules as black-box context and give you no way to tell whether the file was even read, which is the failure mode this entire case study is about.
Whatever tool you pick, run the canary test once at adoption and again whenever you change the rules file or the hosting setup. It costs nothing, and it is the only way to know whether the rule the vendor says is loaded is actually reachable in your environment.
Top comments (0)