DEV Community

Cover image for Claude Code 2.1.283 ships a config linter: what /doctor prompt-audit changes for your CLAUDE.md
Piekwerk
Piekwerk

Posted on

Claude Code 2.1.283 ships a config linter: what /doctor prompt-audit changes for your CLAUDE.md

Claude Code 2.1.283 landed on September 25, and the line that matters for anyone maintaining agent configs is easy to miss in a 90-item changelog: Added /doctor prompt-audit (also /checkup prompt-audit) to audit your CLAUDE.md files, skills, agents and commands for prompting patterns written for older models. A first-party linter for instruction files is new. /context showed what loads, /permissions showed what's allowed, but nothing checked whether the words themselves still work on the models that read them today.

I maintain 12 agent config kits, one per stack. When a vendor ships a config auditor, that's directly aimed at my world. Here's what the feature does, what it won't catch, and what I found when I ran its cheapest checks by hand on my own files.

What the changelog actually says

Three entries in 2.1.283 touch config quality:

  1. /doctor prompt-audit audits CLAUDE.md files, skills, agents and commands for "prompting patterns written for older models".
  2. An improvement to that same audit: "stale paths, stale commands and contradicting instruction files now lead the report, and thinking keywords that Claude Code documents are kept".
  3. deniedModels, a managed setting to block specific models even when availableModels allows them.

The middle one deserves a second read. It tells you the report is ordered by what actually breaks agents: paths that no longer resolve, commands that no longer exist, and two files that say opposite things. Those are mechanical failures. The "older models" patterns are softer: prompt furniture like restating the model's knowledge cutoff, or thinking keywords that the docs now define natively, get flagged but kept when they're documented behavior.

There's also a quiet governance change in the same release worth knowing about: availableModelsMatch: "exact" makes an availableModels entry allow only the named model version, so new releases stay blocked until explicitly listed. Together with deniedModels, platform teams can now pin the model era an agent runs on. Your config files stop being the only layer controlling behavior.

Why this feature exists

The prompt-audit is the visible edge of a real problem: configs written for 2024-era models actively hurt on current ones. Here are the failure classes I see in the wild, all of them born from good intentions:

  • Model-era prompts: "Think step by step" and friends were useful nudges once. Current Claude models have documented thinking controls, and version-pinned configs that hard-code old incantations now add tokens without adding behavior.
  • Stale paths and commands: a config that says "run the tests in src/__tests__/" stops being guidance the day the repo moves to tests/. The agent still tries the path, fails, and burns a tool call discovering what the config should have known.
  • Contradictions between files: user-level CLAUDE.md says "always ask before running migrations", project-level says "apply migrations automatically". Both load. The model picks one, and not always the one you meant.

Config drift is not a hypothetical. I wrote about how configs drift and how to catch it after finding it in my own files, and the cheap heuristic there still works: if you cannot say when a line was last verified against a live agent, it's drift.

The measurement: 24 files, two path warnings, one lesson

I can't run /doctor prompt-audit itself yet; it needs 2.1.283+ and I'm not upgrading my daily driver the day after release. So I ran its cheapest check by hand: extract every backtick-quoted path from my kits, check it resolves in its repo, and count what doesn't.

Scanning 24 instruction files across 12 kits (an AGENTS.md and CLAUDE.md pair each, 1,354 lines total) found exactly two unresolvable path references. Then I looked at both.

The first, in my node-express CLAUDE.md: Start from src/app.ts (or equivalent) to see middleware order. The kit directory contains no src/app.ts. A linter would flag it. A developer reading it would not, because the file lives in the destination repo the kit gets copied into, and the "(or equivalent)" already says so. The second, react-vite's Keep vite.config.ts minimal, is the same story: the file exists in every Vite project this kit lands in, just not in the kit template itself.

Both are false positives, and that's the interesting part. A path check with no repo context produces a report where the only findings are wrong. Now scale that: the audit runs against your actual project, where those references do resolve, so the same lines pass. The lesson cuts both ways:

  • In the template, src/app.ts is a false positive.
  • In your repo, a reference that stops resolving after a refactor is a true positive you want flagged.

Same line of text. The difference is whether the surrounding tree matches it. Any path-level audit, first-party or not, is only as good as the tree it runs against. Run it in the repo where the agent actually works, not against your config source directory.

What the audit won't catch

A linter flags patterns. It cannot judge whether a rule earns its tokens. The things that actually sink configs sit one layer above syntax:

  • Rules the model was going to follow anyway. "Write tests" and "keep functions small" restate preferences current models apply by default. The audit can't tell you your file is 40% restatement, because restatement isn't a pattern, it's a judgment. I count lines instead, and what fits in a 300-line instruction budget is a question the linter will never ask. If you maintain configs for a whole team across stacks, versioned kits like AgentConfig Studio ($29) exist for exactly this.
  • Rules that never fire. An instruction the model reads and ignores does not produce a warning, it produces silence. Detecting that needs before/after runs, not pattern matching. This is why I keep a separate validation layer with behavioral checks the linter doesn't attempt.
  • Prompting for the wrong era in ways that aren't patterns. "Be concise" isn't legacy syntax. But on a model that's already concise by default, it's wasted context and a nudge toward under-explaining. No flag will appear.

The honest framing: prompt-audit is a smoke detector, not a building inspector. It catches the mechanical failures (paths, commands, contradictions, documented-era keywords) that make configs silently wrong. Whether the config is any good is still your problem.

What to do before your first run

If you maintain CLAUDE.md files, skills or agent definitions, three cheap passes before touching the audit:

  1. Resolve your paths. Grep for backtick-quoted paths, check each resolves in the repo the agent works in. My 24-file scan took one shell loop and found the two candidates in under a minute.
  2. De-contradict. If user and project files overlap on the same behavior, decide once, write it once, in one scope. Two rules for one behavior is one rule too many.
  3. Strip era-specific prompting. If a line only made sense on a model you can still name ("for Sonnet 3.5, add.."), it's a candidate for deletion, not migration.

Then run the audit and diff it against what you found. The overlap is your calibration: everything it catches that you missed is the tool earning its keep, and everything you caught that it missed is the part of config maintenance that stays manual.

The bigger shift: configs as managed artifacts

Step back and the direction is clear. deniedModels and availableModelsMatch move model control into managed settings. /doctor prompt-audit moves config quality into the binary. The space your CLAUDE.md occupies is shrinking from both sides: less of it needs to exist because the platform handles more, and more of it is checked automatically.

That's not a reason to stop writing configs. It's a reason to stop writing the parts that are now somebody else's job. The rules that survive are the ones only your repository can justify: the migration order, the "never touch this file" list, the reason a workaround exists. If you want a worked example of that triage, I wrote about which problems your agent config should not solve earlier this year.

Version-pin your configs, run the audit when you upgrade, and treat its report as a diff against your intent, not a grade.

Top comments (0)