Last week Y Combinator posted an interview with Boris Cherny, the engineer who built Claude Code, about the new Opus 5 release (the full talk). It ...
For further actions, you may consider blocking this person and/or reporting abuse
The ablation framing is the useful one. Most of a CLAUDE.md used to be coaching for behaviors the model should already know, like prefer small commits or do not invent APIs, and those lines mostly stop earning their keep once the model improves. What still deserves a permanent seat is the project stuff no model can invent: protected paths, naming that only exists here, the one test pattern that must not get rewritten green. I treat a model upgrade as a cue to re-read every instruction against a real agent session, not as a cue to keep the whole file out of habit.
Good approach and good habit!
the 27% number is the part that stings — tbh matches what you see when you trace which lines actually change model behavior vs which are just there because some older version needed them.
the retrieval strength change is the gnarly one. vague rules used to sit quietly because the model could not reach them for most tasks. now they fire, the model has nothing concrete to bind to, fills with its own judgment. that's not CLAUDE.md working too well, it's working too wide.
did the ablation surface any instruction categories that consistently survived the cut vs got yeeted first?
Not on my corpus, however the validation mechanisms had to be amended. And since I'm building my instructions based on Reporails (github.com/reporails/cli) rules, the instruction surface is tidy and continuously governed. I also have an addition github action that runs alongside with builds that would catch any instruction defects.
validation as a CI step is the right call. the edge i'd watch: if the GH action evaluates on the same model family you wrote the rules for, you might be checking the bias rather than through it — the model fills vague rules 'correctly' because it still carries your authoring intent.
running against a different model family, or a minimal context window, is what actually surfaces instruction drift.
does the action test semantic behavior or schema compliance?
That's why github.com/reporails/cli and it's rules are model agnostic. What happens is that there are 100+ generic rules that are true no matter the model and some additional ones that are specific for certain models.
Rules: reporails.com/rules
Note: The cli was created having adaptability in mind, currently working out the fine details around users being able to define their own rules there too.
Great point! Here is a short and punchy version:
Spot on! Fully agree that "delete-and-observe" is way too blunt for production—watching your own work break just to see what was load-bearing is a costly way to learn.
Your distinction between steering and enforcement is key: a smarter model doesn't mean you drop your hooks or deterministic gates. Also, that 3-line constraint structure (Imperative → Context → Category Restriction) is gold for prompt hygiene.
Awesome write-up! 🚀🔥
thank you ^^
It feels like AI needs some housekeeping now too. As models evolve, cleaning up old instructions seems to become part of the maintenance. I guess useful tools always need a bit of upkeep.
that kind of maintenance is one of the core idea of Reporails (github.com/reporails/cli).