You tell Claude Code not to modify a particular interface.
It acknowledges the instruction. It fixes the code.
A few sessions later, it makes the same mistake again.
You add a rule to CLAUDE.md. The agent reads it. Everything looks good.
Then it happens again.
At some point, you start wondering whether your instruction file needs to be longer, more specific, or rewritten entirely.
But what if the problem isn't the instruction itself?
The problem with telling agents what to do
Files like CLAUDE.md and AGENTS.md are useful. They give coding agents persistent context about a repository, its conventions, and how developers want work done.
But they have a limitation: giving an agent a rule is not the same as getting it to follow that rule when it matters.
Consider a few examples:
- You correct an agent for adding an unnecessary abstraction. It adds another one in a later task.
- You explain how your repository handles testing. The agent ignores the convention in a future session.
- You ask it to verify a claim before reporting success. It confidently summarizes the result without checking.
You can add instructions for all three.
But each new rule adds more information the agent has to navigate. And even when the relevant information is available, the agent can still make the wrong decision.
The obvious response is to keep adding rules.
I'm not convinced that scales.
What should happen after a correction?
Imagine a different workflow.
You correct an agent's architectural decision. Instead of treating that correction as something that only matters to the current task, the system identifies a potentially reusable lesson.
Later, the agent encounters a similar situation.
The system recognizes that the earlier correction may be relevant and surfaces the guidance at that point.
The developer doesn't have to remember to update a Markdown file, and the agent doesn't have to rediscover the same lesson from scratch.
That's the idea.
Of course, there are some hard problems hidden inside that description.
1. Not every correction should become a rule
Sometimes a developer wants a one-off change.
Sometimes a correction reflects a repository convention that will matter for months.
Sometimes the correction only applies to a particular architecture or task.
A useful system needs to distinguish between these situations. Otherwise, it risks turning every correction into permanent baggage.
2. The right lesson needs to appear at the right time
A general instruction like "follow repository conventions" is easy to write but not always useful.
Specific guidance can be much more actionable.
For example:
Instead of:
"Verify claims before reporting success."
A more concrete instruction might be:
"Before reporting that a test passed, check the result of the test command you actually ran."
The challenge is recognizing when a lesson applies and delivering it when the agent is making the relevant decision.
3. More guidance can create more noise
If the system constantly injects irrelevant instructions, developers will start ignoring them.
Learning from corrections is only useful if the resulting guidance improves behavior without overwhelming the agent.
That means measuring not just whether guidance gets surfaced, but whether it helps.
What we're exploring
We're building an open-source solution and exploring ways to make coding agents more reliable across sessions.
Instruction files like CLAUDE.md and AGENTS.md are useful, but they don't always prevent the same mistakes from recurring. We think there's still room to improve how coding agents apply what developers have already taught them.
There are also different kinds of mistakes. Some can be caught with deterministic checks. Others involve judgment, where an agent can have the right information and still make the wrong decision.
Understanding the difference is an important part of the problem.
A question for developers using coding agents
If you use Claude Code, Codex, Cursor, or similar tools regularly, I'd love to hear about your experience.
When you correct a coding agent, how often does that correction actually stick across future sessions?
Do you maintain instruction files? Use hooks? Build your own memory system? Or have you simply accepted that certain mistakes will keep coming back?
I'm especially interested in cases where the agent already had the right instructions and still got it wrong.
We're exploring this with AutoSteer, and I want to understand what actually works before making assumptions about what developers need.
Top comments (0)