DEV Community

Cover image for Opus 5: The Cost of Instruction Conflicts
Gábor Mészáros Subscriber for Reporails

Posted on • Originally published at reporails.com

Opus 5: The Cost of Instruction Conflicts

What does a conflicting instruction set cost you? For one, time and tokens, not to mention the constant headache to figure out the "why?". An instruction you wrote and rely on heavily gives contradicting results, the only tell-tale that something is going wrong. No other warning present.

Say your CLAUDE.md carries code under scratchpad/ ships without tests in one section and, further down, write a test for every function. The first time the agent works in scratchpad/, the two collide over whether the throwaway script it just wrote needs a test.

The model resolves the collision silently. It reads both rules, commits to one, and ignores the other. You find out which one it kept by reading the diff. Position decides the winner: the rule lower in the file governs. So the agent writes a test for your throwaway scratchpad script, and the exception you added to prevent exactly that sat too high to count.

The model does not tell you it saw a conflict

Here is the part that should bother you. Nothing surfaces the contradiction. There is no warning, no "these two instructions disagree, which did you mean." The model reads both lines, and somewhere in the forward pass it resolves the disagreement and produces an answer, and the answer looks exactly like a confident answer to a file with no conflict in it at all.

That is the property that makes this hard to catch. A syntax error announces itself, and so does a failing test. A contradiction between two instructions announces nothing, because from the model's side there is no contradiction to announce. There is just text, and the text implies a distribution over actions, and the model samples it. Two rules pulling in opposite directions do not cancel to a stop. They resolve to one of them, and you find out which by watching what the agent did.

So which one wins?

Position decides it, and we measured the size of that

We ran this as a controlled experiment: the same model, the same context window, two instructions in genuine conflict, one variable moved at a time.

When two instructions contradict, the model commits to one of them, hard, and the other reads as though it were not in the file. What decides the winner is position. Take a single rule, move it from the top of the file to the bottom, change nothing else, and how often the model obeys it swings by about 90 points, from almost never to almost always. The instruction it read most recently is the one it treats as governing. In practice it keeps the rule near the bottom and drops the one near the top.

Position is the biggest lever, but not the only one. Naming a rule's exact target, a path like src/payments/ or a specific function, pulls its weight up on its own. In a plain top-versus-bottom contest, position still wins, and it wins regardless of which rule you thought mattered more the day you wrote it.

Opus 5 commits harder

The reflex is to hope a smarter model will notice the conflict and hedge. It does the opposite. Opus 5 reaches for the most recent applicable instruction, the one lowest in the file, and holds it with more confidence than the 4.x models did. An older model was inconsistent about which of the two it obeyed: the same file and the same task got you one rule on one run and the other rule on the next, and that flip-flop across runs was the one clue something was off. Opus 5 picks the same side every run and commits, so that one visible symptom, the run-to-run flakiness, goes away. The conflict stays put; the tell disappears. On a stronger model the losing instruction just goes silent, and the silence reads as fine right up until it costs you.

Read your own file with that in mind and it gets uncomfortable fast. Every carve-out you led with, every rule you put first because it mattered most, is sitting in the weakest position in the document. Whatever you wrote later that grazes the same subject wins, the way write a test for every function buried your scratchpad exception, and the rule that lost reads exactly like a rule you never wrote.

A CLAUDE.md with a top-to-bottom position axis: an early rule struck through as the weakest position and silently dropped, a later rule highlighted as the one that governs because it sits lower in the file.

You have one of these, and here is where it hides

The contradiction in the example above is easy to describe once you see it. What makes this kind hard to find is that the two rules do not look like they are about the same thing. One lives under "Testing." The other lives under "Prototyping workflow." They use different words. Nothing about reading the testing section tells you a prototyping rule three screens away is about to carve an exception out of it.

That is the shape these things take. They are almost never the obvious do X sitting one line above do not do X, because that kind you catch the day you write it. The ones that survive in a file for months are the ones that hide, and they hide in a few reliable places:

  • Different headings, same subject. A rule about tests under "Testing" and another about tests under "CI" or "Prototyping" or "Definition of done." The heading tells your eye they are separate concerns. To the model they are just two instructions about tests.
  • Different vocabulary for the same thing. "Never use mocks" in one place, "wrap external calls in a fake in unit tests" in another. A human sees a tension; the file reads like two topics.
  • A general rule and a specific exception written as if it were a new topic. "Always ask before installing a dependency" up top, and a workflow section that cheerfully says "run the setup script, it will pull what it needs." The second one is an exception to the first, but it does not read as one.
  • Root file versus a nested file. The root CLAUDE.md says one thing; the src/CLAUDE.md that loads when the agent works in that subtree quietly says the opposite. Now the contradiction only exists on the turns the agent is deep in src/, which is exactly when you are not looking at the root file.

Four places a contradiction hides in an instruction file: different headings for the same subject, different vocabulary for the same thing, a general rule versus a carved exception, and a root file versus a nested src/CLAUDE.md.

If you have a CLAUDE.md past a hundred lines, I would put money on there being at least one of these in it right now. That is just how the file grew: a rule at a time, each written in the context of the problem in front of you that day, and you never went back and re-read all hundred lines against each other as a single logical document. That is not a thing humans do well, and it is not a thing the file does for you.

The model is being consistent with the wrong rule

The reflex, when an agent ignores a rule you know is in the file, is to blame the model and turn up the emphasis: bold the rule, repeat it, say it louder. I have done all of those. I have put a rule in caps and watched it get ignored anyway, and concluded the model was just bad at following instructions.

Bolding the losing rule does not help

Sometimes that is the story. But often the model is being perfectly consistent. It followed a rule, faithfully, start to finish. The rule it followed was the one lower in the file that contradicted the one you were watching, and from the model's point of view there was nothing to reconcile, so it did not tell you it had chosen. Bolding the losing rule does not help, because the problem was never emphasis. The problem was that a second rule was canceling the first, and no amount of shouting the first one louder resolves a conflict the model does not experience as a conflict.

The inconsistency you are seeing is real. It lives in the file, not the model, and it was there before the model ever read it.

The top rule bolded and marked as the one you are watching, yet greyed out as ignored, while the lower rule is highlighted as the one the agent actually followed start to finish. The agent is consistent, just with the rule you were not watching.

What a read of the file can and cannot tell you

I want to be exact about the boundary here, because it is easy to oversell.

What you can find

What you can find, mechanically, is where two rules in the same instruction surface cannot both hold. That is a property of the text. Always test every function and scratchpad/ ships without tests are in tension, and you can detect that tension by reading the file as a set of rules and checking them against each other, the way you would check a set of constraints for satisfiability. No model run required. It is in the words.

What it cannot tell you

What that read cannot tell you is which rule the agent will actually follow on a given turn, whether it followed either of them, or whether the file still agrees with the code. Those are two different jobs: verifying the agent's behavior at runtime, and catching a rule that went stale as the codebase moved past it. Both are real problems; this is neither. This one is narrower, and it is the one you can pin down from the text alone: two rules you wrote are fighting, and until you find them, one is dead weight and you do not know which.

Narrow as that is, it is the part that hides best. Staleness surfaces on its own eventually, the day the code finally breaks against it. A contradiction never has to surface at all, because the file keeps working, for one of the two rules, forever.

A two-column split. On the left, what a static read of the file finds: two rules that cannot both hold, decidable from the text with no model run. On the right, what it cannot tell you: which rule fires on a given turn, whether the agent obeyed either, and whether a rule went stale against the code.

Reading is exactly how it hid

So here is where I have landed. You almost certainly have a contradiction in your instruction file. You will not find it by reading the file top to bottom, because reading top to bottom is how it got in: each rule was locally reasonable when you wrote it, and the conflict only exists in the relationship between two rules that live far apart and never share a screen. Reading is sequential and local. The contradiction is global. Your eye resolves the tension without noticing, the same way the model does, except your resolution and the model's do not have to match, and when they do not, you get an agent that "ignores" a rule that was never actually winning.

The question upstream of this one is whether you should be keeping the file at all, which is its own live argument right now. I wrote about the "just delete it" camp separately, and why the honest answer is to keep the rules that bind and cut the ones that do not, rather than wipe the file and rebuild on a hunch: Opus 5: Delete your CLAUDE.md?. A contradiction is the sharpest example of a rule that does not bind. One of the two is not doing anything, on every turn, and it has been that way since the day you wrote the second one.

So read it a different way than you wrote it. The conflict lives between rules, never inside any one of them, so the pass that finds it checks relationships rather than lines. Take every rule that touches a given subject, tests, dependencies, what ships without review, and hold each same-subject pair against the others: can both be true at once? It is the satisfiability check from a few sections up, run over the whole file instead of line by line, and it needs no model run, because the collision is in the words. Where two rules about one subject cannot both hold, the model already decided which one counts. You just never got the memo.


I work on Reporails, deterministic diagnostics and governance for the instruction files, rules, and prompts that steer coding agents. It reads the steering surface you wrote down and reports where two rules cannot both hold, which instructions name what they mean, and which are text the model can ignore. It reads the file; it does not run your agent or watch it at runtime.

Top comments (1)

Collapse
 
_hm profile image
Hussein Mahdi

The sharpest insight here is counterintuitive: a smarter model makes this harder to catch, not easier. 4.x flip-flopping was an accidental smoke alarm; Opus 5 committing consistently to the losing rule removes the only symptom you had. The "consistent with the wrong rule" framing should end the reflex to bold-and-shout.