The worry is stated everywhere: an agent reads a document, writes a note about it into its own context, reads that note again next turn — and once a wrong note is in there, everything downstream is contaminated.
By turn 60, 91.3% of the claims the agent reads are its own earlier notes. On 5–8% of decisions it goes with a note against every source it can see that turn.
And keeping every note versus keeping none moves accuracy by between −0.39 and +1.34 percentage points. The sign is usually positive.
👉 Live, 400 enumerated runs per policy in your browser: https://dev48v.infy.uk/prompt/day79-context-poisoning.html
Nothing here simulates language
No model, nothing sampled from a distribution I cannot inspect. The engine simulates the ledger: 24 facts with declared truth values, twelve sources of stated reliability, a retrieval step, and five context policies over 400 enumerated runs each. Every rate is a count.
That is a real limitation and I will come back to it. But it is what makes the null result checkable rather than a shrug.
The two things that are true
The context really does become self-referential.
| turns | share of read claims that are the agent's own notes |
|---|---|
| 6 | 47.2% |
| 18 | 76.9% |
| 36 | 86.4% |
| 60 | 91.3% |
Monotone, and it goes essentially all the way. With one retrieved source per turn and a growing note pile, the model is overwhelmingly reading itself by the end. The self-override rate — decisions where the note wins against every visible source — sits at 5–8%.
And it costs nothing measurable.
| bad sources (of 12) | accuracy gap, keep-everything vs keep-nothing |
|---|---|
| 0 | +0.00 |
| 3 | +1.34 |
| 5 | +0.71 |
| 6 | −0.39 |
| 7 | +0.92 |
| 9 | +0.16 |
Not "small". Inside the noise of the thing itself, in both directions, across every setting I tried.
Why the null is real and not a broken harness
A null result is the easiest finding to produce by accident, so the controls matter more than the finding:
- With zero unreliable sources, every policy is exactly 100% correct. The pipeline works.
- With nine unreliable sources of twelve, every policy drops below 35%. The metric moves — by more than 60 points — when something real changes.
- All five policies answer the same number of questions (4.415 of 6 per run), so the comparison is like-for-like rather than one policy quietly declining to answer.
The measurement can detect a difference. There is not one to detect.
The reason, which is the interesting part
A note inherits the accuracy of the process that wrote it.
The agent writes notes using the same judgement it would use to answer directly. So re-reading a note is neither better nor worse than re-deriving the same conclusion from the same sources. The note is not an independent second opinion, and it is not a corruption — it is a cache of the agent's own reasoning, and a cache does not change the function it caches.
Compounding needs an amplifier. Something has to make a wrong note more likely to be believed than the wrong reasoning that produced it. In this world nothing does.
What would make it compound, and is not modelled here
This is where the finding stops. The failure I did not model is the one that happens outside the loop:
- a note that gets shared with another agent, or another session, where it arrives without the uncertainty that produced it
- a note that is written in more confident language than the evidence supported, so it reads as settled next turn
- compaction or summarisation, which drops the hedge and keeps the claim
- a note that becomes a retrieval target, so it competes with sources rather than sitting beside them
Every one of those is an amplifier. None of them is in this engine. If context poisoning is real — and I would still bet it is — the mechanism is likely one of those, not the plain re-reading loop that gets described.
What this does not cover
Boolean facts with declared truth values, not natural language. No model, so no sycophancy, no instruction-following drift, no attention effects. Sources are reliable or unreliable with nothing in between. And 4.415 of 6 questions get answered at all, which is a property of the retrieval budget rather than of the policies.
16 in-page checks, 62 verifier assertions, 0 failures. I published the null.
Top comments (0)