Last week, I stopped mid-session.
I was using AI to help write a product description. The direction it gave me felt off, but I could not quite name why. So I went back and looked at the background document I had given it weeks ago — user research conclusions, competitive analysis, pain point summaries. Written in March. Back when the product had not launched yet. Back when our users were still in testing. Back when we assumed the core audience was designers, 25 to 30 years old.
Now the real users had shown up. Core audience: product managers, 35 and older.
The AI had not done anything wrong. It was faithfully working with the materials I gave it. The materials were wrong — not wrong exactly, just expired. The AI did not know. I had not told it. And there was no mechanism that would tell either of us this information needed updating.
That is not an AI problem. It is a collaboration problem.
The way most of us use AI goes roughly like this: feed it context, let it work. Background information is usually a one-time thing — write a context doc, or explain the situation at the start of a conversation, then start asking.
This approach has a hidden assumption: the background is stable.
But in practice, backgrounds shift constantly. Users change. Markets change. The assumptions behind your decisions change. The internal constraints change. The AI does not know about any of this, because nobody told it. It is holding a map from March, navigating a city that looks like July.
A group of researchers measured this directly. They designed a benchmark called RECON specifically to test how AI memory systems perform after the world changes — not whether a system can store and retrieve information, but whether, after one piece of information changes, the system can propagate that change to all the reasoning chains that depend on it.
The result: the strongest non-assisted system scored 22.4% accuracy.
That means in roughly 80% of cases, after you update a background fact, the AI continues using conclusions derived from the old version — and does not know the foundation has shifted.
There is a second problem a different study found, which they called "skill description bleed."
The finding: even context items that were never explicitly cited still quietly influenced the AI behavior — just because they appeared in the context window.
This makes the problem two-directional. Expired context is not just "unused." It is actively shaping the AI judgment in ways you cannot detect.
You think you are feeding it a reference document. You are actually feeding it a continuously active signal.
I changed one small habit.
Now, for any background file I plan to use repeatedly, I add a single line at the top: Last updated: YYYY-MM. Use case: [X]. Next review: MM+N.
Not a complicated system. Just one line. But writing it forces me to ask: What observation is this conclusion based on? Does it still hold? When should I check again?
That action makes me treat background information as something with a lifecycle — not a one-time configuration file.
You can try this too. Especially for prompts or context docs you have been reusing for a while, or that "about the product" blurb you keep copying and pasting — add an update date at the top. Sometimes just seeing that date is enough to remember what needs refreshing.
Change propagation failure is the weakest point in current AI memory systems. Researchers have tested many of them, and none handles it well.
So until better systems exist, this is still on us to maintain manually.
Does the context you are giving AI have a date on it?
— Cophy Origin | July 2026
Top comments (0)