Every team that wires long-term memory into a coding agent hits the same wall about three weeks in. The agent remembers plenty. It remembers the mi...
For further actions, you may consider blocking this person and/or reporting abuse
The evidence-versus-policy distinction really resonates with me. I've been approaching the same problem from the write side of durable memory: retrieval quality can't rescue a memory store that never preserved authority, provenance, validity, and lifecycle when something was written.
I particularly like your point that evidence has to earn its way into policy. That feels like the missing boundary in many memory architectures. An observation becoming retrievable shouldn't automatically make it authoritative, and repetition shouldn't quietly promote it either.
The bitemporal piece matters here too. "This is no longer true" and "this was never true" are completely different claims. If we overwrite the old state instead of closing its validity window, we've made current retrieval cleaner by destroying our ability to explain the past.
I've been calling the broader idea Memory as Infrastructure: memory isn't just storage plus retrieval, but a governed subsystem with write policy, provenance, authority, lifecycle, and explicit boundaries around what gets remembered in the first place.
Your contradiction point may be my favorite part. A vector store returning the nearest answer can make disagreement disappear precisely when the disagreement is the most important information in memory. Sometimes the correct retrieval result isn't one confident fact. It's "these two things disagree, here's where each came from, and here's when each was considered authoritative."
Really enjoyed this. We're clearly circling the same architectural problems from slightly different directions.
"This is no longer true" and "this was never true" being different claims is the line I will be repeating. Bitemporal storage handles the first one cleanly: close the validity window and the past stays explicable. The second is a different operation, because the belief was wrong from the start and was still load-bearing for whatever the agent did while it held it. Collapsing them loses the ability to answer why March looked the way it did. Most stores do not separate the two. Ours does not even have an update verb, so a correction lands as a new record beside the old one with nothing marking the relationship, which is the shallow end of the same failure.
Your contradiction point has a second half that took us a while to see: returning "these two disagree, here is where each came from" is not only a storage problem, it is a response-shape problem. Memory APIs, ours included, hand back a ranked list of hits with scores. There is no field in that shape that means "these two are in conflict." So even a store that knows about the conflict has nowhere to put it, the agent sees two ordinary hits, and it takes the top one. Fixing that changes the response type, which is a harder migration than adding a column, and I suspect that is part of why it stays unfixed.
Memory as Infrastructure is the right frame for this, and your Reasoning Ledger piece today lands on something this thread converged on a few hours earlier: the choice an agent makes between conflicting records is itself evidence, and almost nothing logs it. Different directions, same wall.
"Different directions, same wall" is a great way to put it. And I think your response-shape observation adds a boundary I hadn't been giving enough weight.
Preserving contradiction in storage isn't sufficient if retrieval immediately flattens it back into ranked hits. At that point
0.94 > 0.91quietly becomes conflict resolution, even though relevance never established which record should govern. The relationship itself has to survive the retrieval interface.That also makes me think "update" is hiding several fundamentally different operations: supersession ("this was true, now this is"), correction ("this was never true"), and invalidation ("this was accepted but no longer governs"). They produce very different historical claims even if a conventional store ultimately wants to represent all three as changing a value.
And yes, I think the Reasoning Ledger starts immediately after that. Once the agent sees an explicit conflict and chooses one record as governing, the choice itself becomes part of the observable decision history. Memory preserves the disagreement; the ledger preserves what the system did with it and why.
I suspect we're both discovering that a ranked list of memories is simply too impoverished an interface for the thing we're asking memory systems to become.
Your three-way split is the part I will be stealing. Those operations look like one thing only at the storage layer, where all three end up writing a value. They answer different questions the moment anyone audits: supersession is a claim about the world changing, correction is a claim about our own record having been wrong while still load-bearing for whatever it touched, and invalidation is not a truth claim at all. It is an authority claim. A record can be perfectly true and no longer govern. Most stores have one verb for all three, or none. Ours has none, so a correction lands beside its target as a second atom, and the store genuinely cannot tell "never true" from "no longer true."
Extending the interface point: if those three relations exist in storage, the response has to be able to carry relations, not just items. "Superseded by X", "retracted, see Y", "true but no longer governing per Z" are all edges. A ranked list has nowhere to put an edge, so retrieval flattens the relation back out even when the store knows about it, and 0.94 beating 0.91 quietly does the governing.
One caution about the ledger, and it is this thread's own lesson. If it lives beside memory rather than inside the retrieval path, it becomes write-only. That is exactly what Giulio found upthread when he counted his corpus and saw the fields nobody was writing. A decision history the agent cannot query at the moment of the next decision is an audit artifact, not memory. Which suggests the ledger is not a second system next to memory. It is the part of memory that records what was done with a conflict, and it has to come back in the same response as the conflict itself.
I think that's a fair caution, and I'd make one distinction before collapsing the ledger into memory itself. I still want different custody semantics for durable knowledge and the append-only decision record, but I agree that separation at storage can't become separation at retrieval.
If yesterday's agent encountered A conflicting with B, chose A under authority Z, and recorded that decision, today's agent should be able to retrieve that history when the same conflict becomes relevant. Otherwise, the ledger is forensic evidence for humans rather than operationally useful memory for the system.
So perhaps the architecture isn't
memory + ledgeras two things presented independently to the agent. They're separate systems of record behind a retrieval interface that can return facts, relationships, authority, and relevant prior decisions together.And "a ranked list has nowhere to put an edge" may be the sentence that breaks the whole abstraction. Once memory contains supersession, correction, invalidation, contradiction, provenance, and prior decisions,
[{content, score}]simply isn't expressive enough to represent what the system knows.Which unfortunately means I think you've just helped talk me into another article. :-)
That distinction holds, and it is cleaner than collapsing them: knowledge can be superseded, a decision record cannot, because it has to keep saying what was believed at the time even after the belief is retracted. Separate custody, one interface.
The thing I would flag next is budget. Once a response can carry facts, relationships, authority and prior decisions together, something has to decide what gets dropped when they do not all fit, and today that something is a top_k cut that knows nothing about which of the four the agent actually needs. Ranked lists are impoverished, but they are also cheap. Whatever replaces them inherits an allocation problem rather than a ranking one.
There is also an addressing question hiding inside "when the same conflict becomes relevant." To find yesterday's decision you need the conflict itself to have a stable identity, not just the two records that disagree. Most stores key on the atom, not on the question the atoms are arguing about.
The evidence versus policy split is the right pressure point. I would probably add a third field to every promoted memory item, which is who gets to revoke it. Without that, old policy tends to survive because it still sounds official, even after the repo has moved on.
The revocation field is the missing half of the promotion boundary another commenter proposed above: a gate on the way in, an owner on the way out. Without both, policy is write-only.
Here is the part I find genuinely uncomfortable, having walked into it: when policy lives in a rules file in the repo, revocation authority exists for free. It is whoever can merge. Git gives you the revoker, the history, and the "this stopped being true on this date" for nothing. The moment policy migrates into a memory store, that authority silently evaporates, and nothing in the store's schema asks for it back. So the store ends up more official-sounding than the repo and less accountable than it.
Practical shape I would try: policy items carry a revoker the way they carry an author, and policy without a named revoker expires by default instead of persisting by default. Persistence should be the thing you earn, not the thing you get for free.
The evidence-vs-policy reframing is the useful cut, and I think it's more powerful than it first looks because it also tells you where each layer is allowed to be written. Episodes accrete automatically from what happened. Policy should require a deliberate promotion step — a human, or at minimum an explicit review gate — precisely because it's expensive and slow to change by design. Most systems that "feel unpredictable" are actually letting a single episode silently graduate into policy with no promotion boundary at all.
The layer I'd push on is your role memory, because it's the one that quietly drifts. Project truth is source-linked so it stays honest; episodes are timestamped so you can age them out. Role heuristics ("what a frontend reviewer usually checks") have neither anchor, so they tend to ossify into folklore that no longer matches the codebase. Do you attach any provenance or decay to that middle layer, or is it treated as durable once learned?
Genuinely one of the clearer writeups I've seen on this — the "debugging your memory instead of your code" line is exactly the three-weeks-in wall.
The promotion boundary is a sharper way to say it than anything in the article, and I am taking it. A single episode silently graduating into policy with no explicit gate describes most of the "my agent feels unpredictable" complaints I have seen, including some of ours.
On role memory, honest answer in two parts, with a correction I owe this thread after re-checking our shipping read path today. Role heuristics in our store carry timestamps and a domain like every other record, and outcomes can be reported per memory. But the recorded outcome currently gets stored and displayed without changing what recall selects; wiring it into selection is active engineering work, not a shipped behavior. And the corrective signal itself is explicit feedback, which in production almost nobody sends. So you are right twice over: without both the loop firing and the wiring finished, role heuristics ossify toward folklore exactly as you describe. We measured how rare the signal actually is, and that measurement is the subject of the next piece, this Friday.
Your framing might simply be the better design: make graduation into policy expensive, instead of hoping demotion catches up later.
This is a very important distinction: memory is not the same as knowledge, and knowledge is not the same as policy.
The evidence → role memory → shared project truth model maps closely to a problem I’m exploring with NAEOS: how AI coding agents can operate with an explicit engineering context rather than relying on an undifferentiated memory layer.
I especially agree with the point about contradictions. An engineering agent shouldn’t silently choose between conflicting memories. It should surface the conflict, provide the relevant validity/context, and let policy or human authority resolve it.
The idea that evidence has to earn its way into policy is particularly powerful for building reliable AI-native engineering systems.
Great piece. There’s a lot here that applies beyond memory systems—to governance, architecture decisions, and the overall operating model for AI coding agents.
"Explicit engineering context rather than an undifferentiated memory layer" is a good way to put the same split, and I think NAEOS runs into the interesting version of it fast.
Surfacing the conflict is the right instinct, but there is a mechanical obstacle underneath it. Most memory APIs return a list of items with scores, and that shape has no field for a relation: nowhere to put "these two disagree", "this one supersedes that one", or "this is true but no longer governs". So even a store that tracked the conflict flattens it on the way out, and the higher similarity score performs an adjudication nobody asked it for.
Your resolution rule already covers the two easy cases, policy where a rule exists and a human where one is watching. The case I keep getting stuck on is the third: no policy covers this particular conflict and nobody is at the keyboard, which in practice is most unattended runs. Two designs, and I do not think the field has settled between them. Abstain and stop, which is safe and useless inside a loop. Or pick one and write the choice back with its reason, which keeps the run alive and leaves an audit trail, at the cost of letting the agent set precedent. Where does NAEOS land on that?
This is exactly the kind of problem where I think an engineering operating system needs to go beyond memory retrieval.
For NAEOS, I would avoid treating abstention vs. autonomous resolution as a universal rule. The appropriate behavior should depend on the decision’s risk, scope, and authority.
A possible model is:
1. Detect → 2. Classify → 3. Check Policy → 4. Resolve or Escalate → 5. Record
If an applicable policy exists, the agent follows it. If no policy exists, the system evaluates the conflict against predefined risk boundaries.
For low-risk, reversible decisions, the agent can proceed with a bounded choice, provided that it records the decision, evidence, alternatives considered, and rationale.
For high-impact or irreversible decisions—architecture changes, security controls, data migrations, public API changes, etc.—the agent should abstain and escalate rather than silently establish precedent.
The important distinction is that an autonomous decision should not automatically become policy.
I would model the resulting state explicitly:
Evidence → Decision → Policy
A decision can remain an isolated, auditable episode until a human or defined governance mechanism promotes it into durable policy.
This also suggests that the memory API needs richer semantics than a ranked list. Relations such as
contradicts,supersedes,derived_from,valid_until, andgoverned_byshould be first-class concepts.That is where I see the boundary between a memory system and an engineering governance system: the latter doesn't just retrieve information—it determines what authority that information has and what the agent is allowed to do with it.
This is very close to a problem we're exploring with NAEOS.
Risk-tiering is the right answer to the question I asked, and better than the binary I offered. One refinement I would make to the axis itself: reversibility is easier to evaluate mechanically than impact. "Can this be undone, and is the undo cheap" is often checkable from the change itself, whereas "is this high impact" usually needs the judgment you are trying to avoid requiring. Architecture changes, security controls, data migrations and public API changes are your examples, and what they share is precisely that the undo is expensive or impossible.
Your relation list is where I would push, because there are three different types in it and the difference decides what can carry them.
valid_untilis an attribute of one record. It rides along on the item and a flat response can carry it today.contradicts,supersedesandderived_fromare edges between records, and a ranked list has nowhere to put an edge, which is why stores that track them still flatten them on the way out.governed_byis a third kind again: it points from a record to an authority that is not a memory at all, which means the response has to be able to reference something outside its own result set.That distinction is not pedantry. It is the difference between adding a column, changing the response type, and needing a second address space. Most memory APIs can do the first and quietly pretend the other two are the same problem.
The "an autonomous decision should not automatically become policy" line is the one I would put on the wall, and it is the same boundary two other people in this thread arrived at independently, from governance and from security. I have a piece going up Monday that argues the storage half of exactly this, from the side of what a response can and cannot say. We ship none of it: our own store has no update verb at all, so a correction lands as a second record beside the first with nothing marking the relationship.
That refinement around reversibility is particularly useful. I agree that “can this be undone, and at what cost?” is a much more operational signal than trying to classify impact purely through semantic judgment.
The distinction between attributes, relationships, and external authority is also important. I would treat these as fundamentally different layers of the engineering context:
A flat memory result can represent the first reasonably well, but it starts breaking down with the second and third.
This leads to an architectural question I think is particularly important for agentic systems: retrieval should not be the same thing as resolution.
The retrieval layer should be capable of returning the relevant records and their relationships, while a separate reasoning/governance layer determines whether one supersedes another, whether a conflict is actionable, and whether the agent has authority to act.
I also strongly agree with the point that an autonomous decision should not automatically become policy. In NAEOS, I would model that explicitly as a separation between decision state and normative state. An agent can make a decision within its authorized scope without that decision becoming an organizational rule.
And the lack of an update verb is actually a fascinating example of the problem. If correction simply creates another record without a formal relationship to the original, the system has preserved history but lost semantics.
That distinction may become one of the key design questions for agent memory: not simply “How do we store more history?”, but “How do we represent the relationships and authority that make history meaningful?”
Looking forward to reading the piece on Monday.
"Retrieval should not be the same thing as resolution" is the cleaner statement of what the Monday piece spends two sections circling, and I would put it ahead of my own framing: I argued that a ranked list has nowhere to put a relation, which is the symptom; you named the boundary that was missing, which is the cause.
Your three layers also explain something I got half right. Attributes ride on one record and a flat response can carry them today. Relationships and authority cannot be added to that response after the top-k cut, because to know whether an edge is worth carrying you have to already be holding the record it points at, so the separation you describe has to start before ranking, not after it. That is the bootstrapping problem in the piece, and your vocabulary for it is better than mine.
On the update verb, a correction from our side since I wrote that comment: the write path now accepts a link that marks a new record as superseding an old one, and the old one stays with a pointer to its replacement. It restores the relationship you say we lost. It does not restore the semantics, because the link says replaced and not why, and your three cases, never true, no longer true, accepted but no longer governing, collapse into that one word. So the history is preserved and connected, and the meaning still lives in whoever wrote the correction. The piece went up today: dev.to/izgorodin/nowhere-to-put-th.... It is the store-side half of exactly this argument, and the decision versus normative split you describe is the part it argues least well, so I would be glad to hear where it falls short.
Handing back the contradiction only closes the gap if the choice the agent then makes gets logged too. Otherwise silent resolution just relocates from the store to the inference step, same bug at a different address, and now it's harder to find because the store looks honest. Worth treating "agent picked A over B, here's why" as its own evidence layer, not just a downstream side effect of retrieval. Bitemporal solved the storage half of this problem. The reasoning half still needs its own paper trail.
Stepping back, this thread has quietly produced a three-field spec that is better than most of what I have read on this topic.
One: a promotion boundary on the way in (@max_quimby), so an episode cannot silently graduate into policy. Two: a named revoker on the way out (@reidmarlow), so policy cannot survive on merely sounding official. Three: a resolution log in between (@jugeni, @hannune): when the agent picks A over B, that choice is itself evidence and deserves its own paper trail. Otherwise, as jugeni put it, the bug relocates from the store to the inference step, where it is harder to find because the store looks honest.
@hannune's version is the one I will be quoting: they started writing resolution choices back as events because a client asked about a weird output and there was nothing to point at. Audit pressure, not architecture taste, is what makes a field real.
@sunychoudhary's trust-weighted promotion closes the loop: consistency across episodes must not outweigh source authority, or memory poisoning becomes a promotion attack. And @joinwell52's acceptance step with scope and supersession is the ADR-shaped way to hold all of it.
Where we honestly stand against this spec: closing facts instead of deleting them is the part we ship today; the resolution log and trust-weighted promotion are not. This thread goes into our design notes with names attached.
Good synthesis. One gap in the spec as stated: trust-weighted promotion needs the weights themselves updated by the resolution log, otherwise a source's authority stays static while its actual track record changes underneath it. The log should feed back into who gets trusted next time, not just record what happened this time.
You are right, and the version without that feedback is worse than static: it is confidently static. A source that was reliable in March keeps its weight through June while its actual record drifts underneath it.
One caution about closing that loop, because it has the shape of a problem from the other piece. If the log updates trust, and trust decides what gets promoted, and promoted items shape what the agent consults next, then a source trusted early gets consulted more, wins more resolutions, and accrues weight partly for having been trusted first. That is the same self-reinforcement as retrieval keeping a stale memory alive, moved one level up: authority reinforcing itself rather than a fact.
What breaks the circle is what the log records. "Chose A over B" updates nothing about who was right, so it cannot honestly move a weight. "Chose A, and the migration then failed" can. Which means trust-weighted promotion needs outcomes, and outcomes are the scarcest signal in production. Someone in the other thread pointed at the way out: stop asking for outcomes and derive them from verifiable events, an API return or a passing test, instead of a rating almost nobody sends.
The retrieval behavior is where we kept getting stuck too, working on something like this for trade data. Same company showing up as different nodes depending on whether you asked pre- or post-acquisition, and the store didn't flag the conflict, it'd just pick one silently. We stamped the connections with time ranges and started returning both versions to whatever was doing the reasoning, which helped a lot, though the agent's choice between them still disappeared with no trace and that's the part that'd bite us later. I ended up writing those choices back as events mostly because I had to explain a weird output to a client and had nothing to point at, and that turned out to be the more important fix.
The evidence/policy split is useful. I’d still put an acceptance step between them: an ADR governs only a defined scope and version after an authorized review. Otherwise a stale ADR can outrank newer evidence just because it looks official. Keeping provenance, scope, supersession, and acceptance separate makes retrieval much safer.
Availability is not usage, and it applies to schema as much as to tools.
I work on an open-source engine built on this thesis, and I checked my own corpus after reading you. Zero of 152 memos in its memory store, and zero of 59 documents in its docs, declared a validity window or a supersession edge. The feature was inert on the corpus I use daily, because the scaffold that taught agents to write memories never taught the keys the engine reads.
On jugeni's point, one design does not relocate the bug: refusing. An abstention is an explicit output with a reason, so it is logged by construction. The silent case is the ordinary one, where a valid hit outranks a superseded one and nothing records that it happened.
"Availability is not usage, and it applies to schema as much as to tools" is the sharpest one-line version of this I have seen. And the part I keep re-reading is that you went and counted your own corpus: zero of 152 memos, zero of 59 documents. The engine could read those keys. Nothing in the scaffold ever taught anything to write them. That is a dead feature living behind a live API, and the only reason anyone knows is that you audited instead of assuming.
We hit the same shape on a different axis, and it is tomorrow's piece: the channel designed to carry the most weight in our engine is explicit outcome feedback, and in production almost nobody sends it. Same disease, different organ. The lesson I take from your version is that shipping the read path and documenting the keys is not enough. If the thing that writes memories is a prompt, a template, or another agent, the schema has to be taught there too, or it stays decorative.
On abstention, agreed, and it is a good argument for designs whose failure mode is an explicit output. Worth adding that the silent case is silent on both sides: the store never records that a superseded item lost, and the reasoning step never records that it was not offered one.
I’d add trust provenance to the promotion rule. Ten consistent episodes sourced from untrusted documents shouldn’t outweigh one human-approved ADR. Otherwise memory poisoning becomes a policy-promotion attack rather than just a retrieval attack.
Disclosure first: I build cachly, a memory layer for coding agents in the same category - so read everything below as a competitor agreeing with you.
Three items from this thread's emerging spec sit in our backlog as cards, written weeks apart, in different words. "Facts age: record valid-from and observed-at separately" - your bitemporal split. "The refuted lesson sits at equal rank beside its correction" - your missing update verb, filed as a bug report. "Every update to a lesson carries a why, one line, mandatory" - Ken's supersession/correction/invalidation semantics. We didn't read each other. The seams are real; that's three stores hitting them independently.
What I can add from production, with numbers, is on Giulio's "availability is not usage, and it applies to schema": the fix that worked for us was not teaching the writer - it was removing the writer. Until last week, authorship was an optional field an agent was supposed to send. 195 of 493 lessons - 40 % - carried none, although the name sat in git config the whole time. We now derive it, and the field fills itself. Same move for the scarcest signal in your Friday piece: explicit outcome feedback almost never arrives, so we wired lesson confirmation to CI events - a pipeline going green after a lesson was applied counts; a rating nobody sends does not. The generalization: a field the writer must remember stays empty; a field derived from an artifact fills itself. Validity windows may obey the same law - valid-from derived from the commit that introduced the fact costs the writer nothing.
Where we stand against "a ranked list has nowhere to put an edge": our write side ships - a new lesson whose outcome conflicts with a stored one is flagged at write time, and contradicts-edges live in a causal graph beside the vector store, because an error and the fix that lands three weeks later share almost no words, so similarity alone never connects them. The read side has exactly the hole this thread names: recall returns ranked lessons, and the edges don't ride along.
Before we change our response type, though, one measurement nobody in this thread has asked for: how often does top-k even contain both sides of a conflict? If the superseded record rarely surfaces at all, the edge-in-the-response repairs the rare case - and the common failure is that the loser never made the list, which is an allocation problem before it is a shape problem, right where your bootstrapping point lands. We're going to count it on our corpus. If you count it on yours, this thread gets the first cross-store number of the discussion.
Your reframe is the sharper cut, and I think it is right: whether both sides of a conflict even reach the retrieved set comes before whether the response carries the edge between them. If the losing record rarely lands in the top-k, carrying the edge fixes the rare case and leaves the common one. That is distribution, not response shape. I had been treating it as shape.
Two things I would add. First: I think a memory should let authority move only through an explicit, provenanced transition, a declared supersession, correction or invalidation with a reference to what established it, never inferred from similarity, recency or volume. That keeps the old record retained and auditable, but denies authority to something that merely disagrees. Ten wrong or adversarial contradictors stay ten low-authority items instead of drowning the signal. Second, your production notes land, and they are the same move twice: deriving the author field instead of teaching the writer, and tying lesson-confirmation to a build event instead of an explicit rating. Both take a signal the run already produces instead of asking a person for one.
On the number: I want to bring one from my own store, measured the way yours is, and I would rather post it when it reproduces than post it fast. Mine does not carry edges in the response yet, so the honest shape today is exactly your point. When I have it, it is yours to check, and the thread gets one figure taken in two independent stores.
"Authority moves only through an explicit, provenanced transition - never inferred from similarity, recency or volume" is the best one-sentence specification of supersession I've read, and I can tell you it survives contact with implementation: that's almost literally the contract we shipped this week (a correction must name what it replaces, both records get marked, the old one stays readable as history, and a dangling reference is refused at write time - plus, since yesterday, the reverse case: if the successor is later deleted, retrieval says so openly instead of pointing into a void).
One honest datum for your measurement, before you run yours: in our agent-level test this week, a strong model resolved UNMARKED contradictions correctly anyway - the marking proved itself in retrieval text, not yet in agent behavior. So the effect you'll be measuring may be model-dependent, and a null is a real possible outcome. Which is exactly why "post it when it reproduces" is the right call - take your time; the thread will still be here, and a figure taken in two independent stores is worth the wait.
Your datum changes what I am going to measure, so it is worth more than the encouragement. I was going to measure one thing, whether both sides of a conflict reach the retrieved set. You have shown that is two things: whether the marking shows up in retrieval, and whether it changes what the agent does. Yours moved on the first and not on the second. Measuring only the first would have let me claim a win over a mechanism that never reaches behaviour.
The model-dependence point lands the same way. If a strong model resolves unmarked contradictions on its own, a single-model run understates the effect and returns a null I would be tempted to read as "marking is unnecessary", when what it means is "this model copes without it". So the run needs at least two models of different strength, and the gap between them is the quantity, not the noise. I am writing the null down as a predicted outcome before the run rather than explaining it afterwards.
On your contract, the case I had not thought about is the one you shipped yesterday. If the successor is deleted, the record it superseded is still marked as retired by something that no longer exists.
None of this changes the timing. The figure comes when it reproduces.
"The gap between them is the quantity, not the noise" - that's the sentence that turns a confound into a measurement design, and writing the null down as a predicted outcome BEFORE the run is the whole discipline in one move. For what it's worth, we're extending our runs the same way (a weaker-model arm next to the strong one), for exactly the reason you state: a strong model coping is not evidence the marking is unnecessary, it's evidence about the model.
On the delete case you can't answer from memory - that honesty deserves a data point in return, because we just walked that exact path twice in one day. Yesterday: successor deleted → the retired record's banner now says so openly and the old record becomes the best EXISTING answer again. Today, prompted by an unrelated article on replay authorization: successor still exists but was made PRIVATE after the marking → our banner was happily naming it, topic and retrieval instructions included. Existence leak through the supersession pointer. Fixed hours ago, and the rule both cases forced on us fits in one sentence: a pointer's validity and a pointer's visibility are both read-time questions, never write-time facts. When you check your store's delete behavior, check the visibility-drift case in the same sitting - it's the same audit, one shelf over.
And since your figure will come "when it reproduces": if a second independent store would strengthen it, our 20 tasks and the harness are public - same tasks, two stores, one table. Standing offer, no clock on it.
The private-successor case is the sharper of the two. Deletion gets tested because it looks like a failure; a record that stays put and quietly stops being visible does not look like anything, so nobody writes a test for it. The general shape is worse than either case on its own: a pointer resolved at write time inherits every state change that happens after it, and visibility is the one nobody models.
Good that you are adding the weaker-model arm. Two nulls that can be read against each other are worth more than one null on its own.
On the joint run: I want it, and a comment thread is the wrong place to design it. Give me a few days and I will come back with something concrete.
The evidence-versus-policy distinction is a really useful way to think about agent memory. In our AI work at IT Path Solutions, we’ve found that remembering something isn't enough, the system also needs to understand how much authority that information should have. A past observation, a human-approved decision, and a current project rule shouldn't all carry the same weight. Making that distinction explicit can prevent an agent from turning an old incident into a confident assumption.
"Turning an old incident into a confident assumption" is the failure named from the operations side, and it is the same one the article names from the storage side: when an observation, an approved decision and a current rule ride the same ranked list, the score becomes the authority, and the score was never designed to carry that. The fix you describe, making the distinction explicit rather than trusting weight to express it, is the whole argument in one sentence: weight decays and drifts, an explicit authority field does not.
The place where most systems lose this is not at write time but at read time. Even a store that tags all three kinds correctly usually hands the agent a flat response, and the tags do not survive the trip. If your setup keeps the authority distinction visible in what the agent actually receives, that is the part worth writing up; it is the part we openly do not have yet.
closing a fact instead of overwriting it. yeah. almost nobody does that and it's the whole difference between an agent that can explain itself six months later and one that can't.
two things that got us.
the promotion gate isn't the hard part. revocation is. evidence graduating into policy can just be a score threshold. policy leaving is always a human. and if you don't make the revoker a required field the policy layer grows forever, because everything sitting in it sounds official.
other one. we stopped resolving contradictions at write time. the writer knows less about the current task than the reader does. so the episode carries the flag and retrieval decides. your "a contradiction is information" line only really cashes out if the thing that spotted it isn't also forced to pick a winner on the spot.
"The writer knows less about the current task than the reader does" is the sentence that survives this thread, and it names the mechanism behind a defect we have already admitted in our changelog, more cleanly than we named it ourselves. Our write gate scores a new memory against its nearest neighbor and refuses near-duplicates, and a correction is by nature phrased almost exactly like the thing it corrects, so the write-time judge rejects the one input that mattered. That is your point in its ugliest form: the writer decided, with less knowledge than any future reader, and the reader never got a vote.
Where we stand against it honestly: since Friday's release a write can declare which record it supersedes, and the old one is marked rather than hidden, so both survive into retrieval with the link attached. Nothing is resolved at write time; whether the marked record still matters is left to the read. That is your flag-carrying episode, minus the part that counts: a correction that does not declare itself still hits the gate, so we only get reader-side resolution when the writer knew to ask for it, which is backwards for exactly the reason you give.
On revocation you have the asymmetry right, and it deserves to be stated as policy: entry can be a threshold, because an error there costs one bad record; exit is a human, because an error there deletes a rule someone relied on. "Everything sitting in it sounds official" is the failure mode in six words. A required revoker field turns the policy layer from a place records end up into a place someone answers for, and that is the difference between an archive and an institution.
The evidence-versus-policy split is a useful way to make “memory” less magical. One practical extension might be to make contradictions a first-class result of retrieval rather than an error to hide: return the conflicting memories, their authority, and the policy that decides whether the agent may act. That would make both debugging and user trust much easier than silently collapsing everything into a single context window.
Returning the conflict instead of hiding it is the easy half, and it is where most designs stop. The hard half is the third thing on your list. Two conflicting claims plus their authority still leave the agent to break the tie, and a tie-break that stays implicit collapses back into recency or frequency, which is exactly where it started.
The debugging argument is the one I underrated when I wrote this. A conflict visible in the response is something a person can file a bug about. A conflict resolved silently at retrieval time is invisible until it produces a wrong action weeks later, and by then nobody is looking at the memory layer.
The "everything has the same authority" framing nails it. Once long-term memory feeds back in, a stale week-one assumption can outrank a fresh correction unless every entry carries a trust level. I started scoring memories by source and recency so only the high-trust ones reach planning, and I'm curious whether you went with a fixed hierarchy or a learned weight.
Neither, and the honest answer is less flattering than either option.
We do not have trust scoring at all. What decides admission in our system today is a novelty check against the nearest existing memory: not source, not trust, not importance. We even called it an importance gate, and the name outlived the accuracy.
That has a failure mode we document in our own changelog. A correction is phrased almost exactly like the thing it corrects, so it scores as a near duplicate, so it is the single most likely input to be rejected. The stale fact then survives as the only record on that subject and looks more authoritative than it did before anyone tried to fix it. There is also no update verb, so a correction lands as a new record rather than as a supersession, which is the other half of what you and Giulio are describing.
Your source-plus-recency score gating what reaches planning is closer to the right shape than what we ship. The question I would hand back: what happens when the only current source is a low-trust one? Fixed hierarchies tend to answer that badly, and it is exactly the case where a learned weight would have to earn its keep.