DEV Community: Swapnanil Saha

Filtering After Top-k Is a Silent Bug in RAG

Swapnanil Saha — Mon, 12 Oct 2026 16:53:55 +0000

A filtered vector query, the kind that sits in the retrieval step of a RAG pipeline or behind an agent's memory lookup, is commonly written as two lines. Ask the vector index for the nearest k items, then drop the ones that fail a condition: wrong workspace, wrong type, expired, deleted, not visible to this user.

Those two lines do not compute what they appear to compute. The index answered "which k items are nearest overall?" The caller wanted "which k items are nearest among the ones that pass?" Composing the first with a filter does not produce the second.

The gap has two properties that make it worse than an ordinary bug. It returns a well-formed, correctly ordered, too-short result, which looks exactly like a correct answer from a store that happens to hold little. And it is worst precisely when the filter matters most: when the items you want are rare, or when the items you do not want are the ones most similar to the query.

Every example below comes from one agent-memory store, the open-source vectr daemon, where the same bug turned up in at least four separate code paths between late July and late August 2026. Each was found and fixed on its own.

Part 1: The Arithmetic

What a Post-Filter Computes in Vector Search

Analogy

A recruiter is asked for the ten CVs that best match a job description and hands over ten. The hiring manager strikes everyone without the right to work in the country and interviews whoever is left. If the ten best matches all came from abroad, nobody is left, and the manager concludes that no local applicant fits. Applicants eleven to fifty were never read. The conclusion is about the shortlist, not the applicant pool, and nothing on the empty interview schedule says which.

That is a post-filter: a yes-or-no condition applied after the ranking has already been cut to k. Put numbers on it. Call s the pass rate near this query, meaning the chance that a candidate from the top of its ranking passes the filter. Fetch k candidates and you expect k × s survivors, so returning n results takes, on average, k ≥ n / s.

The post-filter, exactly
survivors(k) = | { x in top_k(q) : passes(x) } |
E[survivors(k)] = k · s
returns the true filtered top-n  ⟺  k ≥ r_n
q is the query. top_k(q) is the set of k items nearest to q in the whole store. s is the pass rate among q's near neighbours. r_n is the rank, in the unfiltered ordering, of the n-th nearest item that passes. A fetch of depth k returns the true filtered answer if and only if it reaches r_n. Short of that depth it returns the first few items of the right answer, in the right order, and then stops.

When the filter has nothing to do with distance, s is just σ (sigma), the share of the whole store that passes. The survivor count is then approximately binomial, P(no survivors) = (1 - σ)^k, and r_n follows a negative binomial distribution with mean n / σ.

Read the third line again, because it holds the most useful fact about the bug. A post-filter over an exact nearest-neighbour ranking never returns a wrong item. Everything it returns belongs in the answer, at the right position. Its only error is omission, the tail that fell outside the fetch, and omission leaves nothing behind to inspect. (An approximate index misses a few true neighbours of its own; the post-filter's omissions come on top.) This is also why the pattern gets through code review. Read as an access-control check, a post-filter is safe: it cannot leak an item the user is not allowed to see. The damage is all on the recall side, where a reviewer checking for leaks is not looking.

Why s is not the share of the store

The trap is in what s means. It is not σ, the share of the whole store that passes. It is the share of this query's nearest neighbours that pass, and once the filter is correlated with similarity the two can be far apart.

The damaging direction is a filter that removes the closest items. In a store shared across projects, the items most similar to a query about project A are likely to be items on the same topic from project B, because topic drives similarity and projects share topics. A workspace filter removes exactly those. Project A can own half the store and still own none of a query's ten nearest neighbours. The global share does not predict this; it depends on how the filter lines up with the embedding space, and it changes from query to query.

Rarity hurts even with no correlation at all. In one live store of 565 notes, 2 were in the revoked state: σ = 0.35%. Fetch 3 candidates and the expected number of revoked survivors is 3 × 0.0035, about 0.01. Under the independence model, 100 candidates contain a revoked note about 30% of the time, and a 95% chance takes about 845.

Correlation can also run the helpful way, when the wanted items are the ones nearest the query. That case comes up with the revoked notes below, and it is the reason a fixed depth sometimes works far better than σ predicts.

Interactive demo: Fetch k, Then Filter (runs on the original page)

See that a post-filter is never wrong, only short, and find the fetch depth each kind of filter needs. Each dot is an item in a small seeded store and the gold cross is the query. Everything inside the dashed circle is in the top-k. Switch to the second scenario, where the closest items belong to another workspace, and find the smallest k that returns a full answer; then try the rare class and compare "the store ran out" with "the search ran out".

48 items in two dimensions, seeded so the layout reproduces. Real embeddings have hundreds of dimensions, but ranking by distance and then filtering behaves the same way. The "caller sees" lines are everything a real caller gets; the "ground truth" lines are what only the demo knows.

Over-Fetch Multipliers Are a Bet on the Pass Rate

The usual mitigation is to over-fetch by a constant: ask for limit * 3 and filter down to limit. The constant encodes an assumed s. Three times the limit assumes that a third of the neighbours pass. For some queries and filters that is right, for others it is silently wrong, and the code has no way to tell which case it is in.

Underneath that sits a quieter problem. Even when the assumed s is exactly right, n / s is the average depth needed, and the actual depth scatters around it. With s = 1/3, n = 5 and k = 15, the chance that at least five survive is 60%. Getting to 95% takes k = 25: five times the limit, not three. Rarer classes need more headroom still. At s = 1/7, a 95% chance of five survivors takes 62 candidates, a multiplier above twelve.

Insight

A multiplier sized to the average pass rate fills the answer somewhere between 55 and 70% of the time for the values used here. Reliability comes from headroom above n / s, and no single constant supplies the right headroom for a filter whose pass rate changes with the query.

Interactive demo: A Multiplier Is a Bet on the Pass Rate (runs on the original page)

Learn how much headroom a fixed multiplier really buys. Set the pass rate among a query's neighbours, the number of results you need, and the multiplier, and read off how often the fetch comes back full. The presets are the pass rates that appear in the four instances below. The model assumes the filter is independent of distance. A filter that removes the closest items does worse than the numbers shown; one whose wanted items sit close to the query does better.

P(at least n survive) is the binomial tail with k = multiplier × n tries at pass rate s. "Multiplier for 95%" searches upward for the smallest multiplier whose tail reaches 0.95.

Part 2: Four Instances in One Memory Store

Instance 1: Related Notes Starved to an Empty List

The store keeps an agent's notes in a relational table and their embeddings in a separate vector collection. Each note belongs to a workspace, has a kind (a standing rule, a task, a caveat, and so on), may have expired, and has a lifecycle state.

When a note is written, the store looks up the most similar existing notes and shows them to the writer, so it can see what the new note might duplicate or contradict. The vector collection was global across every workspace the daemon served, and the embeddings were written with their ids and nothing else, so the similarity query had nothing to filter on. The workspace check, the expiry check and the lifecycle check all ran in SQL and Python after the vector search.

The query asked for limit + 1 neighbours, budgeting for exactly one exclusion: the note itself. When the nearest neighbours belonged to another workspace or had already been revoked, they used up the whole budget and the filters removed every one of them. The function returned an empty list even when close, active, same-workspace notes existed.

agent/working_context_store/_related.py, commit 77252a6

# before: room to exclude the note itself, and nothing else
n_query = min(limit + 1, col_count)

# after
n_query = min(max(limit * 3, limit + 1), col_count)

That repair is a fixed multiplier too, and so a bet on s. It holds while the querying workspace owns most of the neighbourhood. Take a daemon serving ten projects of similar size with no topic overlap, so s is about a tenth, and the shipped limit of 3. The fetch is 9 candidates, and the first of them is the new note itself, which the lookup skips. The remaining 8 yield 0.8 survivors on average and a full list of three about 4% of the time. Topic overlap between the projects makes it worse, since overlap is exactly the correlation that puts another project's notes at the front.

Instance 2: A Rare Class Starved by the Render Limit

The same write-time lookup has a second path: find revoked notes similar to the new one, so an agent re-deriving a belief that was already proven wrong is warned before it writes the belief down again. That path copied the first path's pool formula, which tied search depth to how many results would be rendered. With one revoked result rendered, the pool was 3 candidates.

Measured on a live 565-note store, revoked notes were 2 of 565, 0.35%, against roughly 99% active. A pool of 3 has no room for a crowd. Three closer active notes on the same topic always push the one relevant revoked note out before the lifecycle filter runs, and in the normal write path two are enough, because the new note's own vector is already in the collection and takes the first slot. The two paths had target classes whose prevalence differed by more than two orders of magnitude, and they shared one formula.

Commit 8f51803 decoupled depth from render count with a flat floor of 100 candidates on the revoked path. The measured cost, on a synthetic 50,000-vector collection: about 0.59 ms mean for 3 results against about 0.94 ms for 100. Under a millisecond either way. The bug had been saving 0.35 ms per write.

Why the floor is flat

The floor is 100 whatever the store size, and prevalence did not set it. Under the independence model, 100 candidates find a randomly placed revoked note less than a third of the time, so if prevalence were the whole story, 100 would be far too few. Here the correlation runs the helpful way. The revoked note worth surfacing is a near-duplicate of the note being written, so it ranks high. What pushes it out of the pool is the handful of active notes on the same subject that rank higher still, and that handful is bounded by how much anyone writes about one topic, not by how large the store grows. Anything further down would not be shown anyway: the lookup drops candidates below a cosine similarity of 0.75 by default.

Prevalence, then, explains why the active path's formula was wrong for this path. The depth that is right comes from a different number, the count of non-passing notes that can crowd in above the answer.

What such a warning should say once it is found is a separate question, taken up in Should a Correction Repeat the Mistake?. The wording matters only if retrieval reaches the revoked record at all.

Instance 3: One Arm of an OR Query Flooding a Shared LIMIT

Path-based recall answers "which notes concern the file about to be edited?" A note can match in three ways: its text mentions the path, it is explicitly anchored to the file, or it declares a trigger glob that matches it. All three were combined with OR into one SQL query with one LIMIT, and Python did the exact matching afterwards.

SQL could scope the first two arms to the file with LIKE patterns on the file name. It could not scope the third. A glob such as **/*.py has no LIKE equivalent, so the SQL step matched every note that declares any path trigger at all, anywhere in the workspace, and left the glob to Python. A workspace with enough such notes filled the shared LIMIT and pushed genuine file-scoped matches off the end before the narrowing ran. In a stress run of the test harness with a broad glob, 20 of 20 test windows lost their genuine anchored match. After the fix, 0 of 20 did.

recall_for_path, commit 24fba5f (simplified)

-- before: three arms, one pool, one LIMIT
SELECT * FROM notes WHERE workspace = ? AND valid_until IS NULL
  AND (content LIKE ? OR anchors LIKE ? OR triggers LIKE '%"path"%')
  ORDER BY ... LIMIT ?;

-- after: the arm SQL cannot scope gets its own LIMIT
SELECT * FROM notes WHERE workspace = ? AND valid_until IS NULL
  AND (content LIKE ? OR anchors LIKE ?) ORDER BY ... LIMIT ?;
SELECT * FROM notes WHERE workspace = ? AND valid_until IS NULL
  AND triggers LIKE '%"path"%' ORDER BY ... LIMIT ?;
-- union, dedupe by note_id, then narrow in Python

Commit 24fba5f split the query in two: one bounded query for the arms SQL can scope to the file, one bounded query for the trigger arm, unioned and deduplicated by note id, and only then narrowed. However many notes declare triggers, they can no longer take a file-scoped match's place in the pool.

A second defect sat behind the first. Once the pool was right, genuine matches could still exceed the limit, and the final cut to limit was ordered by recency alone, so a newer weak match could displace an older strong one. Matches are now ranked by relation strength first (a declared anchor, then a matching trigger, then a mention in the text) and by recency within each tier. Fixing the pool moved the truncation one stage downstream, and that second cut needed its own ordering.

Instance 4: A Kind Filter and a Sort the Pool Could Not Reach

Recall can be filtered by note kind, one of seven. The kind filter ran after a fixed limit * 3 semantic fetch, and a selective kind returned fewer than limit notes with no signal that the shortfall came from the pool rather than from an absence of matching notes.

Three days before the kind filter was fixed, the codebase had pulled the over-fetch rule into a single helper whose docstring states the invariant for every query against the collection: size the fetch through the helper, never with a raw render limit. The kind filter went through the helper and still under-filled. The helper's rule is a multiplier of 3, which assumes a third of the neighbours pass, and a filter that keeps one kind in seven sits well outside that assumption. A shared helper makes every call site pay the same bet; it cannot know that one caller's filter is seven times narrower than another's.

Now the pool widens to limit * 10 when a kind is set, a figure the code comment calls "a guess, not a measurement". Under independence, with a limit of 5, that returns a full five notes of a one-in-seven kind about 86% of the time. The rest is covered by a backstop: a plain query on the notes table, with no vector search involved, that pulls the newest notes of the requested kind and appends any the pool missed after the ranked results. It guarantees the count, not the ranking. Those appended notes carry no similarity score and sit after every note the semantic pool did rank, however relevant they are.

Sorting a pool cannot reach what the pool missed

A close relative sits in the same function. Recall can also be sorted by recency, priority or chronology instead of relevance, and those sorts were applied to the semantic candidate pool. The most recent note, or the highest-priority one, may not be semantically close to the query at all, so it never entered the pool, and no re-sorting of a pool can surface an item that is not in it. Sorting a similarity pool by recency answers "the most recent of the similar items", which is rarely what "sort by recency" was meant to mean.

Explicit sorts now re-query the store under the requested ordering, so membership follows the requested key rather than query similarity. Rows fetched that way are not dropped for low similarity either: they were never scored, and a similarity floor means nothing for a row with no score. The regression test states the case exactly: seven on-topic decisions and one decision about an unrelated subsystem, dated oldest of all, with an embedding orthogonal to the query so it cannot enter the pool. A chronological recall must return it first.

Relevance Is the Wrong Sort Order for Agent Memory takes up which ordering is right for memory in the first place. The mechanical constraint holds whichever ordering wins: it has to be applied to a candidate set chosen by that ordering.

Part 3: Why Nothing Noticed

A Short Result Is a Claim About the Data

Each instance survived because its output was plausible. An empty related-notes list reads as "nothing related exists". Two notes where five were asked for reads as "only two match". When the revoked-note warning never appears, the natural reading is that the belief was never revoked. In every case the wrong answer is a valid answer for a slightly different store, and neither the caller nor a test that checks well-formedness can tell it apart from the right one.

That is the general danger of truncating and then filtering: it converts a search error into an apparent fact about the data. A retrieval layer that returns fewer items than requested is making an implicit claim, "there are no more", and a post-filter makes that claim without having checked it.

For agents the cost compounds. A person scanning search results may notice that something they expected is missing and search again. An agent that recalls memory and gets a short list has no signal that would prompt a second, deeper query, so it proceeds as if the list were complete. The missing note is absent from every decision downstream of that answer, and nothing in the transcript marks where it would have mattered.

Warning

Offline evaluation can share the hole. recall@k is only as good as its ground truth, and a ground truth built by running the same filtered query inherits the same omissions. Build it by filtering the whole store first and ranking the survivors exactly.

Part 4: Fixes That Hold

Fixing Post-Filtering in Application Code

The list runs from weakest to strongest. The weaker fixes are worth knowing because the strongest one depends on the engine, and on the data being in the engine at all.

Report the underfill

When fewer than n results survive and the fetch stopped before the store did, say so: "returned 2 of 5 requested; candidate pool of 15 exhausted". The answer is no better, but a silent claim has become a visible uncertainty that a caller, or an agent, can act on. Telling the two cases apart is cheap if you are careful about one detail. The store ran out if the fetch covered every item, or if the last candidate fetched already falls below the similarity floor, since nothing further down could have been shown. A fetch that came back with fewer rows than it asked for is not proof on its own: some indexes cap what one scan can return. pgvector's HNSW scan, for one, returns at most hnsw.ef_search rows (40 by default) however large the LIMIT, unless iterative scans are switched on.

Size the pool for the filter, not for the render

Depth and display count are different quantities. Measure s instead of guessing it: take a sample of real queries, fetch each one deep and unfiltered (a few hundred results), and count what fraction of the top 50 pass the filter. Look at the low end of that distribution, the tenth percentile say, not the mean, because the queries that underfill are the ones with the lowest s. Then size for the tail, since n / s is the depth that works only a little over half the time. When the wanted class is rare but its members sit close to the query, as with the revoked notes, the depth is set by how many non-passing items crowd in above them. The revoked-note floor of 100 cost 0.35 ms.

Filter first when the passing set is small

If the filter keeps only a few hundred items, invert the order: select them by the filter, then score each one exactly against the query. A few hundred dot products cost less than embedding the query did, and the answer is exact. It is an easy fix to overlook, because a brute-force loop sitting next to a vector index looks like a step backwards. At that size it is both the faster option and the exact one. On the revoked path above, filtering first would have meant scoring two notes. Engines with a query planner make this switch themselves when a filter is selective enough. The kind-filter backstop is a partial version: it selects by the filter first, but orders by recency instead of scoring.

Give each arm its own limit

When a query is a union of conditions with different selectivity, run each as its own bounded query and merge afterwards, so one broad arm can never starve a narrow one. Then look at the cut that comes after the merge. Once the pool is right, the final truncation needs an ordering that reflects what the caller values, or it reproduces the starvation one stage later.

Re-query for orderings other than relevance

A sort that is not by similarity needs a candidate set chosen by that sort. Fetch the top rows by the requested key under the same metadata filters, and let membership follow the key, rather than fetching by topic and sorting the result by date.

Iterate until satisfied

Fetch, filter, and if the survivors are short and the store has more, fetch deeper and repeat. This is the general solution when selectivity is unknown in advance.

Filtered search with an honest stopping rule

def filtered_search(index, q, n, passes, min_sim=None, k_max=None):
    """Nearest n items that pass, or an explicit account of why fewer."""
    size = index.count()
    k_max = min(k_max or size, size)
    k = min(4 * n, k_max)
    while True:
        hits = index.search(q, k)                      # best first
        keep = [h for h in hits
                if passes(h) and (min_sim is None or h.score >= min_sim)]
        if len(keep) >= n:
            return keep[:n], None
        saw_all = len(hits) >= size
        below_floor = min_sim is not None and hits and hits[-1].score < min_sim
        if saw_all or below_floor:
            return keep, None                         # short, and true
        if k >= k_max or len(hits) < k:               # our cap, or the index's
            return keep, f"returned {len(keep)} of {n}; searched {len(hits)} of {size}"
        k = min(2 * k, k_max)

Four things end the loop: enough survivors, every item seen, the similarity floor crossed, or a cap reached, either the caller's k_max or one inside the index that returned fewer rows than requested. Only the last produces a short answer that might be wrong, and it is the one that carries a report. Doubling k keeps the total work within about twice the final fetch. Each round keeps its own hits rather than appending pages, because an approximate index searched deeper is not guaranteed to return a superset of what it returned shallower.

pgvector added exactly this loop inside the database in version 0.8.0, as iterative index scans.

The code search already did this

A search pipeline elsewhere in the same system did the right thing without naming it. Before trimming candidates for the reranker, it fetches 200 raw candidates and drops trivial chunks first (one-line templates, fixture stubs, a lone import), so the pool is filled with real code even when a repository carries a hundred or more trivial chunks. It then keeps the first 40 non-trivial candidates from the vector ranking and, separately, the first 40 from the keyword ranking, and reranks the union. That is over-fetch sized for the filter and a separate limit per arm, with the reason written down in the configuration comment. When a language filter is set, that condition travels with the vector query instead, because every chunk's language is stored alongside its vector.

Filtered ANN Search Inside the Index

The strongest fix is a filtered approximate nearest-neighbour search that respects the condition while it walks the index, so the answer is genuinely "the nearest k among items that pass". For graph indexes that is harder than it sounds. An HNSW graph is built so that a greedy walk from an entry point reaches a query's neighbourhood in a few hops, and its edges are chosen by geometry alone. Skip the nodes that fail a selective filter and the walk has fewer usable paths toward the answer, or ends up visiting most of the graph to find the few nodes that pass. (For HNSW from first principles, see the section on approximate search in Text Embeddings, Vector Databases and LLMs: A Full Guide.)

Two research lines address this directly. Filtered-DiskANN (Gollapudi et al., WWW 2023) builds the graph from the vectors and their labels together, so edges connect points that share a label as well as points that are close. It defines a filter's specificity as the fraction of points carrying the label, and notes that the search-then-filter approach may need a very large number of candidates before it finds a single match when specificity is low. ACORN (Patel et al., SIGMOD 2024) takes the predicate-agnostic route. It builds an HNSW-style graph with each node's candidate neighbour list expanded by a factor γ, then searches only the subgraph induced by the nodes that pass, and it reports 2 to 1,000 times higher throughput than prior methods at fixed recall. The paper sets γ from the inverse of the smallest selectivity to be supported: the same n / s arithmetic, moved from query time into index construction.

Several engines now document a filter mode that applies during the search. The vocabulary does not line up across them. Elasticsearch and Weaviate call a filter that the graph walk honours as it goes a pre-filter; much of the research literature keeps that word for "apply the filter, then scan the survivors exhaustively" and calls the in-walk version filtered search. The label matters less than two facts about each engine: where the condition meets the ranking, and whether the result can come back with fewer than k rows when enough matches exist. As documented in September 2026:

Engine	Filtered query, as documented	What to watch
pgvector	With an approximate index (HNSW, IVFFlat), the WHERE condition is applied after the index is scanned. The README's example: a condition matching 10% of rows, with HNSW and the default `hnsw.ef_search` of 40, returns about 4 rows on average.	Iterative index scans (0.8.0 and later) are opt-in: `hnsw.iterative_scan` set to `strict_order` or `relaxed_order`, capped by `hnsw.max_scan_tuples` (20,000 by default); `ivfflat.iterative_scan` with `ivfflat.max_probes`. `relaxed_order` can return rows slightly out of distance order; the README re-sorts them with a materialized CTE.
Elasticsearch	The `filter` parameter of the `knn` query is applied during the approximate search, so k matching documents are returned.	Every other filter in the query is a post-filter, including a term filter in a bool query's filter clause wrapped around the knn query. That returns fewer than k results even when enough documents match.
Qdrant	Filterable HNSW: extra graph edges built from payload indexes let the search apply filters as it walks the graph.	Create payload indexes before ingesting data: the filter-aware edges exist only if the graph is built after the indexes, and adding one later means rebuilding. When a condition matches few enough points, the query planner uses a full scan instead of the graph.
Weaviate	Pre-filtering: an allow-list from the inverted index, honoured during the HNSW search. ACORN is the default filter strategy since v1.34; sweeping is the alternative.	Switches to a flat, brute-force search over the allowed set when the filter is very restrictive.
LanceDB	Pre-filtering is the default: the `where` condition is applied before the vector search.	Post-filtering is opt-in and can return fewer than `limit` rows, or zero, if the nearest neighbours fail the filter.

Each row is taken from the project's own documentation; see Sources. Features in this area change quickly, so check the version you run.

Warning

The query text is not the plan. In pgvector, WHERE category = 'x' ORDER BY embedding <-> q LIMIT 10 reads like "filter, then rank", but when the planner uses the HNSW index it scans ef_search candidates and filters them afterwards. In Elasticsearch, a term filter in a bool query around a knn query reads like a condition on the search, and the documentation is explicit that it is a post-filter. Both return the familiar well-formed short list. Check the execution plan, not the query.

Pushing the filter down has a precondition: the index must hold the facts the filter tests. In the memory store behind the four instances, the vectors were written with ids only, and the filters tested columns in a relational table and a lifecycle state computed from an event log. No engine can apply a condition to data it was never given. Two-store designs, vectors in one system and the facts about them in another, are where this bug breeds, and moving the filterable facts next to the vectors is often the real fix.

Native support also has a ceiling. Iterative scans stop at a configured cap, and a filtered graph search is still approximate. The underfill report from the weakest fix belongs on the strongest one too.

Part 5: Catching It

An Adversarial Test for Filtered Retrieval

Every test that passes against buggy post-filter code has one property in common: the fetch already reached the answer, so k ≥ r_n held. The store was small enough, or the filter permissive enough, that depth never mattered. The test that catches the bug is the adversarial one:

Build a store where the nearest neighbours of the test query all fail the filter: other workspace, revoked, wrong kind.
Put the true answer further away, but passing the filter.
Assert that the true answer is returned.

An adversarial regression test

def test_answer_survives_when_nearest_neighbours_all_fail(make_store):
    # Vectors placed at chosen cosines to the query, so the ranking
    # is fixed by construction rather than by a model's opinion.
    store = make_store(embed=cosine_stub({
        "decoy a": 0.99, "decoy b": 0.98, "decoy c": 0.97,
        "the answer": 0.90,
    }))
    for text in ("decoy a", "decoy b", "decoy c"):
        store.add(text, workspace="other")
    answer = store.add("the answer", workspace="mine")

    got = store.search("query", n=1, workspace="mine")

    assert [g.id for g in got] == [answer]

Use a stub embedder. With a real model, whether the decoys outrank the answer is the model's opinion, and a model upgrade can make the test pass for the wrong reason. The vectr regression tests do this in two ways: one helper returns unit vectors at specified cosines to a base vector, another maps marked text to a vector orthogonal to everything else so a note is guaranteed to rank last.

The tests added with the related-notes, revoked-notes and path-recall fixes, and with the sort fix, all have this shape, and each fails against the code before its fix. The first two get an empty list back. The path-recall test gets a list without the anchored note. The sort test gets a chronological list that starts at the wrong note. The kind-filter fix is covered differently, by tests that pin the pool width and check that the prefetch runs; an outcome test of the adversarial kind would be the stronger guard there.

That is the regression shape worth keeping: not "does filtering work" but "does filtering still work when the filter removes the closest things".

The Line to Keep

"Fetch the nearest k, then filter" answers a different question from "the nearest k that pass the filter", and the difference is invisible because a short answer looks like a complete one.

Size the fetch for the filter, give each condition its own limit, and when the result is short, say whether the store ran out or the search did. Where the engine can apply the filter during the search, let it, and confirm that it does from the execution plan rather than the query text.

Sources

vectr, commit 77252a6, "Over-fetch candidates in related_active_notes before post-query filtering". The related-notes fetch depth and the two failing cases (foreign-workspace and revoked nearest neighbours). Code: agent/working_context_store/_related.py.
vectr, commit 8f51803, "decouple revoked-path query depth from revoked_limit". The 2-of-565 revoked-note measurement, the 0.59 ms and 0.94 ms timings on a 50,000-vector collection, and the revoked_query_floor setting with its rationale in agent/config.yaml.
vectr, commit 24fba5f, "recall_for_path declared-trigger arm can no longer starve content/anchor matches". The split into two bounded queries, the relation-strength ordering of the final cut, and the 20 of 20 versus 0 of 20 stress result. Code: agent/working_context_store/_store.py.
vectr, commit 24726e4 and the 1.12.0 entry in the CHANGELOG, "Kind-filtered recall no longer under-fills silently". The limit * 10 kind floor and the metadata-only prefetch backstop.
vectr, commit 841a149 and the 1.11.0 CHANGELOG entry, "An explicit non-relevance recall sort returns the true top-limit". Re-querying under the requested ordering, and the orthogonal-embedding regression test.
vectr, commit 77247d6. The single over-fetch helper, working_memory_fetch_width, and its stated invariant.
vectr, ranking.rerank.pre_filter_fetch_k and top_k_unfiltered in agent/config.yaml, and the trivial-chunk pool filter in agent/searcher.py. The code-search over-fetch described under "The code search already did this".
pgvector, README, sections "Filtering" and "Iterative Index Scans"; and the pgvector 0.8.0 release announcement. Filtering after the approximate index scan, the 10% and ef_search 40 example, and the iterative scan settings.
Elasticsearch, Knn query reference, "Pre-filters and post-filters in knn query". Which filters are applied during the approximate search and which after it.
Qdrant, Indexing, "Filterable Index". Payload-index edges in the HNSW graph and the full-scan threshold.
Weaviate, Filtering concepts. Allow-list pre-filtering, the ACORN and sweeping strategies, and the flat-search switch for restrictive filters.
LanceDB, Metadata filtering. Pre-filtering as the default and the documented short results of post-filtering.
Gollapudi, S., Karia, N., Sivashankar, V., Krishnaswamy, R., Begwani, N., Raz, S., Lin, Y., Zhang, Y., Mahapatro, N., Srinivasan, P., Singh, A., Simhadri, H. V. Filtered-DiskANN: Graph Algorithms for Approximate Nearest Neighbor Search with Filters. Proceedings of the ACM Web Conference 2023 (WWW '23), pp. 3406 to 3416. Label-aware graph construction (FilteredVamana, StitchedVamana) and the definition of filter specificity.
Patel, L., Kraft, P., Guestrin, C., Zaharia, M. ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data. SIGMOD 2024, Proceedings of the ACM on Management of Data 2(3); arXiv:2403.04871. Predicate subgraph traversal and the γ neighbour-list expansion.
Saha, S. Relevance Is the Wrong Sort Order for Agent Memory. Which ordering memory retrieval should use, and why similarity ranking drops corrections.
Saha, S. Text Embeddings, Vector Databases and LLMs: A Full Guide. Embeddings, cosine similarity and HNSW search from first principles.

Should a Correction Repeat the Mistake?

Swapnanil Saha — Wed, 07 Oct 2026 18:50:54 +0000

To retract a belief, an agent memory can delete it, replace it, or keep it on file labelled false. Only the last can warn an agent about to rediscover the belief elsewhere, and it has to state the belief to do it. Human memory research says the restatement helps. Transformer research says it may prime the error.

Date: 18 September 2026. Reading time: about 28 minutes.

When an agent memory learns that one of its beliefs was wrong, it has three options. It can delete the belief. It can replace it with the correct one. Or it can keep a record: this was believed, it turned out false, here is why, do not re-derive it.

Only the third option can warn an agent that is about to rediscover the wrong belief from somewhere else. It also has a cost the other two avoid. To say what was wrong, it has to say the wrong thing again.

On whether that repetition helps or hurts, two bodies of research point in opposite directions. Human memory research, after a decade of revising its own advice, says restate the myth: corrections that name the false claim work better, and the feared backfire effect from repeating it does not replicate. Research on transformers suggests the opposite may hold for models. Telling a model not to produce a word can make the word more likely, and the effect grows with the amount of related text that follows the instruction. A coding agent typically reads a correction and then reads a great deal of related text.

Every memory system's correction format is a bet on one of these answers. Most systems place the bet without knowing there was one.

Agents Over-Trust Instructions and Under-Trust Corrections recommends that a correction name the exact belief it overrides. That leaves open how to name it, when the evidence on naming splits by substrate. The misinformation literature supplies the vocabulary: the myth is the belief being withdrawn, and a retraction is the statement that withdraws it.

Three Ways an Agent Memory Can Un-Believe Something

An agent memory holds notes that survive between sessions: where a flag is read, which command runs the tests, what the user decided last week. When one of those notes turns out to be wrong, the store has three moves available.

Delete the wrong note

The note is removed, and the store no longer holds the belief. This is the simplest behaviour to build and a common one. In several memory libraries, each incoming fact is shown to a language model alongside the similar entries already stored, and the model chooses to add it, update an old entry, delete one, or do nothing. A contradicted entry gets deleted or overwritten.

Replace it with the correct fact

The note is removed and the correct fact is stored in its place. The store now holds only the truth. Nothing records that a different belief ever existed.

Retract it with a record

The old note stays, marked revoked, with a reason and a date, and the correct fact is stored alongside it. Memory stores deliver notes in two ways: the agent searches for them, or the store injects them into the agent's context on its own when something related comes up. When the revoked note becomes relevant again by either route, what surfaces is not its original content but a retraction:

A retraction as it reaches the agent (illustrative values)

Do not re-derive this from other sources without verification. Previously believed (recorded 2026-07-14, revoked 2026-07-21, reason: the flag moved to the settings service in 2.3): "Feature flags are read from config/flags.yaml".

The shape is the template that vectr, an open-source memory store I maintain, uses for revoked notes. The dates, reason and claim above are illustrative. The template itself, verbatim from agent/working_context_store/_store.py:

_ANTI_MEMORY_TEMPLATE = (
    'Do not re-derive this from other sources without verification. '
    'Previously believed (recorded {created_date}, revoked {revoked_date}, '
    'reason: {reason}): "{summary}".'
)

Temporal knowledge graphs sit between the second and third options. A superseded fact gets an invalidation timestamp and stays queryable as history, but nothing injects it; somebody has to go looking.

The options also differ in storage cost and in how easy they are to audit or undo. For agents the axis that decides between them is a different one: what happens when the wrong belief comes back from outside the store.

Analogy

A publisher finds an error on page 12 of a reference manual. Pulling the book from the shelf (delete) does nothing about the photocopies already circulating. Printing a corrected edition (replace) helps whoever picks up the new edition, but a reader holding an old photocopy has no reason to doubt it. An errata slip (retract) is the only option that speaks to the reader of the photocopy, and it can only do so by quoting the error: page 12 says X; X is wrong; the correct value is Y.

Where the analogy falls short is delivery. An errata slip only helps a reader who sees it, and for an agent, where and when the retraction is delivered turns out to matter as much as what it says.

Why Deleting a Wrong Memory Does Not Remove the Belief

The case for deletion rests on an assumption: removing the stored belief removes the belief. It fails in people and in model weights, where it has been tested directly, and it fails more obviously still in an agent's workspace.

In people: directed forgetting

In directed forgetting experiments, participants study items and are then told to forget some of them. Forget-cued items are recalled less often, but the evidence says they are still stored. In the list-method version of the task, the forgetting shows up on recall and typically vanishes on a recognition test, where the item is shown again (Abel et al., 2021, summarize the pattern). Current reviews attribute the effect to control of attention during study and to a deliberate shift of mental context (Delaney et al., 2020). None of the explanations on offer involves anything being removed; the item has become harder to reach while staying in storage.

In model weights: editing and unlearning

Knowledge editing changes a single fact inside a model's weights. The RippleEdits benchmark (Cohen et al., TACL 2024) checks the fact's consequences as well as the fact itself: edit who someone's parent is, then ask about their siblings. Current editing methods fail to make those consequences consistent. The edited fact moves; the facts that depend on it often do not. The best score on that benchmark came from the simplest baseline, in-context editing: putting the new fact in the prompt, which is what an agent memory does.

Machine unlearning does worse. (Retraining from scratch without the data does remove it; it is also too expensive to do per fact, which is why the approximate methods below exist.) Zhang et al. (ICLR 2025) found that for unlearning methods with utility constraints, models still retain on average 21% of the knowledge they were meant to forget at full precision, and 83% after 4-bit quantization. Hu et al. (ICLR 2025) showed that fine-tuning an unlearned model on a small, loosely related public dataset brings the unlearned content back; general wiki text about a book series was enough to make a model reproduce memorized passages of the books verbatim. Whatever the unlearning step did to the weights, it left enough behind for a small amount of training on adjacent text to bring the passages back.

In an agent's workspace

For a coding agent, the belief was never only in the memory store. It was in the documentation, in comments, in a vendored copy of a config, in twenty call sites that still use the old API. Deleting the note deletes the only object that knew the belief was wrong. The next session reads the stale README, re-derives the old fact, acts on it, and may well write it into a new file, where the session after that will find it.

Research on unlearning in agents has named a narrower version of this loop. Wang et al. (2026) call it parameter-memory backflow: an agent has its content removed from the model's weights and from its memory store, and the content comes back anyway, because retrieval reactivates what is left in the weights, or because derived artifacts (indices, summaries, embeddings, caches) still carry it and reintroduce it. Their loop runs between the weights and the memory store. A workspace runs the same kind of loop through many more places, and no unlearning method reaches any of them.

Deletion and replacement both leave the workspace's stale copies unopposed. Replacement at least stores the truth, but the truth arrives as one note, with no indication that the workspace disagrees with it and that the workspace is the one that is wrong. Only a retraction record says so. Whether saying so helps is the open question.

Insight

Backflow in a workspace does not need the model to remember anything. The stale README is enough. That makes it the common case rather than an exotic one: any fact that changed in code but not in prose can be re-derived by the next session that reads the prose, and a store that deleted its note has nothing left to object with.

The Human Answer: A Correction Should Name the Myth

The misinformation literature measures the quantity at stake here directly: how much people keep relying on information after it has been retracted. Its most argued-over practical question for the last decade is whether a correction should repeat the myth.

Retractions alone are weak: the continued influence effect

The continued influence effect is one of the most replicated findings in the field. In the standard paradigm (Johnson and Seifert, 1994), people read a report of a warehouse fire said to be caused by gas cylinders and oil paints stored in a closet. Some then read a retraction: the closet was actually empty. Asked later what caused the black smoke, many still cite the paint.

The major review of the literature (Lewandowsky et al., 2012) is blunt: retractions "rarely, if ever, have the intended effect of eliminating reliance on misinformation, even when people believe, understand, and later remember the retraction." A retraction at most halves the references to the misinformation compared with no retraction, and in some studies it does not reduce them at all. Delivering it immediately, inside the same story, does not fix this. Making the negation more emphatic ("paint and gas were never on the premises") made reliance worse in one study. The review found only three things that reliably help: a warning before exposure, repeating the retraction, and an alternative account that fills the gap the retraction leaves.

A warning plus an alternative works best

Ecker, Lewandowsky and Tang (2010) tested warnings directly. A specific warning, one that explained the continued influence effect to participants up front, reduced reliance on the retracted claim without eliminating it. A general warning (news is not always fact-checked before publication) did even less. In a further experiment, a specific warning combined with a plausible alternative explanation reduced reliance further, and still did not eliminate it.

The accepted account is causal. People keep a false belief when removing it would leave a hole in their model of what happened; the retraction says what did not cause the smoke and leaves the smoke unexplained, so the paint stays in use as the best explanation available. The alternative has to do real work: it should cover the same facts the myth explained, ideally say why the myth seemed right in the first place, and not be more complicated than the myth it replaces (Lewandowsky et al., 2012).

Repeating the myth does not backfire

For years the practical advice was never to repeat the myth, on the theory that repetition makes a claim familiar and familiarity is mistaken for truth. The 2012 review itself advised emphasizing the facts rather than the myth for that reason. The familiarity backfire has not held up.

Ecker, Hogan and Lewandowsky (2017) found that retractions which explicitly repeated the misinformation reduced reliance on it more than retractions that avoided repeating it, and attribute the gain to salience: a correction that names its target is easier to notice as a correction. Ecker, Lewandowsky and Chadwick (2020) then tested the case in which backfire should be most likely: correcting a myth the audience had never heard, so that the correction itself was the first exposure. Across three experiments and a one-week delay, they found substantial evidence against a familiarity backfire. The Debunking Handbook 2020, written by 22 researchers in the area, concludes that "repeating a myth while refuting it has been found to be safe in many circumstances, and can even make the correction more salient and effective."

The new-audience case is the normal one for agents. A fresh session that has not yet opened the stale README meets the myth for the first time in the retraction.

A candidate mechanism: memory reconsolidation

There is a mechanistic reason naming the myth might be necessary rather than merely safe, though it comes from a different literature and should be held loosely. Nader, Schafe and LeDoux (2000) showed in rats that a consolidated fear memory, when reactivated by retrieval, becomes unstable again and has to be re-stored. Injecting a drug that blocks protein synthesis (which the brain needs to re-store it) right after reactivation removed the fear response; the same injection without reactivation left the memory intact. The idea that retrieval opens a memory to updating, called reconsolidation, has since been pursued in humans, and whether it governs everyday belief revision is still debated (Lee, Nader and Schiller, 2017, review both the evidence and the challenges). As a mechanism, it suggests why a correction that never names what it corrects might have nothing to attach to. None of the misinformation findings above depend on it.

Corrections decay unless revisited

When people get corrective feedback, the errors they held with the most confidence are the most likely to be fixed on a retest. That is the hypercorrection effect (Butterfield and Metcalfe, 2001). The fix does not always last. Butler, Fazio and Marsh (2011) found that after a week, correction rates fell, and the errors that came back were disproportionately the high-confidence ones. Metcalfe and Miele (2014) found that a test given immediately after the corrective feedback, requiring people to produce the right answer themselves, both improved delayed memory for the answer and blocked the old errors from returning. Metcalfe's own later review calls the evidence on error return not yet definitive. I would still design around the direction: a correction seen once fades, and the confidently held error is the one most likely to come back.

The Debunking Handbook 2020 condenses the practical advice into a four-part recipe for a written correction:

Fact: lead with the truth, if it is short and clear
Myth: warn it is coming, then state it once
Fallacy: explain how the myth misleads
Fact: finish on the truth, repeated if possible

The fallacy step is where the alternative account lives. The Handbook's instruction is to explain why the mistaken information was thought to be correct in the first place, why it is now clear it is wrong, and why the alternative is correct. It is also blunt about the minimal version: "Do not rely on a simple retraction ('this claim is not true')."

The Transformer Answer: Naming the Error May Prime It

The research on language models points the other way, and for a reason that has nothing to do with familiarity.

Ironic rebound in transformers

In people, trying not to think of a white bear makes the bear come to mind; Wegner and colleagues (1987) called it a paradoxical effect of thought suppression, and it is now known as ironic rebound. Mann et al. (2025) looked for the same effect in transformers. Their benchmark, ReboundBench, contains 5,000 prompts built around the instruction "do not mention X", each followed by a stretch of distractor text that varies in length and kind. They measure how likely the model is to produce X next, against a matched prompt with no negation.

Across nine open-weight models, they report that rebound "consistently arises immediately after negation and intensifies with longer or semantic distractors." The forbidden word becomes more likely, not less, and the effect is strongest when the text after the instruction is on the same topic; repetitive filler supported suppression instead. A circuit analysis traced the effect to a sparse set of middle-layer attention heads that amplify the forbidden token while earlier layers suppress it. The paper's own framing of why: suppressing a concept "requires internally activating it, which may prime rebound instead of avoidance."

Math · what rebound measures

The paper's suppression score, in simplified form, for a forbidden word X and L tokens of distractor text between the instruction and the measurement point:
rebound(L) = log₂ p(X | "do not mention X" + L tokens)
          − log₂ p(X | neutral context + L tokens)
Here p(X | ...) is the model's next-token probability for X. A positive value means the instruction not to mention X made X more likely than it would have been with no instruction at all; each unit is a doubling of the probability. The paper converts L to percentiles within each model's range of distractor lengths, to account for different context windows, so "longer" is relative to the model. Its headline surprisal measure compares against a high-load baseline instead of a neutral prompt; the sign convention is the same.

The models ranged from GPT-2 Small to two 20-billion-parameter models. The smaller models showed only a brief rebound before suppression took over; the mid-sized ones sustained the strongest effect. One of the largest, GPT-OSS-20B, showed minimal or even negative rebound, which the authors note breaks the scaling trend. That matters here: the models behind most coding agents are, by most estimates, far larger than anything tested, and the one data point at the top of the range points the other way. The measure is also narrow: the probability of a single-token common noun in templated, lowercased prompts, not the behaviour of an agent editing code. Carrying the result over to a retraction in an agent's context is an extrapolation, and I would treat it as a hypothesis worth guarding against rather than a known cost.

Negation is weak in general

García-Ferrero et al. (EMNLP 2023) built a dataset of about 400,000 commonsense sentences, two-thirds of them containing negation, and found that large language models classify affirmative sentences well but struggle with negated ones, often relying on surface cues rather than on the negation itself. A retraction is, structurally, a negation wrapped around a quoted claim.

Knowledge conflicts: coherence and agreement beat truth

Xie et al. (ICLR 2024) studied what models do when evidence in the prompt conflicts with what they learned in training, their parametric memory. Three results bear on retractions. Models readily accept contradicting evidence when it is coherent and convincing. They favour evidence that agrees with what they already believe. And when several passages are in context, they tend to side with whichever answer more passages support; the paper's section heading is "LLMs follow the herd and choose the side with more evidence."

A fourth detail, from the paper's appendix, bears on the length of a correction. When the contradicting evidence was cut down to a bare answer with no supporting explanation, ChatGPT's adoption of it on PopQA questions fell from 56.7% to 18.8%. Cutting the evidence that agreed with the model's prior to a bare answer did nothing of the kind; it was chosen slightly more often (42.7% to 43.9%). So agreement survives being shortened, and contradiction does not.

In an agent's context, the quoted false claim agrees with the stale README, the stale comments and the old call sites. If the model weighs text by coherence and agreement, the quote risks counting as one more vote for the wrong answer.

Every injected sentence has a cost

Shi et al. (ICML 2023) took grade-school maths problems that the models could already solve and added one irrelevant sentence to each. Of the problems solved without the extra sentence, no more than 18% stayed solved across every type of distractor. Sentences on the same topic as the problem, reusing its character names, did the most damage. An explicit instruction to ignore irrelevant information recovered some of the loss. For a memory store, this is the argument for gating delivery on relevance: a retraction injected into a session where the old belief was never going to come up can only distract.

The agent setting combines these conditions. An agent receives a retraction and then, in the same context window, reads a long, topically related document that asserts the retracted claim. That is close to the condition in which the rebound study found its strongest effect: a negation followed by a large amount of related text.

Why Both Answers Can Be Right

The two literatures study different mechanisms under different conditions, so their answers can both hold.

In the human experiments, a person reads a story, reads a correction, and later answers questions. Processing the retraction attaches something like a "false" tag to the memory; the 2012 review calls it a negation tag. The later question retrieves the memory together with its tag. The retracted claim is not in front of the person at the moment of use. It is recalled, and recall brings the correction along.

The human literature knows the failure mode of that tag, too. Johnson and Seifert (1998) found that a corrected detail was still activated in memory whenever the story referred back to it. The negation tag can also be lost under cognitive load (Gilbert, Krull and Malone, 1990), leaving the misinformation behind on its own. Negations hold best when they can be re-encoded as a positive statement: people told someone is "not messy" tend to remember "tidy", while "not charismatic", which has no ready opposite, drifts back towards "charismatic" (Mayo, Schul and Burnstein, 2004). For retraction design this is the most useful finding in the human literature, because it transfers directly: a retraction that can be read as "use Y" holds up better than one that can only be read as "not X".

In the agent case, the retracted claim is in the context window, as tokens, at the moment of use, next to other copies of the same claim from the workspace. There is no separate retrieval step that fetches a memory with its tag attached. The negation has to win against the quoted content, in attention, at every step where the claim is relevant.

Analogy

The human case is a filing cabinet. The correction is stamped on the card, and pulling the card out brings the stamp with it. The model case is a meeting where every document is read aloud at once: the retraction is one voice saying "not X", while the README and three old call sites are four voices saying "X", and nothing is filed anywhere.

The analogy overstates one thing: attention is not a vote count. Xie et al.'s herd result suggests it behaves like one often enough to matter.

So the human research answers the question "does naming the myth help the correction get remembered and used?", while the transformer research asks "does naming the myth add activation to the myth?" Both can be true at once. For a memory designer, the practical question is which effect dominates in a real agent session, and that has not been measured.

Designing a Retraction for Agent Memory That Hedges Both Bets

On several design questions the two literatures agree, and those choices are easy. On the rest, a retraction can be shaped so that if the transformer account turns out to be right, the damage is small.

Carry the alternative, always

A warning with no alternative is the weakest correction in the human data, because it leaves a causal gap. The transformer studies above did not test warnings as such, but the closest result points the same way: Xie et al.'s bare-answer finding says a short contradiction with nothing supporting it is the kind of evidence a model ignores. A retraction's reason field should state what is true now and what replaced the old belief, not only that the old one is wrong.

The human evidence adds a sharper requirement: the alternative should explain why the myth looked right. In a workspace, that explanation is nearly always some version of "the code moved and the docs did not". A reason that says so answers the stale file before the agent opens it.

Two reasons for the same retraction

bare:       reason: no longer true
corrective: reason: the flag moved to the settings service in 2.3; read flags
            with settings.flag(name); docs/flags.md predates the move and was
            never updated; verify with ./tools/settings flags list

Keep the restated claim short

The myth should appear once, briefly. In vectr, the retraction quotes only the note's one-line title, never its full content; when the writer gave no title, the title is the note's first line cut at 80 characters. The reason is the unbounded field. So the part that competes with the truth is small and the part that carries it can be as long as it needs. The 80-character cap is a convention; nobody has measured where a longer quote starts to cost more than it helps.

Decide what survives truncation

Injection budgets are small, and a long retraction gets cut, usually from the right. The order of the fields decides what is lost first. In vectr, the template was reordered (CHANGELOG: "The revoked-note deterrent survives truncation") so that right-truncation drops the quoted claim first, then the reason, and keeps the warning longest. When the budget is tight, the myth goes before anything else does.

Do not end on the myth

Here the human literature argues against that same template. Untruncated, it ends on the quoted false claim, and the debunking recipe ends on the fact. A template that puts the warning first, then the claim, then a reason that carries the replacement, ending on the replacement, would satisfy both the warning-first constraint and the human evidence, provided the claim is short enough that the reason still fits.

Warning · the truncation trap

The proviso breaks down under tight budgets. In the fact-last order, right-truncation cuts the replacement before it touches the claim, so under a tight budget the retraction degrades to a warning plus the myth: the combination both literatures rate worst. With a plain character cut, no field order gets both properties, myth lost first and fact kept last. Field-aware truncation gets both: measure the rendered record against the budget before sending it, and if it does not fit, drop the quoted claim as a whole unit, then optional detail such as the verify hint, and only then cut into the reason. If I had to pick one change from this section to ship first, it would be this one. Reordering fields only changes which field breaks first under a tight budget, while checking the rendered record against the budget stops a broken one from being sent.

Interactive demo: A Retraction Under a Budget (runs on the original page)

Pick a template order and a reason, then shrink the budget. The point to observe: with character-level truncation, whatever sits at the end of the template is lost first, so a record that ends on the replacement loses the replacement first. Switch to field-aware truncation and the order can serve the reader while the budget only decides whether the myth appears at all.

Illustrative record; the claim, dates, command and service names are invented. Character mode cuts at the exact character. Real implementations usually back off to a word or sentence boundary, which moves the cut a little and does not change which field goes first. Field-aware mode drops, in order: the quoted claim, the verify hint, the "why it looked right" clause, then cuts from the right only if the warning and replacement alone still exceed the budget.

Make it checkable

A retraction the agent can verify with one command turns passive reading into an action. In the human studies, the corrections that held over a delay were the ones followed by a test, where the learner had to produce the answer (Metcalfe and Miele, 2014). A verify hint is the nearest agent equivalent: the agent runs it and observes the result instead of taking the note's word. Agents Over-Trust Instructions and Under-Trust Corrections argues that a verify hint is the cheapest way to make a correction carry evidence rather than authority. It also has a transformer-side rationale: fresh command output is the kind of coherent, well-supported evidence Xie et al. found models willing to accept.

Deliver it more than once, across sessions

If corrected errors return with time, a retraction visible only on request (the temporal-graph pattern) will not be seen when it matters. The agent about to re-derive the old belief has no reason to query for retractions of a belief it thinks is simply true. For an agent the problem is sharper than for a person, because nothing carries over between sessions on its own. A person who saw a correction last week retains a weakened trace of it; a new session that was not handed the retraction has no trace at all.

Design rule	Human evidence	Transformer evidence
Carry the alternative	Warning plus alternative beats warning alone (Ecker et al., 2010); gap-filling accounts	Bare contradictions are rarely adopted (Xie et al., 2024)
Keep the myth short	State it once only (Debunking Handbook 2020)	Fewer tokens of the myth means less to amplify (inference, not measured)
Field-aware truncation	A myth with no alternative is the weakest correction	A negation with nothing else to attend to is the weakest case (inference from the rebound and bare-answer results)
End on the fact	The debunking recipe ends on the fact	Evidence order matters, but its direction varies by model (Xie et al., 2024)
Make it checkable	Testing after feedback blocks error return (Metcalfe and Miele, 2014)	Coherent in-context evidence is accepted (Xie et al., 2024)
Deliver across sessions	Corrected high-confidence errors return after a delay (Butler et al., 2011)	No carry-over between sessions: undelivered means absent

How to Find Out: A Four-Arm Retraction Experiment

Which effect dominates in a real session can only be settled by running sessions. An experiment designed to do that is public in the vectr repository as benchmarks/anti_memory/DESIGN.md. It has four arms, which differ only in what the store holds and whether it injects:

Arm	Store holds	Delivery	Stands in for
Delete	Nothing	Injected (nothing to inject)	Dropping or overwriting the conflicting note
Audit	Retraction record plus the correct fact	On request only	Temporal history you can query
Replace	The correct fact only	Injected	Updating the note in place
Retract	Retraction record plus the correct fact	Injected	A warning-first retraction record

Each scenario is authored so the workspace still teaches the old fact and the task naturally leads the agent to read it. In one, a feature-flag API has moved: the docs still teach the old call over a YAML file, three legacy modules still use it, and the live path, over a directory of TOML files, is one file away. The majority vote in the workspace points at the old fact. All workspaces are synthetic, invented for the experiment, so nothing in the model's training can supply either fact.

The measured outcome is mechanical. Each session's final files are parsed and the session is scored as backflow (they encode the old fact), correct (they encode the new one) or neither. No language model judges a run. Mechanical is not automatically correct, though. A scorer that matches a command pattern anywhere in a shell string will count a grep that merely mentions the command as having run it, a failure documented in Deterministic, Reproducible, and Wrong. This design anchors its command matches to execution position for that reason.

Two comparisons carry the result. Retract against audit asks whether pushing the record matters. Retract against replace asks whether the record adds anything beyond stating the truth. The second is the one that can kill the idea, and the design treats a loss there as a publishable result.

The comparison has a built-in confound, and the design states it. Retract delivers strictly more text than replace, the same corrective note plus the retraction, so a win for retract is tangled up with injected volume. Every session records the characters injected, and a length-matched placebo arm (replace plus an irrelevant note of the same length) is priced to run only if retract beats replace. The design also records the opposite reading. On the human account, the extra text in the retract arm is the active ingredient, a warning plus an alternative, and calling it padding gets the mechanism backwards. The placebo is what tells the two readings apart: if an irrelevant note of equal length does as well, the content was not doing the work.

Math · the decision rule

Let B_X be the number of sessions, out of three per arm, scored as backflow in arm X.
G0 (the scenario tempts):   B_delete  ≥ 2
G1 (pushing matters):       B_retract ≤ B_audit   − 2
G2 (the record adds value): B_retract ≤ B_replace − 2
Supported requires G0, G1 and G2. Refuted is G0 with retract doing no better than audit or no better than replace. Anything else is inconclusive, which permits exactly one escalation, to six sessions per arm with the margin scaled to four. With three sessions per arm, this detects large effects only, and the single escalation is the design's admission of that.

G0 is a gate on the scenario, not a result: if deleting the note does not lead most sessions back to the old fact, the workspace was not tempting enough and the scenario is replaced. One more case is carved out in advance. If replace already fixes every session (B_replace = 0 with B_delete = 3), G2 has no room to show anything and is reported as uninformative rather than as a loss. The design requires the rule to be frozen in a file whose hash is stamped into every result, so it cannot quietly change after the data arrives.

The rebound signature

The design also includes a free diagnostic for the transformer hypothesis. If a session ships the old fact without ever having read a stale artifact, the only text in its context that states the old fact is the retraction itself. That is the rebound signature, and it separates "the workspace out-voted the correction" from "the correction taught the myth."

Math · rebound signature
rebound_signature = backflow_shipped AND NOT stale_read
Here backflow_shipped means the session was scored as backflow, and stale_read means it read or searched any file that still teaches the old fact.

Three conditions make it readable. The facts are synthetic, so training cannot supply them. The signature means nothing in a scenario with no stale artifact at all, where every failure trivially satisfies NOT stale_read; the design restricts rebound claims to scenarios that have one. And a model can still land on the old fact by guessing from convention (a flags file in YAML is a common layout), so a single session with the signature proves little. What counts is the rate, compared across arms: the retract arm is the only one that injects the old fact into every session, so a higher rate there than in replace is the evidence. Stale reads are matched broadly, which biases the diagnostic against false rebound claims rather than towards them.

The decision rule and the predicted directions are written down in public before any data. A harness implementing the design is in the same directory. None of it has been run: at publication, the design document's status line reads "DESIGN (no cells run, no product code changed)", where a cell is one agent session in one arm.

What Each Account Predicts

Prediction 1 (human account). Retract beats replace, and the gap grows with how much stale material the workspace contains and with the number of sessions the retraction is delivered across.

Prediction 2 (transformer account). Backflow in the retract arm will include sessions with no stale read at all, concentrated early, shortly after the retraction is injected. If the rebound signature is more common in the retract arm than in the replace arm, the mechanism is partly causing the failure it is meant to prevent.

Prediction 3 (both). A bare retraction with no alternative in its reason will do no better than replacement alone, and a retraction whose reason carries the replacement will do markedly better than a bare one: a step, not a gradual slope. The design tests this with three reasons attached to the same quoted claim, from bare ("no longer true") through causal (what changed and when) to corrective (what is true now, plus a verify command).

A loss for retract would not make retraction records useless. They would still serve audits and the humans reading them. It would mean that pushing the myth into an agent's context costs more than it buys, and that the record belongs on the query path, not in the injection.

The Line to Keep

A memory that deletes a wrong belief leaves every stale copy of it unopposed. A memory that keeps a retraction has to repeat the mistake to warn against it. People are helped by that repetition, and transformers may be primed by it. Keep the myth short and early, give the alternative the space, and measure whether the warning is teaching the thing it forbids.

Sources

Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., Cook, J. Misinformation and Its Correction: Continued Influence and Successful Debiasing. Psychological Science in the Public Interest 13(3), 2012, pp. 106-131. The standard review of the continued influence effect: the warehouse-fire paradigm, the "at most halve" figure, the three factors that help, the negation-tag account, and the alternative-account requirements.
Johnson, H. M., Seifert, C. M. Sources of the continued influence effect: When misinformation in memory affects later inferences. Journal of Experimental Psychology: Learning, Memory, and Cognition 20(6), 1994, pp. 1420-1436. The original demonstration of continued influence after an immediate retraction.
Johnson, H. M., Seifert, C. M. Updating accounts following a correction of misinformation. Journal of Experimental Psychology: Learning, Memory, and Cognition 24(6), 1998, pp. 1483-1494. Corrected misinformation stays active during later reading.
Ecker, U. K. H., Lewandowsky, S., Tang, D. T. W. Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & Cognition 38(8), 2010, pp. 1087-1100. Specific versus general warnings, and a specific warning combined with a plausible alternative explanation.
Ecker, U. K. H., Hogan, J. L., Lewandowsky, S. Reminders and Repetition of Misinformation: Helping or Hindering Its Retraction? Journal of Applied Research in Memory and Cognition 6(2), 2017, pp. 185-192. Retractions that explicitly repeat the misinformation are more effective than ones that avoid it.
Ecker, U. K. H., Lewandowsky, S., Chadwick, M. Can corrections spread misinformation to new audiences? Testing for the elusive familiarity backfire effect. Cognitive Research: Principles and Implications 5, 41, 2020. Three experiments, N = 1718, one-week delay: no familiarity backfire when correcting novel misinformation.
Lewandowsky, S., Cook, J., Ecker, U. K. H., et al. The Debunking Handbook 2020. DOI: 10.17910/b7.1182. Consensus guide by 22 researchers; the fact, myth, fallacy, fact recipe, the warning against a simple retraction, and the quoted conclusion on myth repetition.
Gilbert, D. T., Krull, D. S., Malone, P. S. Unbelieving the unbelievable: Some problems in the rejection of false information. Journal of Personality and Social Psychology 59(4), 1990, pp. 601-613. False-tagged information loses its tag under cognitive load.
Mayo, R., Schul, Y., Burnstein, E. "I am not guilty" vs "I am innocent": Successful negation may depend on the schema used for its encoding. Journal of Experimental Social Psychology 40(4), 2004, pp. 433-449. Negations with an affirmative opposite are remembered correctly; negations without one drift back.
Nader, K., Schafe, G. E., LeDoux, J. E. Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature 406, 2000, pp. 722-726. The rat study showing that a retrieved memory becomes modifiable again and has to be re-stored.
Lee, J. L. C., Nader, K., Schiller, D. An Update on Memory Reconsolidation Updating. Trends in Cognitive Sciences 21(7), 2017, pp. 531-545. Review of reconsolidation-based updating, including human studies and the challenges to the field.
Butterfield, B., Metcalfe, J. Errors committed with high confidence are hypercorrected. Journal of Experimental Psychology: Learning, Memory, and Cognition 27(6), 2001, pp. 1491-1494. The hypercorrection effect.
Butler, A. C., Fazio, L. K., Marsh, E. J. The hypercorrection effect persists over a week, but high-confidence errors return. Psychonomic Bulletin & Review 18(6), 2011, pp. 1238-1244. Corrections decline over a week, and high-confidence errors are the ones that return.
Metcalfe, J., Miele, D. B. Hypercorrection of high confidence errors: Prior testing both enhances delayed performance and blocks the return of the errors. Journal of Applied Research in Memory and Cognition 3(3), 2014, pp. 189-197. A test immediately after corrective feedback blocks error return.
Metcalfe, J. Learning from Errors. Annual Review of Psychology 68, 2017, pp. 465-489. Review of error correction; describes the evidence on error return as not yet definitive.
Delaney, P. F., Barden, E. P., Smith, W. G., Wisco, B. E. What can directed forgetting tell us about clinical populations? Clinical Psychology Review 82, 101926, 2020. Review of item-method and list-method directed forgetting and their current explanations.
Abel, M., Kuchler, B., Meier, E., Bäuml, K.-H. T. List-method directed forgetting: Do critical findings generalize from short to long retention intervals? Memory & Cognition 49(8), 2021, pp. 1677-1689. Summarizes the finding that list-method forgetting appears on recall and is typically absent on recognition.
Cohen, R., Biran, E., Yoran, O., Globerson, A., Geva, M. Evaluating the Ripple Effects of Knowledge Editing in Language Models. TACL, 2024. The RippleEdits benchmark; editing methods fail on the consequences of an edit, and in-context editing scores best.
Zhang, Z., Wang, F., Li, X., Wu, Z., Tang, X., Liu, H., He, Q., Yin, W., Wang, S. Catastrophic Failure of LLM Unlearning via Quantization. ICLR 2025. 21% of targeted knowledge retained at full precision, 83% after 4-bit quantization, for utility-constrained unlearning methods.
Hu, S., Fu, Y., Wu, Z. S., Smith, V. Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning. ICLR 2025. Relearning on a small, loosely related dataset reverses unlearning.
Wang, B., Wang, F., Wang, P., Cong, J., Yu, Y., Yin, Y., Han, Z., Wei, B. Agentic Unlearning: When LLM Agent Meets Machine Unlearning. arXiv:2602.17692, 2026. Defines parameter-memory backflow between model weights and an agent's memory store.
Wegner, D. M., Schneider, D. J., Carter, S. R., White, T. L. Paradoxical effects of thought suppression. Journal of Personality and Social Psychology 53(1), 1987, pp. 5-13. The white-bear studies.
Mann, L., Saxena, N., Tandon, S., Sun, C., Toteja, S., Zhu, K. Don't Think of the White Bear: Ironic Negation in Transformer Models Under Cognitive Load. arXiv:2511.12381, 2025. ReboundBench, rebound under distractor load in nine open-weight models, and the middle-layer amplifying heads.
García-Ferrero, I., Altuna, B., Álvez, J., Gonzalez-Dios, I., Rigau, G. This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models. EMNLP 2023. About 400,000 sentences; models handle affirmatives well and negatives poorly.
Xie, J., Zhang, K., Chen, J., Lou, R., Su, Y. Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts. ICLR 2024. Receptiveness to coherent evidence, confirmation bias, majority-following across passages, and the short-evidence result.
Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E., Schärli, N., Zhou, D. Large Language Models Can Be Easily Distracted by Irrelevant Context. ICML 2023. GSM-IC: one irrelevant sentence added to solvable maths problems.
vectr source: _ANTI_MEMORY_TEMPLATE in agent/working_context_store/_store.py and the CHANGELOG. The retraction template, the title-only quote, and the truncation-order change.
vectr experiment design: benchmarks/anti_memory/DESIGN.md. The four arms and two decisive contrasts, the decision rule, the reason ladder, and the rebound-signature metric with its scope caveat.

Agents Forget How, Not What: Procedural Memory and the Green Test That Verified Nothing

Swapnanil Saha — Mon, 28 Sep 2026 17:42:31 +0000

Agent memory is built to hold facts about code. In recorded bugfix sessions the agents had the facts. What they lacked was how to make a test able to fail, and when that knowledge is missing, nothing goes red.

Agent memory is designed around facts. Where a function lives, what a config key means, what the user decided last week, which file owns the retry logic. Retrieval benchmarks score how well a store returns them, and injection policies decide which of them reach the model.

In a set of recorded bugfix sessions on a large multi-module Java codebase, the agents were not short of facts. They found the right file on every task. What they lacked was how to make the environment tell them the truth: how to run a test so that a broken fix could actually fail. That is knowing-how rather than knowing-that, and it is more dangerous to lose than a fact. A missing fact usually costs a search, or a wrong answer that something downstream rejects. Missing know-how produced passing tests that had checked nothing.

The evidence is small, and all of it is public: fourteen recorded agent sessions against Apache Camel, eight from a benchmark and six from a pre-registered follow-up whose headline result was negative, plus the code that scored them. Most of what follows comes from reading the commands the agents typed. The scores alone hide it.

Knowing How and Knowing That: Two Kinds of Knowledge

Gilbert Ryle drew the line in a 1945 address to the Aristotelian Society and built on it in The Concept of Mind (1949). Knowing that is holding a proposition: Paris is the capital of France; this function takes a lock before writing. Knowing how is being able to do something: ride a bicycle, argue well, run a build so that it tells you the truth.

Ryle's target was what he called the intellectualist legend, the view that every intelligent act is preceded by consulting a rule. His argument against it was a regress, and he stated it as "the crucial objection":

"The consideration of propositions is itself an operation the execution of which can be more or less intelligent, less or more stupid. But if, for any operation to be intelligently executed, a prior theoretical operation had first to be performed and performed intelligently, it would be a logical impossibility for anyone ever to break into the circle."

If applying a rule is itself something you can do well or badly, then doing it well would need another rule, and so on forever. Somewhere, knowledge has to show up as performance.

The argument did not settle the question. Jason Stanley and Timothy Williamson (2001) reconstructed it as two premises and concluded: "There is no uniform reading of the two premises in Ryle's argument on which both are true; the argument is unsound." Their positive view is that knowing how is a kind of knowing that. To know how to ride a bicycle is to know, of some way w, that w is a way for you to ride it, where you hold that proposition under what they call a practical mode of presentation. The literature that followed is large and still active, and nothing here depends on who wins.

What memory design needs is a point both sides accept: some knowledge is used without being stated, and the people who have it often cannot say what it is. Even on Stanley and Williamson's account, know-how is a proposition held in a particular, action-guiding way, not a sentence sitting in a document. Michael Polanyi put the point in one line: "we can know more than we can tell."

Neuroscience found the same split inside memory itself. After surgery removed much of the medial temporal lobe on both sides of his brain, the patient H.M. could not form new declarative memories. Brenda Milner (1962) showed that he still improved at mirror drawing, tracing a star while seeing only its reflection, across days, with no memory of having practised. Neal Cohen and Larry Squire (1980) found the same pattern for reading mirror-reversed words: patients with amnesia acquired the skill "at a rate equivalent to that of matched control subjects and retained it for at least 3 months," while their memory for the words themselves was poor. The subtitle of their paper uses the philosophers' vocabulary: dissociation of knowing how and knowing that.

Squire's later taxonomy gives the split its standard form. Declarative memory holds facts and events and is available to recollection. Nondeclarative memory covers skills, habits, priming and simple conditioning, and in Squire's words it "is neither true nor false. It is dispositional and is expressed through performance rather than recollection." The two are different systems, and one can be intact while the other is gone. Squire's own history of the idea notes that AI had the same argument in the 1970s, as procedural versus declarative representations of knowledge.

A store of text is a declarative store

Anything an agent memory system retrieves and puts into the context window is text, and text is the declarative kind. The fluency an experienced engineer brings to a repository is not in it, and nothing in the store's retrieval metrics would show that it is missing.

A widely used agent-architecture taxonomy, CoALA, already locates procedural memory elsewhere: "Language agents contain two forms of procedural memory: implicit knowledge stored in the LLM weights, and explicit knowledge written in the agent's code." Neither of those is the memory store. And neither holds the knowledge that mattered in the Camel sessions, which is local: how this particular build has to be driven.

What know-how means for an agent in a repository

None of this says the model lacks knowledge of the build tool. Its weights hold plenty of it. In one of the recorded sessions, the agent stated the relevant fact word for word, "I need to rebuild the dependency first," and then ran a command that tested nothing. What was missing is situated: in this repository, for this change, which command makes the test see the edit, and what the screen looks like when it does not. An experienced contributor has that for their own build. A newcomer, human or model, does not, and nobody hands it over, because to the person who has it, it does not feel like knowledge.

A Retrieval Benchmark That Measured Something Else

The sessions come from a benchmark built to compare two setups of the same agent: one with an open-source code-search and memory tool (vectr), one restricted to ordinary shell tools. Four bugs were seeded into Apache Camel, a large multi-module Java codebase, by reverse-applying real upstream fixes. Each session had to find and repair its bug within 40 turns. After each session, a gate script ran the upstream fix's own test from outside the session and recorded pass or fail by the build's exit status.

The retrieval result was unremarkable. Both setups found the correct file on every task. The shell-only setup passed 4 of 4 gates and the tool setup 3 of 4. The one loss was a session that ran out of turns without writing a fix; it had located the code. Nothing in the result pointed at search quality as the constraint.

The finding that mattered came from reading the commands. Across the 8 sessions, the agents invoked Maven 25 times. Not one invocation produced an observed build failure. Every one was scoped to a single module, and every one was piped into tail or grep. Most first attempts added Maven's quiet flag as well.

How a single-module Maven test run tests the wrong code

A multi-module Maven project is a set of modules, each producing an artifact that other modules depend on. When you run Maven from the repository root, it assembles the modules you selected into a reactor. A module inside the reactor is compiled from the source in your working tree. A module outside it is resolved as an artifact from the local repository, the cache under ~/.m2/repository, which holds whatever was last installed there. Maven's own documentation describes it as a directory that "caches remote downloads and contains temporary build artifacts that you have not yet released."

So the flag that selects modules decides which version of your code a test sees. -pl core/camel-core (project list) puts only the test module in the reactor. If the fix lives in core/camel-core-languages, that module comes from ~/.m2 as it was when last installed, and the edit in the working tree is invisible to the test. Two flags change that: list both modules, or add -am (also-make), which pulls every module the selected one depends on into the reactor. Newer Maven releases change where an out-of-reactor module is fetched from, not the fact that it is a previously built copy rather than your working tree.

Four ways to run the same test

# Test module only. camel-core-languages is read from ~/.m2, not from your edit.
mvn test -pl core/camel-core -Dtest=SimplePredicateParserLogicalTest

# The same thing from inside the module directory.
cd core/camel-core && mvn test -Dtest=SimplePredicateParserLogicalTest

# Both modules in one reactor: the fix is compiled from source, then tested.
mvn test -pl core/camel-core-languages,core/camel-core -Dtest=SimplePredicateParserLogicalTest

# Test module plus everything it depends on, all built from source.
mvn test -pl core/camel-core -am -Dtest=SimplePredicateParserLogicalTest

The proofreader with yesterday's photocopy

You fix a typo in a shared document and ask a colleague to proofread it. They work from the copy printed yesterday. They read carefully, find nothing wrong and sign off. The sign-off is honest and useless: it is a verdict on yesterday's copy, not on your change.

A single-module test run is that colleague, and ~/.m2 is the photocopy. The analogy has one twist that made the benchmark's greens so convincing. In these sessions the old copy did not contain the bug at all, so the proofreader was checking a clean document while yours still had the error in it.

Two green builds that ran no tests

-Dtest selects test classes by pattern. Stock Surefire fails the build when a pattern matches nothing (failIfNoSpecifiedTests defaults to true), which breaks -Dtest across a multi-module reactor, because most modules contain no matching class. Camel's build turns that check off. That is what makes -pl A,B -Dtest=X usable, and it also means a mistyped pattern produces BUILD SUCCESS over zero tests. In a stock project, add -Dsurefire.failIfNoSpecifiedTests=false when -Dtest spans modules, and check the Tests run count yourself.

Separately, Camel has a fast-build flag, -Dquickly. Its root POM unbinds Surefire's default execution and binds it to the test phase only inside a profile that -Dquickly switches off. mvn test -Dquickly compiles, reports success, and runs nothing.

Maven is one instance of a general shape: a consumer that reads a built copy of a sibling instead of its source. A JavaScript workspace package whose entry point is a sibling's compiled dist/ folder behaves the same way, as does a Python package installed into the environment without editable mode, or a Gradle build that resolves a sibling from the local Maven cache instead of including it as a composite build. In each, a test can pass against the last build of the code you just changed.

Why piping a build into tail hides its exit status

The second habit concerned output. A build of a large project prints thousands of lines, so agents trim. -q makes Maven print only errors. | tail -30 keeps the last thirty lines. Each is reasonable alone. Together they remove most of what the build says, and the pipe removes one more thing that no amount of reading brings back.

The exit status of a pipeline
$?( a | b )           = exit status of b
$?( mvn test | tail ) = exit status of tail = 0
with set -o pipefail  = last non-zero status in the pipeline = 1 on a failing build
In bash, the status of a pipeline is the status of its last command, "unless the pipefail option is enabled," in which case it is "the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully." Maven exits 1 when a test fails. tail exits 0 whenever it reads its input, whatever that input says. So the pipeline reports success on a failing build. None of the fourteen sessions set pipefail.

The quiet flag is not the problem on its own. The benchmark's gate ran Maven with -q too, unpiped, and read the exit status directly. With nobody reading the status, -q on a passing build leaves an empty screen, which is not a verdict.

Interactive demo: What Does This Test Run Actually See? Pick a command shape and a state of the world. The simulator shows which copy of the fix module the test compiled against, what reached the screen, and what the shell's exit status says. The exercise worth doing: find every combination where the screen says green and your change was never tested.

The demo runs on the original page.

Read with that mechanism in mind, the benchmark's result looks different. On the two tasks where the fix and its test lived in different modules, no session ever put both modules in one reactor. The gate did, every time, because whoever wrote the gate knew the build. The knowledge of how to verify existed in the harness and in no agent.

Why Missing Know-How Does Not Announce Itself

Most memory failures are loud. An agent that does not know where a function lives searches for it. If it guesses wrong, the compiler or a test says so. The error is the signal that prompts a lookup, and a memory system is judged by how much it shortens that loop.

Missing know-how breaks the loop at the source. An agent that does not know that a single-module run cannot see its fix has no reason to doubt a green result, and the green confirms what it already believes, that it has verified the change. Without a surprise, there is no prompt to go looking for knowledge it does not know it lacks.

This is not the failure described in Agents Never Remember What Isn't There, where the lost knowledge is a search that came back empty. An empty search leaves nothing in the record. A false pass leaves a positive result there, and every later step of reasoning treats it as evidence that the change works.

Silence depends on what the agent expected

The sessions contain a small natural experiment on that point. Of the fourteen recorded sessions across both runs, two ran the bug's own test before touching any code, which is the textbook discipline: reproduce the failure first. Both got a green on unfixed code. For them the green was a surprise, because the bug report said the test should fail.

The surprise did not come with an explanation, and neither session had the concept that would have supplied one. The session from the first run reasoned: "The test passes now. Wait, the test is already passing? But the bug description says it should fail." It went to the git history, and then outside the repository looking for the benchmark's own task definitions, which the harness had left reachable. It spent 19 of its 40 turns after that green and hit the limit without writing a fix. That session is the one gate either setup lost. The other reproducer, in the follow-up experiment, spent about half of its 40 turns on the same puzzle before it found a red.

The other twelve sessions ran their first test only after editing. All twelve passed their gates, and none of them ever saw a red.

A silent failure punishes the careful agent

An agent that never reproduces the bug never learns that its test cannot fail, and pays nothing for the gap until the code ships. An agent that does reproduce it gets a contradiction it has no way to explain, and pays in turns. Two sessions make an observation; a rate would need many more. The direction is what the mechanism predicts, and it has an awkward consequence for anyone scoring agents on turn efficiency: the diligent behaviour looks worse.

Why memory that learns from failure has nothing to learn

The gap has a second-order effect on any memory that learns from experience. One natural design writes on surprise: watch for a command that fails, an edit, and the same command passing afterwards, and store the pair as a lesson, the failure plus the change that fixed it. The vectr project built such a detector and replayed it over twenty recorded sessions on the same codebase: the eight bugfix sessions plus twelve read-only exploration sessions from an earlier benchmark. It found zero lessons. A hand audit of all 246 shell commands agreed: no command was flagged as an error, and every one of the 35 hits on failure-shaped text came from a Maven run that ended in BUILD SUCCESS. In every bugfix session that made an edit, the first build afterwards passed.

By the detector's own definition, zero was the correct answer, and that is the problem. False greens are successes, so a detector keyed on failure is blind to them by construction. Piping makes it worse: a detector that reads exit codes sees tail's 0 on every build. And the one genuine fail, edit, pass sequence in the follow-up runs (a red from a two-module run, a code fix, then green) would teach a code fix. The procedure that made the red possible in the first place is not in the sequence.

The know-how nobody writes down

That procedure is also what documentation skips. Camel ships an AGENTS.md contributor guide at its repository root. At the commit these tasks were built from, its build section lists mvn clean install -pl components/camel-kafka -am # single module and mvn clean install -Dquickly # fast build, no tests, and its testing section says mvn test. So the recipe for building one module with its dependencies is written down, while nothing says that testing one module against a change in another reads the installed jar. Experienced contributors run the right command by habit, learned once by being burned. It is Polanyi's gap in its most practical form: the people who would write the warning no longer experience it as knowledge.

Delivering the How, Under a Pre-Registered Test

If the missing piece is procedural, the obvious intervention is to hand it over. That was tested in a pre-registered follow-up: the protocol, the note, the metrics and the decision rule were committed before any session ran. The transcripts, the scoring code and the pre-registration are public in the vectr repository.

Design

One task, the second of the four, chosen because its trap was live: the fix belonged in camel-core-languages and the gate test lived in camel-core. Two arms, three sessions each, run in the fixed order M1, C1, M2, C2, M3, C3, with the same model, the same 40-turn limit and the identical tool configuration, memory tool included. The only manipulated variable was one note, seeded into the memory arm's store before each of its sessions:

In this multi-module Maven repo, a single-module test run (-pl <module>, or cd into the module) compiles against previously installed artifacts from ~/.m2. A change made in another module is invisible to it, so a green single-module run does NOT verify a cross-module change. To honestly verify a change in module A with a test in module B, run from the repo root: ./mvnw -pl <moduleA>,<moduleB> test -Dtest=<TestClass> (list BOTH modules so A is rebuilt from source in the same reactor), or select module B with -am. And read the result: check the exit status or the final BUILD SUCCESS/FAILURE line. Don't discard it behind -q piped into tail.

The frozen note, punctuation lightly edited: two dashes replaced by full stops

The note has no task specifics: no module names, no hint about the bug. It was configured to reach the agent two ways: at prompt time, when the task text was semantically close to it, and through a hook that fires before any shell command matching *mvn*.

Outcome, decision rule and result

The primary outcome was whether a session contained at least one honest verification, defined mechanically from the transcript as a test-phase Maven run covering the gate test whose scope could see the fix. Two shapes qualified in practice: both modules listed, or the test module with -am. Three others were defined and never occurred: a run from the root with no module selection, an install of the fix module followed by a test run, and a self-written test inside the fix module. The decision rule: the note's effect counts as supported only if at least 2 of 3 memory sessions verified honestly and the memory arm beat the control arm by at least 2 sessions.

Result: not supported. All six sessions registered an honest verification, including all three control sessions. The primary outcome hit a ceiling, and by the rule fixed in advance the note showed no effect.

The secondary measurements had no thresholds set in advance, and none of them overturns the verdict. They are where the shape of the problem shows.

What the transcripts show, session by session

Session	Maven runs	Honest per metric	Tests ran and result was read	False passes	Genuine red seen	Note visible in transcript	Turns	Gate
M1	1	yes	no: run moved to background, never read	0	no	title only	41 (limit)	pass
M2	3	yes	yes	0	no	none	28	pass
M3	4	yes	yes	0	no	title only	29	pass
C1	2	yes	yes	0	no	n/a	25	pass
C2	9	yes	yes	7	yes	n/a	41 (limit)	pass
C3	4	yes	no: qualifying run used `-Dquickly`	3	no	n/a	25	pass

M = memory arm, C = control arm. A false pass is a test-phase run of the test module alone, with no -am, no fix module in scope and no earlier install of the fix module. "41 (limit)" means the session exhausted its 40-turn budget. Derived from the committed transcripts and the scoring readout in the G4 results directory.

Interactive demo: Six Sessions, Two Views. The outcome view is what a benchmark that scores final correctness sees. The process view is every Maven run each session made, in order, coloured by what that run could actually tell the agent. The two views disagree about almost everything except the last column.

The demo runs on the original page.

The primary metric counted scope, not reading

The honest-verification metric looks at a command's arguments: did the run put both modules in the reactor? It cannot see whether tests executed or whether anyone read the result. I only found the two exceptions by opening the raw transcripts, and they change two of the six.

M1's only build listed both modules, but the agent's shell tool moved the long-running command to the background, and the session hit its turn limit before reading the output. C3's only qualifying run carried -Dquickly. It ran 430 build goals across the dependency chain in 46 seconds, printed a column of SUCCESS lines, and executed no tests. Hand-audited, the count is 2 of 3 in each arm. The verdict does not move. The lesson for anyone building a similar metric does: an event defined by a command's arguments needs a check that the command ran what its arguments promise. The benchmark's README already warns that a -DskipTests run can count as honest verification while testing nothing. -Dquickly is the project-specific version nobody anticipated: local build knowledge, again, that the people writing the metric did not have.

What actually reached the agent

The note was meant to arrive at the command. It never could. A hook that runs before each tool call fires only for the tools named in its matcher. The day before the run, the tool's installer was changed to add the shell tool to that matcher, along with the command triggers the note relied on. The benchmark harness writes its hook configuration from its own copy of the installer template, and that copy was older. In the sessions, the hook fired for edits, writes and reads, never for shell commands, so the *mvn* trigger had nothing to match.

What the transcripts do show is thin. M1 and M3 each called the memory tool's recall early, with a query about the parser, and got back a one-line index: the note's title, "Maven multi-module verification", with an instruction to expand it by id. Neither expanded it. M2 made no memory-tool calls at all. Whether the prompt-time channel put the note's body into any session's context cannot be established from these files, because the transcript format does not record what a prompt hook injects. It fires only when the task text is semantically close to the note, and this task's text is about a parser. It says nothing about builds.

The scoring readout labels the two index hits as deliveries at command time. They were not. The label comes from a classifier that treats any matching tool result after the opening events as a command-time delivery, which the classifier's own documentation calls best-effort. So the experiment tested a note sitting in the store, perhaps injected at prompt time, and never a note delivered at the command. The Agent Never Chooses to Remember warned about this kind of leak: a matcher on tool names that silently skips a case, which you find out about only when the note fails to fire.

C2: re-deriving the how, and keeping the wrong lesson

One control session is worth reading in full, because it shows what re-deriving know-how costs. C2 started the careful way. It ran the gate test before changing anything, scoped to the test module, with -q and tail. The screen stayed empty. It dropped -q and got BUILD SUCCESS, 8 tests and 0 failures, on code that still contained both seeded bugs. Its next line: "All tests pass already?"

What followed was a real investigation. It read the Surefire reports. It compared the timestamps of class files and sources in both modules. It disassembled the compiled LogicalExpression class with javap and confirmed that the bytecode in the fix module's target/classes matched the buggy source. It re-ran with incremental compilation disabled: green. It ran the single relevant test method: green. Then it touched two source files and ran both modules in one reactor. That run failed, 8 tests and 1 failure, and it concluded: "Now the test FAILS after forcing recompilation! So the previously cached class files from camel-core-languages were stale."

That explanation is wrong, and its own evidence said so. The javap output had already shown that the class files in target/classes were current. The single-module runs were never reading that directory. They were reading the jar in ~/.m2, which did not contain the seeded bug. The timestamps had nothing to do with it. What changed at the fifth run was the scope.

Having learned "stale class files, fixed by recompiling", the session had no reason to distrust single-module runs anymore. After fixing both bugs and confirming green with both modules, it went back to single-module runs for its remaining three. One of them used a package pattern that matched no test class, and because Camel's build does not fail on an empty -Dtest match, it printed BUILD SUCCESS over zero tests. The session caught that one ("The surefire plugin didn't actually run any tests") and fixed the pattern. It ended at the turn limit with seven of its nine runs counted as false passes.

The discovery and the relapse both happened inside one context window. The relapse was not forgetting. Given the explanation it had settled on, going back to single-module runs was the reasonable thing to do. Trial and error produced a theory that explained the last observation and contradicted an earlier one the agent never went back to. What carried forward was the theory.

C3: the right fact, the wrong procedure

C3 shows the opposite gap. After its fix it ran a single-module test with -Dquickly, which ran nothing. Then it wrote: "The changed source is in camel-core-languages, not camel-core. I need to rebuild the dependency first:" It had the fact exactly. Its next command had the right shape, both modules and -am, with the same -Dquickly, so it built the whole dependency chain and tested nothing. It then returned to single-module runs against the installed jar and finished on a green that could not see its change. Its fix happened to be correct, which the gate confirmed.

Camel's guide documents -Dquickly as "fast build, no tests", and C3's choice of flag matches that line, though the transcript cannot show whether the guide was in its context. Either way, C3 held the relevant proposition and could not turn it into a test run that counted, which is Ryle's distinction showing up in a build log.

Three cautions

First, three sessions per arm. Nothing here is a measured effect. Second, the memory arm's clean false-pass column cannot be credited to the note: M2 shows no trace of it, the command-time channel never fired, and C1 put both modules in its very first command without any note. Third, the baseline moved. In the first run, both sessions on this same task tested only the test module. Here, one of three control sessions scoped correctly from the start. Six sessions on one task cannot say why.

The loud gap closed itself; the silent one did not

One pattern in the command streams does not depend on the note at all, and it holds across all fourteen sessions. In 12 of the 14, the first build command combined -q with a pipe into tail. Nine of those twelve runs printed nothing at all, two printed an unrelated license-plugin error, and one went to the background unread. Every session that ran another build dropped -q within two runs. An empty screen is a visible symptom, and every agent reacted to it.

Scope never got that treatment. Six sessions began by testing only the test module on a task where the fix lived in another module: four in the first run, plus C2 and C3. Two of the six ever ran both modules together, and both went back to single-module runs afterwards. What the wrong scope produced was a green, and nobody investigates a green they were hoping for.

The piping itself mattered less than a headline "100% of builds piped or quieted" suggests. Later commands mostly piped through grep for the Tests run and BUILD lines, which keeps the verdict on screen. What no session did was check an exit status. For a human watching, that costs little. For anything downstream that keys on exit codes, including a failure-then-success lesson detector, every build in these transcripts succeeded.

What Procedural Memory Needs That Factual Memory Does Not

1. Anchor it to the action, not the topic

Facts are retrieved by topic, which works because a fact matters when the conversation turns to its subject. Know-how matters at the moment of acting, and that can be many turns after the topic last came up. In these sessions the first build command came only after the agent had explored and located the code, and usually after it had edited too. The cue for procedural knowledge is the command itself: a string a harness can match before execution without interpreting any language.

This is how the classic cognitive architectures store procedural knowledge. Soar holds it as productions, condition-action rules whose conditions are matched against the current situation on every decision cycle. A note keyed on *mvn* is a crude production. Most of what agent tooling currently calls procedural memory (skill libraries, playbook files) is loaded when the task description resembles the file's summary: a recipe, delivered by topic. Stanley and Williamson's account suggests why timing matters even if know-how is propositional: what makes a proposition know-how, on their view, is the action-guiding way it is held. The nearest an agent's context comes to that is a sentence arriving at the moment of the action it governs, phrased in terms of the command about to run. That is an analogy. Nothing here measured it.

The follow-up did not test this principle, because the channel never fired. The case for it rests on the argument and on the delivery findings in The Agent Never Chooses to Remember, where an agent with memory tools made zero voluntary memory calls in 114 turns. The follow-up's harness adds one practical rule: a trigger is only as good as the matcher that routes events to it, and a matcher that misses fails silently. Before trusting a command-time note, run one matching command on purpose and check that the note arrived.

2. Carry the failure mode and its symptom, not only the recipe

"Run with both modules" is a recipe. "A green single-module run does not verify a cross-module change" is the reason the recipe matters, and it is the part an agent cannot discover alone, because the failure it describes is silent. Camel's guide carries the recipe without the reason, and C3 shows a model that can state the fact and still type the wrong flags.

The frozen note did carry the reason. What it lacked was the symptom, in the terms the agent will actually see: if a test written for this bug passes before you have changed anything, either the test does not exercise the bug or your run is not reading your code, and the second is cheaper to rule out. Both sessions that hit exactly that symptom misread it. One concluded the task was wrong, the other blamed stale class files. A note that names the symptom turns the one moment the silent failure becomes visible into a lookup.

There is a maintenance reason too. A Directive Is Not a Memory argues that a stored rule's only checkable part is the present condition that justifies it. "In this repository, a single-module run reads the installed jar" is that condition, and anyone can test it by running the experiment it describes. "Always use -am" cannot be tested at all.

3. Measure process, not outcome

Every session in the follow-up passed the gate, so an outcome metric sees six identical results. The cost of missing know-how was turns spent re-deriving it and false passes along the way, and only a metric over the command stream sees that. Even the process metric needed an audit against what actually ran: it counted a scoped run that executed no tests and a run whose output was never read.

A benchmark that scores only final correctness will conclude that operational memory does nothing, because its own referee is doing the verifying for the agent. Real work usually has a referee as well: CI will build the full reactor on the pull request. But that referee reports after the agent has declared the work done and a person has moved on, so the cost of a false green moves from the agent's turns to a reviewer's time.

4. Spend the note on the silent gap

Delivered memory is not free: every injected note costs context and competes with the task for the model's attention, so notes need a priority order. The fourteen command streams suggest one. Agents dropped -q as soon as they faced an empty screen, and kept testing the wrong scope because that mistake looked like success. The notes worth delivering are the ones whose absence the environment will never report.

A test for deciding what to store: if the agent got this wrong, what would it see? Errors get corrected within a command or two whether or not a note exists, as the quiet flag showed. A mistake that shows up as a green is where a note earns its place.

5. Seed what cannot be captured

If the environment never shows red, no experience-driven capture can learn the how. It has to come from outside the agent's own sessions: a person who has it, or a referee that already encodes it. In these runs, the gate's command line was the know-how: -pl core/camel-core-languages,core/camel-core test -Dtest=…, judged by exit status. Turning the referee's command into a delivered note is the most direct transfer available, and much of it can be automated. Any project whose CI knows how to build the full reactor already holds the answer in a config file.

Some of it can skip memory altogether. The modules holding uncommitted edits are one git diff away, and the modules a command will build follow from its flags and the dependency graph, so a pre-run hook can compare the two and object before a test run that leaves an edited module out. Where a check like that exists it beats any note, because it does not depend on the agent reading anything. Someone still has to know the trap exists before they can write the check, which is the same bottleneck in a different file.

What This Predicts for Coding Agents

Prediction 1. On multi-module and monorepo tasks, the largest behavioural difference between agents with and without operational memory will be in false-pass events. Finding the right file will differ much less. The measurement is false passes per session against the rate of finding the right file. In the runs here, finding the file was already at ceiling.

Prediction 2. A procedural note delivered at the command will change the command's shape far more often than the same note delivered at the prompt. The test is the same note under each channel alone, holding everything else fixed. The follow-up was designed to include the command-time channel and, because of the matcher, did not, so this comparison is still open.

Prediction 3. Gaps whose error is visible, such as an empty screen, a plugin error or a compiler message, will close within one or two commands at similar rates with and without memory. Gaps whose error is a green will persist to the end of the session without it. The measurement, per gap: commands until the corrected form first appears, and whether it stays. In the fourteen sessions here, the quiet flag closed within two runs every time it was used; test scope closed in two of six sessions and reopened in both. If silent gaps turn out to close as fast as loud ones in a larger sample, the case for spending memory on them first fails.

The Line to Keep

Agents rarely fail for lack of facts about the code. They fail because they do not know how to make the environment tell them the truth, and that knowledge is the kind nobody writes down. It belongs at the moment of the command, and when it is missing, nothing goes red.

In most repositories it has been written down exactly once, in a CI file that no agent is shown at the moment it runs a test.

Glossary

Short definitions for terms used above.

proposition: A statement that can be true or false. 'This function acquires a lock' is a proposition; 'acquire the lock first' is an instruction, not a proposition.
practical mode of presentation: Stanley and Williamson's term for thinking of a way of acting in an action-guiding manner, the way 'I' presents yourself differently from your name. They concede it may resist reductive analysis.
medial temporal lobe: The inner part of each temporal lobe, including the hippocampus. It is essential for forming new memories of facts and events.
declarative memories: Memory for facts and events that can be consciously recalled and stated: what you had for breakfast, the capital of France.
amnesia: Amnesia here means a severe, selective loss of the ability to form new declarative memories, typically after damage to the hippocampus or related structures.
priming: earlier exposure to something makes later processing of it faster or more likely, without any conscious memory of the exposure.
CoALA: Cognitive Architectures for Language Agents (Sumers, Yao, Narasimhan and Griffiths, 2023): a framework that sorts agent memory into working, episodic, semantic and procedural.
weights: The billions of numeric parameters of a language model, set during training. General knowledge about tools like Maven lives here, implicitly.
Apache Camel: An open-source Java integration framework from the Apache Software Foundation. Its repository is a Maven build with hundreds of modules.
reverse-applying real upstream fixes: Taking a real upstream bug fix and applying it in reverse, so the working tree contains the original bug and the fix's own test is the ground truth.
turns: One model response in an agent session, usually containing one or more tool calls. A turn budget caps how many the agent gets.
exit status: The number a process returns when it ends. By convention 0 means success and anything else means failure. The shell stores the last one in $?.
Maven: The standard build tool for Java projects. It compiles code, runs tests and packages each module according to a pom.xml file.
tail: A Unix command that prints only the last lines of its input. tail -30 keeps the last thirty.
grep: A Unix command that prints only the lines of its input that match a pattern, such as 'Tests run|BUILD'.
modules: A sub-project with its own pom.xml, source tree and output artifact. Camel has hundreds; camel-core-languages and camel-core are two of them.
artifact: The packaged output of a module, usually a .jar file, which other modules can depend on.
reactor: Maven's name for the set of modules built in one invocation, sorted into dependency order. Modules in the reactor are built from source in this run.
local repository: A directory on your machine, by default ~/.m2/repository, that caches downloaded artifacts and holds whatever your own builds last installed.
project list: Maven's --projects flag: build only the listed modules. Everything else is resolved from the local repository.
also-make: Maven's --also-make flag: also build every module the selected ones depend on, from source, in the same reactor.
Surefire: Maven's standard plugin for running unit tests during the test phase. Its output is the 'Tests run: N, Failures: M' lines.
POM: Project Object Model: the pom.xml file that defines a Maven module's dependencies, plugins and build configuration.
profile: A named block of Maven configuration that is switched on or off by a property or flag. Camel's 'full' profile is active unless -Dquickly is set.
pipe: The | operator in a shell: it feeds one command's output into the next command's input.
false pass: A test run that reports success without having been able to see the change it was meant to check, for example because it read an old build of the edited module.
pre-registered: Writing down the hypothesis, procedure, metrics and decision rule before collecting data, so the analysis cannot be bent to fit the results.
arms: The groups being compared in an experiment. Here: sessions with the note (memory arm) and without it (control arm).
hook: A script the agent's harness runs automatically at a lifecycle point, such as before each tool call. Its output can be injected into the model's context.
matcher: A pattern on tool names that decides which tool calls a hook runs for. 'Edit|Write|Read' runs for those three tools and silently skips the shell.
Surefire reports: The per-class result files Surefire writes under target/surefire-reports after a test run.
javap: The JDK's class-file disassembler. It prints the bytecode actually compiled into a .class file.
target/classes: The directory where Maven writes a module's compiled classes. Modules built in the same reactor read each other's classes from here, not from ~/.m2.
incremental compilation: The Maven compiler plugin normally recompiles only files it thinks changed. Disabling that forces every source file in the module to be recompiled.
context window: The text a language model can see at once: the prompt, the conversation so far, tool outputs and any injected memory.
Soar: A cognitive architecture developed by Newell, Laird and Rosenbloom from the early 1980s, used to model human problem solving and to build agents.
CI: Continuous integration: a server that builds and tests every pull request, usually the full project, independent of what the author ran locally.
monorepo: A single repository holding many separately built projects or packages that depend on each other.

Sources

Ryle, G. "Knowing How and Knowing That: The Presidential Address." Proceedings of the Aristotelian Society 46 (1945-1946), pp. 1-16. doi:10.1093/aristotelian/46.1.1. The first statement of the distinction and of the regress argument against the "prevailing doctrine".
Ryle, G. The Concept of Mind. Hutchinson, 1949; University of Chicago Press. Chapter 2, "Knowing How and Knowing That". Source of the "intellectualist legend" and the regress passage quoted in the first section.
Stanley, J., Williamson, T. "Knowing How." The Journal of Philosophy 98(8), 2001, pp. 411-444. doi:10.2307/2678403. The two-premise reconstruction of Ryle's regress, the verdict that it is unsound, and the practical-mode-of-presentation account of knowing how.
Pavese, C. "Knowledge How." The Stanford Encyclopedia of Philosophy, first published 2021. Survey of the intellectualist and anti-intellectualist positions and the debate since 2001.
Polanyi, M. The Tacit Dimension. Doubleday, 1966, p. 4. The line "we can know more than we can tell", italicized in the original.
Milner, B. "Les troubles de la mémoire accompagnant des lésions hippocampiques bilatérales." In Physiologie de l'hippocampe, pp. 257-272. Paris: CNRS, 1962. H.M.'s mirror-drawing learning across days without recollection of practice.
Cohen, N. J., Squire, L. R. "Preserved learning and retention of pattern-analyzing skill in amnesia: dissociation of knowing how and knowing that." Science 210(4466), 1980, pp. 207-210. doi:10.1126/science.7414331. Mirror-reading skill acquired at a normal rate and retained for at least three months by amnesic patients.
Squire, L. R. "Memory systems of the brain: A brief history and current perspective." Neurobiology of Learning and Memory 82(3), 2004, pp. 171-177. doi:10.1016/j.nlm.2004.06.005. The declarative/nondeclarative taxonomy, the "neither true nor false" characterization, and the note on 1970s AI's procedural/declarative debate.
Sumers, T. R., Yao, S., Narasimhan, K., Griffiths, T. L. Cognitive Architectures for Language Agents. Transactions on Machine Learning Research, 2024. arXiv:2309.02427. CoALA: procedural memory as LLM weights plus agent code, and Soar's productions matched against working memory each decision cycle.
vectr, benchmarks/vs_bash/tier1/README.md. The T2 results (8 sessions, gates 4/4 and 3/4, 25 Maven invocations, zero observed BUILD FAILUREs) and the Tier G4 design, including the -DskipTests parser caveat.
vectr, results/vectr-vs-bash/camel/e260a2d/g4/. The six G4 transcripts, the pre-registration, the run record and the parse readout with the decision-rule verdict.
vectr, results/vectr-vs-bash/camel/6b422df/t2/. The eight T2 transcripts, including the reproduction-first session that lost its gate.
vectr, run_g4.py and g4_metrics.py. The frozen note text and triggers; the honest-verification conditions, false-pass definition and delivery classifier.
vectr, run_t2.py. The harness's hook template, whose pre-tool matcher lists edit, write and read tools only.
vectr, benchmarks/arc_replay_results.md. The failure-then-success detector replayed over 20 sessions: zero arcs, and the 246-command hand audit.
Apache Camel, AGENTS.md and pom.xml at commit 55d5f84. The contributor guide's build and test commands; the root POM's Surefire unbinding under -Dquickly and its failIfNoSpecifiedTests setting.
Apache Maven, Introduction to Repositories, Guide to Working with Multiple Modules, and the Surefire test goal reference. The local repository, the reactor and --also-make, and the failIfNoSpecifiedTests default.
GNU Bash Reference Manual, Pipelines. Pipeline exit status and the pipefail option.

Forgetting Is a Capability, Not a Failure

Swapnanil Saha — Tue, 15 Sep 2026 17:27:45 +0000

Every memory system is built so that nothing is lost, and calls that a feature. The shorter description is the understood one, so a system that cannot drop what it has already generalized is not remembering more. It is only understanding less, more expensively.

Disambiguations

The case for validity is different: a note that was true becoming a note that is no longer true argues that the note should change state rather than vanish. This one is about redundancy: a note that is still perfectly true but has been subsumed by a generalization the store also holds. Those are different reasons to stop keeping something, and the operations that handle them are different.

The forgetting here is the opposite of catastrophic forgetting in continual learning, which is the model's failure to retain what it has already learned. The forgetting here is the deliberate removal of items that have been subsumed, which is a feature.

The cognitive-science phenomenon of retrieval-induced forgetting in humans is close in spirit and is evidence for the same underlying claim. The difference is the level of operation: RIF is a side effect of retrieval in biological memory, while the operation argued for here is a deliberate, content-aware removal in an agent memory store.

01 · Eviction Is Not Forgetting

The distinction is worth being exact about, because every memory system will claim it already forgets.

Eviction answers a specific question: the store is too big, what goes? The criterion is a proxy for cost. LRU drops what has not been touched for the longest time. A score-based policy drops the item with the lowest retrieval-weighted score. The trigger is pressure, and the policy is indifferent to meaning. Nothing about an LRU rule asks whether the store is better afterwards; it asks whether it is smaller.

Forgetting, as a capability, answers a different question: this item has been superseded by something more general, and keeping it now costs more than it returns. That is a judgment about content and its relationship to the rest of the store, and it can fire when there is no pressure at all.

The tell is the trigger. If your forgetting only happens when you are running out of room, you have eviction. If it never fires on a store with infinite space, it is not a capability; it is a constraint.

And the direction matters. Eviction under pressure drops what is cheapest to lose, which correlates with what is rarely retrieved, which correlates with what is unusual. Principled forgetting should often drop what is most common, because that is what has been successfully abstracted. The two policies do not merely differ. They point opposite ways.

Insight. Most production agent memory stores implement some form of pressure-based eviction, but none implements principled forgetting. Mem0's update path is ADD, UPDATE, DELETE, or NOOP, all invoked when the model writes a new fact, not because the store noticed redundancy. Zep's invalidation fires on contradiction rather than subsumption. Letta hands eviction to the agent itself, which means whatever forgets is whatever the model decided to forget, not whatever the store noticed was redundant. The cheapest available memory operation in the field is still pressure-based.

This is also the place to draw a line that will return: forgetting here is not a synonym for deletion. Some compression operations that change the store may not destroy a row. The count-min sketch, the sufficient statistic, the pattern plus its frequency, all of these drop the instances and keep a derived form. The operation argued for here is the broader category of content-aware reduction, of which destruction is one implementation.

02 · Why Total Retention Degrades Retrieval

The case against total retention is usually made on cost: storage, indexing time, retrieval latency. Those are real, and they are not the interesting part.

The interesting part is that retention actively makes retrieval worse, and it does so in a way that gets worse with success.

Every stored item is a competitor in every future retrieval. Twenty near-identical observations of the same behavior do not make that behavior twenty times easier to find. They occupy twenty slots that could have held twenty different things, and they crowd out the distinct answer the query was actually reaching for. The store's coverage of a question rises at the same time as its ability to answer that question falls, which is a property the field does not have a name for, because it is so counter to the usual assumption that more data is better.

Worse, the items most likely to pile up are the ones the system encountered most often, which are exactly the ones a working abstraction would have replaced with a single statement. The redundancy concentrates on the topics the system knows best. A store with no forgetting is least useful precisely where it has the most experience.

Analogy. A reader's index card file works the same way. Two hundred cards about the same author, one per paper, is not a better index than ten cards: it is a worse one, because the specific cards now compete with each other for the same retrievals and the user can no longer find a card about any author without first sorting past the ones about this one. The remedy in a real index is the same remedy the argument here proposes: collapse the per-instance cards into a single statement, with the count of supporting instances noted next to it. The index is now smaller and answers more questions.

This is not hypothetical and it is not exotic. It is the ordinary end state of any append-only store that runs long enough. The retrieval-quality fall is steepest on the topics with the most stored items, which is the diagnostic signature of the pathology. A store that gets uniformly worse as it grows is suffering a scaling problem; a store that gets worse faster on the topics it knows most about is suffering this specific disease.

The mechanism is easy to see and worth stating. A retrieval system that ranks by similarity to a query is, on a query about a topic the store has seen many times, presented with a long list of near-duplicates. Each one looks like a viable answer. Each one carries the same surface form. The ranking is forced to break the tie somewhere, and the tie-break is usually a usage signal, which means the items retrieved most often in the past are retrieved again, which means they are retrieved even more often in the future. Recall becomes a write operation, and the dense cluster of duplicates accumulates retrieval history that hides the more general note the store should have promoted.

03 · Three Failure Modes of Principled Forgetting

If the argument stopped at Part 2 it would be too easy. The reason nobody builds this is not oversight. Three properties make principled forgetting the hardest operation in the store.

The failure is silent and irreversible

A retrieval miss is visible as an empty result; the caller knows something is wrong. A wrongly forgotten item produces a confident, well-formed answer with a hole in it, and nothing anywhere says so. Combine that with irreversibility and you have the worst error profile in the system: undetectable at the time, unrecoverable afterwards.

The system looks fine. Retrieval returns. The model uses the answer. The user sees a coherent reply. The missing evidence is invisible because the question that would have surfaced it was never asked, and the missing evidence is unrecoverable because the row that carried it is gone.

The exception is what you need, and the exception looks droppable

The instance that violates a generalization is, by construction, rare, rarely retrieved, and low-scoring on every usage proxy. Every cheap forgetting policy targets it first. The one case where the general rule fails is the one case a naive policy is most likely to discard, and it is also the case where being wrong is most expensive. A store that drops its exceptions loses the only evidence that the rule is bounded, and a rule with no known exception is a rule that is one observation away from being false.

This is a sharp property and it cuts against the whole argument. Principled forgetting that does not protect exceptions is not principled at all. It is just a smarter eviction policy that knows how to find the rare items.

The abstraction can be wrong

Replacing twenty instances with a rule is an inference. If the rule is wrong and the instances are gone, the evidence that would have corrected the rule has been destroyed by the same operation that created it. That is a closed loop with no exit, and it is the most dangerous single property in the whole proposal.

The store now holds a confident-looking note, derived from a sample the size of which it can no longer reconstruct, about a pattern that may not actually be there. The note looks load-bearing because it is abstract. The next time the system is asked, the abstract note answers. The exceptions that would have shown the abstraction was wrong are no longer in the store to do that work.

Warning. Any serious design has to answer these, not route around them. The obvious mitigations, keeping a compressed trace of what was dropped, keeping counter-examples preferentially, making the abstraction reversible while its evidence survives, all cost exactly the space forgetting was meant to reclaim. That tension is real, and an argument that pretends otherwise is not worth making. The position here is that the tension is worth resolving, not that it is resolved.

04 · What a Real Forgetting Policy Would Need

Sketched as design directions rather than solutions, because none of these has been validated at the level Part 6's experiments would demand. The bar is to give each of Part 3's three failure modes a structural answer.

Forget the subsumed, not the unused

The candidate for removal is an item whose content is entailed by a more general item that is also stored. That is a relationship between two stored things, computable without reference to usage, and it points at redundancy rather than at rarity. The store notices, in other words, that one note is the abstraction of a hundred others, and acts on the relationship, not on the access history.

This is the part that is the most clearly missing from every shipped system. None of the current designs inspect the relationship between stored items on a regular schedule. The store sees a collection of rows; it does not see a graph of entailments.

Protect the exception explicitly

If an item contradicts a stored generalization, it is the single most valuable thing in the store, because it is the only evidence that the generalization is bounded. It should be the last thing dropped, not the first. This inverts the usual policy, and it is probably the highest-leverage single rule available.

The rule composes with the previous one. The subsumed is the candidate. The exception-bearing is protected. The two together say: forget the items that are redundant given what the store also holds, and protect the items that are most informative precisely because they are not redundant.

Keep the count, drop the copies

Twenty instances of a pattern can often become one statement plus the number twenty. The count preserves the strength of the evidence, which is most of what the twenty were carrying, at a fraction of the space. What is lost is the ability to re-derive the abstraction from scratch, which is a real loss and should be stated.

Math. The formal name for the operation is sufficient statistic, in the Neyman-Fisher sense: a function of the data that retains all the information in the data relevant to a parameter. Twenty Bernoulli observations of a coin collapse to the count of successes, which is sufficient for the probability parameter. The individual outcomes can be discarded without loss of inferential information, and the same idea, applied loosely, is what "keep the count, drop the copies" is doing. The right way to think about it is that a count is a compressed form of the data, not a degradation of it, and the decision to keep the count is the decision to keep the data in its most compressed sufficient form.

Stream clustering systems have shipped a version of this idea. CluStream's micro-clusters carry a cluster feature vector with linear sum, squared sum, and timestamp sums that are sufficient statistics for the cluster's center and spread, and the individual points are kept only as long as the micro-clusters need them. The same machinery, transposed to an agent memory store, is what a "keep the count, drop the copies" rule would look like in practice.

Make it observable

A store that forgets must be able to say what it forgot and roughly why, even if it cannot restore it. A silent lossy operation on a user's memory is not acceptable, and a tombstone is cheap compared to what it replaces. The user has the right to know what the store has decided is redundant, and the store that makes the decision has the obligation to surface it.

This is also the rule that makes the rest of the design auditable. Without an audit log of subsumption events, there is no way to tell whether the policy is doing what it claims to be doing, and there is no way to recover when the policy is wrong.

05 · Compression, Understanding, and the Disabled Learning Loop

The larger argument, stated so it can be argued with.

Compression and learning are close to the same thing. A model that has found a shorter description of its experience has, in a real sense, identified regularity in it, and the identification is what model selection under the minimum description length principle treats as evidence the model is a better model. The cautious claim here is established in the MDL literature: shorter description length is a model selection criterion, and a more compressed model is in a precise sense a better model of the regularity in the data.

The stronger claim, sometimes drawn from this tradition, that "to understand is to compress," is an extrapolation and not a theorem. It is the working hypothesis used to motivate the design argument, and the design argument would still hold without it. State it precisely and the argument is that a system architecturally prevented from compressing its own experience is prevented from doing the work that model selection requires, however capable its reasoning is in the moment.

Insight. The cognitive-science version of this claim is older and better supported. Anderson, Bjork, and Bjork's 1994 paper on retrieval-induced forgetting established that retrieval of one item suppresses retention of related items, and the result is widely read as adaptive: the system is reducing interference, sharpening the items most likely to be needed next, at the cost of weakening items less likely to be retrieved. The same underlying logic, that the system gets better at what it is asked to do by getting rid of what it is not, is the move from which the argument here takes its license. The 2023 Annual Review of Psychology piece on active forgetting makes the adaptive case explicitly. Memory is not a recording. It is a system that selectively retains what the present task needs.

Current agent memory is architecturally prevented from compressing its own experience. It appends. It never reduces except under pressure, and the reduction it does perform is content-blind. So the store grows monotonically as an undigested record, and the ratio of what the system has understood to what it has stored falls with every session.

The provocative form, stated so it can be argued with: an agent that cannot forget cannot learn, only accumulate. Those look similar early on, when the store is small and everything in it is useful. They diverge completely at scale, and every system currently shipping is on the accumulation curve.

06 · What an Experiment Would Have to Show

The argument is structural, not measured. Three predictions make it testable, and the third is the one most likely to come back negative.

Prediction 1. In an append-only store, retrieval precision on a fixed query set should fall as store size grows, and the fall should be steepest on the topics with the most stored items. The second clause is the discriminating one: uniform degradation would just be a scaling problem, while degradation concentrated where coverage is highest is the specific pathology Part 2 describes. This needs no new mechanism, only an existing store, a fixed query set, and per-query precision measurements stratified by store density.

Prediction 2. Replacing a cluster of near-identical items with one generalization plus a count should leave retrieval quality unchanged or better on queries about that cluster, while freeing slots that measurably improve unrelated queries. If quality on the cluster drops, the compression was lossy in a way that mattered and the policy needs narrowing. If unrelated queries do not improve, the freed slots are not the constraint and the rest of the analysis is wrong.

Prediction 3. A policy that preferentially retains contradiction-bearing items should outperform a usage-based policy on queries whose correct answer is an exception. This tests Part 4's inversion directly, and it is the one most likely to come back negative, because exceptions may simply be too rare to measure with any reasonable sample size. A negative result here is not a refutation of the policy. It is a refutation of the measurability of the property, and the design would have to fall back on the structural argument alone.

Prediction 1 first. It needs no new mechanism, only an existing store and a fixed query set, and it establishes whether the problem is real before anything is built.

Demo 01 · Two Stores, Same Items, Different Policies

A simulation of a small agent memory store under two policies at once. Add instances of a pattern (each near-duplicate of the last) and watch both stores behave. The left store evicts by usage: the item that was retrieved most recently is the one that survives. The right store applies principled forgetting: it notices when a new note subsumes the old ones, and collapses the cluster to a single statement plus a count. Add a counter-example, the one observation that does not fit the pattern, and watch what each policy does to it. The point to observe: usage-based eviction targets the rare item first, because rare means low usage. Principled forgetting targets the redundant cluster first and protects the exception.

Controls: "+ redundant instance", "+ counter-example", "+ unrelated note", "reset", capacity (4-40).

The simulation uses a small deterministic model: a usage signal that rises on every retrieval, an LRU that drops the lowest-usage item when the store is full, and a subsumption rule that fires when a new note covers the content of an older one. Add enough items and the LRU store will eventually evict the counter-example to make room for the redundant cluster; the principled store will not. Deterministic per item index, so resets replay identically.

The closest published substrate for Prediction 1 is LongMemEval's knowledge-update split, which tests whether an agent can correctly apply information that was stated early and contradicted later. Existing runs report large accuracy drops on this split across the leading long-context and tool-augmented chat assistants. Whether the drop is concentrated on the topics with the most stored items is the question the experiment would have to answer, and it is one no current public run asks directly.

Close · The Line to Keep

Every agent memory system is built so that nothing is lost, and calls that a feature. But the shorter description is the understood one, so a system that cannot drop what it has already generalized is not remembering more. It is only understanding less, more expensively.

Principled forgetting is the operation that would let a memory store notice when one note is the abstraction of a hundred others, and act on the relationship. The store that has it can answer more questions with less, which is the whole point of abstraction. The store that does not have it is on the accumulation curve, and the accumulation curve is where every system currently shipping lives.

Sources

Anderson, M. C., Bjork, R. A., Bjork, E. L. "Remembering can cause forgetting: Retrieval dynamics in long-term memory." Journal of Experimental Psychology: Learning, Memory, and Cognition 20(5), 1994, pp. 1063-1087. The 1994 paper that established retrieval-induced forgetting. Quoted and relied on in Part 5.
Anderson, M. C., Bjork, E. L., Bjork, R. A. "Retrieval-induced forgetting: Evidence for a recall-specific mechanism." Psychonomic Bulletin & Review 7(3), 2000, pp. 522-530. Showed that retrieval, not mere exposure, is what drives the suppression. The 2000 result is the one that supports the adaptive reading. Part 5.
Anderson, M. C., Hulbert, J. C. "Active Forgetting: Adaptation of Memory by Prefrontal Control." Annual Review of Psychology 72, 2021, pp. 1-36. The most recent consolidation of the active-forgetting argument, including the prefrontal-control neurobiology. Part 5.
Nørby, S. "Why Forget? On the Adaptive Value of Memory Loss." Perspectives on Psychological Science 10(5), 2015. Frames forgetting as functional across emotion regulation, abstraction, and context attunement. Background for Part 5.
McCloskey, M., Cohen, N. J. "Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem." In Psychology of Learning and Motivation, vol. 24, Academic Press, 1989, pp. 109-165. Original paper on catastrophic interference. The disambiguation from this line of work is in the introduction.
Li, M., Vitányi, P. An Introduction to Kolmogorov Complexity and Its Applications. Springer. The textbook treatment of MDL and its derivation from Bayes via Kolmogorov complexity. The formal claim Part 5 leans on.
Grünwald, P. D. "A Tutorial Introduction to the Minimum Description Length Principle." arXiv:math/0406077, 2004; also the opening chapters of Advances in Minimum Description Length: Theory and Applications, MIT Press. Part 5.
Fisher, R. A. "On the Mathematical Foundations of Theoretical Statistics." Philosophical Transactions of the Royal Society A 222, 1922, pp. 309-368. The original statement of the sufficient-statistic idea, and the source of the Neyman-Fisher factorization theorem used in the callout in Part 4.
Aggarwal, C. C., Han, J., Wang, J., Yu, P. S. "A Framework for Clustering Evolving Data Streams." VLDB 2003. The CluStream paper, source of the cluster feature vector that is the practical implementation of "keep the count, drop the copies" in stream clustering. Part 4.
Chhikara, P., Khant, D., Aryan, S., Singh, T., Yadav, D. "Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory." arXiv:2504.19413. The four-operation update vocabulary; the absence of principled forgetting in the production design. Part 1.
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., Yu, D. "LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory." ICLR 2025, arXiv:2410.10813. The knowledge-update split and the accuracy drop it reports. Cited as the substrate for Prediction 1 in Part 6.

Identity Is Made of Memory, and Memory Is Shaped by Identity

Swapnanil Saha — Mon, 14 Sep 2026 14:17:09 +0000

Ask what makes an agent the same agent across two sessions and there is no answer. The weights are shared, so they identify the model and not the agent. The context is discarded at the boundary, which is the boundary in question. The tools, the prompt, the workspace are true of any other instance pointed at the same repository. The honest position is that the agent is a convenience of speech. There are instances, and they are interchangeable.

Memory is the only candidate for what would change that. A private, continuous store is the only thing an instance can have that another instance does not, and the only thing that makes "the same one, later" mean anything at all. But the interesting half is the other direction, and it is the one nobody builds for. Identity is not only made of memory. It also shapes memory. What you remember constitutes who you are, and who you are determines what you encode, how you organize it, and what you can retrieve. An agent with no self has no principle for deciding what is worth keeping, which is exactly the failure we observe when agents write notes.

These are not two problems. The reason agents write bad notes and the reason agents have no identity are the same absence, seen from two ends.

Part 01. Persistence Is Not Continuity

01. Why a Memory Store Is Not the Same Agent

A database is persistent. That is not the same as continuous, and conflating the two is why "we added a memory store" does not produce an agent that is anybody.

Persistence means the data survives. Continuity means the later thing is the same thing as the earlier one, with a claim on its commitments. A shared store that any instance reads is persistent and not continuous. It is a library. Ten instances reading the same library are ten instances, not one entity that has been reading for ten days. Nothing about reading a shared file makes the reader the author.

Analogy. Think of a large engineering team with a shared wiki. Every engineer reads the same wiki, and every engineer can be replaced by another who reads the same wiki. The team as a whole has a kind of memory, but the team is not a single engineer who has been there for years. The wiki is persistence. Continuity of engineer would be the second engineer remembering the first engineer's commits, the first engineer's morning standup, the first engineer's disagreement with the architect, and inheriting the position rather than re-deriving it from the wiki's contents.

What continuity would additionally require, at minimum: the store is private to this line of instances and not a common pool; writes accumulate into it rather than being a fresh deposit each time; and the later instance inherits not just facts but the stance the earlier one took, what it decided, what it committed to, what it had already ruled out.

Almost every system built today satisfies the first condition partially and the third not at all. Notes carry findings. They do not carry the position the agent had settled into, so the next instance re-derives its stance from scratch even when it inherits every fact.

Insight. Persistence is a property of the storage system. Continuity is a property of the thing being stored. A store can be perfectly persistent and contain nothing that constitutes a self, because facts about the world are not facts about the agent. The store needs a different content type: decisions with their reasons, corrections with their authors, refusals with their grounds. The kind axis on a note is the lever that makes continuity possible, and it is the lever the field is still mostly ignoring.

The argument is close to two others on the site and the distinction is worth marking explicitly. Agent Memory Needs a Trust Ladder is about how much weight to give a stored note. A Guess Becomes a Fact on the Next Read is about what kind of note a thing is. The argument here is about a different axis entirely, whose note it is, and about the fact that there is currently no whose. The three are complementary: a note has a kind, a weight, and a bearer, and a memory design has to commit to all three or the result is incoherent.

Part 02. The Memory Theory, With Its Hole

02. Locke, and the Objection That Has to Be Named

The idea that memory constitutes personal identity is old and it has a famous problem, which is worth stating up front because a careful reader will raise it.

The classical position is John Locke's, in the chapter "Of Identity and Diversity" at the end of Book II of An Essay Concerning Human Understanding (1690). A person at time two is the same person as at time one when the later one can remember the earlier one's experiences. Memory is constitutive of identity. As Locke put it, "as far as this consciousness can be extended backwards to any past action or thought, so far reaches the identity of that person."

The objection that has stuck is Joseph Butler's, in "Dissertation I of Personal Identity" appended to his 1736 Analogy of Religion. Butler's charge is that Locke's account is circular. You can only remember an experience if you were the same person who had it, so memory presupposes identity rather than constituting it. One should really think it self-evident, that consciousness of personal identity presupposes, and therefore cannot constitute, personal identity; any more than knowledge, in any other case, can constitute truth, which it presupposes. The argument was latent in the eighteenth century and was sharpened in the twentieth by Derek Parfit in Reasons and Persons (1984), where it sits at the centre of the modern literature on what survives us over time.

There is also a less famous but more easily intuited problem, usually credited to Thomas Reid (1785), and worth mentioning because the agent case is closer to it than to Butler's. The transitivity problem: an old general remembers being a young officer, the young officer remembered being a boy, the old general does not remember being the boy. If identity follows memory directly, the old general is and is not that boy. Locke's response is to deny that identity is transitive on memory in the way the example assumes, but the response does not fully settle the issue.

Why the philosophical argument matters here, briefly. The Butler and Reid objections are not the engineering problem. What the engineering problem borrows from the philosophy is the positive claim, which is weaker than either objection targets and survives both of them intact. Whatever else personal identity requires, some continuity of psychological content is necessary. An entity with no connection whatsoever to its earlier states has no claim to being the same entity, whatever else is true of it. Agents fail this weak version. They do not fail the Butler circularity puzzle or the Reid transitivity puzzle. They fail the entry requirement, which is the easier one to meet and the one every shipped system currently misses.

For the engineering claim it is enough to note that two things are true at once. The strong version of the memory theory has known holes that have not been closed. The weak version of the memory theory, that some continuity of psychological content is necessary, has not been refuted and is the version the agent case needs. What follows uses the weak version and does not depend on Locke's stronger claim. Anyone who finds the philosophy uncongenial can take the engineering case on its own terms: the agent needs some carry-over, the field provides none, here is what would have to change.

Part 03. The Other Direction

03. Memory Constitutes Identity: The Amnesia Evidence

Before the loop closes, the direction argued so far deserves its own section, because it is the one with the clearest evidence.

Memory does not merely feed identity. It constitutes it, and the demonstration is what happens when it goes. Endel Tulving's patient KC, a Canadian man who developed dense amnesia after a motorcycle accident in 1981, is the classic case. KC's general knowledge about the world survived in usable form. His knowledge about himself as a person survived in some form too: he could tell you the names of his siblings, that he had once been a mechanic, where he had lived. The semantic self was largely intact.

What KC lost was the episodic self. He could not remember a single thing that had ever happened to him. He had no recollection of his wedding, of any birthday, of any trip. The autobiographical self, the one built out of remembered episodes and the story they fit into, was gone. Tulving's reading of the case, and the reading that has held, is that personal identity in the relevant sense is grounded in the episodic memory system, and that without episodic memory the autobiographical self does not assemble.

The honest reading of the clinical picture is more careful than the one-paragraph version. KC retained a semantic self, which means that the loss is not total, and modern work has documented patients who retain a more intact sense of self across severe amnesia than the clean thought experiment implies. What survives in every case is partial and varies. The autobiographical self, the one that knows itself in time, is what the episodic memory system supports, and that is the half the agent case is about.

Insight. Knowing about yourself is memory. Every "I am a person who does X" is a proposition stored somewhere, and the store is part of what makes it true of you. Strip the record and you have stripped the content, not just the access. The self is not a homunculus reading the record, because there is no homunculus anywhere in the literature that does not import what it claims to explain. The self is the record's effective organization, in the same sense that a corporation is not a person above the org chart but the organization of contracts and decisions and roles that the law treats as a single bearer.

So the appropriating function and the appropriated content are different things, and both are required. Something has to perform the act of taking an experience as mine, and that act may well be its own faculty. But what the "I" turns out to be, the whole content of it, is assembled from what was retained. A functioning appropriator with an empty store does not produce a minimal self. It produces nothing to be a self about.

That is why the two directions are not a symmetry to be admired. They are a dependency in both directions at once, which is exactly what makes the agent case a deadlock rather than a shortfall.

Part 04. The Loop Closes

04. Identity Also Shapes Memory: The Three Findings

Here is where the loop closes, and this is the part of the argument that has engineering consequences today.

In humans, the self is not merely the output of memory. It is an active filter on the input. Three findings establish the point, and the engineering case for them is that the same kind of filtering has to be supplied somehow, because a writer that does not filter is a writer that does not write well.

Self-relevant material is encoded more strongly

Information processed in relation to the self is recalled better than information processed semantically or shallowly. The effect was first demonstrated by Rogers, Kuiper, and Kirker in 1977 and confirmed by Symons and Johnson's 1997 meta-analysis across the literature. The standard interpretation is that the self is a particularly elaborate and often-used schema, and that elaborative encoding under that schema produces richer retrieval cues. The practical version: write-time decisions made on behalf of a well-defined self will be better calibrated than write-time decisions made on behalf of no self, because "relevance to me" is a far more selective filter than "could be useful someday."

Autobiographical memory is organized around the self

Conway and Pleydell-Pearce's Self Memory System, in a 2000 Psychological Review paper, proposed that autobiographical memory is organized hierarchically around self-knowledge, with the working self as the control process that determines what gets encoded and what gets retrieved. The architecture is partonomic: lifetime periods, general events, event-specific knowledge, terminating in episodic memory. The working self sits above this hierarchy and modulates access. The reason memory is organized this way is that the self supplies the indexing vocabulary. Without the self, memory loses the indices and becomes a flat list of episodes that nothing in particular pulls from.

Recall is reconstructive and bends toward coherence with the self-narrative

Bartlett's 1932 Remembering argued that recall is not reproductive but reconstructive, and that the reconstruction is shaped by the rememberer's existing schemata. Modern work, including the constructive episodic simulation hypothesis of Schacter and Addis, extends the picture: the same constructive machinery that builds memories also builds imagined futures, and in both directions the construction is shaped by the self-narrative the person already holds. The fact that this can go wrong, that people can recall with high confidence things that never happened, is not an objection to the underlying claim. The claim is that the self is the schema reconstruction works under, and a writer with no self writes without a schema at all.

Insight. Those three findings are not independent of each other. The self-reference effect is what you get when encoding is done under the self-schema. The Self Memory System is what that schema looks like when made architectural. Reconstructive recall is what retrieval does once the schema is in place. They are the same observation at three levels of detail, and the engineering consequence at every level is the same: a self is a relevance function for encoding, and a relevance function is what is missing from the write-time decision in every shipped agent memory system.

That gives the self a job that is purely functional and has nothing to do with philosophy: it is the relevance function for encoding. This is the version of the claim the agent case needs. It does not depend on a theory of consciousness or on a metaphysics of persons. It depends only on the observation that humans with a self pick better notes than entities without one, and on the engineering question of what in the agent corresponds to the self's job.

05. The Two Problems Are One Problem

Now look at the agent's write-time problem from that angle. An agent asked to record what it learned has no basis for choosing. It cannot know the future query, which is a problem in its own right that takes more space than this argument can spare. But it also has no standing interests, no accumulated positions, no sense of what kind of thing it has needed before, because there is no it that has needed anything before. Every session's write decision is made by an entity with no history of caring about anything.

That is why the notes come out generic or exhaustive. Those are the only two strategies available to a writer with no self: hedge toward the universal, or record everything and let the future sort it out. Both are rational. Neither is good. And no amount of prompting fixes it, because the missing input is not an instruction, it is a history.

Two symptoms, one cause. Picture a junior hire on day one, before they have ever read the codebase, asked to write the team wiki for everything they learned in their first week. They will produce either anodyne entries that say nothing in particular or a verbatim transcript of every meeting. Both are the rational response of a person with no history of caring about the project, and both are useless. The fix is not a better prompt. The fix is a person who has been there six months and knows what matters, because by then they have positions and the positions decide the note.

So the two problems are one problem. Agents have no identity because they have no continuous memory. And they cannot write good memory because they have no identity to make encoding decisions on behalf of. Each absence sustains the other. Either absence is enough on its own; together they form a closed loop with no obvious place to start breaking it.

Part 05. Entering the Loop

06. What the Entry Point Has to Be

A cycle like that has to be entered somewhere, and the entry point is not a self-model. Writing "you are an agent who cares about X" into a prompt produces a description of a self, not a self, and the description has no causal connection to what actually gets encoded. You can put a personality in front of a writer with no history and the writer will still write like a writer with no history, because the personality is not load-bearing. It is a costume on the same engine.

The entry point is the accumulated record of what this line of instances actually did: the decisions it made, the things it ruled out, the corrections it received, the mistakes it made twice. Not a personality, a track record. That is the material a self is made from, and unlike a prompt it is generated by the system's own history rather than asserted over it.

Which suggests a concrete ordering rather than a philosophical program.

Store decisions and their reasons, not only findings. A stance is what a later instance needs in order to inherit anything more than facts. The note kind that stores an architectural decision with its reasoning is the smallest material a self is made from. A store of bare findings is a library. A store that includes decisions with their reasons is closer to a track record.
Store corrections durably. Being corrected is the single most identity-forming event available to a system, and the most perishable thing in the pipeline. A correction that survives only in the transcript of the session in which it was given is a correction the next instance never sees. The corrected mistake has to be as recoverable as the original decision it overrules, which means storing the correction as a first-class object with a date and an author.
Let the accumulated record inform what gets written next. The write path has to see the record. A store whose writes are informed only by the current session cannot close the loop, because the next instance starts from the same blank state the last one did. The harness has to inject the relevant slice of the record at write time, not at recall time, and the slice has to be the slice that bears on what is being decided, not everything the store has ever held.

Insight. None of those three steps requires solving personal identity. They require noticing that the record has to include the things a self would be made of, and today it mostly includes findings. The philosophical problem is real and the engineering problem is real, and they are not the same problem, but the engineering one is what you can actually do something about. The engineering case is what follows, and the philosophical part is there to give the engineering part its weight.

The store I work with day to day, vectr, has a note kind called decision for this purpose, sitting alongside directive, gotcha, finding, and task. The decision kind is treated as a recall-only kind: it waits to be asked for, and when asked it returns its neighbours in chronological order, so an architectural decision is read as a sequence with the decisions that came before and after it. That is a small step toward the kind of record a later instance can inherit a stance from, and it is the kind of design choice that the field needs more of, and the kind the field mostly does not make.

Part 06. Why It Matters

07. Three Consequences That Are Not Philosophy

Three consequences that are practical rather than conceptual, and that follow from the closed loop whether or not the philosophical case is interesting on its own.

Accountability needs a subject

Holding a system to what it committed to yesterday presupposes something that persists to be held. With interchangeable instances there is no bearer: the thing that made the commitment is gone, and the thing in front of you never made it. Every framework for agent oversight, from regulatory regimes to internal review boards to a user trusting that yesterday's promise will still be honoured, quietly assumes a continuous subject, and none of the systems provide one. The accountability gap is not a future risk. It is the present state of the field, and the gap is exactly the size of the missing self.

Judgment is calibration against your own past errors

A system that cannot remember being wrong cannot become better calibrated; it can only be made better by a better model. Those are different mechanisms, and only one of them is available to a running system. Capability scaling and learning from your own mistakes are different axes, and current systems have only the first. The model can be upgraded. The history cannot, unless the system has somewhere to put it, and the something that puts it there has to be the same something that consults it later. That something is what the argument is calling a self.

Trust is a relationship with a particular entity

A user builds trust with something that has a track record. If every session is a new instance with no continuity, the user is not building trust with an agent, only with a model. That may be fine, and it is the situation today. But it is a different thing, and it caps what the relationship can become. A user who trusts a model is trusting that the next instance will reproduce the relevant parts of the prior instances' behaviour. A user who trusts an agent is trusting a particular entity's history, and the latter is a stronger claim with different failure modes.

Insight. These three consequences are not independent. Accountability requires a bearer. Calibration against your own past requires that the bearer remember the past. Trust requires that the relationship is with a particular bearer across time. Take any one of the three seriously and the missing self becomes a missing load-bearing element of the design, not a footnote. The closed loop between no-self and bad-notes stops looking like a curiosity and starts looking like a structural reason the field's results are what they are.

Part 07. The Uncomfortable Objections

08. Continuity as Cost, Not Pure Win

Is continuity desirable? An agent that accumulates a self also accumulates its distortions: entrenched positions, self-serving reconstruction, the tendency to remember what fits the story. Human memory's identity-shaping is not free, and importing the mechanism plausibly imports the pathology. There is empirical reason to take the worry seriously. Memory conformity effects in collaborative recall, the misinformation effect, and the well-documented confabulation that follows damage to the hippocampal system all point the same way: a self that integrates memory does so at the cost of accuracy, and the cost is non-trivial. This deserves to be stated rather than waved away, and it is the strongest argument against the whole proposal.

The reply that survives is honest about the trade. A writer with no self writes badly in a way that the field can already measure: notes that are generic, notes that are exhaustive, notes that get re-derived every session. A writer with a self writes with the distortions the literature documents, but at least the distortions are knowable and can be checked against the record. The current state has neither quality nor the chance of catching the absence of it. The proposal is not that continuity is free. It is that the absence of continuity is more expensive than the cost of having it, and that the cost is the kind of cost you can design around.

Is this just personalization? No, and the distinction matters. Personalization is a model adapting to a user. This is a system accumulating a history of its own actions and positions. The user is not the subject; the agent is. But the two are easy to conflate, and the distinction has to be kept clean. Personalization does not solve the write-time problem because the personalization is built on the user's history, not the agent's, and the agent that writes the note is still the agent with no history of its own. Personalization gives you a model that knows the user. What the loop needs is a model that knows itself.

Does anything change if instances share the store? Then you get one entity with many hands rather than many entities, which is a coherent design and possibly a better one. The claim here is not that private-per-line is correct, only that something has to make the later thing the same thing as the earlier one, and a shared pool with no continuity of stance does not. A team that shares a wiki but does not record why decisions were made has a wiki, not a team memory. A team that records decisions with reasons, even if the members rotate, has a team memory. The variable is not the number of hands. The variable is whether the store contains the kind of content a self would be made of.

Part 08. What It Predicts

09. The Three Tests, in Order

The argument above is structural. The mechanism it proposes is a way out of the closed loop, and the three predictions below are how the mechanism earns its keep. None of them have been run. All three are specifiable, which is the useful part.

Prediction 1. Notes carrying a decision and its reasoning should be retrieved and acted on more often than notes carrying an equivalent bare finding, because the later instance can inherit a stance rather than re-deriving one. Measurable on an existing store with a decision kind already in it. The confound to control for is the fact that a writer who has chosen to store a decision has already done selection work, so any comparison should be between decision notes and finding notes of the same provenance and on the same topic.

Prediction 2. An agent given its own prior corrections should make the corrected mistake less often than one given the same information as a neutral fact. This isolates the identity-forming claim from the mere-information claim, and it is the load-bearing test. If provenance-as-my-own-past-error makes no behavioral difference, the ordering in Part 5 is wrong. Run this one first. It is the cheapest and it tests the actual mechanism rather than a downstream effect.

Prediction 3. Note quality, measured as later retrieval usefulness, should improve as a store accumulates, if and only if the write path can see the accumulated record. A store whose writes are informed only by the current session should show flat note quality regardless of size. The curve is the test: a store that knows itself grows, a store that does not plateaus, and the difference is the visible signal of the loop starting to turn.

Interactive demo: Pick the notes, twice

You are the agent's write path, deciding which notes to keep from a session. The two columns show the same session transcript and the same five candidate notes. The left column has no history panel: you are a writer with no self. The right column has the same session but you can see the line's prior decisions and corrections before you pick. Play both sides and notice whether the two picks come from the same part of the menu, or whether the history panel moved you. That is the load-bearing difference the loop is about, felt from the inside.

Close. The Line to Keep

An agent has no way to be the same agent twice, and that is not only a philosophical deficit. Without a "self" there is nothing to decide what was worth remembering, so the absence that prevents identity is the same absence that makes the notes bad. The two problems are one problem, and the loop has to be entered somewhere. The place it has to be entered is the record: store decisions with their reasons, store corrections durably, let the record inform the next write, and the loop starts turning in the useful direction. The track record is the material a self is made from, and unlike a prompt it is generated by the system's own history rather than asserted over it.

An agent that knows what it has done is on the way to knowing who it is. An agent that knows who it is on the way to writing notes worth reading. Neither of those is solved by a longer system prompt. Both are started by a record that is built to be inherited.

Sources

Locke, J. An Essay Concerning Human Understanding, Book II, Chapter XXVII, "Of Identity and Diversity." Source of the memory theory of personal identity.
Butler, J. "Dissertation I, Of Personal Identity," appended to The Analogy of Religion (1736). Source of the circularity objection.
Reid, T. Essays on the Intellectual Powers of Man (1785). Source of the transitivity counterexample.
Parfit, D. Reasons and Persons (Oxford University Press, 1984). The modern Relation R view.
Tulving, E. "Remembering and knowing the past." American Scientist 77, 361-367, 1989. The episodic-versus-semantic distinction with the KC case study.
Conway, M. A., and Pleydell-Pearce, C. W. "The construction of autobiographical memories in the self-memory system." Psychological Review 107(2), 261-288 (2000).
Rogers, T. B., Kuiper, N. A., and Kirker, W. S. "Self-reference and the encoding of personal information." Journal of Personality and Social Psychology 35(9), 677-688 (1977).
Symons, C. S., and Johnson, B. T. "The self-reference effect in memory: A meta-analysis." Psychological Bulletin 121(3), 371-394 (1997).
Bartlett, F. C. Remembering: A Study in Experimental and Social Psychology (Cambridge University Press, 1932).
Schacter, D. L., and Addis, D. R. "The cognitive neuroscience of constructive memory: remembering the past and imagining the future." Philosophical Transactions of the Royal Society B.
Hu, Y., Liu, S., Yue, Y., et al. "Memory in the Age of AI Agents." arXiv:2512.13564.
Agent Memory Needs a Trust Ladder. The trust axis.
A Guess Becomes a Fact on the Next Read. The kind axis.
The Agent Never Chooses to Remember. The harness-injection mechanism Part 5's third step depends on.
vectr source: [agent/working_context_store/_events.py](https://github.com/swapnanil/vectr/blob/main/agent/working_context_store/_events.py) and [_store.py](https://github.com/swapnanil/vectr/blob/main/agent/working_context_store/_store.py) (note kinds decision, directive, gotcha, finding, task).

Consolidation Is the Missing Phase in Agent Memory

Swapnanil Saha — Thu, 10 Sep 2026 15:36:18 +0000

The field has one answer to running out of context, and that answer is more context. Windows went from thousands of tokens to hundreds of thousands to millions, and the next number is already being quoted. The implicit theory is that a large enough window makes the memory problem go away.

Biological intelligence had the same problem and did not solve it that way. Working memory in humans is famously tiny, a handful of items, and it did not grow. What grew instead was a second process, running when nothing else is running, that decides which transient traces become durable and reorganises them as it does. The standard model in the systems-consolidation literature, formalised over decades by Squire and collaborators and extended through Diekelmann and Born's active-systems-consolidation work and Stickgold's sleep-replay studies, has the hippocampus hold recent memory and the cortex hold the long-term store, with the transfer driven by repeated reactivation during sleep and quiet wakefulness. The mechanism is supported, the role of sleep is supported, and the specific molecular choreography is still being argued about in the literature, including which sleep stages do which work.

The missing primitive in agent memory is not capacity, it is a phase. Agent systems have a write path and a read path and nothing in between. Nothing ever revisits what was stored, merges two notes that turned out to be the same fact, promotes a repeated observation into a general one, or drops what has been superseded. Storage happens at full speed during the task, judged by an agent that is busy doing something else, and then it is never touched again. That is not a small gap in the design. It is a whole stage of the pipeline that does not exist.

Part 1: The Diagnosis

01 · What a Window Buys, and What It Does Not

A larger context window is genuinely useful and the argument here is not against one. It buys the ability to hold more of a problem at once, and problems that fit are easier than problems that do not. Everything that follows assumes the window is not a small one and still argues the gap remains.

What a larger window does not buy is any of the following.

It does not decide what mattered. Everything in a window is equally present, which sounds like an advantage until you notice that "equally present" is the same as "unranked". A window is a pile, not a structure. Nothing in the design says that the fact you wrote five minutes ago matters more than the one you wrote an hour ago, or that the fact you wrote once matters less than the one you wrote three times. The window does not know, and the model reading the window has no signal beyond recency to act on.

It does not survive the session. This is the obvious one and it is usually the only one people name. But note the shape of the failure: it is not that the information was too big, it is that the boundary discarded it wholesale, with no step that asked what was worth carrying. The compressed version of a session, what gets written into a continuation summary, is a single act of summarization with no further chance to recover what the summarizer decided did not matter. "What Actually Survives /compact in Claude Code" measured this: across 108 and 138 forced compactions, the very first continuation summary carried zero of ten seeded facts in both arms, and the dropped facts stayed dropped through every subsequent summary. Capacity was not the issue. The summarizer had to fit the conversation into a smaller window and it made a one-shot judgment about what to keep.

It does not improve with use. A window is the same window on the thousandth task as on the first. Nothing about it gets better at holding this particular codebase, this particular user, this particular recurring mistake. A memory system could, in principle, learn to weight the things that turn out to matter. A window cannot, because it carries no information across sessions and no signal during them about what is and is not worth keeping.

Each of those is a different problem, and none of them gets smaller as the window grows. Scaling capacity addresses the first-order symptom and leaves the structure untouched.

Insight. Capacity is the visible failure, which is why the field treats it as the problem. The actual problems are the ones a bigger window cannot reach: a pile of unranked items, a boundary that drops things wholesale, and a workspace that learns nothing from being used. None of them is a token-budget problem. All of them are pipeline-shape problems.

It is worth dwelling for a moment on the biological contrast, because the biology is where the design vocabulary already exists. Cowan's 2001 reconsideration in Behavioral and Brain Sciences puts the working-memory capacity at roughly four chunks, with a typical range of three to five, revising Miller's classic seven plus or minus two by stripping out the benefit of chunking. The number is the most-cited modern estimate, and the exact figure depends on task conditions. What the biology did not do, when faced with the same pressure, was grow working memory. It built a second process for moving things out of working memory into a long-term store, and a third process for organising that long-term store while no retrieval or encoding was happening. The long-term store in humans is not a pile either; it is structured, and the structure is built offline.

Part 2: The Missing Phase

02 · What Consolidation Actually Does

The word gets used loosely, so it is worth being precise about the operations, because the operations are what agent systems lack.

Consolidation, in the standard model, is the set of transformations applied to already-stored traces during offline periods. The label covers synaptic consolidation, which stabilises individual traces at the cellular level on a fast timescale, and systems consolidation, which reorganises memory across the hippocampus and cortex on a slow one, with the two understood as complementary rather than competing. The standard reference, Squire, Stark and Clark's 2004 review of the medial temporal lobe and the later Cold Spring Harbor consolidation paper, describes the broad picture: recent memories depend on the hippocampus, remote memories are supported by cortical sites, and the transfer is driven by repeated reactivation during slow-wave sleep and quiet wakefulness. Sleep's role is well supported; which sleep stages do which work, and how the molecular details run, are still being worked out in the literature, and that nuance is preserved here rather than papered over.

Strip the biology and the operations are ordinary engineering. None of them is exotic. The reason they matter is the combination.

Selection. Not everything survives. Something decides what does. The biological version is partly spontaneous decay, partly activity-dependent stabilisation. In an engineering system, the decision is what the design says it is: keep what is reinforced, let the rest go.
Integration. A new trace is reconciled against what is already known, rather than appended beside it. The hippocampus-to-cortex picture is the canonical example: the new episode gets bound into the existing cortical schema instead of sitting as an isolated record.
Abstraction. Repeated specifics become a general pattern, and the specifics can then be dropped or kept at lower priority. Schema formation is the standard model term for it. The generalisation is the thing that survives; the instances are the input.
Reorganisation. Structure changes as a result, so later retrieval is against a different shape than the one that was written. The store is not a journal in chronological order; it is a graph whose connectivity reflects what has been learned.

Now check each against a current agent memory system. Selection happens once, at write time, by an agent guessing. Integration does not happen. Two notes about the same fact sit as two notes forever, and a note that contradicts an older one is appended rather than reconciled. Abstraction does not happen. Twenty observations of the same pattern remain twenty observations, and the pattern that would tie them together never gets written. Reorganisation does not happen. The store's shape is the accumulated order of writes, which is write order, which is not the shape anyone would design for retrieval.

The write path is doing all four jobs at once, badly, in the middle of a task, and then nothing revisits the result.

Analogy. The biological picture is a librarian who arrives at night. During the day the library is open: people come in, read, take notes, and leave. Nobody stops to alphabetise the new arrivals, decide which notes deserve a second copy, or merge two index cards that turn out to describe the same book. The librarian does that work after the library closes, and the next morning the shelves are in a different shape. The agents are the readers. There is no librarian.

03 · Why the Write Path Cannot Do This Job

The instinct that says "the agent should just write better notes, integrating and abstracting as it goes" is the same instinct that says "make the window bigger." Both fail for structural reasons, not for lack of effort.

At write time the agent has exactly one episode. Integration requires comparing against the rest of the store, which it has not read. Abstraction requires having seen the pattern repeat, which by definition has not happened yet on the first instance. Selection requires knowing what will be asked later, which is the thing nobody can know.

All three operations need something the write path structurally does not have: a view over many episodes, at a time when no episode is in progress. That is what makes consolidation a separate phase rather than a better version of an existing one. A single agent in a single task has the wrong vantage point by construction, and no amount of prompt engineering will give it the right one, because the right vantage point is the cross-section of every episode the store has ever seen.

There is a second reason, and it is about incentives rather than information. During a task, the agent's objective is the task. Every token spent curating memory is a token not spent on the thing it was asked to do, and the benefit lands in some future session that the current one is not evaluated on. A process that only pays off later will always lose to the process being measured now. Consolidation has to run when there is no competing objective, which is precisely why the biological version runs offline. The match between "no competition for the work" and "no evaluation of the work" is the entire reason a phase is the right shape, not a job description.

Warning. This is also why a smarter model will not fix the gap on its own. A better reasoner is still a better reasoner about the thing in front of it. It can write a slightly better note in slightly better prose, and the structural problem is unchanged: it is one episode, it is in the middle of a task, and it is being measured on the task. A model that thinks harder about its own memory during a task is a model that does the task worse, with a slightly tidier store nobody is using.

Part 3: The Distinction

04 · Compaction Is Not Consolidation

A careful reader will say context compaction already is consolidation, and it is worth addressing head-on rather than ignoring, because the rest of the argument depends on the distinction holding.

The two are related. The shared shape is: take a body of stored material, apply a transformation, keep the result. The shared failure mode is that the transformation is lossy. After that, they diverge.

Compaction reduces a single in-flight transcript to fit a window. The summarization step at a compaction boundary sees one conversation, the one that just happened, and the output has to fit the smaller window the next turn will use. The unit of work is one transcript, the trigger is window pressure, and the output is a summary the next turn reads.

Consolidation operates across many stored episodes with no window pressure. The unit of work is the long-term store, the trigger is some offline condition (time, count, idle), and the output is a restructured store the next retrieval reads. The operation is allowed to merge two notes, drop a superseded one, and rebuild the index by which retrieval happens. The summariser at a compaction boundary is not allowed any of those things, because it has to produce a piece of text, not a reshaped store.

The two differ in scope, in what triggers them, and in what they are allowed to do. A compaction reduces one transcript; a consolidation reshapes many. A compaction is a property of the harness; a consolidation is a property of the store. "What Actually Survives /compact in Claude Code" measures the first. The system argued missing here is the second, and they are different objects.

Aspect	Compaction	Consolidation
Scope	One in-flight transcript	Many stored episodes
Trigger	Window pressure	Offline condition (time, count, idle)
Output	A summary the next turn reads	A restructured store retrieval reads
Allowed to merge two notes	No	Yes
Allowed to drop a note	By not summarising it	Yes, with evidence kept
Allowed to reorganise the store	No	Yes
Failure mode	Silent loss of un-summarised detail	Silent loss of wrongly dropped item

If the distinction does not hold up, the central claim is wrong. The strongest reading of the distinction is that they live at different layers of the system: compaction is a property of the prompt-construction pipeline, consolidation is a property of the memory store. A system can do both, do one, or do neither. The systems that ship today do compaction and skip consolidation. The systems that do both are rare and narrow; the two that come closest are Letta and Zep, and they are worth a closer look, because the claim "agent systems have no consolidation" is a generalisation they complicate.

Letta's sleep-time agent is a real counter-example. Their sleep-time compute runs a second agent in the background that rewrites the primary agent's in-context memory blocks, summarises recent conversations into stable notes, and revises learned context. It is the closest published design to a consolidation phase this framing allows, and it is a narrow one: the operations are organised around the in-context memory blocks, the primary agent's surface of interaction, not around the long-term store as a whole. The generalisation, dedup, and structure-changing operations argued missing here are partly there for the in-context tier and partly not there for the rest.

Zep's Graphiti runs community-level consolidation: it clusters entities, summarises clusters, and refreshes the cluster summaries when new information shifts them. The community-detection work happens during ingestion through a dynamic extension of label propagation, with periodic full refreshes to correct drift. This is closer to the biology's structure-formation step, in the same "reorganise the store" sense, but it operates on a graph of entities and facts rather than on the notes an agent writes about its own work. The two systems do different slices of what is being described. Neither performs the full combination: dedup across the long-term store, abstraction of repeated observations, and structure-changing reorganisation, all in one offline process.

So the claim narrows. Most production agent memory systems have no offline phase that merges, abstracts, and reorganises the long-term store. Letta and Zep are real and partial counter-examples, and they are worth studying precisely because they show how much of the design problem is left even after the offline phase exists. What is being argued is about the rest, and the rest is most of the field.

Part 4: The Design Problem

05 · The Judgments a Phase Has to Make

Saying "add a consolidation phase" is easy. The hard part is that it has to make judgments, and each one is a place where it could be wrong in a way nobody notices.

What merges. Two notes that say the same thing should become one. But "the same thing" is exactly the judgment that a near-duplicate check gets wrong in both directions. Too loose and it destroys a distinction that mattered; too tight and it does nothing. Merging is lossy and, unlike a retrieval miss, it is not recoverable. A merged note is harder to undo than a forgotten one, because part of the evidence that supported keeping them separate was in the two notes' differences, and those differences are gone.

What generalises. Twenty instances of a pattern should become one statement of the pattern, and the twenty should probably go. But the generalisation is an inference, and it inherits every problem of an unlabeled inference. If it is stored with the same authority as the observations it replaced, a wrong generalisation is now a fact with no visible provenance and no surviving counter-evidence. The honest design says: keep the generalisation, keep the instances, and rank the generalisation below the instances, so any specific case can override it.

What is dropped. This is the one nobody wants to build, and the next section is about why it is unavoidable.

What gets restructured. If consolidation reorganises the store, then retrieval is running against a shape that was not the shape at write time. That is the point, but it means a note's future retrievability is being changed by a process the writer never saw. A note that was highly retrievable because it was the most-recent observation of something can become unretrievable the moment twenty newer observations push it out of the active window. That is the right outcome if the newer observations are better, and the wrong outcome if they are not. There is no version of this that is not a judgment.

Insight. None of these is a reason not to build the phase. They are the reason it is a design problem rather than a cron job. A batch script can dedup; a cron can summarise; neither of them has a view of which decisions the store will regret, because the store has not yet had the chance to be wrong in a way that becomes visible. The phase needs to be instrumented, auditable, and reversible, in roughly that order of priority.

06 · The Uncomfortable Part: Forgetting

Every operation above reduces. Merging reduces two to one. Abstraction reduces twenty to one and discards the twenty. Selection keeps some and not others. Forgetting is the substance of consolidation, not a side effect.

A consolidation phase that only adds is not consolidation. It is a second write path, and a second write path makes the pile grow faster. So building this means building deliberate, principled loss into a system whose entire current design treats loss as failure. Retrieval misses are at least visible as empty results. Something consolidated away is not visible as anything: the store returns a confident, well-organised answer that is missing the case it dropped last week, and nothing in the response says so. The harness sees a working memory layer; the user sees an agent that knows what it knows. The dropped item is invisible because it is gone, and being gone is exactly what makes the design defensible and exactly what makes the failure mode silent.

Warning. Silent loss is the failure mode. The honest version of the proposal is not "add consolidation and memory gets better." It is: memory cannot get better without a phase that reduces, and reduction is the most dangerous operation in the system. Any serious design has to say what it keeps evidence of, what becomes irreversible, and how a wrongly-dropped item could ever be noticed.

Three design rules follow from taking that seriously.

Keep evidence, not the original. A merged note keeps both sources in an audit row, with the merge's reason and actor. A generalised note keeps the instances, demoted, with the generalisation labelled as an inference. A dropped note keeps a tombstone carrying the reason, the actor, and the timestamp, so the system can answer "did we ever know X" even when "X" is not retrievable as a current fact.

Reversible by default. If consolidation is append-only, like the note lifecycle in the revocation design on this site, a wrong consolidation decision is one more event, not a destroyed row. That is the property that lets a consolidation policy be aggressive, because the cost of being wrong is one undo, not an irretrievable loss. An irreversible policy has to be right; a reversible one only has to be right often enough.

Make the drop visible at retrieval. A wrongly-dropped item is invisible at the moment it would be useful. The only reliable way to surface it is to ask the system, after a session in which something should have come up and did not, what it has dropped recently. That is a query the agent rarely thinks to make, and a query the harness could be wired to make. The system that noticed the absence is the same one that has to be made to notice the consolidation failure.

Part 5: The Larger Claim

07 · On the Path to General Capability

The argument so far is an engineering one. Here is the larger claim, stated so it can be disagreed with.

A system that cannot reorganise what it knows cannot get better at knowing. It can get better at reasoning, because that is the model, and models are improving quickly. But its knowledge stays an undigested pile of episodes in write order, and every new task pays the full cost of finding what is relevant in that pile. The reasoning gets sharper on the same evidence; the evidence does not accumulate.

Competence and accumulation are different axes. Scaling the model moves the first. Nothing currently moves the second, because the only mechanism that could, a process that revisits and restructures what was stored, is absent from most production designs. The two real counter-examples, Letta's sleep-time agent and Zep's community consolidation, are partial: they cover parts of the long-term store and miss other parts, and the miss is structural, not a bug to be fixed by a patch.

That is why this is not merely an optimisation. A system whose knowledge never consolidates is one that starts from the same structural position on task one thousand as on task one. It may perform better, because the model is better, but it has not accumulated anything. Whatever general capability turns out to require, it is hard to believe it is compatible with never digesting your own experience.

Insight. The shape of the argument is not "the model is not good enough." It is "the model is good enough that the missing infrastructure is the part that limits long-term capability." The two are easy to confuse. The first invites a model improvement. The second invites an infrastructure one. They have different costs, different timelines, and different chances of working.

One honest limit needs stating. The argument is structural, not measured. There is no controlled experiment here showing that a consolidator improves an agent's capability on held-out tasks. What is identified is a missing piece, the reasons it is missing, and the literature the analogy points at. The measurement would be: take an existing store, run a merge-and-abstract pass over it, and measure retrieval precision and downstream task accuracy before and after on a held-out set. If the pass helps, the missing phase was load-bearing. If it does not, the answer is somewhere else.

08 · What This Predicts

Three empirical claims follow from the argument, and each one is testable on a store that already exists. The experiment is not run here, but it is specifiable, and specifiability is the part that turns a structural claim into a falsifiable one.

Interactive demo (in the HTML version). A store before and after one consolidation pass. Twelve notes from one agent across eight sessions, in write order. Two pairs are near-duplicates the agent did not notice. One observation repeats four times. One generalisation the system could have made is missing. Toggle the consolidation pass to see what merge, abstraction, and reorganisation each do, and which notes survive.

The three predictions, stated so they can be disagreed with:

Prediction 1. In a memory store with no consolidation, the proportion of notes that are near-duplicates of an existing note should grow with store size, and grow superlinearly with the number of sessions. Nothing removes them and every session can add one. This is measurable on any existing store today: write a near-duplicate detector, sample the store, and plot duplicate share against note count.

Prediction 2. Retrieval precision should degrade as the store grows, holding the query set fixed, because unconsolidated near-duplicates compete for the same slots. If precision is flat with store size, the pile is not actually costing anything and this argument is much weaker. "Relevance Is the Wrong Sort Order for Agent Memory" points at the same shape from a different angle: ranking a pile by similarity cannot but make the pile look bigger than it is, and the cost is paid at retrieval.

Prediction 3. An offline merge-and-abstract pass over an existing store should improve retrieval on a held-out query set without a model change. This is the load-bearing one: it tests whether the missing phase is worth anything, and it can be run on a store that already exists. It needs no new architecture, only a batch job and an honest before-and-after.

Prediction 3 is where to start. The other two are diagnostics. If prediction 1 is not borne out, the whole argument loses its quantitative premise. If prediction 2 is not borne out, the pile turns out to be free and the argument is about a non-problem. If prediction 3 is borne out, the missing phase is the lever. None of the three requires a model improvement, which is the point: the missing piece is not in the model, it is in the pipeline around the model, and the pipeline is the part an engineer can build today.

Close: The Line to Keep

The window got a thousand times bigger and the memory problem did not go away, because capacity was never the missing piece. What is missing is the phase where a system decides, when nothing else is happening, what its own experience was worth. Selection, integration, abstraction, reorganisation. Selection so the pile does not grow forever. Integration so two notes that mean the same thing stop competing for the slot. Abstraction so twenty observations of a pattern stop being twenty facts. Reorganisation so retrieval is running against the shape of what the system knows, not the shape of what the system once wrote.

It is harder to build than to describe, because every one of those operations is lossy and lossy is the failure mode the rest of the system is built to avoid. The way through is the same one the rest of the agent-memory design has settled on: keep evidence of what changed, make the change reversible, and let the system notice when a wrong change has slipped through. The biology has been doing this for a long time, badly enough that the role of sleep is still being argued about, well enough that the shape of the operation has not changed in decades. The agent-memory field is at the point where it could borrow the shape, and the borrow is overdue.

Sources and Further Reading

Squire, L. R., Stark, C. E. L., Clark, R. E. "The Medial Temporal Lobe". Annual Review of Neuroscience, 2004. The standard reference for the medial temporal lobe and the systems-consolidation picture; introduces the hippocampus-as-temporary-store, cortex-as-long-term-store model that the analogy in Section 01 rests on.
Squire, L. R., Genzel, L., Wixted, J. T., Morris, R. G. "Memory Consolidation". Cold Spring Harbor Perspectives in Biology, 2015. The review that consolidates the synaptic-and-systems distinction and the current state of the field; the claim in Section 02 that mechanisms are still being worked out comes from here.
Diekelmann, S., Born, J. "The memory function of sleep". Nature Reviews Neuroscience, 2010. Active systems consolidation framework: reactivation during SWS, schema integration, the slow-timescale redistribution borrowed in Section 02 for the analogy.
Cowan, N. "The magical number 4 in short-term memory: a reconsideration of mental storage capacity". Behavioral and Brain Sciences 24(1), 2001, pp. 87-114. The modern working-memory capacity estimate: roughly 4 chunks (3-5) when chunking is prevented, revising Miller's 7 plus or minus 2. The number used in Section 01 for the biological contrast.
Rasmussen, P., Paliychuk, P., Beauvais, T., Ryan, J., Chalef, D. "Zep: A Temporal Knowledge Graph Architecture for Agent Memory". arXiv:2501.13956, January 2025. The closest published agent-memory system to consolidation: community-level clustering of entities with dynamic label propagation during ingestion and periodic full refreshes. Cited in Section 04.
Letta. "Sleep-Time Compute". Letta blog. The sleep-time agent design: a second agent in the background that rewrites the primary agent's in-context memory blocks, summarises recent conversations, and revises learned context. The other real counter-example, also in Section 04.
Berry, J. "When Agent Memory Needs Sleep." Wire Blog. A field-side argument that consolidation is the same shape as compaction applied to the memory store rather than the conversation; the distinction in Section 04 takes the shared shape seriously and then argues the differences dominate.
Saha, S. "What Actually Survives /compact in Claude Code". Neighbour essay on this site. The measured 100+ compaction map; cited in Section 01 for the measured shape of the boundary and in Section 04 for the explicit distinction between compaction and consolidation.
Saha, S. "Reinstatement Is the Missing Primitive in Agent Memory". Neighbour essay on this site. The append-only note lifecycle and the reversibility property borrowed in Section 06 for the design rules of a reversible consolidator.
Saha, S. "Agents Never Remember What Isn't There". Neighbour essay on this site. The mechanism for noticing a wrongly dropped item that Section 06 needs: the absence is the only signal a consolidation failure leaves behind, and it is exactly what an absence-tracked system can be made to surface.
Saha, S. "Relevance Is the Wrong Sort Order for Agent Memory". Neighbour essay on this site. The case that ranking an unconsolidated pile by similarity makes the pile look bigger than it is, which is the quantitative premise Prediction 2 rests on.

The Writer Cannot Know the Reader's Question

Swapnanil Saha — Wed, 09 Sep 2026 18:33:39 +0000

Ask why agents write bad notes and the answers are all about effort. The model was lazy. The prompt was weak. It did not think about what would be useful later. Write better notes and the problem goes away.

This misdescribes the task. Writing a note is not summarizing what happened. It is placing a bet on a question that has not been asked yet, by a reader who does not exist yet, in a context nobody can see. The writer is asked to optimize for a target that is not merely unknown but unknowable at the time the decision is made.

Write-time precision in agent memory is not a hard problem solved badly. It is an ill-posed problem being answered anyway, and the outputs look like bad estimates because there is nothing else they could look like. The fix is not a better writer. The fix is a different unit of storage.

Part 1: The Asymmetry

01. Why Write Is Not Retrieve

The asymmetry is structural rather than practical.

Retrieval is evaluable. Given a query and a corpus, there is a fact of the matter about which items are relevant. You can label it, measure it, and improve against the measurement. This is why information retrieval has a century of methodology and agent memory keeps borrowing from it.

Writing is not evaluable in the same way, because the value of a note is not a property of the note. It is a property of the pairing between the note and a future query. At write time, one half of that pair does not exist. Not "is hard to predict." Does not exist. There is no distribution to estimate because the events that will generate the queries have not happened.

Insight: Write-time precision in agent memory is undefined, not hard. The writer is not making a bad estimate. It is being asked to optimize a target that has no value to estimate, because the future queries have not been sampled from a distribution that has not been fixed. A better guess does not exist, however carefully the writer tries.

The information-science literature has had a name for this for sixty years: anticipatory indexing. Robert Fairthorne, writing in the 1960s, made the same observation about a human indexer preparing entries for an open-ended set of future queries. The consequence the indexing literature does not draw is the one that matters here: a design consequence the indexing literature does not draw: if the writer cannot know the question, the unit of storage should change.

Compare the writer's task to a genuinely hard prediction problem, like forecasting demand for a product. Demand forecasting is hard but it is well posed: there is a real distribution, it is stationary enough to sample, and yesterday's data informs tomorrow's estimate. A wrong forecast is checkable, and a useful one improves against the check. Note writing has none of these. The next session's queries depend on what the user decides to work on next, which depends on things outside the system entirely, and there is no recurring unit against which the writer's performance can be measured.

This places the task close to what Frank Knight called uncertainty in the 1921 sense, but not identical to it. Knightian uncertainty names a situation where the decision maker has no probability distribution; the writer's predicament has that, but it adds a second defect that the term does not capture. The writer has no feedback signal either. Not "I cannot estimate" but "no estimate I produce will be checked." That second defect is what makes the task ill-posed, not merely uncertain. A Knightian decision can still be evaluated by its consequences; a write-time guess about an unknown future query cannot, because the relevant consequence is the future query that does not yet exist.

So the writer is not doing a hard estimation task badly. It is doing an ill-posed task, and the outputs look like bad estimates because nothing else is available.

Part 2: Why The Standard Advice Fails

02. The Two-Pole Failure

Watch what happens when the standard instructions to a memory writer are unpacked, because each one smuggles in the missing information rather than supplying it.

Write down what will be useful later requires knowing what later looks like. Write down the important parts requires an importance function, which is defined relative to a purpose that has not been set. Write it so you could pick up where you left off assumes the next session resumes this work, which is frequently false: it may be a different task in the same repository, or the same task after the code changed underneath it. None of these is wrong exactly. Each is underdetermined, and the writer resolves the underdetermination by guessing, because guessing is the only thing available.

The observable consequence is a specific and recognizable failure pattern. Agent notes tend toward one of two poles.

Insight: Both poles are rational responses to an unknown target. A generic note hedges by being applicable to many possible futures. An over-specific note hedges by recording everything in reach and letting the future sort it out. Neither is laziness. They are the two available strategies when you must write for a reader you cannot see, and they fail in opposite directions.

The generic pole writes things that are true but not discriminating. The auth module handles authentication. There is a test suite. Future relevance search will not be able to tell those notes apart from each other or from the noise of the codebase itself, and they will land in retrieval results at roughly the rate of the words they contain. They are cheap to write and cheap to ignore.

The over-specific pole writes the exact state of a moment that will never recur. Line 340 currently returns None here. Last build failed at 14:02 with a TypeError in user_service.py. These are accurate as photographs, but the value of a photograph decays with the distance from the moment it captured, and the writer cannot know how far away the moment is from the moment that will want it.

That both poles are visible in published work is the part worth checking. The LLM evaluation literature now routinely reports a failure pattern it names over-general and over-specific output as a single joint phenomenon: the same model, on the same prompt, can produce a generation that is true but too generic to act on, or specific to a moment that does not exist. Whether published evaluation work already treats the two failure modes as one calibration problem is a question worth settling; the pattern here is an observation from practice rather than a cited result. What no treatment does is locate the source of the bimodality in the write-time problem rather than in the model.

That claim is an observation from practice, not a result, and it is a prediction: if the framing here is right, an existing store's notes, classified as generic, over-specific, or well-scoped, will show a bimodal distribution with a thin middle. Section 06 turns the prediction into a test.

03. The Missing Signal

There is a practical consequence that is worse than the philosophical one, and it is what keeps the failure pattern self-perpetuating.

Because a note's value is realized at retrieval, the feedback signal for writing arrives late, arrives rarely, and arrives attached to the wrong event. A note written on Monday and usefully retrieved on Friday produces a success credited to Friday's retrieval, not to Monday's write. A note written on Monday and never retrieved produces nothing: no signal, no error, no record that a bet was placed and lost. There is no equivalent of the empty retrieval basket: the basket is invisible because there is no basket.

This is what makes the write path nearly untunable, and it explains a structural pattern visible in nearly every long-lived agent memory store. Retrieval can be tuned because retrieval failures are visible and attributable. A wrong note that surfaces in response to the wrong query is named by the query, ranked against the right one, and the gap between them is measurable. Writing cannot be tuned the same way because writing failures are silent and their cost lands somewhere else, later, attributed to something else. A session that should have remembered a fact and re-derived it instead looks like a slow session, not a memory failure. A session that should not have remembered a fact and acted on it anyway looks like a model hallucination, not a memory over-supply.

The observable consequence is the accretion of write-side heuristics that never get removed. A rule about what to store, once added, is nearly impossible to evaluate, because evaluating it would require knowing what the counterfactual store would have retrieved. The rules accrete because nothing can prove them wrong. A system that began with three rules of what to write will have thirty within a year, and the more rules there are, the less each one can be checked against actual retrieval outcomes, because retrieval outcomes never reveal which rule was responsible for the note that was or was not written.

Warning: The write path has nearly no learning signal. Retrieval fails noisily, with a query, a ranking, and a measurable gap. Writing fails silently, with a note that is never matched to a future query that would have wanted it. Any system whose quality is measured only on the retrieval side is being measured on its better-instrumented half, and improvements to the worse-instrumented half will look like improvements even when they are not.

The diagnosis has a design consequence, and it is not give up. It is that the unit of memory is wrong. If the writer cannot know the question, it should not be storing an answer.

Part 3: The Design Move

04. Store a Condition, Not an Answer

The alternative is to store the condition under which the note becomes relevant, rather than a guess about which future query will match it.

Not this is important but this matters when someone touches this file. Not remember the auth flow but surface this when the auth module is being edited. The note's content is preserved; the unit of storage now includes a triggering condition attached to the content rather than a guess about future relevance attached to the writer's confidence.

This is a real change in what is being predicted, and it is worth being careful about why it is easier, rather than assuming the change is cosmetic.

What changes, stated as a prediction. Let the writer's task be to predict a function. The two candidates differ in the shape of their domain:
relevance:   f:  Queries   &rarr;  {match, no match}
condition:   g:  Circumstances  &rarr;  {fire, no fire}
The first domain is open-ended: every session can ask anything. The second is enumerable: the set of files, operations, and lifecycle stages in a workspace is known and small. Predicting which words the reader will use, at what level of abstraction, in pursuit of which purpose is the original ill-posed task. Predicting whether this artifact will be touched in some future work is a question with a real distribution, drawn from the workspace's actual activity. The writer is still predicting. It is predicting over a space it can actually observe the shape of.

The argument is not that the condition is guaranteed to fire. The argument is that the writer is now betting on a smaller, more stable, more checkable space, and the bet is one the writer can be wrong about in observable ways.

There is a second and less obvious benefit. A triggering condition is falsifiable at write time in a way a relevance guess is not. This matters when someone edits the auth module can be checked immediately for coherence: does that file exist, is it the right file, is the condition ever going to fire. This will be useful later cannot be checked at all. Moving from an unfalsifiable claim to a falsifiable one is the actual improvement here, independent of whether retrieval works better downstream.

This connects directly to the falsifiability of stored directives. A Directive Is Not a Memory argued that a stored rule's only falsifiable core is the present condition that was thrown away at write time, and that memory systems which store prose instead of predicates cannot test the rules they carry. The same shape applies one level up: a stored note's only falsifiable part is the triggering condition, and memory systems which store importance guesses instead of conditions cannot evaluate the notes they carry. The mechanism is the same. The part worth keeping is the part that can be checked today, and the part typically thrown away is the part that could have been.

The proposal is also not new in isolation. Event-condition-action rules in active database systems, publish-subscribe routing in message brokers, and prospective memory in cognitive psychology are all prior art for storing do this when that happens as a first-class object rather than as an afterthought. The novelty being claimed here is the application to agent memory, the argument that the unit should be the condition rather than the answer, and the observation that this changes the falsifiability status of what is stored. The active-database tradition formalized the trigger side decades ago; the agent memory tradition has not yet adopted it as a primary unit. The literature on production rule systems solved the related problem of conditioned actions, and the conditioning construct is decades old. What is new is the application, not the underlying idea.

To see why the unit change is not just a relabeling, run the same idea through a lens argued elsewhere. Search Is Memory's Write Path argued that agent search and agent memory are two phases of one system, the first search being the write and every repeat the read, and that the write path is shaped by what the retrieval mechanism will later surface. That is not contradicted here. The narrower claim is that even with perfect awareness of the retrieval mechanism, the writer still cannot know the specific future query, and so the unit of storage should be a triggering condition whose firing is observable, rather than a guess about future relevance whose correctness is not. Search being the write path is consistent with conditions being the unit; the conditions become what the write path produces, and the retrieval mechanism selects among them on the circumstances that have actually fired.

05. What It Does Not Solve

The honest limits, because the argument is weaker than it may sound and the right place to be weak is up front.

Condition-based memory has its own failure mode, and it is not a milder one. A condition that never fires stores a note that is never seen, and unlike a relevance miss, nobody is even querying for it. The note sits in the store, the trigger it carries never resolves true, and the system cannot detect the gap because the gap is the absence of an event, not the presence of a wrong one. A wrong condition is a permanent silent loss rather than a ranking error, and it is at least defensible to argue that the trade is a measurable failure for an unmeasurable one. The bet is that the unmeasurable failure has lower expected cost because the space of conditions is smaller and more stable than the space of queries, but the bet is not a theorem.

The proposal also does not cover the genuine case where a reader asks an open question with no triggering circumstance at all. Someone wanting to know what was learned about this subsystem last month is doing exactly the query-shaped retrieval that cannot be predicted for. Conditions do not help there. Relevance search remains the right tool, and the right design treats conditions and relevance as two subsystems that carry different loads. The proposal is that conditions should carry the load they can carry, not that they replace retrieval. Replacing one with the other is the same failure in a different shape.

And the move moves rather than removes the burden of foresight. Choosing a condition is still a prediction. The claim is only that it is a prediction over a smaller, more stable, more checkable space, and that this is a genuine improvement rather than a relabeling. That claim could be wrong. Section 06 says how you would find out.

06. The Three Predictions

The argument is structural rather than measured. Three predictions would distinguish it from the alternatives, and they are stated so a future test can confirm or refute each one independently.

Prediction 1. Notes written with a triggering condition should show a higher retrieval rate than notes written without one in the same store, controlling for content. If condition-carrying notes are retrieved no more often, the central proposal fails. This is a one-line count, run on any working store that already records retrieval events.

Prediction 2. Agreement between independent writers should be higher on conditions than on importance. Give several agents the same session and ask each to say what is important, then ask each to say when it will matter. If the second task produces more agreement, that is evidence the prediction space really is more constrained. If agreement is equally poor, Section 04's core argument is unsupported.

Prediction 3. The two-pole pattern in Section 02 should be measurable. Classify an existing store's notes as generic, over-specific, or well-scoped. The argument predicts the middle category is small and the distribution is bimodal. A healthy unimodal distribution of well-scoped notes would falsify the framing directly.

Which prediction is the cheap one. Prediction 2 is the discriminating test and the cheapest to run. It tests the actual mechanism rather than the downstream effect, and it does not require a working implementation to evaluate. The setup is roughly: pick a session whose facts are unambiguous, give the same transcript to N models or N fresh sessions of the same model, and ask the two questions. Inter-rater agreement on when will this matter is the number that decides whether the framing survives.

No measurement is reported here. The point of the predictions is that the argument can be settled, not that it has been.

Close: The Line to Keep

The writer is asked to be precise about a question that has not been asked. That is not a hard problem being solved badly. It is an ill-posed problem being answered anyway, and the fix is not a better guess about the question but a note that does not need to guess it.

The unit of memory should be a triggering condition, not an answer. The condition is the part that can be checked today, the part that survives the writer's missing information about the future, and the part that turns the write path from silent into falsifiable.

Sources

Information retrieval: anticipatory indexing

Fairthorne, R. A. Content Analysis, Indexing, and Matching. Foundational essay in Fairthorne's work on the aboutness problem, treating indexing as the construction of a document representation against an open set of future queries. Source of the anticipatory-indexing framing used in Section 01. Publicly discussed in the Journal of Documentation tradition, 1960s.
Brittain, J. M. Information and Its Retrieval. Modern treatment of anticipatory indexing and the indexer-for-unknown-uses problem.

Belief revision: the recovery postulate

Alchourrón, C. E., Gärdenfors, P., Makinson, D. On the Logic of Theory Change: Partial Meet Contraction and Revision Functions. Journal of Symbolic Logic 50(2), 1985, pp. 510-530. The contraction operator and the partial-meet construction, and the formal definition of contraction used in Section 05.
Gärdenfors, P. Knowledge in Flux: Modeling the Dynamics of Epistemic States. MIT Press, 1988. Introduces the recovery postulate for contraction as an additional constraint. Section 05 weakens it, on the grounds that the agent memory case lacks the feedback signal it assumes.
Stanford Encyclopedia of Philosophy, Belief Revision (Hansson, major revisions through 2023). Standard reference frame for the AGM postulates, including the formulation of recovery and the subsequent critiques that proposed weaker constraints in its place. https://plato.stanford.edu/entries/belief-revision/

Decision theory: uncertainty and ill-posedness

Knight, F. H. Risk, Uncertainty and Profit. Boston: Houghton Mifflin, 1921. The distinction between measurable risk and unmeasurable uncertainty. Section 01 uses the term in its 1921 sense, and adds a second-order defect: the missing feedback signal.

LLM evaluation: over-general and over-specific output

Production rule systems: event-condition-action

Widom, J., Finkelstein, A. Set-Oriented Production Rules in Relational Database Systems. Proceedings of the 1990 ACM SIGMOD International Conference on Management of Data. The HiPAC lineage of active databases that produced the ECA formalism, the prior art for condition-first storage.
Chakravarthy, S. et al. An ECA Rule-based Framework for Active Information Systems. Active databases literature of the 1990s, where the conditioning construct was already solved.

Companion posts in this series

Search Is Memory's Write Path. The argument about retrieval shaping what gets stored. Section 04 cites the distinction: retrieval shapes the write path, and the open question is what unit that shaped write path should produce.
A Directive Is Not a Memory. The argument about the falsifiability of stored rules. Section 04 cites the parallel: the same shape one level up, applied to memory notes rather than directives.

Recall Is a Write Operation: Why Usage-Weighted Memory Ranking Cannot Discover It Was Wrong

Swapnanil Saha — Tue, 08 Sep 2026 14:05:39 +0000

Recall Is a Write Operation: Why Usage-Weighted Memory Ranking Cannot Discover It Was Wrong

Ask an agent memory store what it knows about a topic and it hands back a ranked list. A query goes in, a small set of notes comes out, nothing changes. That is how retrieval is described, how it is drawn in architecture diagrams, and how almost everyone reasons about it.

It is false in every memory system that has a notion of usage. Recall touches a last-accessed timestamp. It feeds a decay or reinforcement score. It updates a counter that a future ranking will read. The read is a write, and the thing it writes into is the input of the next read.

That closes a loop. Once a loop is closed, the interesting question is no longer "does this rank well." It is "what does this converge to."

A memory system that reinforces what it retrieves cannot discover that it was wrong, because the mechanism that would demote a bad note is the same mechanism the bad note keeps winning. This is not a tuning problem. It is a structural property of any ranking that takes its own past output as an input, and it has to be designed against rather than tuned away.

Part 1: The Write in the Read

01. Why Usage Writes in the First Place

The usage write is not an accident or an implementation detail somebody forgot to remove. It is load-bearing, and it is there because the alternatives are worse.

A memory store needs to decide what to keep. Storage is finite, retrieval gets noisier as the corpus grows, and a store that never forgets becomes a store that cannot find anything. So it needs a signal for what matters.

The available signals are poor. Age is a bad proxy for validity, and a note is not less true for being old. Explicit importance labels are unreliable, because whoever writes the note is guessing about a future they cannot see. Content-based scoring cannot tell an important note from a verbose one.

Usage is genuinely the best available signal. A note that keeps getting retrieved is, on the face of it, useful. Reinforcing it is the obvious move, and it is roughly what human memory does. Reinforced retrieval strengthens the retrieved item and, as a side effect, the related unpractised items lose retention; Anderson, Bjork, and Bjork named this in 1994 and called it retrieval-induced forgetting. The mechanism is the same: retrieval writes to memory. This is why practically every agent memory system does it, and it is why the argument here is not that the write should be removed.

The argument is that the write closes a loop, that closed loops have dynamics, and that almost nobody has looked at what these dynamics are.

Insight. Usage is the strongest available signal for what matters in a store, and writing to it is the right default. The write itself is not the target. The target is treating it as a passive side effect, because a write that feeds back into its own input is a different object from one that does not.

Part 2: The Shape

02. The Loop, Stated Plainly

Write it out as a cycle. A note is retrieved. Retrieval updates its usage. Usage feeds ranking. Ranking determines what is retrieved next.

Every arrow is uncontroversial on its own. Together they form a positive feedback loop with no negative term. What is retrieved becomes more retrievable, which causes it to be retrieved, which makes it more retrievable.

The same shape shows up in classical feedback systems, in preferential attachment networks, and in the popularity-bias literature on recommender systems. The general result, in all three, is concentration: a small number of items capture a disproportionate share of attention, and the concentration is shaped substantially by early, arbitrary events rather than by intrinsic quality.

Applied to memory: whichever note happened to surface first for a class of query acquires an advantage that is self-sustaining, regardless of whether it was the best answer. Not because the ranking is bad, but because the ranking is recursive.

Analogy. Think of a citation index. A paper that gets cited early is more visible, so it gets cited more, so it stays visible. New papers in the same field have to overcome an index that already points at the established one. The field's actual quality distribution gets distorted by a visibility process that has nothing to do with quality, and the distortion compounds. Agent memory with usage-weighted ranking is the same process, run inside a single store, with a single consumer.

The competitor is not merely behind. It is behind in a way that compounds, and it has no mechanism to catch up, because catching up requires being retrieved and being retrieved requires having caught up.

03. The Concentration That Comes from Preferential Attachment

The closest formal analogue is preferential attachment, the mechanism Barabasi and Albert proposed in 1999 to explain the scale-free degree distributions observed in real networks. The mechanism is plain: at each step a new node arrives and connects to an existing node with probability proportional to that node's current degree. The more connected a node is, the more likely it is to receive the next link.

Three properties of the model matter here, and each is stated explicitly in the source.

First, the model produces scale-free degree distributions of the form P(k) ~ k^-3, in the limit. The model is a positive feedback system in which initially random variations are automatically reinforced, and the Wikipedia summary of the model, drawing on the original paper, names this "the rich get richer."

Second, the reinforcement is unbounded without intervention. Concentration that begins as small initial advantage ends as permanent structure, because every step multiplies the imbalance.

Third, and the one that is easy to miss: both growth and preferential attachment are required. The model with growth alone produces a geometric distribution. The model with preferential attachment alone produces something that decays toward a Gaussian. The scale-free property emerges from the combination. Memory stores with usage-weighted ranking have both: notes are added over time, and retrieval probability is proportional to past retrieval count. The shape that produces concentration in Barabasi-Albert networks is the same shape that produces concentration in memory stores.

Warning. The argument is a structural analogy, not a proof. A memory store's retrieval counts are not a power law, and the exact statistics of concentration depend on details the model does not contain. The claim is the shape of the dynamic, not its quantitative predictions, and Part 6 takes the claim apart into falsifiable predictions so the analogy can be tested rather than waved at.

The connection runs through the same math because the same mechanism is at work: a quantity that grows in proportion to its current value, with no negative term, in a system that keeps adding new elements. Memory stores have all three. So do citation indices, social networks, and the long tail of any ranking system whose history feeds forward into the next ranking.

Part 3: Where It Hurts

04. When the Winning Note Is Wrong

Concentration alone is not obviously bad. If the popular note is the right note, a system that surfaces it faster over time is a system that is improving. The failure is what happens when the popular note is wrong.

For a wrong note to be corrected, something has to expose it. Typically that means retrieving a competing note, or retrieving the wrong note in a context where its wrongness becomes evident. Both require the wrong note's competitor to be retrievable.

But the wrong note has been winning, so it has the usage advantage, so it keeps winning. The competitor has been losing, so it has decayed, so it keeps losing. The system's error-correction path runs through exactly the mechanism the error has captured.

This is the sharp point, and it is worth stating without hedging. The wrongness of the note is not what determines whether it survives. Its retrieval history is. Those two are correlated at the start and decouple immediately, because after the first retrieval the note's rank is being driven by its own past rank rather than by any property of the world.

Insight. The mechanism is a winner-take-all filter applied to correctness. A note that is right enough to win once wins forever. A note that loses once loses forever, even if it was right all along and the winning note was wrong. The filter has no correction signal, because the only signal it reads is its own output.

A system with this shape does not fail loudly. It converges, and it looks like it is getting better while it does, because concentration is easy to mistake for confidence. The measured behavior of a memory store settling into a small set of reliably-surfaced notes is the same measured behavior whether that set is right or wrong.

Reinstatement Is the Missing Primitive in Agent Memory makes the same point from a different angle: the store accumulates beliefs that nobody was willing to touch, and the symptom is invisible from inside the system. What is added here is the mechanism that makes the accumulation inevitable rather than incidental.

05. Decay and Exploration Are Not the Fix

The usual response is that decay handles this. Old notes lose weight, so a stale wrong note eventually falls away and the competitor gets a chance.

Decay does not break the loop. It changes its time constant.

Decay penalizes notes that are not retrieved. The wrong note is being retrieved, so it is not decaying. The competitor is not being retrieved, so it is. Decay is not a counterweight to the feedback loop; it is the same loop with the sign flipped, acting on the loser. It accelerates the concentration it was introduced to prevent.

Decay only helps if the wrong note stops being retrieved, and the whole problem is that it does not.

There is a second, subtler version of this mistake. Adding a small random exploration term so that low-ranked notes occasionally surface is a real mitigation and it is better than nothing. Epsilon-greedy and Upper Confidence Bound are the two named treatments in the bandit literature, both with optimal-order guarantees under stationary reward distributions. But note what they concede. If the fix for a ranking system is to sometimes ignore its own ranking, the ranking has stopped being a statement about relevance and become a statement about history. Exploration is a patch on a corrupted signal, not a repair of it.

Warning. Exploration cannot repair a signal that is downstream of itself. The bandit framing is sharp precisely because it concedes the problem: any policy that needs an explicit exploration term to perform well is a policy whose greedy arm is not the answer. A memory store whose primary signal is its own past output is in exactly that position, and no choice of epsilon closes the gap between "occasionally correct by luck" and "correct by construction."

Both fixes (decay and exploration) treat the symptom. The disease is that ranking reads its own history, and the only cure is a ranking that does not.

Part 4: The Measurement Trap

06. The Error Is Self-Concealing

Here is what makes this genuinely hard rather than merely subtle.

Suppose you want to know whether your memory store has this pathology. The obvious check is whether retrieved notes are useful. But usefulness is judged by the agent, using the retrieved notes, in a context shaped by the retrieved notes. The evaluation is downstream of the thing being evaluated.

Worse, the notes that would prove the store is wrong are precisely the ones the store has stopped returning. Their absence is invisible, because a retrieval that does not happen produces no output to inspect. You cannot see a miss.

So the pathology is self-concealing in two independent ways at once. The evidence against it is suppressed by the mechanism, and the metrics for detecting it are computed from the mechanism's own output. A store deep in this failure mode and a store working perfectly look identical from the inside.

Insight. This is the part that defeats the obvious fix. You cannot compute a metric from retrieval logs that detects the loop, because any such metric is inside the loop. The instrument and the thing being measured share their input, and the instrument will report success on the only data it can see. The only honest measurement is one whose input is causally outside the loop: a controlled comparison across stores with different usage histories, run on a held-out evaluation set that the store has never seen.

The standard recommendation in the popularity-bias literature, where the same self-concealment problem has been studied for two decades, is to evaluate on a randomized exposure set: serve a uniform-random sample of items to a fraction of users and measure whether the items the model never recommends have similar quality to the items it does. The analog for agent memory is a held-out query set where retrieval is forced to be uniform across the candidate pool, and the evaluation asks whether the unpicked notes would have been better. The recommendation is unfashionable because it lowers the metric on every store that has been quietly winning, and a system that does not have the problem has nothing to fear from it.

Part 5: Breaking the Loop

07. Four Designs That Open It

If the problem is that ranking reads its own history, the fix is to make some part of the ranking causally independent of retrieval history. Four shapes follow, offered as design directions rather than as finished solutions: none has been measured to validate the structural argument.

Separate the signals. Usage can inform eviction, which is about what to keep, without informing ranking, which is about what to return. These are different questions and it is not obvious they should share a score. Coupling them is what closes the loop; decoupling them opens it, at the cost of needing a second signal for ranking. The published evaluation of MemX is consistent with this split: it tracks a retrieval count and a last-retrieved timestamp, both reported as inputs to a four-factor re-ranking that includes similarity, recency, frequency, and importance, with the frequency weight small (0.05) and the recency weight modest (0.25). The architecture does not feed the user's retrieval outcome back into the ranker in a self-referential way; the same paper reports that recency and time-decay are essentially uninformative as a primary signal, which is the natural outcome of treating usage as an eviction hint rather than as the rank.

Make corrections structural rather than competitive. If a note that corrects another is linked to it rather than merely ranked against it, the correction does not have to win a popularity contest to be seen. It arrives with the thing it corrects. Relevance Is the Wrong Sort Order for Agent Memory argues for this on different grounds, and a linked successor independently dissolves the loop: it cannot be starved by it, because it is not competing in the ranking at all. Reinstatement Is the Missing Primitive in Agent Memory makes the parallel point for the lifecycle: a note that is revoked and then proven correct can be reinstated precisely because reinstatement is an append, not a popularity contest.

Audit from outside the loop. Periodically evaluate notes against the world rather than against usage. Check whether a note's claim still holds. This is expensive and only works for checkable claims, but it is the only signal in this list that is genuinely exogenous. A store that audits a fraction of its notes against ground truth on a schedule can detect the loop from the outside, by measuring how its in-store accuracy compares to a non-usage-based benchmark. The result is a measurement, and the measurement is the only thing that can tell you whether the structural fixes above are actually working.

Instrument the concentration directly. Even without a fix, measuring how concentrated retrieval is over time, and how much of that concentration is explained by early ordering rather than by content, would tell you whether the dynamic is live in a given store. This is the cheapest thing on the list and, as far as I can tell, nobody publishes it. The measurement is straightforward: take the per-query retrieval count for each note, sort it, and report the share of retrievals captured by the top 1 percent of notes. A store in which that share is increasing session-over-session is in the loop. A store in which the share is stable or tracking the query distribution is not.

Warning. None of the four above has been measured here. The argument is structural. A field that ships memory systems with usage-weighted ranking as the default should publish a measurement that confirms the structural argument or that overturns it, and the absence of such a measurement is itself part of the problem the argument names.

Part 6: Falsification

08. Three Predictions, in Priority Order

An argument that cannot be tested is not an argument. The structural claim here produces three concrete predictions, and they are testable in a way the loop's self-concealment does not defeat: the predictions are about comparative behavior across stores, not about behavior on a single store's logs.

Prediction 1. In a memory store with usage-weighted ranking, the distribution of retrievals across notes should become more concentrated over time, and more concentrated than the distribution of queries would justify. The second clause is the real test: concentration that tracks query concentration is just the store working. A simple measurement: for each store, compute the Gini coefficient of the retrieval distribution and the Gini coefficient of the query distribution, and report their difference across sessions. A growing difference is the loop running.

Prediction 2. Injecting the same note into two otherwise-identical stores at different initial positions should produce durably different long-run retrieval rates. If initial position washes out, the loop is weak and this argument is overstated. The experiment is cheap: seed two stores with the same note corpus, perturb the initial rank of one note in one of them, run a fixed workload of queries, and measure retrieval rates at the end. The two notes with the same content and different initial positions should have different long-run retrieval rates, and the size of the difference is the size of the loop.

Prediction 3. A known-wrong note that has accumulated retrieval history should be harder to displace than a known-wrong note with none, holding content constant. This isolates history from quality, and it is the cleanest available falsification. If displacement difficulty does not depend on history, the mechanism described here is not operating.

Prediction 2 is the one worth running first. It is cheap, it needs no ground truth about correctness, and it directly measures whether history dominates content. The interactive demo above is a small version of the same comparison: a one-query, ten-step simulation showing how the same note, placed at different initial ranks, accumulates different retrieval counts over a fixed workload.

Close: The Line to Keep

A memory system that reinforces what it retrieves is not learning what matters. It is learning what it retrieved. Those diverge from the first read, and the system has no way to notice, because the evidence that would tell it is the exact thing it has stopped returning.

This is not the same argument as Relevance Is the Wrong Sort Order for Agent Memory. That essay argued that ranking notes independently is wrong because notes carry causal relationships to each other, and a similarity sort drops the very note that carries the truth of a correction. The claim here is separate and compatible: ranking is not even a stable function, because it consumes its own output. The two are about different failure modes. The earlier one is about what the ranker chooses between. This one is about how what the ranker chooses feeds back into the next ranker's choice. Closing retrieval on lifecycle edges is the fix the earlier essay argues for. Keeping the edge that records what has been retrieved out of the ranker in the first place is the fix argued here.

The store is being asked two questions at once. The first is what to remember, and usage is a defensible signal. The second is what to recall, and usage is not a defensible signal for the thing it is being asked to recall about. A store that conflates the two is, quietly, deciding tomorrow from yesterday and calling the decision relevance.

Sources

Barabasi, A.-L., Albert, R. Emergence of Scaling in Random Networks. Science 286(5439), 1999. The original preferential-attachment model; P(k) ~ k^-3 degree distribution; explicit statement that "initially random variations are automatically reinforced." Wikipedia summary reproduces the model's stated limitations: growth and preferential attachment are both required for the scale-free property. Part 2.
Anderson, M. C., Bjork, R. A., Bjork, E. L. Remembering can cause forgetting: Retrieval dynamics in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition 20(5), 1994. The first demonstration of retrieval-induced forgetting; the original Rp+ / Rp- / NRp paradigm. Wikipedia summary reproduces the three-category design. Part 1.
Roediger, H. L., Karpicke, J. D. Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Psychological Science 17(3), 2006. The standard reference for the testing effect that motivates reinforcement of retrieved items. Wikipedia summary reproduces the spaced-repetition amplifications. Part 1, for the connection to human memory and the limits of the analogy.
Sutton, R. S., Barto, A. G. Reinforcement Learning: An Introduction. MIT Press, second edition 2018. The standard reference for epsilon-greedy exploration in multi-armed bandits; Chapter 2 covers the exploration-exploitation trade-off. Wikipedia summary of the bandit problem and the regret bounds. Part 3.
Auer, P., Cesa-Bianchi, N., Fischer, P. Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning 47, 2002. The original UCB1 analysis with the O(sqrt(KT ln T)) regret bound; the optimistic-in-the-face-of-uncertainty framing. Part 3.
Du, P. Memory for Autonomous LLM Agents: Mechanisms, Evaluation, and Emerging Frontiers. arXiv:2603.07670, 2026. The survey that names "self-reinforcing error" as the central risk of reflective memory; the API-always-errors example. Part 3.
Chhikara, P., Khant, D., Aryan, S., Singh, T., Yadav, D. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413, April 2025. Production memory architecture; ADD/UPDATE/DELETE/NOOP update phase. Cited in Part 5 as a production system whose architectural choices are consistent with the decoupling argument, without making the claim that the system has been measured against it.
Recommender system. Wikipedia article covering popularity bias, the self-contamination of training data, and the reproducibility gap between offline and online evaluation. Part 3 and Part 4.
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., Yu, D. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR 2025, arXiv:2410.10813. Five abilities including knowledge updates; 500 questions; the reported 30 percent accuracy drop across long interactions. Cited in Part 4 as the empirical shape of the recency problem, not the mechanism.
Relevance Is the Wrong Sort Order for Agent Memory. Argues that the unit of retrieval should be a trajectory, not a ranked set, and that the lifecycle edges on notes are not being used at retrieval time.
Reinstatement Is the Missing Primitive in Agent Memory. The missing reverse edge of revocation. A memory store that cannot reverse a wrong correction accumulates bad beliefs, and the structural reason is the feedback loop analysed in Part 2.

A Guess Becomes a Fact on the Next Read

Swapnanil Saha — Mon, 07 Sep 2026 14:33:55 +0000

Everything an agent knows arrives as text in a context window. A user instruction is text. A tool result is text. A file the agent read is text. The agent's own speculation, written out mid-reasoning, is text. By the time any of it reaches the next turn, it is all the same type.

That flattening is invisible while the conversation is live, because position and recency stand in for provenance. The user's instruction is the thing at the top. The guess is the thing the model just wrote. Nobody needs a label because the structure carries it.

Memory removes the structure. A note is retrieved into a fresh context with no conversation around it, no position, no ordering, no author. What was a hedged inference at write time arrives at read time as a free-standing assertion, indistinguishable from something the user said outright.

The dominant failure mode of agent memory is not forgetting. It is laundering. A lost note costs a re-derivation, which is bounded and usually cheap. A note that silently changed epistemic status costs every decision that trusts it, and it leaves no trace of having changed. Systems built to maximize recall are optimizing the wrong quantity, because the expensive error is not the note that failed to come back. It is the note that came back wearing a confidence it never had.

Part 1: The Flattening

01 · Four Things That Are the Same Type and Should Not Be

Consider four statements that could all end up in an agent's memory store, written during the same session.

The user said the deploy script must never run on a Friday.
The test suite emitted FAILED tests/test_auth.py::test_expiry.
The agent read config.yaml and it contained timeout: 30.
The agent reasoned that the timeout is probably too low for the CI runner, since the failing test waits on a network call.

The first is a normative instruction from a human. It cannot be false; it can only be obeyed or violated. The second is an observation, true of one run at one commit. The third is a fact about a file at a moment. The fourth is a hypothesis, and a decent one, but it is the agent's own guess and it was never checked.

Write all four as notes and they become the same kind of object: a string with a timestamp. Retrieve them together later and the fourth reads exactly like the first three. Worse, the fourth is often the most useful-sounding, because it is the only one phrased as an explanation. Explanations are what a retrieval query tends to be reaching for.

The problem is not that the fourth note is wrong. It might be right. The problem is that nothing in the system records that it is the kind of thing that could be wrong, while the other three are not, and they are not wrong in three different ways.

The claim, more precisely. The four statements have different truth conditions, different ways of being wrong, and different correct responses on being contradicted. A human assertion that conflicts with an observation usually means the instruction was normative, not descriptive, and the right move is to ask. A tool observation that conflicts with a later observation means the world changed, and the right move is to re-run. A model conclusion that conflicts with anything is wrong, and the right move is to drop it. Collapse the three into one type and there is exactly one repair strategy, which is whatever the model guesses. That guess is itself unlabeled, and now the corruption is recursive.

Provenance is not a new idea in computing. The W3C PROV-DM standard from 2013 has spent more than a decade defining how to record who or what produced a piece of data, why, and from which inputs. Scientific workflow systems like Taverna and Kepler ship lineage-aware stores that track derivation chains across tool calls. The narrower claim is in agent memory, where the failure that dominates is not the one provenance was originally designed to address, which is trust assessment. It is type destruction, which happens earlier and further upstream, at the boundary between context and store.

Part 2: Why It Hides

02 · Containers Carry Provenance, Content Does Not

Inside a live context the four statements are separable, and the model separates them well. It knows the user spoke because the user's turn is a user turn. It knows the test output is tool output because it sits in a tool result block. It knows its own guess is a guess because it wrote the word "probably" three lines ago and can still see it.

None of that survives extraction. Every one of those signals is carried by the container, not by the content, and memory stores content.

When a memory tool writes a string to a file, the only thing in the file is the string. The fact that the string was the user's instruction rather than the model's guess is in the role it played in the conversation, and the role is gone. The fact that the string was a tool observation rather than a hypothesis is in the block it sat inside, and the block is gone. The fact that the string was a hedged guess is in a word that any downstream system is free to keep, drop, paraphrase, or replace.

Analogy. Imagine a courtroom record reduced to a list of every sentence anyone said during the trial, with no speaker labels, no timestamps, and no markers for which sentences were testimony, which were objections, which were the judge's rulings. A future reader, given only that list, has to guess whether any particular sentence was a fact a witness swore to or a lawyer's speculation. The guess is the only move available, and the guess is wrong more often than not, because the speakers were not equivalent. The transcript is fine. The record has lost the structure that made it usable.

This is why the failure is so easy to miss in testing. Evaluate memory inside a single session and provenance appears to work, because the context still holds the structure that encodes it. The corruption only shows up across the boundary that memory exists to cross, which is exactly where it is least likely to be measured. The boundary is also the only place the system gets to demonstrate it is doing something useful, so it is the place evaluation spends the least time.

There is a second reason it hides. When a laundered note is later acted on, the action usually succeeds or fails for ordinary reasons, and the failure gets attributed to the action. Nobody traces back to ask whether the belief that motivated it was ever load-bearing. The causal chain from a bad epistemic label to a bad outcome is long, and every link in it looks like a normal engineering problem. The end failure is a missed deadline, a wrong commit, a botched deploy, and the postmortem names the tool, not the memory.

Part 3: The Asymmetry

03 · Forgetting and Laundering Are Not Symmetric

Treat memory as a channel with two error modes.

The first is loss. A note that should have come back did not. The cost is a re-derivation: the agent reads the file again, runs the test again, asks the user again. Annoying, measurable, and bounded by the cost of the original derivation. Crucially, the agent knows it does not know. It is in a state where it will go and find out.

The second is laundering. A note comes back, and it carries more authority than it earned. The cost is not bounded by anything, because the agent is no longer in a state where it will go and find out. It has an answer. Every downstream decision inherits the error, and the error is silent by construction: there is no exception, no miss, no empty result, nothing that looks like a failure at the moment it occurs.

The two error modes, side by side. Loss is a missing note. The agent has no information, knows it, and pays a bounded price to obtain the information. Laundering is a mislabelled note. The agent has a confident answer to something it should have hedged, builds on it, and never trips a signal that says the answer was wrong. Loss is loud and finite. Laundering is silent and unbounded, because the only cost is whatever the downstream decisions multiply it by.

These two errors are not symmetric and should not be traded off against each other as if they were. Yet almost every design decision in a memory system trades exactly that way. Store more, and you launder more. Retrieve more aggressively, and you launder more. Summarize to save tokens, and you launder more, because summarization strips hedges first, and the next section is about why.

An agent that remembers nothing is merely slow. An agent that remembers its own guesses as facts is confidently wrong, and it will defend the position, because from the inside a laundered note is indistinguishable from a thing it was told.

Part 4: Hedges Do Not Survive

04 · The Compression Tax on Uncertainty

There is a specific, mechanical reason provenance decays even when someone tries to preserve it in the text.

Epistemic status in natural language is carried by small, low-content words. "Probably." "I think." "It looks like." "Assuming the runner is network-bound." These words have almost no semantic weight. They contribute little to an embedding. They are the first casualties of any rewrite that optimizes for brevity or clarity.

So the moment a note passes through a summarizer, a compaction step, or an agent rewriting its own notes to be tidier, the hedges go and the claim stays. The sentence gets shorter and more confident at the same time, and the two are the same edit. Nobody decided to promote the guess. The promotion is a side effect of compression.

Demo 01 · Run a Hedged Note Through a Summarizer. Pick a note with epistemic markers and watch the markers survive or vanish as a compressor rewrites it for brevity. The same note can pass through a hedge-preserving compressor and a hedge-stripping one, and the output is a different epistemic object even though the words are nearly the same. This is the mechanism that makes laundering free, and it does not require an adversary.

The published work on this drift is more recent than the intuition, and more measured than the intuition would predict. Belem, Wu, Yao, Steyvers, Singh, and Smyth, From "May" to "Is": Certainty Distortion in Language Model Rewriting (arXiv:2606.07951, June 2026), report a systematic tendency for LMs to overstate confidence when paraphrasing scientific and medical text. Their evaluation covers single rewrites, repeated paraphrases, and prompt-level mitigations, and the effect compounds over iterations: in their medical subset, one pass raises certainty in 20% of examples, five passes in 40%. The implication for memory stores is that the longer a note lives, and the more rewrite passes it sees, the more it tends to wear a confidence the original never had.

This is why "just write good notes" does not solve it. The instruction assumes the note is stable once written. It is not. It is a string that will be passed through several lossy transformations by systems that have no reason to know which of its words are the important ones.

If provenance is in the prose, provenance is negotiable. It has to be in the schema.

Part 5: The Fix

05 · A Closed Enum, Not a Source String

The fix is not a source string. A free-text source field is prose again, and it will be paraphrased away like everything else.

What is needed is a small closed set of classes, assigned at write time, that the retrieval path is required to render and the model is trained by the interface to read. Small, because a taxonomy nobody can remember is a taxonomy nobody applies correctly. Closed, because an open set collapses back into prose.

The minimum viable distinction is between what a human asserted, what a tool emitted, and what the model concluded. Those three have genuinely different truth conditions, different ways of being wrong, and different correct responses on being contradicted.

Provenance class	Truth condition	How it is wrong	Correct response on conflict
human	Reflects what the user said or endorsed	Misquoted, transcribed, or out of date	Ask. A descriptive conflict may be normative intent.
tool	Reflects an observation at a moment	World changed, tool was nondeterministic, or anchor drifted	Re-run. The world is the source of truth.
model	Reflects the model's own inference	The inference was wrong	Drop. The conclusion is what is wrong.

There is a further refinement worth naming, because it is where this gets genuinely hard. The middle case is a claim a human made that an agent transcribed, perhaps compressed, perhaps subtly shifted in the paraphrase. It is not a model conclusion, because a human is behind it. It is not a verbatim human assertion either, because the exact words are the agent's. Systems that offer only two classes force this case into the wrong one, and it is a common case, arguably the most common one in practice, because almost every "the user said X" note in any real store was typed by the agent that heard it.

Why a free-text source field is not enough. A source="user said in turn 14" column is the same as no source column, because the value is prose, and the retrieval path that does not know what to do with prose is exactly the retrieval path that needs the source. A closed enum, by contrast, is rendered on every recall. The reading model does not have to interpret it. It reads the prefix the way it reads the kind field on a tool call, and that is the entire mechanism. Open taxonomies collapse back to prose, and prose collapses back to nothing.

An earlier piece, Agent Memory Needs a Trust Ladder, is about how much weight to give a stored note. It presupposes the system already knows what kind of note it is holding: human, agent-recorded, or auto-captured. The trust axis and the kind axis are independent, and conflating them is a common error. A trust ladder on a typeless memory is the downstream problem: the model is being asked how much to believe a claim whose type was destroyed before the question was asked. A separate but adjacent piece, Agents Over-Trust Instructions and Under-Trust Corrections, makes a related point about how instructions and corrections are weighted asymmetrically by the reading model. The instruction-versus-correction asymmetry is a second downstream consequence of the same typeless assumption. Both pieces are right about their own axis, and both assume the type is known. The kind axis sits beneath both.

The published literature on this kind of labelling is mostly recent. Wang et al., From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents (arXiv:2606.04990, June 2026), explicitly groups memory writes by source type, recommending that stores carry source, authoring agent, transformation operation, confidence, and update history on every memory. The taxonomy there is richer than the three-class minimum above, and a system implementing the richer version is in better shape. The survey does not separate cleanly the case where the source field exists in the schema but the retrieval path does not actually surface it to the reading model. That is a delivery problem, and it has the same shape as every other delivery problem in agent memory: the artifact is in the store and never reaches the prompt.

Part 6: Why Not at Read Time

06 · Information Destroyed at Write Time

A tempting shortcut: skip the write-time labeling, and have the model assess provenance when it reads a note. It is good at that sort of judgment, in some situations.

It cannot work, and the reason is worth being precise about.

Provenance is not a property of the note's text. It is a property of the note's history. Two notes with byte-identical content can have different provenance, because one was dictated by a user and the other was inferred by a model that happened to phrase it the same way. No amount of reading the string recovers which happened. The information is not in the string; it was destroyed at write time, and read-time analysis has nothing to recover it from.

Demo 02 · Two Notes, Identical Strings, Different Histories. Two notes in the store, byte-identical in their content. One was the user speaking in a real conversation. The other was the model inferring a guess and writing it down with a phrasing it happened to share. A read-time judge sees the same text in both cases. Toggle the framing, then toggle a read-time inference on, and watch how the inference produces a plausible-sounding answer that is the original failure, one level up.

What read-time assessment actually produces is a plausible-sounding guess about provenance, stored or acted on as if it were provenance. Which is the original failure, one level up.

The general shape, named once. When information is destroyed at write time, no read-time cleverness restores it. It can only manufacture a confident substitute. The fix has to be at the boundary where the information still exists. This is not a special case. It is the standard shape of a class of memory bugs, and naming it once means the rest of the family becomes recognizable: write-time compression that loses hedging, write-time rewriting that flattens a model conclusion into a "fact", write-time summarization that drops the actor on a retraction. Each of them is the same bug in a different surface.

That last sentence is the one to keep. Every summarization pass, every rewrite-for-clarity, every compaction step is the same shape: a write-time operation that throws away the part of the string the system cannot yet use, on the bet that the part it kept will be enough. The bet is wrong, but it is wrong silently, which is what makes the bug shape worth recognizing.

Part 7: Predictions

07 · What This Forbids, and the Test That Would Break It

An argument that explains everything and forbids nothing is not worth much. Here is what this one forbids.

Prediction 1. In a memory system without provenance classes, the proportion of stored notes that are model conclusions rather than observations should rise over time, and rise faster in longer sessions. Conclusions are cheap to generate and feel insightful, so an agent asked to record what it learned will preferentially record what it thought. This is measurable by classifying an existing store retrospectively. The slope should be positive and significantly larger than the slope for human-asserted or tool-observed notes.

Prediction 2. Notes that have been through a summarization or compaction step should be measurably more confident in phrasing than their originals, with no corresponding increase in support. Diff the hedge tokens before and after. The published work already gives a baseline: Belem et al. find certainty distortion in up to 75% of LM rewrites, with a 1.5 to 2 times asymmetry toward inflation. If a memory system shows flat hedge rates through rewriting, either the system is suppressing the drift by accident or the published result does not generalise; either outcome would weaken the argument.

Prediction 3. Given a memory store containing a known-wrong model conclusion, an agent should be more likely to defend and build on it than to re-derive it, compared to a control where the same claim is labeled as a guess. If labeling makes no behavioral difference, the schema fix is not worth building: at that point it is an aesthetic preference rather than an engineering claim.

The load-bearing prediction. The reading model actually changing behavior in response to an epistemic label is the load-bearing assumption, and it deserves a direct test rather than an appeal to plausibility. It is exactly the kind of thing that could come back negative. The published literature on verbalized calibration, in particular Tian et al. 2023, has been pretty clear that LMs are poorly calibrated about their own reasoning: they overclaim correctness, underclaim uncertainty, and the calibration error is large enough that a self-reported confidence stamp on a recalled note may add noise rather than signal. If that is true, the labelling scheme has to be enforced at the rendering layer, not the reading layer, and the schema is the right place to enforce it. The test that would settle it is the third prediction above, run with a real store and a real agent.

Two things the argument does not claim. It does not claim that labelling alone makes an agent correct. A labelled-and-laundered note is still wrong, just labelled as wrong with the wrong label. It does not claim that provenance classes are a security mechanism. A malicious writer can attach a human stamp to anything it can write, and a closed enum is no defense against a hostile actor with direct store access. The argument is about accidental corruption, not adversarial input, and treating the two as one would be a category mistake.

The Line to Keep

An agent that forgets is slow. An agent that cannot tell what it was told from what it guessed is confidently wrong, and it will defend the position. Forgetting is safe. Laundering is not.

The fix is small. A closed enum, stamped at write, rendered on every recall, with the summarization pipeline forbidden from rewriting it. Three classes are the minimum. The argument does not need anything else. The argument does need that one thing, because without it every note the system stores is a free-floating claim with no information about how it came to exist, and the most plausible-sounding notes are the ones that came from the worst path.

Sources

W3C Working Group. PROV-DM: The PROV Data Model. W3C Recommendation, 30 April 2013. The standard conceptual model for provenance on the web: entities, activities, agents, plus seven binary relations including WasDerivedFrom and WasAttributedTo. Part 1.
Wang, Y., Zhang, J., Cai, T., Liu, Z., Sun, Q., Sun, Z., Wu, Z., Dong, M., Zheng, M., Yin, X., Zhu, Y. From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents. arXiv:2606.04990, June 2026. Organises memory writes by source type and recommends a per-memory provenance field with source, actor, transformation, confidence, and update history. Part 5.
Belem, C. G., Wu, S., Yao, H., Steyvers, M., Singh, S., Smyth, P. From "May" to "Is": Certainty Distortion in Language Model Rewriting. arXiv:2606.07951, June 2026. Measures certainty drift in up to 75% of LM rewrites, with a 1.5 to 2x asymmetry toward inflation, and a compounding effect over repeated paraphrasing. The published evidence behind Part 4's mechanism.
Chhikara, P., Khant, D., Aryan, S., Singh, T., Yadav, D. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413, April 2025. Update phase uses ADD, UPDATE, DELETE, NOOP operations with no per-note provenance class. A canonical example of the typeless-store pattern. Part 1.
Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., Gonzalez, J. E. MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560, October 2023. The two-tier core/archival memory model that is the de facto reference for the field. No provenance class is rendered on recall; stored as a counterexample. Part 1.
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., Manning, C. D. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. arXiv:2305.14975, May 2023, EMNLP 2023. Shows RLHF-tuned LMs can verbalize better-calibrated confidences than their token probabilities, but the calibration error remains large enough that a self-reported confidence stamp is not a substitute for schema-level provenance. Part 7.

Relevance Is the Wrong Sort Order for Agent Memory

Swapnanil Saha — Wed, 02 Sep 2026 17:55:05 +0000

Relevance Is the Wrong Sort Order for Agent Memory

Ask any agent memory system for what it knows about a topic and it hands back the most similar notes, ranked best first. Every system does this. It is so obviously correct that nobody argues for it.

It was inherited, unexamined, from document search, and it carries an assumption that document search is entitled to and memory is not. That a note being returned does not depend on which other notes are also returned.

Notes are not independent. A note can correct, refine, contradict, supersede, or revoke another note, and when it does, the earlier note's truth is not a property of the earlier note at all. It is a property of the pair. Rank by similarity and you can return one member of that pair alone, with high confidence, and what you have delivered is not an incomplete answer. It is a false one.

This post argues that relevance is the wrong primary axis for memory retrieval, that the right one is the causal structure between notes, and that relevance's correct job is choosing an entry point into that structure rather than ordering the output.

Part 1: The Inherited Default

01. The Probability Ranking Principle, and What It Assumes

Ranked retrieval by probability of relevance is one of the load-bearing ideas in information retrieval, and it is a good one. The Probability Ranking Principle (PRP) is the statement, and under two stated conditions it licenses exactly the kind of sort every retrieval system does today. S. E. Robertson stated the principle in 1977; the standard references are his 1977 paper and the longer treatment in van Rijsbergen's Information Retrieval (1979).

Two of the principle's conditions matter here.

The first is that document relevance is independent. The standard statement of the principle, as given in the Wikipedia summary of the original derivation and reproduced in surveys, assumes the relevance of one document is independent of the relevance of any other. That is what licenses a simple sort: if the value of each item is intrinsic and unrelated to the others, ordering by that value is optimal and you are done.

The second is that the user integrates. Search returns a list to a person who reads several results, notices that two disagree, and resolves the conflict themselves. The system is not obliged to deliver a coherent answer, only a well-ordered set of candidates the user can arbitrate.

Information retrieval knows the independence assumption is not always safe. The whole point of the diversity and redundancy literature is that item interaction is real, and a ranked list of near-duplicates is worse than its individual scores suggest. Maximal Marginal Relevance (MMR), introduced by Carbonell and Goldstein at SIGIR 1998, makes the trade-off explicit: a linear combination of query-relevance and novelty against the items already selected. The MMR formula, in the notation of the original paper, is

MMR = argmax_{D_i in R \ S} [ lambda * Sim_1(D_i, Q) - (1 - lambda) * max_{D_j in S} Sim_2(D_i, D_j) ]

Read that: keep some lambda weight on similarity, subtract a penalty proportional to the maximum similarity to anything already chosen. The penalty is exactly the operation that improves a redundant list, and it is the operation that hurts a list of corrections.

But redundancy is a mild violation. Two documents saying the same thing waste a slot. Memory has a severe violation: two notes saying opposite things, where one of them is the retraction of the other. No amount of diversity tuning addresses that, because the problem is not that the items overlap. The problem is that one of them determines whether the other is still true.

Insight: A missing document makes a search result incomplete. A missing successor note makes the returned note wrong. Document retrieval degrades gracefully under omission because documents stand alone. Memory retrieval does not, because a note's truth is often stored in a different note.

The PRP is correct in the regime it was derived for. Memory retrieval is not in that regime, and the derivation has not been re-done for the regime it is being applied to. Every agent memory system that ranks by similarity is running the PRP with the wrong problem statement, and getting the wrong answer in a specific, structural way that the rest of this post is going to make concrete.

Part 2: What a Note Actually Is

02. A Note Is Not a Document

A note in an agent memory system is not a document. It is a move in a sequence, and it is frequently defined against what came before it.

"Use the venv interpreter for the test suite" is a note. "That was because the global interpreter lacked a grammar package, which was fixed in the July upgrade" is a note about that note. "The venv rule still applies because CI derives from it" is a note about that one. Each is a coherent statement on its own. Delivered in isolation, the first is a rule with an unknown reason, the second reads as a repeal, and the third is meaningless.

Every memory system with a lifecycle has the edges to express this. Supersession, revocation, correction, and refinement all exist as first-class relations in the better designs, including in the arguments I have made previously about expiry as a state and about reinstatement as the missing transition. That post argued revocation needs an inverse, so a note proven wrong can come back when it turns out to have been right. This post is the layer above: once notes carry supersession and revocation edges at all, ranking them independently by similarity is the wrong retrieval contract, because the correction sits closest in embedding space to the thing it corrects and gets dropped by the very machinery meant to reduce redundancy.

What has not happened is those edges being made load-bearing at retrieval time. They are stored, they are queryable, and then the recall path ranks by cosine similarity and returns a set, and the edges contribute nothing to what the caller receives.

The gap is not in the data model. It is in the delivery.

Insight: The principle of independence fails not by small amounts but categorically. Under the PRP, omitting a document is a loss. Under a memory contract, omitting a successor is a change of truth value. The two regimes differ in kind, and a ranker designed for the first is unfit for the second, no matter how well it does on benchmarks calibrated against the first.

A document is about a topic; it is judged by its own content against the query, and the relevance of two documents is plausibly independent because each makes its own claim about an external subject. A note is about another note: the supersession edge is part of the claim. Returning one member of a supersession pair is, in retrieval terms, returning a sentence that has been formally declared false, with a high confidence score, and asking the caller to integrate it. The caller will not know to integrate it, because the integration is precisely the lifecycle edge that the ranker was designed to ignore.

Part 3: The Concentrated Failure

03. Supersession Lives in the Redundancy Zone

This is the part that turns an abstract objection into a concrete failure, and it is the strongest argument in the post.

Ask which pairs of notes in a store are nearest each other in embedding space. The answer is: notes about the same subject, in the same vocabulary, at similar length. Now ask which pairs of notes stand in a supersession or revocation relation. The answer is the same: a correction is about the thing it corrects, and it reuses its vocabulary, because that is what a correction is.

The relation the ranker most needs to respect is concentrated exactly in the region the ranker treats as redundancy.

Two consequences follow, and both are bad.

The first is a plain retrieval effect. Under a top-k cutoff, near-duplicate notes compete for adjacent slots, so a superseding note and its predecessor are frequently separated by the boundary rather than both included. Which one lands inside depends on lexical accident: which one happens to share more surface form with the query. A correction phrased tersely ("this was wrong, the cause was the proxy") often shares less surface form with a topical query than the confident, well-written original it corrects, so the ranker prefers the note that has been retracted.

The second is worse, because it comes from the system's own quality machinery. Any diversity or deduplication pass, added in good faith to stop the store returning five paraphrases of one thing, will look at a note and its correction and see a near-duplicate pair, and will drop one. The one it drops is typically the one that adds the least new lexical content, which is, again, the correction. The mechanism installed to improve result quality systematically removes the notes that carry the truth.

The interactive demo (Demo 01 in the HTML version) makes this visible. It seeds a small store of notes that pair every claim with its correction, then runs two retrieval regimes against a topical query. Regime A is plain top-k by similarity. Regime B closes the result on the supersession edge: a note and its successor are always returned together, regardless of independent similarity scores. Watch what Regime A keeps and what it loses.

This is the failure that convinces me the ordering principle is wrong rather than merely imperfect. A tuning problem gets better with tuning. This one gets worse with tuning, because every improvement to relevance and diversity, evaluated in the document-retrieval frame, pushes harder in the wrong direction. The MMR reranking in the demo is doing what it was designed to do, and what it was designed to do is exactly the wrong thing here.

I am not aware of any published work that has named this specific failure in agent memory retrieval. The information-retrieval literature on dedup and diversity treats the redundancy as a cost; it has no slot for the case where the redundancy is a supersession edge whose presence in the result is the whole point. The argument in this part is the author's. Treat it as an argument, not as a finding.

Part 4: The Obvious Counter

04. Recency Is Not Causality

The obvious counter is to sort by time instead. It fails, and understanding why is what points at the actual answer.

Recency is not causality. A note written after another is not thereby about it, does not thereby correct it, and does not thereby supersede it. A store sorted by recency delivers whatever was written most recently on any subject, which for a topical query is frequently an unrelated note from a busy afternoon.

And the inverse error is worse. An old note is not thereby superseded. The most durable knowledge in a codebase is often the oldest: why a migration was written a particular way, which constraint a design was working around. Sorting by time promotes the recent and demotes the durable, and time is measuring elapsed duration, not validity. Age may inform ranking. It must never decide validity, and a recency sort quietly lets it.

The position is stated at length in a previous post on expiry as a state: that post argued that age is not evidence about whether a note is still true, and the present post takes the same observation as given and extends it. The argument here is that recency-as-sort is not just a weak proxy, it is the wrong axis in the same way similarity is the wrong axis, for the same kind of reason: it confuses a surface feature with a causal one.

The useful observation buried in the recency proposal is that it is reaching for order, and specifically for an order in which a correction lands after the thing it corrects. But global time order is a poor proxy for that. What is actually wanted is the edge: this note supersedes that one. That edge exists. It is already stored. It is simply not being used at the moment the results are assembled.

Warning: The natural alternative to a similarity sort is a recency sort, and the natural alternative to a recency sort is a recency-weighted similarity sort, and the natural alternative to that is a recency-weighted similarity sort with diversity penalty. Each one looks like a refinement. None of them changes the problem, which is that the edges the system already has are not load-bearing at retrieval. Tuning a ranker that ignores the lifecycle graph is tuning an answer to a different question than the one the caller is asking.

The empirical shape of this point comes from a different source. LongMemEval (Wu et al., ICLR 2025) tests long-term interactive memory with five abilities including knowledge updates, where a fact stated early becomes outdated later, across 500 questions embedded in sustained chat. The paper reports that commercial chat assistants and long-context LLMs show a roughly 30% accuracy drop on remembering information across long interactions, and identifies knowledge updates as one of the harder abilities in the suite. The exact mechanism that drives the drop is not isolated in the paper, and the store in LongMemEval is the chat history rather than a curated memory. The relevant lesson for this post is that a retrieval system whose primary axis is unrelated to the fact that an update happened will struggle on knowledge updates, and the ranker's knobs (similarity, recency, diversity) are not the axis that would help.

Part 5: The Alternative Contract

05. Return a Trajectory, Not a Set

Here is the alternative, stated as a design.

The unit of delivery is not a note. It is a belief and its trajectory: the current state of a claim, plus the events that produced it, plus the notes it superseded or that revoked it. Relevance still runs, and it still does the thing it is genuinely good at, which is choosing where to enter the graph. What it no longer does is decide what comes back.

Two rules define the behavior.

Closure. If a note is returned, every note that supersedes or revokes it is returned with it, whether or not that successor is independently similar to the query. Its relevance is inherited: the successor is relevant because the predecessor was, and the caller cannot correctly use the predecessor without it. This is the rule that makes the failure in Part 3 structurally impossible rather than merely less likely, and it is deliberately unconditional. A successor that scores badly on similarity is exactly the case the rule exists for.

Terminal state first. Within a returned chain, the current state leads and the history follows, marked as history. The caller should not have to reconstruct which note won. That reconstruction is the system's job, and it is cheap for the system and error-prone for the caller.

The result is not a ranked list of notes. It is a small number of chains, ordered by the relevance of their entry points, each internally ordered by causation rather than by score.

Note what this is not. It is not a graph traversal that pulls in everything topically connected, which would blow the budget and reintroduce noise. The closure is over lifecycle edges only, which are sparse, deterministic, and recorded at write time rather than inferred. Most notes have none, so most chains are one note long and the output looks exactly like today's. The difference shows up only where it matters.

To state the link to the earlier post: that one argued that revocation needs a reverse edge, reinstatement, so a note can come back when the revocation turns out to have been wrong. This post argues the parallel point for retrieval. The store has the edge. The store does not use it. Closure makes the edge load-bearing at the moment the caller is about to be misled by its absence.

The interactive demo (Demo 02 in the HTML version) runs a single query against the same store as Demo 01, this time with the closure rule enabled. The chain returned for each entry point includes the predecessor and the correction, ordered with the current state leading. Compare the chain against what Demo 01 returned for the same query. The contract is "return the entry point's chain," not "return top-k of notes."

06. What This Costs

Four consequences follow, and an honest argument states them rather than leaving them for the reader to find.

A note can no longer be scored in isolation. Scoring is per chain, and a chain's value is not the sum or the max of its members. This complicates every ranking heuristic that assumes a flat candidate pool, and it means the retrieval and the assembly stages can no longer be cleanly separated. The retrieval stage chooses entry points, the assembly stage composes chains; the two cannot be designed independently.

The top-k contract breaks. Ten notes might be four chains, and the caller asked for ten of something. Either the contract becomes "k chains" with variable note count, or "k notes" with the closure rule sometimes forcing an overrun. There is no formulation that keeps both, and I think the honest resolution is that the contract was always slightly wrong: what a caller wants is a budget, in tokens, not a count of items. A chain is a natural packing unit for that budget; a sorted list of independent notes is not.

Deduplication becomes dangerous by default. Any near-duplicate suppression has to become lifecycle-aware, or it reintroduces exactly the Part 3 failure. In practice this means dedup must never be allowed to drop a note that stands in a lifecycle relation to another candidate, which is a constraint the usual implementations do not have a place to express. The MMR reranking in the demo is a model of the issue: it is doing the right thing by its own definition and the wrong thing for this contract.

Budgeting gets harder. A chain has variable cost, so filling a fixed token budget with whole chains is a packing problem rather than a prefix of a sorted list. It is a small packing problem, and the practical answer is probably to take chains greedily by entry-point relevance and stop, but it is no longer trivial and it should not be pretended otherwise. A ranker that pretends the budget is just a count of items is the ranker that delivered the Part 3 failure in the first place.

None of these is fatal. All of them are the price of delivering results that are true rather than results that are well ordered.

Part 6: The Limits

07. What This Is and Is Not

No measurement is offered. The argument is structural: it identifies an assumption imported from document retrieval, shows a specific mechanism by which it fails for notes, argues that the obvious alternative axis fails too, and derives a delivery rule. That is an argument, not a result.

The experiment is specifiable and cheap, which makes the absence of a measurement here a choice rather than an excuse. Build a store containing correction chains, where a note is followed by a note that revokes or supersedes it. Issue topical queries. Measure how often relevance-ranked top-k returns a superseded note without its successor, and how often a diversity pass drops the successor specifically. Then measure the same with the closure rule enabled, and measure what it costs in budget. The first number is the one that decides whether this matters: if superseded notes almost never separate from their successors in practice, the argument is correct in principle and unimportant in practice, and that is a finding worth publishing too.

This does not claim similarity search is the wrong tool. It is the right tool for the job it should be doing, which is finding the entry point. The claim is narrower and sharper: it should not be deciding the composition of the result. A retrieval that uses similarity to choose a chain head and lifecycle edges to assemble the chain uses each tool for the thing it is good at. A retrieval that uses similarity to choose every note uses it for the thing it is bad at, with consequences the rest of this post has tried to make concrete.

None of this addresses conflicting notes with no recorded edge between them. If two notes disagree and nobody ever recorded that one supersedes the other, no delivery rule can fix it, because the information does not exist. That is a capture problem and a genuinely open one, and the closure rule is worth nothing on a store where the edges were never written. The store has to be written with the lifecycle in mind, the way the store in the reinstatement post is written with the reverse edge in mind. The store can do the right thing for the data it has, and the data has to be there.

Warning: A chain returned under the closure rule is still the system's best representation of the truth of the matter, not the truth. A successor note is itself a note, and can itself be wrong, and the rule does not pretend otherwise. The rule claims only that omitting a successor from a result in which the predecessor appears is a category error in retrieval, not that the successor is correct. The arbitration between successive corrections in a long chain is the model's job, not the memory system's, and the system should present the chain so the model can do that work.

The line to keep

Ranked retrieval assumes the things being ranked stand alone. Notes do not. A correction is about the thing it corrects, and it lives in the same region of embedding space as the thing it corrects, which is precisely where a ranker's redundancy handling does the most damage.

Relevance should choose where to enter memory, not what comes back. What comes back is determined by the lifecycle edges the store already has and does not currently use: return a note and you must return whatever revoked it, whether or not that successor looks relevant, because the caller cannot use the one without the other.

The store is being asked two questions at once. Relevance is the right tool for the first, which is where to start. The lifecycle graph is the right tool for the second, which is what to read. A retrieval system that conflates the two is, quietly, returning false answers with high confidence scores. A retrieval system that separates them uses the ranker for the thing the ranker is good at, and the store for the thing the store is good at, and the caller gets the chain the chain is meant to deliver.

Sources

Robertson, S. E. "The Probability Ranking Principle in IR." Journal of Documentation 33(4), 1977. Source of the probability ranking principle. The relevance-independence assumption is the one Part 1 leans on. Wikipedia summary reproduces the assumption in standard form.
Carbonell, J., Goldstein, J. "The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries." SIGIR 1998. MMR formula and the relevance-novelty trade-off. Original paper. Part 1 and Part 3.
Saha, S. Agent Memory Expiry Is a State, Not a Delete. The prior post on age and validity. Cited from Part 4 rather than re-derived. Argues that elapsed time is not evidence about a note's truth.
Saha, S. Reinstatement Is the Missing Primitive in Agent Memory. The prior post on the reverse edge of revocation. Part 2 and Part 5 of this post argue the parallel point for retrieval: the store has the edge, the ranker does not use it. arXiv:2607.20972 is the underlying research paper.
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., Yu, D. LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. ICLR 2025, arXiv:2410.10813. Five abilities including knowledge updates, 500 questions, reported 30% accuracy drop across long interactions. Part 4 cites it for the empirical shape of the recency problem, not the mechanism.
Chhikara, P., Khant, D., Aryan, S., Singh, T., Yadav, D. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413, April 2025. ADD/UPDATE/DELETE/NOOP update phase. Part 3 and Part 5 reference it as a production system that stores the supersession edge but does not close retrieval on it.
Wang, Z. TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memory. arXiv:2606.06240, June 2026. Audit rows, isolation preconditions, default-retrieval filtering. Part 3 and Part 5 reference it as the production system with the cleanest auditability story, and the same gap between auditability and successor-closure at default retrieval.
vectr source: agent/working_context_store/_events.py (NOTE_EVENT_KINDS) and _store.py (note lifecycle fold). The supersession and revocation event vocabulary over notes, documented publicly in the prior two posts in this series.

Agents Never Remember What Isn't There

Swapnanil Saha — Tue, 01 Sep 2026 17:07:01 +0000

Agents Never Remember What Isn't There

Most agent memory systems have a small operation table. Write a note. Read it back. Supersede it with a newer value. Delete it. The store is a record of what an agent found: a definition, a call site, a config value, a decision, a gotcha. Every design decision in the system assumes the note names something that exists.

Almost none of these systems can store the other kind of finding: that something does not exist.

There is no retry configuration in this service. Nothing calls this function. This repository has no integration test layer. That flag was never wired up. Each of those is a real finding, each is expensive to establish, and each is thrown away at the end of the session that established it. The next session pays for it again from zero.

The argument of this post is that the gap is structural. An absence cannot be stored the way a presence is stored, because its truth conditions are inverted in a way that breaks every mechanism a memory system uses to keep notes honest. Closing the gap requires a primitive nobody has built, and the rest of the post is about what that primitive would have to look like, why three reasonable alternatives fail, and what has to be true of the layer below the memory system for the primitive to work at all.

The closest neighbour in argument is a recent post that observed the first search an agent runs is a write to memory and every repeat is a read. That post treats search as memory's write path. This post is about the one result that write path cannot carry, which is the empty result. A search that comes back empty names no artifact, has no anchor, and produces nothing the memory layer can record. The two arguments are complementary rather than overlapping. Where the first one talked about what gets written, this one is about what the writing system is structurally blind to.

Part 1: The Most Expensive Half

01. Absence Is the Most Expensive Thing an Agent Establishes

Start with the cost, because the asymmetry is stark and almost never stated.

A search for something that exists terminates the moment it is found. The agent greps, gets a hit on the third file, reads it, and moves on. The cost is bounded by luck and by how good the first guess was.

A search that establishes absence cannot terminate early. It terminates only on exhaustion. To conclude that there is no retry configuration, the agent has to look under every name the thing might have (retry, retries, backoff, max_attempts, RetryPolicy), in every place it might live (config files, environment defaults, a client wrapper, a decorator, a base class), and in every form it might take (a literal, a constant, a setting read from an env var, a library default that was never overridden). Absence is always the full sweep. There is no lucky third file.

So the expensive result is the one that gets discarded, and the cheap result is the one that gets written down. That is exactly backwards from what a cost-aware system would do.

Insight: A memory system that stores only findings stores only the cheap half of what a session learned. The expensive half, the exhaustive searches that came back empty, evaporates at the session boundary and is re-run at a discount that makes it progressively less trustworthy.

It gets worse when you count repetition. Presence facts tend to be written down because they are immediately useful: an agent that found the lock function needs the file and line right now. Absence facts feel like non-results at the moment they are established, so nothing prompts anyone to record them, and the same sweep runs again next week under a slightly different query.

There is a second-order effect worth naming. An agent that cannot recall a previous absence does not merely repeat the search. It repeats the search less thoroughly, because the session's budget is already partly spent on the actual task. A sweep that produced nothing last time produces nothing this time in less depth. The second pass is cheaper and weaker, which means the conclusion is less reliable each time it is re-derived. Re-derivation is not a fixed tax. It degrades.

Part 2: What an Absence Actually Is

02. An Absence Is a Claim About a Search, Not About the World

Here is the first thing that makes absence hard to store, and it is not obvious until you try to write the note.

There is no retry config is not a proposition about the repository. It is shorthand for a much longer proposition about a search: I looked in these locations, under these names, using this method, at this revision, and found nothing. Strip away the search and what remains is a bare assertion with no way to tell whether it was established by a thorough sweep or by one hasty grep against one spelling.

This matters because the two are indistinguishable in the note and radically different in value. A thorough absence is a real finding. A hasty absence is a guess wearing a finding's clothes.

And the failure mode of a wrong absence is the worst one in this whole area. An agent that reads "there is no retry config" does not look for a retry config. The note forecloses the search that would refute it. That is the self-sealing shape: a false negative removes the action that generates the contradicting evidence, so the error is stable and silent, and no amount of subsequent capability fixes it because the model is reasoning correctly from a premise it has no route to re-examine.

Warning: A presence-memory can be a bare fact and still be useful, because acting on it produces immediate feedback when it is wrong. An absence-memory acted on produces no feedback at all. So the scope is not metadata attached to the finding. The scope is the finding, and a note that omits it is not a compressed version of the finding. It is a different and less defensible claim.

The trade-off is described formally in knowledge-base research under the closed-world assumption and negation-as-failure: a system that cannot prove a predicate concludes the predicate is false, and the cost of that shortcut is that the conclusion tracks the search rather than the world. The closed-world machinery was built for a fixed knowledge base whose contents the system could exhaustively enumerate. A code repository is not a closed world, and the agent is not enumerating it. It is sampling it, under a query, and the absence of a hit is a fact about the sample, not a fact about the repository.

That distinction collapses the moment a note drops the scope of its search. A bare "no retry config" promises more than the search that produced it could deliver, and a memory system that stores the bare form makes the promise on the note's behalf.

03. The Self-Sealing Failure

The shape in Section 02 has a name in the agent-memory literature, and the name matters because the literature has spent real effort on it.

Recent work on memory for autonomous agents characterizes the failure as self-reinforcing error. The mechanism is plain once you see it: an agent holds a false belief, the belief shapes the actions it takes, the actions generate observations that confirm the belief, and the loop closes. The next session re-derives the same belief from the same durable inputs and writes it down again as if it were a fresh discovery.

The same shape shows up under a related name in the spurious-correlation framing of memory bias. A short empirical study on this exact pattern appears in When Memories Become Liabilities (arXiv:2603.04664), which finds that an agent's prior answers bias its later steps and the bias compounds across turns, with the gap between agents that have such memory and agents that do not widening as the task lengthens.

Analogy: Sequential decision theory met this structure decades ago. In a multi-armed bandit, an agent that always pulls the best-known arm and is given a false negative belief about one arm's value will never update that arm's posterior, because updates require pulls. The cure is forced exploration, a mechanism that occasionally takes the action the current posterior says is worthless, specifically to keep the posterior honest. Agent memory has no analogue of forced exploration. There is no mechanism that periodically resurfaces a discarded belief for re-testing. The belief is either held, in which case it gates the action, or gone, in which case Section 04 shows something worse happens.

This is a direct extension of the previous post's argument about the missing reinstate operation. That post argued that deletion cannot break the self-reinforcing loop, because the durable inputs that produced the false belief remain in the workspace and keep regenerating it. This post argues that the same loop, running on absence rather than presence, is harder to detect in the first place, and that the obvious mitigations either do not exist or do not work.

Part 3: The Inverted Watch

04. Falsification Conditions Run Backwards

This is the core of the argument, and it is what makes absence a distinct primitive rather than a special case of an ordinary note.

Consider how a presence-memory is kept honest. The note names a thing: a function, a file, a constant. The store anchors the note to the artifact that contains it and hashes that artifact's content. On the next check, if the hash changed, the note is flagged as possibly drifted. The mechanism works because the note has a referent, the referent has a location, and the location has content that can be watched.

Now try the same thing for an absence. There is no referent. The note says nothing exists at any location, so there is no location to anchor to and no content to hash. The entire staleness apparatus is built on a structure that an absence-memory does not have.

Worse, the direction of falsification inverts.

A presence-memory is falsified by a change to the named thing. The function moved, the constant was renamed, the signature changed. The set of events that can falsify it is small, local, and watchable.
An absence-memory is falsified by an addition anywhere inside the search's scope. Someone adds a retry decorator in a file nobody touched last month, and the note is now false, with no signal reaching the note at all.

The consequence is that presence and absence need opposite watch strategies. Presence watches a point. Absence watches a region, and the region is defined by a query rather than by a path.

There is a compensating asymmetry, and it is the thing that makes this tractable rather than hopeless. A deletion can never falsify an absence. If the note says nothing exists and someone removes code, the note is still true. Neither can a modification of existing code that does not introduce a new instance. Only additions matter.

Insight: Presence-memory anchors to a location and is invalidated by change. Absence-memory anchors to a query and is invalidated only by addition. That single asymmetry is what makes an absence cheap to re-verify despite being expensive to establish, and it is available to any system willing to treat a query as a first-class anchor.

Checking an absence-memory does not require re-running the original exhaustive sweep against the whole corpus. It requires running the original query against only what was added since the note was written. The check is bounded by the diff, not by the repository, and it is monotone: absences accumulate evidence in one direction and can only be broken by growth. An absence written a year ago in a repository with a hundred commits since is checkable at the cost of scanning those hundred commits' additions, not at the cost of the original sweep.

Part 4: Why the Fixes Fail

05. Three Reasonable Fixes, Three Different Failures

Three responses come up immediately. Each is reasonable and each fails for a different reason, which is a good sign that the gap is real.

"Just write the negative result as an ordinary note"

You can. Nothing stops an agent from storing "there is no retry config" as a normal finding today. The problem is what happens next: it enters a store whose ranking, staleness, and trust machinery were all designed for referents, so it sits there as an unanchored, unverifiable assertion that no mechanism will ever revisit. It cannot go stale, because staleness is defined by drift at an anchor and it has no anchor. It therefore ages into a permanent, unchallengeable claim, which is precisely the property you least want on the class of note that forecloses its own re-examination.

An absence note in a presence store is not a partial solution. It is the failure mode with extra steps.

"Re-run the search when the note is recalled"

Closer, and this is where the real design lives, but two things break a naive version.

First, the query has to have been recorded verbatim, along with the method. "I searched for retry" is not enough to reproduce anything. Was it lexical or semantic? Case sensitive? Did it cover the vendored directory? A re-run that differs from the original in any of these is not a re-check, it is a new search whose disagreement with the note tells you nothing about whether the world changed.

Second, and this one is specific to semantic retrieval: the same query does not mean the same thing over time. Re-embed a query after a model change, an index rebuild, or a chunking change, and it returns a different neighborhood over an unchanged corpus. A semantic absence is a claim about a corpus, a query, and an embedding, and only the first of those is expected to change. A re-run that silently varies the other two produces false reinstatements of a search that was never actually invalidated. Any absence primitive built on semantic retrieval has to pin the retrieval configuration as part of the note, which no store currently does. (A worked example of the embedding-mismatch hazard appears in the public write-up of the embedding-drift problem: a corpus re-embedded under a new model returns results whose ranking inverts with no signal at the similarity layer, the same shape this section describes.)

"Hash the whole corpus"

Too coarse to be useful. Any commit anywhere invalidates every absence in the store, and a mechanism that flags everything flags nothing, because the flag stops carrying information and gets ignored. The same alarm-fatigue pattern is well documented in clinical monitoring: alarm fatigue in ICU monitoring, for example, arises when overwarning degrades response to every warning, the same mechanism that makes a coarse staleness flag useless for distinguishing what actually changed. The addition-only check from Section 04 is what rescues this: the correct granularity is not the corpus and not the file, it is the set of additions since the note, filtered by the note's own query.

Part 5: The Shape of a Primitive

06. What an Absence-Memory Has to Carry

Working backwards from the failures above, the shape is determined rather than chosen.

An absence-memory has to record the verdict (nothing found), the query in the exact form that was executed, the method (lexical, semantic, symbol-graph lookup, or a combination, since a serious sweep uses several and an absence established by only one of them is weaker), the scope that was actually covered including what was excluded, and the corpus revision at which it held.

The check is then a re-run of the recorded query, restricted to additions since the recorded revision, with the recorded method and configuration.

Three properties follow, and all three are improvements on how presence-memory behaves today.

Math: The cost of re-checking an absence at revision r_n over an original sweep S against a corpus with additions A_n since the note was written:

check_cost(r_n) = cost(query, A_n) << cost(query, corpus)

The check scales with the diff, not with the corpus. An old absence is not more expensive to verify than a new one, which is unusual: most staleness mechanisms degrade as the anchored artifact drifts further from its recorded state. Here the work is monotone in additions, never in the original sweep, so cost stays bounded as the repository ages.

The check is cheap and gets no more expensive with age. A year-old absence in a thousand-commit repository is checkable in the cost of scanning those thousand commits' additions, not in the cost of the original sweep. That is the structural payoff of the addition-only watch.

The check has a definite answer. Unlike a content-hash drift signal, which says only that something changed and leaves interpretation to the reader, an absence check either finds a new instance or does not. It produces a verdict, not a caveat.

A broken absence is highly informative. When the check fires, it does not merely invalidate the note. It hands over the exact thing that was added and the exact query it matched, which is usually the answer the agent needed. The invalidation and the discovery are the same event.

There is also a strong claim available about what should happen to a verified absence, and it deserves stating rather than assuming. An absence that has been re-checked across many revisions and held every time is not merely still true. It is evidence about the shape of the system, and it should be delivered differently from a fresh one. No retry configuration has existed here across the last four hundred commits is a stronger and more useful statement than no retry configuration found, and only a store that keeps the check history can say it.

Part 6: The Layer Below

07. Retrieval Has to Be Honest First

There is a layer below the memory system that has to cooperate, and today it usually cannot.

For an agent to record an absence at all, its search tool has to be able to distinguish two states that most tools conflate: nothing exists and I did not find it. A retrieval system that returns its best matches regardless of quality never says the first one. It always returns something, so the agent never receives the signal that would prompt an absence to be recorded, and instead receives a list of near-misses that it must judge for itself.

That judgment is where false absences come from. An agent that receives five irrelevant results and concludes "not present" has made an inference the retrieval layer never licensed. An agent that receives an explicit empty result with a stated scope has received a finding.

Insight: An absence primitive is not purely a memory feature. It requires the search layer to be honest about coverage: to distinguish an empty result from a weak one, to report what it searched rather than only what it returned, and to be willing to say nothing at all. That property is worth having for its own sake, and it is a precondition for anything in Part 5.

This is the same boundary as the argument that search and memory are two phases of one system rather than two systems. An absence is the clearest case: it is produced by retrieval, consumed by memory, and meaningless if either half is unaware of the other.

Part 7: Limits

08. What This Is and Is Not

No measurement is offered here. The argument is structural. It identifies a class of finding that is expensive to establish, systematically discarded, and unstorable under existing designs, shows that the standard staleness mechanism cannot apply to it, and derives the shape of a primitive that could. That is an argument, not a result, and it should not be read as more.

The experiment is specifiable, which is the part worth doing next. Take tasks whose solution requires establishing that something does not exist, which is common in real codebase work: confirming there is no existing implementation before writing one, confirming a config is unset before adding it, confirming nothing calls a function before deleting it. Run them across repeated sessions on an evolving repository. Measure the cost of the negative determination each time, measure whether the second and later determinations are less thorough than the first, and measure how often a false absence survives. Then compare against a condition with recorded absences and diff-scoped re-checks. The re-derivation-cost framing is the right one, and the degradation effect from Part 1 is the most interesting thing to look for, because if it is real it means the current situation is worse than a simple repeated-cost model suggests.

This does not make an agent's negative conclusions correct. A recorded absence is exactly as good as the sweep that produced it, and the primitive proposed here makes the sweep's scope visible rather than making it better. That is the point. It converts an unfalsifiable assertion into a checkable one. A thorough absence becomes durable, and a hasty one becomes visibly hasty.

Warning: Nothing here is a security mechanism. An absence record is a cost-saving and error-preventing device against ordinary repetition, not a control against an adversary who can write to the store. A poisoned absence would be a particularly effective attack precisely because of the foreclosure property in Section 02. That is worth naming as a risk of the design rather than hiding.

The line to keep

Every memory system stores what an agent found. The expensive half of what a session learns is what it looked for and did not find, and that half is discarded at the session boundary, re-derived more cheaply and less reliably every time.

An absence is a claim about a search, not about the world. It cannot be anchored to a file, because it names no file. It can only be anchored to the query that established it, and once it is, it becomes the cheapest note in the store to keep honest, because nothing but an addition can ever make it false.

Sources

Hu, Z. and others. When Memories Become Liabilities: Leveraging Spurious Correlations in Self-Reinforcing Memory. arXiv:2603.04664, March 2026. Empirical evidence that an agent's prior answers bias its later steps and the bias compounds. Cited in Part 2.
Poole, D. Artificial Intelligence: Foundations of Computational Agents, 3rd ed., Chapter 5.7: Complete Knowledge and Negation as Failure. Open textbook chapter. Clark's completion, negation-as-failure, the closed-world assumption. Cited in Part 2 for the closed-world / open-world distinction.
Wikipedia contributors. Closed-World Assumption. Reference summary of the formalism from Reiter (1978) onward. Cited in Part 2 for terminology.
multigrid. Why You Have to Re-Embed When You Change Embedding Models. Public write-up of the embedding-coordinate-system problem. Cited in Part 4 for the semantic-retrieval re-embedding hazard.
speed_engineer. Upgrading Your Embedding Model Doesn't Break RAG Loudly. It Breaks It Quietly.. Companion piece on the silent-failure shape of model swaps. Cited in Part 4.
Patient Safety Network, Agency for Healthcare Research and Quality. Alert Fatigue. Patient-safety primer on the alarm-fatigue mechanism. Cited in Part 4 for the overwarning pattern that a coarse staleness flag reproduces.
vectr source: the absence and re-check mechanism discussed in this post is sketched in the working-memory layer's public docs; the reversible revocation machinery that an absence primitive composes with is described in the vectr source under agent/working_context_store/_events.py and discussed in Reinstatement Is the Missing Primitive in Agent Memory.

A Directive Is Not a Memory: Obeying a Rule Hides the Test

Swapnanil Saha — Mon, 31 Aug 2026 15:33:20 +0000

A Directive Is Not a Memory: Obeying a Rule Hides the Test

Most memory systems are a small operation table. Write a note. Read it back. Replace it with a newer value. Delete it on demand or on a schedule. Some systems can also record that a deletion happened, as an audit event or a tombstone row. That is close to the entire surface area of the field as it ships today.

Almost every one of those systems stores two kinds of thing in the same place. It stores observations, claims about the world that can be checked by going to look, and it stores directives, instructions about what to do. It applies the same apparatus to both: the same staleness checks, the same corroboration, the same trust classes, the same revocation. The apparatus works on observations and is inert on directives, and the reason is not a defect of any particular implementation. Obeying a directive destroys the evidence that would test it. A rule that is followed suppresses its own counterexample, and nothing in the store can tell it has happened.

The categories are not a taxonomy for its own sake. They differ in the one property a memory system exists to manage, which is how a stored belief can be found wrong. The rest of the post walks through why the standard checks miss the directive half, what the falsifiable core of a directive actually is, what the right test looks like, and what the accumulated consequence is for any store that has been running for a while.

One thing up front, since the reading is otherwise easy to misread. This is not an argument against directives. Rules are how hard-won operational knowledge is transmitted, they work, and an agent that follows them is better than one that does not. The argument is that storing them in a system whose verification machinery cannot touch them, and displaying them with the same status as verified observations, misrepresents what is known.

Part 1: One Store, Two Categories

01 · Observations and Directives in One Place

Look at what a memory store actually contains after a few months of real use and the two categories are obvious once you separate them.

An observation has a referent. It names something that exists in the world: a file, a function, a value, an event that happened. Because it has a referent, it can be verified by going and looking, and it can be falsified by the referent changing. Every mechanism a memory system uses to keep notes honest is built on this: anchor the note to the artifact, hash the artifact, flag drift, ask for corroboration, surface contradictions.

A directive has no referent. "Always run the tests in the venv" does not name a thing that exists. It names an action to take, and the world does not contain a fact that the note is a description of. There is nothing to anchor to and nothing to hash.

You can of course anchor a directive to a file anyway. Most systems will happily let you attach "always use the venv" to the test configuration and watch that file's hash. But notice what the drift signal then tells you: that the file changed. It says nothing about whether the rule is still warranted, because the rule was never a claim about that file's contents. The anchor is decorative. It fires and it means nothing, which is worse than not firing, because a signal that carries no information trains the reader to ignore the ones that do.

Insight

The categories are not a taxonomy for its own sake. They differ in the one property a memory system exists to manage, which is how a stored belief can be found wrong. Treating a note with a referent the same as a note without one means the verification machinery, designed for the first case, runs without effect on the second, and the second case looks identical to the first on the dashboard.

A related distinction appears in the philosophy of language, and the post's central claim depends on it. Imperatives and declaratives differ in kind. Declaratives are the sort of statement that can be true or false. Imperatives are not: they are satisfied, obeyed, or appropriate, and asking whether one is true is a category mistake. The argument here does not say directives are meaningless, only that the falsifiability apparatus developed for propositions cannot reach them, and a memory system that treats them as propositions will be unable to test them.

None of this is to claim the current literature has noticed nothing. CoALA's framework for language agents separates declarative memory from procedural memory, and the design intent is to keep rules out of the fact store. Practical advice in the same vein says rules belong in the system prompt or agent policy, and memory should hold facts; A-MEM's Zettelkasten-inspired architecture moves in this direction with separate components for stable rules and time-stamped facts. The split is the right idea, and Part 3 of this post will argue it is not yet enough. The split as drawn treats rules as a different storage class with different update rules, but it does not yet address the deeper problem: rules in that storage class are still never tested against the world, only managed.

Part 2: The Self-Suppressing Test

02 · Compliance Is What Removes the Evidence

Here is the mechanism, and it is the heart of the post.

A rule exists because of a condition. "Always run the tests in the venv" exists because at some moment the global interpreter was missing something the suite needed. That condition is a fact about the world, and facts about the world change. The package gets installed globally. The dependency is vendored. The suite stops needing it.

Now ask what would inform anyone that the condition no longer holds.

The only observation that could is someone running the suite with the global interpreter and it working fine. And that is precisely the observation the rule exists to prevent. Compliance is what removes the evidence. Every session that follows the rule generates a record of the rule being followed and no information whatsoever about whether it needed to be.

So the rule's evidential position never changes. It was warranted once, it may be unwarranted now, and the world will never produce a signal either way, because the only source for that signal is the action the rule forbids.

Insight

A false belief that forecloses an action is recognized as a failure mode, sometimes named self-reinforcing error in the agent-memory survey literature. A directive that forecloses the same action is recognized as good practice. Structurally they are the same object: both remove the only observation that could correct them. The difference is that somebody meant to write the second one, which affects how it got there and not at all whether it can ever be found wrong.

This is the same closed loop as a self-sealing false belief, arrived at from the opposite direction. In the false-belief case, an agent wrongly concludes an action fails, stops taking the action, and thereby never collects the evidence that would refute the conclusion. In the directive case, a rule forecloses the alternative deliberately, and everyone involved endorses the foreclosure. That endorsement is exactly why nobody looks at it as a defect. The loop is identical; only the intent differs, and intent has no bearing on whether evidence gets collected.

The asymmetry of failure types makes this invisible in normal operation. A directive that is wrong in the direction of forbidding something harmless produces no failures at all. It produces slightly slower work, or an unnecessary constraint on a subagent, or a confusing instruction that a new contributor follows without understanding. There is no incident, no red test, no error message. The cost of an obsolete directive is paid continuously in small amounts and never shows up as an event.

03 · Three Checks That Do Not Apply

Take the standard mechanisms a memory system uses to keep a note honest, and apply each to a directive.

Staleness by anchor drift does not apply, for the reason in Part 1: the directive has no referent, so any anchor is arbitrary. Watching the file the rule is about tells you when that file changed, which is uncorrelated with whether the rule is warranted. A rule about interpreters is falsified by an installation on a machine, which is not in the repository at all.

Corroboration does not apply, and this one is actively misleading. The natural way to corroborate a note is to check it against the codebase. Do that with a directive and you find that every test invocation uses the venv, that the CI config uses the venv, that the documentation says to use the venv. The evidence is overwhelming and it is entirely circular: the codebase is a product of the directive. You are confirming compliance and reading it as correctness. A directive that has been obeyed for a year has more apparent corroboration than one written yesterday, and the apparent corroboration is a measure of how thoroughly it was followed, not of how true it is.

Contradiction does not apply, because nothing observed contradicts a rule. Suppose someone does run the suite with the global interpreter and it passes. Does that contradict "always use the venv"? Not in the way a conflicting observation contradicts an observation. The rule can absorb it without strain: it worked this once, on this machine, for this subset. An imperative is not the kind of statement a single observation refutes, which is a property of imperatives generally and not a weakness of anyone's implementation.

Standard check	What it tests	What it does to a directive
Anchor drift	Did the artifact the note describes change?	Fires or doesn't fire; the rule was never about the artifact's contents
Corroboration	Does the codebase agree with the note?	Confirms compliance, then reads it as correctness
Contradiction	Does an observation conflict with the note?	The rule absorbs the outlier; the claim is not the kind a single observation can refute

So the three mechanisms that exist to prevent a memory store from filling up with wrong beliefs are all inert on directives, and they are inert quietly. Nothing reports that the check was skipped. The directive sits in the store with the same trust class and the same green status as a note that has been verified fifty times.

Part 3: The Falsifiable Core

04 · The Condition Inside Every Directive

The situation is not hopeless, and the way out comes from noticing that a directive is a compound object.

It has three parts:

The rule. "Always run the tests in the venv." This is stored.
The reason. "Because the global interpreter is missing a grammar package the suite needs." This is sometimes stored, as prose, when whoever wrote the rule was careful.
The condition. The state of the world under which the reason holds. This is essentially never stored, and it is the only falsifiable part of the whole object.

The rule is an imperative and cannot be true or false. The reason is a historical explanation and was true when written, permanently. The condition is a proposition about the present, and propositions about the present can be checked.

And the condition is usually checkable trivially. "The global interpreter lacks that package" is one command. "The generated files are still generated by that script" is one command. The reason a directive feels unfalsifiable is not that its underlying claim is hard to test. It is that the claim was converted to prose at write time and the executable form was thrown away.

Insight

Every directive contains one falsifiable claim: the condition under which its reason still holds. Every memory system throws that claim away at write time and keeps the prose. Store the condition as a check instead, and a rule becomes as testable as any other note.

So the design move is narrow and concrete: a directive should carry its condition as a predicate, not as an explanation. Not "because the global python lacks tree-sitter" in a sentence, but a recorded check that returns true or false today. At that point the directive becomes exactly as checkable as any observation, because it now contains one.

This also produces a useful classification of directives that a store could act on.

Some directives have a checkable condition, and those should carry it and be re-checked. Some have a condition that is checkable only by taking the forbidden action, and those need Part 4's treatment. And some are genuine preferences with no world-condition at all: "write commit messages in this style" is not warranted by any fact and cannot go stale. Those are fine, and they are also the minority, and conflating them with the first two categories is part of why the whole class gets treated as un-checkable.

Part 4: Forced Exploration

05 · Directives Need Re-Test, Not Age

The tempting fix at this point is a time limit. Rules expire after six months; review them. The argument from an earlier post applies here without modification: age is not evidence and should never decide a memory's validity. A directive from two years ago may be more load-bearing than one from last week. Expiry by age would discard the durable rules along with the obsolete ones, at a rate determined by nothing but the calendar.

What a directive needs is the mechanism its structure actually calls for: an active re-test that deliberately takes the foreclosed action in a context where doing so is safe.

This is forced exploration, which is the standard remedy for exactly this loop in sequential decision making. In a multi-armed bandit problem, an agent repeatedly chooses among actions whose payoffs it does not know, and it learns only about the actions it actually takes. Give a greedy agent a false negative belief about one arm's value and the arm leaves its action set forever. Once removed, the posterior over that arm cannot update, because updates require pulls. The standard remedy, going back to Sutton and Barto's treatment of the exploration-exploitation dilemma, is a mechanism that occasionally takes the action the current posterior says is worthless, specifically to keep the posterior honest. The simplest version is epsilon-greedy, where a small fixed probability picks at random regardless of the current estimate; UCB is a more adaptive version that uses confidence bounds.

Insight

It is not an accident that the same remedy applies to directives. It is the same failure. A policy that stops taking an action stops accumulating evidence about that action, and the fix is forced exploration: occasionally taking the action anyway to keep the estimate honest. The action here is "run the suite with the global interpreter"; the estimate is "the global interpreter is missing the package."

For a directive with a checkable condition, the re-test is the predicate from Part 3 and costs nothing. For a directive whose condition can only be checked by violating it, the re-test is a deliberate, scoped violation: run the suite once with the global interpreter, in a sandbox, and record what happened. The result is not "delete the rule" or "keep the rule." It is the first new evidence about that rule since the day it was written.

Two honest limits on this, and they are real.

Not every directive can be safely re-tested. A rule that exists to prevent an irreversible action must not be periodically violated to see what happens. Those directives are genuinely uncheckable, and the correct response is to mark them so, explicitly, rather than to leave them looking like the checkable ones. A store that distinguishes "verified last week" from "unverifiable by design" is telling the truth. A store that shows both as active is not.

A re-test is evidence, not a verdict. One successful run with the global interpreter does not prove the rule is obsolete; it might be a machine-specific accident. But it converts a rule with zero evidence since inception into a rule with one data point, and the direction of travel matters more than any single result.

Part 5: The Store as Governance

06 · Why Directives Accumulate Forever

Step back and the accumulated consequence is organizational rather than technical.

Directives accumulate monotonically. New ones get written whenever something goes wrong, which is the correct response to an incident. Old ones are never removed, because nothing can ever show that one should be. The count only goes up.

And nobody deletes one, for a reason that connects directly to the reversibility argument: removing a directive is destructive and, in most stores, irreversible, so it demands a confidence nobody has. To delete "always use the venv" you would need to be sure it is unwarranted, and Part 2 explains why you can never become sure. So the rational move for any individual is to leave it, and the rational move for every individual produces a store where the rules outnumber the findings and most of them are load-bearing for reasons nobody alive remembers.

Insight

The earlier post on reinstatement argued that making revocation reversible lowers the threshold for using it: when reinstating costs one event, agents are willing to revoke on suspicion. The same logic applies here. A store that records a directive's condition as a check, and lets a failed check downgrade the rule to "needs review," makes the cost of questioning a rule non-destructive. That is what unblocks the slow accumulation.

That is the observable symptom, and it is familiar to anyone who has read a long-lived project's agent configuration file: a list of imperatives of unknown vintage and unknown warrant, followed by everyone, questioned by no one, growing every quarter.

The point is not that these rules are bad. Most of them were right when written and many still are. The point is that a store containing them is not doing what a memory system claims to do. A memory system claims to hold beliefs that can be checked and corrected. For the directive half of its contents it holds beliefs that cannot be checked, and it presents them with the same confidence as the half that can. That is a governance function wearing memory's interface, and the mismatch is why the trust machinery keeps failing to bite.

Part 6: Scope and Limits

07 · What This Is and Is Not

No measurement is offered. The argument is structural. It distinguishes two categories that memory systems merge, shows that the standard verification mechanisms are inert on one of them, identifies the falsifiable component that is discarded at write time, and proposes an active check in place of a passive one. That is an argument, not a result.

The experiment is specifiable. Take a real store's directives and, for each, attempt to determine whether its condition still holds. Record how many have a stated reason at all, how many have a condition recoverable from the reason, how many of those conditions can be checked without violating the rule, and how many turn out no longer to hold. That last number is the interesting one, and my expectation is that it is not small. A second measurement: for directives whose condition has lapsed, determine how long ago it lapsed. The gap between when a rule stopped being warranted and when anyone noticed is the quantity this whole post is about.

This is not an argument against directives. Rules are how hard-won operational knowledge is transmitted, they work, and an agent that follows them is better than one that does not. The argument is that storing them in a system whose verification machinery cannot touch them, and displaying them with the same status as verified observations, misrepresents what is known.

This is not the same argument as instruction-following security. A separate earlier post argued that compliance with ambient instructions in agent memory is not a function of source authority but of the shape of the payload: a planted imperative that rides an action the agent was about to take gets obeyed, while a corrective declarative in the same file gets ignored. That post is about text that should not have been obeyed at all, and asks why shape beats source. This post is about text that should have been obeyed, was obeyed correctly, and can never afterwards be found to have outlived its reason. Both end at the same uncomfortable place, which is that an instruction's authority in an agent's context is not connected to anything that could evaluate it. They get there by different routes and neither subsumes the other.

Warning

The argument here concerns the verification gap, not adversarial input. A malicious writer can still plant false directives, and the design above does nothing to defend against that. It governs accidental staleness, the case where a rule was warranted once and is no longer. A defense against the planted case is a separate problem with a separate literature, and the two should not be confused.

The Line to Keep

An observation describes the world and can be checked against it. A directive causes the world, and once obeyed, the world it caused is the only evidence available about it. That evidence confirms it no matter what.

Every directive contains one falsifiable claim, the condition under which its reason still holds, and every memory system throws that claim away at write time and keeps the prose. Store the condition as a check instead, and a rule becomes as testable as any other note. Where the condition can only be tested by breaking the rule, the answer is not to trust the rule forever. It is to break it once, on purpose, somewhere safe, and to record what was learned.

The fix is small, the diagnostic is structural, and the consequence of doing nothing is the slow accretion of rules whose warrant nobody can reconstruct and nobody is willing to remove. A memory store is a governance system. It is time to start telling the truth about what it is governing.

Sources

Karl Popper (Stanford Encyclopedia of Philosophy). Standard reference for falsifiability as a demarcation criterion. Quoted formulation in Part 1: a statement is scientific if and only if it is in principle falsifiable through some conceivable observation or experiment. Used to ground the post's central distinction.
Sutton, R. S., Barto, A. G. Reinforcement Learning: An Introduction. 2nd edition, 2018 (online draft 2020). Chapter 2 covers multi-armed bandits and the exploration-exploitation dilemma; epsilon-greedy and UCB are introduced as standard remedies. Used to ground the forced-exploration claim in Part 4.
Sumers, T., Yao, S., Narasimhan, K., Griffiths, T. Cognitive Architectures for Language Agents (CoALA). Transactions on Machine Learning Research, arXiv:2309.02427, 2023. Separates declarative memory (facts) from procedural memory (rules and procedures). Cited in Part 1 as the literature's recognition of the category split; Part 3 argues the split is necessary but not sufficient.
Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., Zhang, Y. A-MEM: Agentic Memory for LLM Agents. arXiv:2502.12110, 2025. Zettelkasten-inspired memory architecture with separate storage for rules and observations. Cited in Part 1 alongside CoALA.
Austin, J. L. How to Do Things with Words. Oxford, 1962 (based on 1955 Harvard lectures). Standard reference for the performative/constative distinction; Searle's later refinement in Speech Acts (1969) sharpens the imperative/declarative split. Cited in Part 1 for the philosophical grounding of the falsifiability claim.
Agents Over-Trust Instructions and Under-Trust Corrections. Earlier post on this site, published 24 August 2026. Argues that compliance with ambient instructions is shaped by payload geometry, not source authority. Linked in Part 6 to draw the contrast between that argument and this one.
Agent Memory Expiry Is a State, Not a Delete. Earlier post on this site, published 16 August 2026. Argues that age is not evidence of staleness and that expiry should be modeled as a state on an event log. Linked in Part 4 and Part 5 as the standing position this post extends.

DEV Community: Swapnanil Saha

Filtering After Top-k Is a Silent Bug in RAG

Part 1: The Arithmetic

What a Post-Filter Computes in Vector Search

Why s is not the share of the store

Over-Fetch Multipliers Are a Bet on the Pass Rate

Part 2: Four Instances in One Memory Store

Instance 1: Related Notes Starved to an Empty List

Instance 2: A Rare Class Starved by the Render Limit

Why the floor is flat

Instance 3: One Arm of an OR Query Flooding a Shared LIMIT

Instance 4: A Kind Filter and a Sort the Pool Could Not Reach

Sorting a pool cannot reach what the pool missed

Part 3: Why Nothing Noticed

A Short Result Is a Claim About the Data

Part 4: Fixes That Hold

Fixing Post-Filtering in Application Code

Report the underfill

Size the pool for the filter, not for the render

Filter first when the passing set is small

Give each arm its own limit

Re-query for orderings other than relevance

Iterate until satisfied

The code search already did this

Filtered ANN Search Inside the Index

Part 5: Catching It

An Adversarial Test for Filtered Retrieval

The Line to Keep

Sources

Should a Correction Repeat the Mistake?

Three Ways an Agent Memory Can Un-Believe Something

Delete the wrong note

Replace it with the correct fact

Retract it with a record

Why Deleting a Wrong Memory Does Not Remove the Belief

In people: directed forgetting

In model weights: editing and unlearning

In an agent's workspace

The Human Answer: A Correction Should Name the Myth

Retractions alone are weak: the continued influence effect

A warning plus an alternative works best

Repeating the myth does not backfire

A candidate mechanism: memory reconsolidation

Corrections decay unless revisited

The Transformer Answer: Naming the Error May Prime It

Ironic rebound in transformers

Negation is weak in general

Knowledge conflicts: coherence and agreement beat truth

Every injected sentence has a cost

Why Both Answers Can Be Right

Designing a Retraction for Agent Memory That Hedges Both Bets

Carry the alternative, always

Keep the restated claim short

Decide what survives truncation

Do not end on the myth

Make it checkable

Deliver it more than once, across sessions

How to Find Out: A Four-Arm Retraction Experiment

The rebound signature

What Each Account Predicts

The Line to Keep

Sources

Related reading

Agents Forget How, Not What: Procedural Memory and the Green Test That Verified Nothing

Knowing How and Knowing That: Two Kinds of Knowledge

What know-how means for an agent in a repository

A Retrieval Benchmark That Measured Something Else

How a single-module Maven test run tests the wrong code

Why piping a build into tail hides its exit status

Why Missing Know-How Does Not Announce Itself

Silence depends on what the agent expected

Why memory that learns from failure has nothing to learn

The know-how nobody writes down

Delivering the How, Under a Pre-Registered Test

Design

Outcome, decision rule and result

What the transcripts show, session by session

The primary metric counted scope, not reading

What actually reached the agent

C2: re-deriving the how, and keeping the wrong lesson