Every agent memory layer can store a fact. Over a weekend I went looking for one that could answer a different question, and could not find it:
A document you ingested three weeks ago turns out to be wrong. What does your agent now believe because of it?
In a vector store the honest answer is nothing. The bad fact was embedded,
reasoned over, and the conclusions were written back as new memories with no link to where they came from. You can delete the source document and the agent will carry on asserting the conclusions, confidently, and cite them.
That is not a retrieval problem. It is a graph problem, so I built it on one. The result is Recant, built for Graph Hacks with FalkorDB.
The agent says yes, and the panel on the right walks each claim it used back to the document that stated it. That chain is what makes retraction possible: when one of those documents turns out to be wrong, the graph knows exactly what else has to go.
If you would rather watch the cascade than read about it, here is the whole thing in three minutes:
There is also a live deployment you can click
through yourself. It starts in its September state, so you can run the demo
end to end.
The setup
I wrote a small synthetic corpus for a fictional company. Seven documents from September: an infrastructure inventory, a load test report, a platform audit, a Black Friday capacity plan, a decision record, an on-call rota, a vendor contract. There is one deliberate trap in them. The load test measured a sustained ceiling of 1,200 requests per second, and four later documents quietly build on that number.
Ask the agent "are we ready for Black Friday?" and it says yes, with a chain of reasoning that walks back to that load test. It is not being careless. That genuinely is what September's documents say.
Then October's incident and postmortem arrive. The load test ran against a
staging database with a ten times smaller product catalog and a pre-warmed
cache. The number does not transfer. A corrected test measured 520 rps.
Now: what does the agent believe?
Provenance as the data model
The schema is deliberately small.
(:Source)-[:STATES]->(:Claim)
(:Claim)-[:DERIVED_FROM]->(:Claim)
(:Claim)-[:ABOUT]->(:Entity)
(:Claim)-[:CONTRADICTS]->(:Claim)
(:Answer)-[:BASED_ON]->(:Claim)
Two decisions carry most of the weight.
Claim ids are a content hash of the normalised text. Two documents asserting the same fact converge on one node with two STATES edges. Corroboration is structural rather than a similarity heuristic.
DERIVED_FROM points from a conclusion to its foundation. Retraction walks it forwards, and PageRank accumulates weight backwards onto the foundations. One edge direction serves both.
Retraction then becomes a traversal. I run it as a fixpoint of single hop writes rather than one big variable length write, because that terminates safely on cyclic reasoning and gives every claim an honest hop distance from the retraction:
MATCH (d:Claim)-[:DERIVED_FROM]->(p:Claim)
WHERE coalesce(p.tainted, false) = true AND coalesce(d.tainted, false) = false
SET d.tainted = true,
d.taintDepth = coalesce(p.taintDepth, 0) + 1,
d.taintReason = 'derived from a retracted claim: ' + p.text
RETURN count(d) AS n
Loop that until it returns zero. On my demo graph the entire cascade settles in single digit milliseconds.
Past answers inherit the damage, which is the part that makes it feel real:
MATCH (a:Answer)-[:BASED_ON]->(c:Claim)
WHERE coalesce(c.tainted, false) = true
WITH a, collect(c.text) AS broken
SET a.stale = true,
a.staleReason = 'relies on ' + toString(size(broken)) + ' retracted claim(s)'
RETURN count(a) AS n
The answer the agent gave you two minutes ago gets struck through.
The first thing I got wrong
My headline feature was supposed to be that corroborated claims survive. A claim should only collapse when every source that asserted it has been retracted:
MATCH (src:Source)-[:STATES]->(c:Claim)
WITH c,
count(src) AS total,
sum(CASE WHEN coalesce(src.retracted, false) THEN 1 ELSE 0 END) AS retractedCount
WHERE total > 0 AND retractedCount = total
SET c.tainted = true, c.taintDepth = 0
RETURN count(c) AS n
That is a comparison over the edges into a node, and it has no equivalent in a key value store. I was pleased with it. Then I dumped the graph the extractor had actually built and found this edge:
[infra-audit] Checkout API runs on 3 application nodes in eu-west-1.
<- [infra-inventory] Checkout API runs on 3 application nodes in eu-west-1 behind the shared ALB
The audit document restates a fact the inventory already established. Because the wording differed slightly the content hash differed, so instead of becoming a second STATES edge onto one node it became a second claim, and the model helpfully linked it as derived from the first.
That one edge destroyed the guarantee. Retracting the inventory would now
cascade into the audit's claim, even though an independent auditor still vouches for it.
The fix is not a better prompt. The fix is to stop trusting the model with graph structure. Before any DERIVED_FROM edge is written I compare the two texts and reject the edge if they are substantially the same assertion:
export function restatesSameFact(a, b, threshold = 0.7) {
const A = contentWords(a);
const B = contentWords(b);
if (A.size === 0 || B.size === 0) return false;
let shared = 0;
for (const w of A) if (B.has(w)) shared += 1;
return shared / (A.size + B.size - shared) >= threshold;
}
Restating a fact is corroboration. Only a genuinely new conclusion that reasons from a fact is a derivation. The same check later caught a "contradiction" where the model's own note read "these claims are redundant but compatible, not conflicting". It proposed the edge anyway. The guard drops it regardless of what the model says.
The second thing I got wrong, which turned out to be the interesting one
With corroboration working I ran the demo again, and the cascade was tiny.
The reason is worth sitting with. The bad 1,200 number had been copied into three documents: the load test that measured it, the capacity plan that quoted it, and the postmortem that debunked it. Because corroboration now worked properly, retracting the load test did nothing. Two other sources still asserted the number, so the belief stood.
My instinct was to call this a bug and tune it away. It is not a bug. It is exactly how wrong facts spread through an organisation, and the graph was telling me something true: deleting the document is not the same as withdrawing the belief.
So I added claim level retraction. You can retract a source, or you can retract a belief wherever it was asserted from, and both run through the same recomputation so they compose. The UI now shows you this before you commit to anything:
Blast radius of one belief
Checkout API safe sustained ceiling is 1,200 rps for the current 3-node configuration
asserted by 2 source(s): Load test report; Black Friday capacity plan
5 downstream inferences, 1 past answer at risk
Retracting the belief collapses it and everything standing on it, four hops deep: the capacity figure, then "can absorb the peak", then "no additional capacity required", then the decision not to scale up, and then the two things nobody thinks about. The eighteen thousand dollars that decision claimed to save. The on-call rota that went down to one engineer because of it.
Ask the same question again and it says no, citing the corrected 520 rps ceiling. Meanwhile "Checkout API runs on 3 nodes" is untouched, because the platform audit still vouches for it.
Testing semantics, not vibes
Demos are easy to fake, so the test suite builds its own fixture graph through the same queries the app uses with no LLM involved, and asserts the behaviour in both directions:
PASS cascade taints the root and all three inferences
PASS propagation took three hops
PASS hop distances are honest
PASS retracting one source leaves a belief a second source still asserts
PASS withdrawing the belief itself collapses it and everything downstream
PASS answer goes stale when its foundation collapses
PASS the most load-bearing claim is the one everything rests on
39 passed, 0 failed
The fourth one matters as much as the first. Proving the cascade fires is half a guarantee. Proving it does not fire when a claim still has support is the other half, and that is the one I would have shipped broken.
Graph algorithms as an agent tool
One more thing that fell out almost free. Running PageRank over DERIVED_FROM
ranks the beliefs the most reasoning rests on:
CALL algo.pageRank('Claim', 'DERIVED_FROM') YIELD node, score
Because edges point from a conclusion to its foundation, incoming weight lands on the load bearing claims. That is directly the question "which of my beliefs would hurt most if it were wrong", and it is roughly ten minutes of work when the algorithms live in the same engine as the queries.
What I would do next
FalkorDB holds thousands of isolated graphs in one instance, so per user memory is selectGraph(tenantId) rather than a schema change. After that, temporal validity on claims so superseded facts expire without manual retraction, and confidence penalties propagated along weighted edges instead of a boolean.
The deeper point I came away with is that agent memory is not a storage problem. Everyone can store. The hard part is what happens when something you stored turns out to be false, and answering that requires knowing what was built on top of it. That is an edge, and edges want a graph.
- Code: github.com/imarpanpatra/recant
- Live demo: recant.onrender.com
- Video walkthrough: youtu.be/0MGY-S4MVZI
I used an AI coding assistant during the build. The graph model, the retraction semantics and the two design mistakes above are mine, including the time spent finding them.
Top comments (0)