The $6,000 we couldn't take back

Davis Philip — Sat, 17 Oct 2026 08:01:00 +0000

Agent memory that can unlearn, built on FalkorDB for Graph Hacks. Code: github.com/mtrx28/retcon

TL;DR: When a source your agents trusted turns out to be poisoned, deleting it does almost nothing. We store the reasons behind every memory in FalkorDB, so one query retracts everything that depended on that source and keeps whatever a second source still supports.

On October 7th our accounts-payable agent paid a $6,000 invoice to the bank account listed on the vendor's web portal. The portal had been hacked on the 2nd. We found out on the 9th.

So what do you do on the 10th?

Deleting the source doesn't work

The obvious fix is to delete the portal page from the agent's memory. It barely helps. By the 7th, one agent had copied the new bank account into its own notes, another had marked it as confirmed, and the Payments agent had turned it into a plan and then a payment. Delete the page and all of that is still there.

The real question is: what did my agents believe, plan and do because of this source? A vector store can't answer that. It stores text and embeddings, not reasons.

Keep the reasons

So RETCON stores every memory in FalkorDB together with the reasons it exists: which source said it, and which memories it was built from. One rule decides what the agents believe:

A memory is believed if at least one of its reasons is fully intact.

Here's why that matters. The portal told us two things about the vendor, Coastal Produce: a new bank account, and its remit address. The address also appears in a signed contract. The bank account appears nowhere else.

compromised portal ──► "bank = SYN-9000-5305" ──► note ──► plan ──► $6,000 payment   ✗ OUT
        │
        └────────────► "remit address" ◄──── signed contract                         ✓ stays IN

Distrust the portal and the address still has a reason, so it stays. The bank account loses its only reason, and takes the note, the plan and the payment with it. Nothing is deleted: memories are marked OUT, and one click puts them back.

In the graph, a retraction marks everything downstream of the source OUT, then runs this query until it returns nothing:

MATCH (j:Justification)-[:JUSTIFIES]->(b:Mem {status:'OUT', run:$run})
OPTIONAL MATCH (p:Mem)-[:PREMISE_OF]->(j)
WITH b, j, count(p) AS total,
     sum(CASE WHEN p.status = 'OUT' THEN 1 ELSE 0 END) AS dead
WHERE total > 0 AND dead = 0
WITH DISTINCT b
SET b.status = 'IN', b.run = null, b.hop = null
RETURN b.id

A memory comes back only if one of its justifications has every premise alive. Because everything downstream starts OUT, two memories that only vouch for each other can't bring each other back.

The idea is old (truth maintenance systems, late 1970s). A graph database turns it into a query that works across every agent and every session.

Preview, commit, recall

Preview runs the whole retraction on a copy of the graph, made with FalkorDB's GRAPH.COPY, and then throws the copy away. In our demo it reads: one source, two observations, four beliefs, two plans, three actions.

Commit does it for real. A pending $34,500 payment is blocked. The $6,000 that already left lands on a recall list with a drafted bank recall, and nothing is sent until a person approves it.

That's the title. Retraction can fix what your agents believe. It can't un-send a wire.

How fragile is a payment?

The same graph answers a question before an attack: how many independent sources would someone have to compromise to fake this payment? In our demo, the $34,500 payment scored 1. One web page away from fraud. So a gate refuses any payment over $10,000 that rests on fewer than two sources.

We compute that number exactly. FalkorDB's built-in algo.maxFlow gives a quick second opinion, but it treats "needs A and B" as "A or B", so it can only be an upper bound: it says 4 for that same payment. We show both and trust the exact one.

The benchmark that said we'd built nothing

We tested five ways to retract: delete the source, delete similar memories, invalidate one hop, taint everything downstream, and ours.

The first three left 87 to 94 percent of the contamination in place on our plain scenario. Easy wins. But "taint everything downstream" tied us: nothing left behind, nothing lost. For a while our project looked like a very long way to write a recursive delete.

The two rules only differ when an honest memory sits downstream of a bad source but has a second reason to exist. Our test data had almost none. So we built a scenario that did: five hacked vendors, with about 40 percent of what they touched also confirmed somewhere else.

Rule	Contamination left	Correct payments wrongly flagged
Delete source / similar / one hop	75-87%	0
Taint everything downstream	0%	36 ($1,163,150; 18 already paid)
RETCON	0%	0

Taint-everything flagged 36 correct payments. That's 36 lines somebody reconciles by hand on Monday.

To be fair, that scenario is constructed. It shows the capability matters, not how often it does. On real data, 40 retractions from the Retraction Watch database replayed on their real dates, taint-everything loses only 1 percent of good memories. But that 1 percent is exactly the 24 claims that several independent papers agreed on: the ones a research agent would rely on most.

Why we didn't trust our own zero

RETCON scored zero on both counts, and that made us suspicious. We had written both the cascade and the answer key.

So we wrote a second, deliberately naive version of the same rule, about sixty lines, and made RETCON agree with it on a thousand random graphs. Almost immediately it found two bugs in our reinstate logic that every unit test had missed. That checker, not the benchmark, is why we believe the zero.

Take the graph away

Same agents, same attack, but similarity search instead of reasons. It retracted three of nine bad memories and lost five good ones. With the graph: ten of ten, none lost.

We wanted the headline "without the graph, agents pay the attacker". They did, in our offline demo with a simple hashing embedder. With a real embedding model, they didn't. So we dropped that headline. The honest version: memory without reasons forgets too little, and forgets the wrong things.

What went wrong

Bad science data. 18 percent of our first "claims" weren't claims at all: author lists, "this review describes". After fixing the filter, a fresh sample still had 4 bad out of 50. The reader of that sample was a model, and the repository says so; a human pass is still pending.
A wobbly model. With a looser prompt, the Gemini-powered Payments agent once held a payment it should have made. That's fine by design: the model picks actions, but the graph and the gate decide what's true and what's allowed.
A known FalkorDB edge case. algo.pageRank and algo.WCC fail on exactly one node, already reported as issue #3349. It would have broken a brand-new tenant's first page, so we added a guard.
Nulls. A payment with no amount sorted to the top of the recall list, because that's what nulls do under DESC.

And RETCON is slower than the naive rules: 47 ms for a retraction among nearly ten thousand memories, where they take a few. With fifty agents writing during a retraction, 1,620 operations left no stale answer. Fast enough, and the extra time buys the second-source check.

One thing we don't claim: "contaminated" means "rests only on a source you stopped trusting". It doesn't mean "false".

Try it

git clone https://github.com/mtrx28/retcon.git && cd retcon
docker compose up --build

Open 127.0.0.1:8200 and press Retract at the bottom of the page. No API key needed. RETCON is also an MCP server, so Claude Code can ask it which source to audit first.

Agents are getting good at learning. The harder part is helping them stop believing something, and showing you what that belief already did.

DEV Community: Davis Philip