DEV Community

Cover image for How can I let an AI assistant learn from user corrections while keeping context lean?
Edward Izgorodin
Edward Izgorodin

Posted on Originally published at mnemoverse.com

How can I let an AI assistant learn from user corrections while keeping context lean?

A correction appended beside an old memory leaves two statements that disagree, and both can enter the context. A correction discarded as a duplicate leaves no correction at all.

TL;DR

  • Write a correction as a replacement that names the memory it replaces, not as a note beside it. The store can then hold one current answer per subject, and the context receives one line instead of two conflicting lines.
  • Key each memory by what it describes, using a subject plus a predicate, and treat its sentence as replaceable. A correction then finds its target by key rather than wording similarity, following the scheme Max Quimby described.
  • A correction can disappear during deduplication, during the assistant's turn, or during retrieval for a broad question. UyLe encountered all three while building a chatbot memory and wrote, "Each needed its own fix."
  • Lean context depends on what the assistant reads by default, not on deleting history. Serve the current version and retain the replaced versions as a record of what changed, when, and on whose word.

Mnemoverse (the author's company) makes one of the memory systems named below.

A correction is a write that says an earlier memory was wrong or out of date and names the memory it replaces.

A memory key is the stable identity of what a memory is about, such as a subject and a predicate, kept apart from the sentence that states it.

The current version of a memory is the one no later write has replaced; it is the only version a default read should put into the context.

Why does appending corrections make the context grow and the answers worse?

Appending preserves both claims as current:

user:editor = "I use VS Code"
user:editor = "I no longer use VS Code. I use Zed."
Enter fullscreen mode Exit fullscreen mode

The assistant must resolve the contradiction each time it reads them. Retrieval can also return the older claim without the newer one.

Haoning Kan reported this behavior in a NeatMem run while "running PointWise + Rewrite with recall threshold 0.8". After a move from Seattle to Austin, he found that "two contradictory memories coexist in the store". In that run, only pairs above 0.8 received a model judgment. Similarity between the two statements fell between 0.71 and 0.75, so the judgment was skipped and the new statement became another memory. Kan explained the underlying problem: "Contradictory statements are often worded very differently (one says Seattle, the other Austin), so their similarity is naturally low."

Kiell Tampubolon described a related failure: "A correction written from a different angle does not just fail to link to the old entry, it can paraphrase the mistake into a new entry that reads as fresh insight." Replacement semantics cannot help if the write never identifies what it replaces.

Prompt space can impose a hard limit. The Hermes Agent documentation says, "Memory has strict character limits to keep system prompts bounded". As documented on 2 November 2026, MEMORY.md allows 2,200 chars (~800 tokens), while USER.md allows 1,375 chars (~500 tokens). Its documentation also states:

"Memory does not auto-compact: when a write would exceed the limit, the memory tool returns an error instead of silently dropping entries."

Replacing one value uses less prompt space than appending each correction. For a broader treatment of this rule, see How do I stop my AI assistant giving outdated answers after code refactoring?

How should a correction be written so it replaces the old memory?

A correction needs four fields: the identifier or key of the memory being replaced, the new value, the source of the correction, and the time of the correction. The write should create a replacement record and link it to its predecessor. It should not silently edit the earlier record in place.

Aryapathak documented what happens when a repair changes the sentence but retains the bad support beneath it: "My memory repair had fixed the sentence and kept the lie underneath it, at confidence: 1.0." Their correction flow marks the earlier claim as replaced, creates a linked replacement, and updates dependent decisions. The reason is provenance: "A correction isn't an edit. It's a new piece of evidence from a different source, a human reviewer, and it needs its own provenance like any other source."

A correction should also name the version it read. The ai-memory source describes an optional expected_page_id that writes only when that page remains current. If the target changed first, "a failed precondition writes nothing and returns reason: "precondition_failed" with the current version". That visible failure prevents one correction from overwriting a later one.

The replacement must be narrow enough. Hermes Agent documents why it stopped offering a whole-entry replacement action:

"There is no replace action. It overwrote the whole matched entry, and models used it as a find-and-swap, which wiped the other facts in multi-fact entries."

Its patch action instead "replaces only the old_text span inside the matched entry and keeps everything else." This matters when one entry contains several facts and only one is wrong.

The word update can hide distinct operations. Memory API response shape: Why one update result is not enough explains why callers need the operation's actual outcome.

On the Mnemoverse REST API, a write can pass supersedes to mark this write as the correction for one or more earlier atoms. The target remains fetchable by ID and gains a pointer to the replacement. A write carrying that field is not refused by the novelty check. The caller must first read and identify the target. One limitation matters here: read-side filtering is off by default and must be enabled by the organisation. Until enabled, replaced records can still appear in reads, marked by superseded_by. Long correction histories also consume storage quota, even when filtering later hides them from search.

What should a memory be keyed on so a correction finds its target?

Max Quimby gave an operational rule:

"We key a memory on a stable slug (a normalized subject + predicate) and treat the natural-language body as replaceable."

For example:

key: user:editor
value: "Zed"
replaces: atom_older_editor
source: user
Enter fullscreen mode Exit fullscreen mode

The key identifies the question. The value answers it. A write looks up the key before deciding whether to append, replace, or reject.

Quimby's process merges new wording into the existing record instead of appending another synonym. A model is called only when values conflict, rather than whenever wording differs. The key alone cannot decide whether a statement narrows an earlier fact or contradicts it. As Quimby observed, "works at a startup" vs "works at Acme" aren't dupes and aren't contradictions, and an overwrite loses information either way. The write operation must express whether the new value refines or replaces the old one.

Keys create their own failure mode. Heinrich Neb, who disclosed that he works on a competing memory layer for coding agents, wrote: "Misspell the slug and you have created a second truth rather than fixing the first, and nothing anywhere objects." He added that in his own layer, closely related keys such as deploy:cache and deploy:caching may remain separate, with nothing to notice that they name the same subject.

Ken W Alger proposed narrowing predecessor candidates by stable subject, claim type, or authority domain before semantic comparison. He also argued that the system should preserve what it searched. Otherwise, "no predecessor found" can mean either "there wasn't one" or "we looked in the wrong place."

A practical sequence is:

  1. Normalize the subject and predicate.
  2. Search for a predecessor by that key.
  3. Use similarity as a secondary check for key spelling or normalization errors.
  4. Return no predecessor found with the searched key and scope.

Where do user corrections get lost, and how do I check each place?

UyLe summarized the problem directly: "Corrections were lost three ways." This was an observation from UyLe's hippo project, not a cross-system measurement. UyLe reported that since September 22, 12 people had used the chatbot on the web and Telegram, and it had stored 26 memories for them on mainnet.

Place What happens Check Passing result
Deduplication on write The correction is discarded as a copy, or admitted beside the old claim Write a fact, a similarly worded correction, then a differently worded correction. List every record for the subject One current record. The old record is marked as replaced
The assistant's turn The model acknowledges the change but stores nothing After the acknowledgment, inspect the store rather than the reply A new record names the record it replaces
The broad question A broad query recalls the old fact but misses the correction In a fresh session, ask both a broad and a specific question Both answers use the correction

UyLe's concrete deduplication observation was: Dedupe discarded “I no longer use VS Code” as a duplicate of “I use VS Code”, 0.24 apart, because distance cannot see “no”. That value describes one observation in that project. It is not a general threshold.

The demo also changed one memory, discarded the conversation, and asked questions in a fresh session. UyLe reported that 0 of 5 answers stated pnpm without bun.

A fresh session matters for systems that snapshot memory. Hermes Agent documents: "The system prompt injection is captured once at session start and never changes mid-session." A fact saved during the current session becomes visible to the next session, not to another session that was already running. Testing only in the writing session therefore proves neither success nor failure.

There is a fourth check before storage: verify the correction itself. Mihai Perdum warned that a diagnosis can be wrong while still looking plausible for the case that produced it. Ali Khater recommends sending low-confidence contradictions to review rather than silently replacing existing evidence.

For deeper write-path tests, see Write Gate: What Never Gets Stored and RAG vs agent memory.

What happens to the old version when a correction is applied or undone?

An undo operation must state which outcome it intends:

  1. Keep the replacement chain and mark a newer correction.
  2. Withdraw the correction and restore its predecessor as current.
  3. Withdraw both versions because neither remains valid.

These outcomes are not interchangeable.

The Hy-Memory account below is the author's reading of its source at commit fcb1bcd, with the same behavior at 8f1c404, both read on 2 November 2026.

The update method writes node.content = content, replacing the text and embedding on the same record. It writes a history entry with change_reason="client.update" when history is enabled by configuration. In the author's reading of that method, it sets no supersession field.

A separate data-model comment, in memory.py at the same commit, says, in the author's translation, that fact changes are tracked through status transitions and a SUPERSEDED_BY edge rather than direct content edits. The delete path can repair a chain by restoring the predecessor as current, logging chain repair: restored {pred_id} as chain head.

An update can preserve old text in the optional history journal without leaving a replacement chain on the read path. Deleting a chain head can restore the earlier belief. Restoration is correct when the replacement was the mistake. It is wrong when both claims have become invalid.

The ai-memory source takes another approach. Its README says, "The source of truth is a git-backed wiki of ordinary .md files". The database is a rebuildable derived index. Marking a page stale or wrong lowers its salience without deleting it, and a later rewrite clears that mark. Administrators can list the Git checkpoints that changed a path.

Raghu Juluri condensed the design requirement: "Memory your agent writes is memory you will eventually need to correct. Pick the format you can correct."

For six more source-based examples, see When a newer memory contradicts an older one, what happens to the older one?

How do I keep context lean without losing the record of what was corrected?

Default reads should return current records. Historical reads should reconstruct earlier states. Ali Khater describes this split directly: "A contradiction should usually close the old fact's validity window and link it to the replacement, while the default read returns only currently valid facts."

Keep each current memory compact. UyLe's project stores each memory as one line with a type tag. The history can retain the longer account of who changed what and why.

Some corrections apply only to one kind of work. Hermes Agent's documentation says that for a location the agent needs on every run of a recurring task, a skill "is often the better home than a memory entry" because "it loads only when relevant and does not compete for the 2,200-character budget." The documentation discusses locations for recurring tasks. Applying the same placement rule to task-specific corrections is this article's inference.

The correction record should remain available in storage. It answers why the assistant changed its behavior and helps prevent a rejected decision from returning. See AI agent memory and decision records for that separate problem.

Automated review may also separate a proposed correction from an active one. Hermes Agent's source says background patch and remove operations are never applied unattended. This is an example of why "stored" and "currently applied" need separate states.

How do I test this on my own assistant?

Write the expected result before each run.

  1. Store I use VS Code. Then write I no longer use VS Code, I use Zed. Expect one current editor record, one replaced record, and a replacement link from the new record to the old one.
  2. Repeat with My editor is Zed now. If the system uses keys, also introduce a key typo. Check whether it finds the predecessor or creates a second current answer.
  3. Start a fresh session. Ask Which editor do I use? and What do you know about me? Expect Zed in both answers. VS Code should appear only in a history query.
  4. Undo the correction. Before running the operation, write down which version should become current. Compare the stored result with that expectation.
  5. Inspect the assembled context and count the editor claims. Expect one current line.

Illustration, not data: the editor names are examples.

What this article does not claim

No performance figure is presented for any system. The numbers quoted above are practitioners' own runs and observations, with the bounds they stated, and none was rerun here.

The systems named are examples, not a complete list, and none is ranked. Statements about Hy-Memory, ai-memory, and Hermes Agent source code are the author's reading of the commits linked above, not vendor statements. Those repositories can change after the cited commits.


Sources and method. Every quotation was read on 2 November 2026 with two instruments, a browser-like request and a reader service with its cache off (dev.to also through its API, Medium also through RSS, GitHub files also raw and through jsDelivr at the commits named); a made-up address on each host answered 404 as a control, except Medium, which answers 200 to any address, so there only the post text counts. The text describes those pages and files as of that date, and the numbers are the practitioners' own, with the bounds they stated. A longer version, with every source, is in the Mnemoverse documentation library: How can I let an AI assistant learn from user corrections while keeping context lean?

Edward Izgorodin · Mnemoverse · 2026-11-02

Top comments (1)

Collapse
 
aifrontierpost profile image
AI Frontier Post •

Your 2,200 character MEMORY.md limit shows replacement alone does not fix bounded prompts, since a corrected entry still occupies that same capped space and can push out other current facts. The 0.24 dedupe distance is the sharper risk, because a correction that close to the original can be discarded before replacement logic ever sees it.