I'm a self-taught robotics engineering student in Bengaluru. I built Neo, a fully offline AI assistant that remembers facts about you across sessions. It runs llama3.1 through Ollama, stores vectors in LanceDB and facts in SQLite, and does web search through a self-hosted SearXNG. No cloud, no API keys.
The problem
When you tell the assistant "I stopped liking chess", it has to decide what that means. Is it a new fact, an update to an old one, or a retraction?
My first design used one rule: if cosine similarity with an old fact is above 0.8, treat it as a contradiction.
What I measured
I tested 10 real sentence pairs with my actual embedding model through Ollama. The 0.8 rule missed paraphrases and updates. Unrelated pairs scored only 0.29 to 0.41, so the useful range sat much lower than I had assumed.
I also found an earlier mistake. My retrieval eval was measuring L2 distance, not cosine. After fixing the metric, Recall@k came out at 0.95 and MRR at 0.925.
The fix
I stopped asking cosine to make the decision. Now it only finds candidates, using a low threshold of about 0.5. Then the flow is:
new fact
↓
find candidates (cosine ≥ 0.5, active facts only)
↓
no candidates → ADD
candidates found → LLM returns {action, target_id}
↓
code validates the answer
↓
write to SQLite + LanceDB
The LLM picks one of ADD, UPDATE, DELETE or NONE, plus a target id. Plain code then checks the answer:
- An unknown or malformed target id falls back to ADD.
- An UPDATE whose target text is identical to the new fact becomes NONE.
The model proposes and the code decides what is allowed. This mattered because llama3.1 sometimes returned UPDATE for a fact that already existed, or picked the wrong target. Prompt tweaks helped, but the guards in code are what made it reliable.
What I'd tell others
- Test your threshold on real sentence pairs before trusting it.
- Check which metric your eval actually uses (L2 vs cosine).
- Keep the LLM's output narrow and validate it in code.
- Write tests with fake LLM calls so you can check the gate logic without Ollama running.
Honest notes
v1.9 was mostly written with Claude and turned out unstable, so I rebuilt v1.9.5 by hand from v1.8.5 to understand every part. The README credits the AI help. Known limits are listed in KNOWN_ISSUES.md. For example, comparative preferences aren't extracted, and follow-up web queries lose context. It also needs hardware that can run an 8B model locally.
Repo: https://github.com/Neo-X7/neo-AI
How would you handle contradicting facts in a memory system? I'd like to hear other approaches and how can I improve things in the future.
Top comments (1)
The fallback from an unknown target_id to ADD is worth testing with the chess retraction example. If the model correctly recognizes an update but returns an invalid identifier, adding the new statement can leave both active facts in memory: likes chess and stopped liking chess. The malformed decision becomes a durable semantic conflict.
I would distinguish a valid ADD decision from an invalid edit proposal. The latter could remain pending for another bounded resolution step, while the existing active fact stays explicit. A fake-LLM test returning UPDATE with an unknown ID would show whether validation preserves uncertainty or silently converts it into a new assertion.