All numbers in this post come from reproducible evaluation scripts. Protocol: full LoCoMo benchmark, 10 sessions, 1536 answerable QA pairs, recall@10. Public repo at commit
ad04fb2.
The Human Brain Does Not Organize Memories While Awake
Here is a well-known fact from neuroscience: memory consolidation happens during sleep. Events from the day are first stored in raw form in the hippocampus; once you fall asleep, the brain starts "replaying" them — connecting isolated fragments of experience and stitching them into the existing network of knowledge. When you wake up feeling that you "figured it out," it is actually your brain doing the work for you overnight.
Mainstream memory engines have no such mechanism. Write is the end; retrieval is everything — from the moment a memory hits disk, it is dead, waiting to be looked up.
Last week we installed this "sleep module" in NylonME. Internally it is called Async Reflection. The result: full-benchmark LoCoMo recall@10 went from 84.6% to 85.3%, and the hardest category — commonsense reasoning (Cat3) — reached its all-time best of 60.9%.
This post explains three things: why Cat3 needs a new mechanism, how async reflection works, and which two paths we tried that turned out to be wrong.
First, the Medical Record: Why Cat3 Is the Last Weak Spot
Our LoCoMo scores have climbed steadily, but Cat3 (commonsense reasoning) has always been the weakest of the four categories. In the run that hit 84.2% overall, Cat1 was 80.5, Cat2 86.6, Cat4 88.0 — and Cat3 only 53.3.
The cause lies in the nature of Cat3 questions. A typical example:
The dialogue says "she quit coffee after getting pregnant," and another segment says "she ordered a decaf americano yesterday." Question: "Why does she not drink regular coffee now?"
The answer requires a piece of knowledge nobody ever said out loud: pregnant women should avoid caffeine. That sentence is not in the dialogue. No retrieval technique — lexical, vector, or graph diffusion — can find an answer in words that were never spoken. No matter how strong the retrieval system is, it cannot retrieve a bridge that does not exist in the corpus.
This is not a retrieval problem. It is a knowledge gap problem. Either give up, or make the engine build the bridge itself.
Async Reflection: Consolidating Memories During Idle Time
We chose the second option, with one strict engineering constraint: bridge-building is heavy work and must never sit on the write path.
That gave us the full async reflection pipeline:
WeaveSession writes session facts
→ reflection task enters the queue (occupies no request thread)
→ engine idle for 10 minutes (NYLON_REFLECT_IDLE_SECS=600)
→ background task starts: LLM distills 1-3 neutral commonsense facts from the session
→ each fact becomes a special node, linked to every fact of that session
A concrete example: for a dialogue, the LLM distilled bridge nodes like "pregnant women should generally avoid caffeine" or "people on business trips usually value window seats" — neutral, self-contained world knowledge that contains no private information.
These nodes have three special design properties in the graph:
-
Tagged
__world_knowledge__: they exist as an intermediate layer for graph diffusion and never enter the final recall results themselves — a bridge is for walking on, not for looking at. - Low confidence, low tension, slow decay: confidence 0.6, tension 0.6, decay rate 0.01 — they are auxiliary structure and should never overpower real memories.
- Edges of weight 0.8 to the whole fact bundle: stronger than auto-built edges (0.5), weaker than explicit inter-layer edges (1.0) — enough to conduct activation, not enough to steal the show.
At retrieval time, two previously isolated fact nodes ("quit coffee," "decaf americano") are now connected through a shared commonsense bridge node. A Cat3 question activates one of them as a seed; diffusion travels across the bridge to the other, and the context is complete.
Why must it be async? Because LLM distillation is slow and expensive. Put it on the synchronous write path and write latency doubles, while handing LLM providers a hand around your throat. And reflection is not urgent by nature — consolidation happens offline, not at the moment of experience. Let the engine think slowly when it is idle, and hang the results up when they are ready. This continues our old principle: the LLM is an enhancement, not a dependency; if the LLM goes down, the engine keeps reading and writing.
The Data: 85.3%, and One Piece of Key Evidence
Full benchmark: 10 sessions, 1536 questions, official results:
| Category | Score |
|---|---|
| Cat1 multi-hop reasoning | 84.8% |
| Cat2 temporal | 86.6% |
| Cat3 commonsense reasoning | 60.9% (previous best 59.8%) |
| Cat4 single-hop detail | 87.6% |
| Overall recall@10 | 85.3% (1310/1536) |
Beyond the numbers, there is a more interesting piece of evidence. In this experiment the seed recall for Cat3 was 64.1% — lower than the keyword-expansion scheme we had tried (66.3%) — yet the final recall ended up higher.
This shows the gain does not come from "finding more seeds." It comes from the graph structure itself after bridging: the seeds are the same seeds, but there are more places you can reach from a seed. Mechanism and data line up, which is why we dared to lock in this result.
(A pitfall worth recording: always probe the API once before running an LLM evaluation. When the DeepSeek account balance runs out it returns HTTP 402, session fact writes silently become +0, the whole benchmark score is void, and an hour of compute is wasted.)
Two Failed Paths, Worth Publishing Just the Same
The tradition of this series: the failure list is as long as the success list. After locking in 85.3%, we tried two "obviously better" directions. Both died:
Failure one: precise linking. Intuitively, a bridge should not link to every fact in a session — only the few it truly supports. We had the LLM output source_fact_indices for precise linking. Result: 85.1% overall, -0.2 total, Cat3 -2.2. Rejected. The lesson is counterintuitive: the value of a bridge comes from its connectivity. Narrowing the links narrows the bridge. Coarse linking is noisy, but it contributes more to reasoning.
Failure two: confidence-weighted edges. Weighting bridge edges by the confidence the LLM reports should, in theory, make "reliable bridges" louder. Quick 2-session test: 78.8%, no positive gain. Withdrawn.
After two rounds of failure, our judgment is: stop piling up prompt tricks and linking tricks. The next optimization surface is the quality of the bridge text itself (dedup, filtering out vacuous bridges — a "people have preferences" bridge is nonsense and must be blocked) and the trigger policy (switch from "reflect on every session" to "only reflect on sessions with high perplexity and low recall," saving LLM calls). These are already queued on the engine optimization branch.
The Full Picture Now
With async reflection, the NylonME memory lifecycle is complete:
- At write time: LLM weaving and decomposition, six-thread structuring, dual-layer graph writes (raw text + abstracted facts)
- At idle time: async reflection, LLM-built commonsense bridges, isolated facts stitched into a network
- At retrieval time: dual seed channels → adaptive-depth diffusion (bridge nodes as intermediate layer) → tension scoring → vector reranking
- Throughout: natural tension decay, conflict detection on write, automatic fallback if the LLM is absent from any step
From the 47.1% lexical baseline to 85.3% — every point of the 38-point gain has a controlled experiment and a public script behind it. All evaluation switches are environment-variable-driven (NYLON_WORLD_BRIDGES_ASYNC=1 reproduces this result), the code is at ad04fb2, and we welcome being proven wrong.
Cat3 at 60.9% is still a weak spot, and we do not intend to hide it. But the account of the direction is settled: what a memory system is missing is not stronger retrieval — it is the knowledge that retrieval cannot reach. And the way to fill it is to let the engine do the thinking for you while it sleeps.
The human brain spent hundreds of millions of years evolving sleep. The memory engine learned this lesson in one week.
NylonME is open source at github.com/nylon-memory/NylonME, Apache-2.0. pip install nylon-sdk is also live on PyPI.
This is post #9 in the NylonME technical blog series. Previous posts: 01 Phase 1 benchmark / 02 The Memory Engine Is the Core of AI Agents / 03 The Second Half of AI / 04 vs TencentDB-Agent-Memory / 05 From 47 to 84, the Full Record / 06 Choosing an Agent Memory Engine / 07 Two-Minute Agent Integration / 08 Memory Engines, the Next Database. Performance numbers are self-measured by the author (2026-08); evaluation scripts are open-sourced with the repo and reproducible.
Top comments (0)