Disclosure: we are the authors of NylonME. This is naturally a bit of "the seller praising their own melons" — but we will lay the evaluation framework, the measured data, and when you should NOT choose us all out on the table. The choice is yours; our job is to give you all the facts.
Think First: What Are You Paying For?
In 2026, building an Agent, the memory layer is no longer a "whether or not" question — it's a "which one" question. But most teams' selection process is a mess: read three vendor blogs, run one demo, and decide. Six months later, when memory volume grows and query types get complex, they discover the ceiling of their choice was right above their heads the whole time.
So before talking products, let's talk about the evaluation framework. Whether a memory engine is worth choosing comes down to six dimensions:
- Memory model — how does it organize information? A flat list, tiered compression, or a graph network? This caps retrieval quality.
- Retrieval quality — are there measured numbers on public benchmarks? Full-run basis? Reproducible?
- LLM dependency — if the LLM goes down, times out, or raises prices, does your memory system still work?
- Production performance — write throughput, query latency, resource footprint: can it carry real load?
- Deployment and data sovereignty — can you self-host? Does data leave your data center?
- Long-term risk — license, vendor lock-in, project sustainability.
Below, we walk through the mainstream options along these six dimensions, then state clearly where NylonME stands on each — including our weaknesses.
The Five Types of Solutions on the Market
As of August 2026, Agent memory solutions fall into roughly five schools:
Type 1: DIY RAG assembly. A vector database + prompt templates — shippable in two weeks. The problem is the ceiling arrives just as fast: no concept of time, no conflict detection, no forgetting mechanism. "The user said last week they don't eat spicy food; this week they can't live without it" — in such a system, those two memories fight forever. Fine for validation; not for production.
Type 2: Managed memory cloud services (Mem0, Zep, etc.). Fastest to integrate, friendly SDKs. The price is data leaving your domain — your users' conversations flow to someone else's servers, which is an instant veto for finance, healthcare, government, and legal scenarios. And under the high-frequency read/write patterns of Agent architectures, the metered pricing model deserves a careful look at the cost curve.
Type 3: Framework-builtin memory (Letta/MemGPT family). Memory is deeply bound to the framework — works out of the box. But choosing the framework means choosing the whole stack: if you ever want to switch LLM orchestration frameworks, the memory layer gets kidnapped along with it.
Type 4: The tiered-compression school (TencentDB-Agent-Memory as the representative). L0–L3 progressive compression; the token-saving is real, and pip install with zero dependencies is genuinely convenient. We analyzed the structural problems in detail in Part 4 of this series: compression is lossy, and letting the abstract layer replace the raw layer in retrieval loses details — in our own ablation, keeping only the abstract layer cost 3.4 recall points.
Type 5: The heavy academic stack (MemOS as the representative). Neo4j + Qdrant combo, theoretically complete — but deploying it means maintaining a graph database cluster plus a vector database cluster. Small teams, beware.
Where NylonME sits: the graph-network school — single binary, self-hosted, with all benchmark numbers published. Let's go dimension by dimension.
Dimension 1: Memory Model — the Cap Is Set by Structure
The ceiling of a flat structure is "find what looks similar." The ceiling of tiered compression is "find what the summaries kept." The ceiling of a graph network is "find what is related."
NylonME weaves its graph with the six-silk model: every memory carries six silks — fact, relation, temporal, emotion, frequency, provenance — and memories are explicitly linked to each other through relation silks. Retrieval is "contextual resonance": diffuse from seed nodes along the graph, ranked by tension.
This structure delivers three capabilities no other model provides:
- Associative recall: "my internet is lagging" can activate a broadband-outage work order from three months ago, because they're connected by an edge in the graph — no keyword hit required.
- Conflict detection: when a new memory is written, it's automatically checked against existing memories for contradictions, and contradictions are returned with the response instead of silently overwriting.
- Natural forgetting: tension decays exponentially with time; frequently mentioned memories resist forgetting — no manual database cleanup needed.
Plus one architecture upgrade that landed in August: dual-layer writes. The leaf layer stores verbatim turns; the abstract layer stores LLM-distilled session-level facts; both coexist and both participate in retrieval. This design came directly from experimental lessons (see Part 5): the abstract layer cannot replace raw text — compressed artifacts are an increment, not a replacement.
Dimension 2: Retrieval Quality — Full-Run Basis, Open Scripts
This is the dimension we most welcome being scrutinized on.
LoCoMo long-conversation memory benchmark: recall@10 = 84.6%, on the full 10 sessions, 1,536 answerable questions. The evaluation scripts are open-sourced with the repo; the embedding model is bge-m3 (1024 dimensions, runnable on a LAN Ollama) — anyone can reproduce it.
We deliberately refuse to speak on small-sample bases. This field has too much "90% on 2 sessions" marketing — with a small sample, the variance of 11 questions can lie to you at will; the full run is the mirror that reveals the demon. Our own 2-session figure was 80.1%; pushing to the full run actually rose to 84.2% — but even that "actually" contains noise: the HNSW index fluctuates ±1.5 points, and we label this honestly in the blog.
From the 47.1% lexical baseline to 84.6%, every step has a controlled experiment on record, including four failed designs (LLM query expansion, explicit inter-layer edges, tension floor, reserved seed quotas). The failure list is as long as the success list — that's what real research looks like.
Dimension 3: LLM Dependency — an Enhancement, Not a Dependency
Many memory systems put the LLM on the critical path: LLM extraction for writes, LLM rewriting for reads. That means LLM jitter = memory-system jitter, LLM bill = memory-system bill.
NylonME's principle is pluggable, fallible LLM:
- With an LLM endpoint configured (any OpenAI-compatible API; we tested with DeepSeek): you get three enhanced capabilities — weaving decomposition, conflict detection, and session-level abstraction.
- Without one, or when the endpoint fails: automatic fallback to heuristic decomposition and rule-based detection; the core read/write path is unharmed.
Your memory engine should not go down because your LLM provider went down. That sounds like a truism — but go check the solution you're using now and see if it dares to run with the LLM unplugged.
Dimension 4: Production Performance — Hard Numbers from a Rust Kernel
All figures measured on a development machine (release build; numbers reproducible via the repo's benchmarks):
| Metric | Measured | Notes |
|---|---|---|
| Write throughput | 12,494 TPS | WAL group commit, 32-way concurrency, fsync on |
| 3-hop graph query | P50 = 3.6ms / P99 = 7.2ms | 100K-node scale |
| Vector retrieval | Top-100 P50 = 0.67ms | Self-built HNSW, recall@10 ≥ 0.9 |
| Memory footprint | 720 MB (real text) / 432 MB (minimal) | 1M nodes + 10M edges |
| Deployment form | Single binary | Zero external dependencies; embeddable or as a service |
The cost of a Rust kernel is ours to bear (slower development); the benefits are yours (single binary, no GC jitter, predictable memory). Compared with the "install Neo4j, then install Qdrant" school, the operational complexity is not on the same order of magnitude.
Dimension 5: Deployment and Data Sovereignty
- Self-hosting: a single binary, served via gRPC + REST gateway, or embedded as a library in your process. Data never leaves your data center.
- Embedding-model freedom: swap any OpenAI-compatible endpoint at will — official API, Azure, or bge-m3 on a LAN Ollama. Our production setup runs Ollama on an Ubuntu box in the LAN: zero API cost.
- Multi-client access: the gRPC protocol is public; clients can be generated in any language. We've already wired it into our own Codex workflow — resonate for historical decisions at the start of each task, weave to deposit new facts at the end. 29 project memories in service. Using your own system to dig up your own memories is the harshest test of whether it works.
Dimension 6: Long-Term Risk
- License: Apache-2.0 — an enterprise-friendly permissive license, no restrictions on commercial use.
- CLA: CLA Assistant is configured; the IP chain of community contributions is clean. Your legal team will like this.
- Lock-in risk: the data format (.nylon snapshots), the gRPC protocol, and the evaluation scripts are all open-sourced. Even if you abandon NylonME tomorrow, your memory data exports intact — no hostage situation.
- Sustainability: this is the most honest item — we are a new project; our star count and ecosystem richness can't compare with big-vendor projects. Choosing us means choosing to grow alongside a project. What we can commit to: development activity, data transparency, and the fact that we ourselves use this path every single day.
When You Should NOT Choose NylonME
The melon-selling is over; here comes the dissuasion. In the following situations, we'd suggest looking elsewhere:
- You're a Python full-stack team shipping in two weeks. Choose TencentDB-Agent-Memory or Mem0 — pip install really is fast. Our Python SDK is still on the way.
- You want a managed service and don't want to operate any process. Choose the cloud-services school. NylonME's cloud version is still on the roadmap.
- You need team-shared memory (multi-user sharing, permission isolation). We don't have that yet; Tencent's Team Memory is production-ready.
- Your scenario is purely "save tokens for the LLM," and you don't care about the retrieval-quality ceiling. The tiered-compression school fits better — their 61% token compression is real.
- You can't afford to bet on a new project. Understandable. Take our evaluation scripts and run them — until the numbers convince you, no commitment is needed.
There is no silver bullet — only fit. We write out our weaknesses because trust in the selection process is worth more than conversion.
Decision Cheat Sheet
| Your situation | Recommendation |
|---|---|
| Hard self-hosting requirement; data must not leave your domain | NylonME (or build your own) |
| Pursuing the retrieval-quality ceiling with complex query types | NylonME (84.6% full-run, reproducible) |
| Production load with high-concurrency writes and low-latency queries | NylonME (12K TPS / millisecond queries) |
| Python team, quick validation | TencentDB-Agent-Memory / Mem0 |
| Want a managed service with zero ops | Mem0 / Zep Cloud |
| Team-shared memory | TencentDB (Team Memory) |
| Academic-grade completeness, dedicated ops staff | MemOS |
A POC Path: Verify Our Claims in Half a Day
- Run it (10 minutes): clone the repo, cargo build, start the single binary.
- Reproduce the numbers (1 hour): run the open-sourced LoCoMo evaluation scripts and see with your own eyes how 84.6% comes about.
- Pour in your data (2 hours): connect a slice of real conversation logs via gRPC or REST, and watch how resonance retrieval performs on your own distribution.
- The unplug test (5 minutes): cut both the LLM endpoint and the embedding endpoint, and confirm the core path is still alive.
After these four steps, you'll know whether to choose it more clearly than after reading ten blog posts (including this one).
Final Words
A memory engine is the foundation of an Agent, and a foundation chosen wrong will reveal itself in the most expensive way, six months later. Our advice is not "choose NylonME blindly" — it's: choose with a framework, verify with data, decide with a POC.
If you try it and find something wrong, open an issue on GitHub and tell us — that feedback, like the 84.6%, is a real part of this project.
NylonME is open-sourced at github.com/nylon-memory/NylonME, Apache-2.0.
This is Part 6 of the NylonME technical blog series. Earlier parts: 01 Phase 1 benchmarks / 02 The memory engine is the core of the Agent / 03 The second half of AI / 04 vs TencentDB-Agent-Memory / 05 From 47 to 84, the full record. All performance figures are the authors' own measurements (Aug 2026, development machine); evaluation scripts are open-sourced and reproducible. Competitor descriptions are based on public information as of Aug 2026; where discrepancies exist, each vendor's official statements prevail.
Top comments (0)