Hybrid vector + graph recall in pure TypeScript, zero dependencies

Every AI agent forgets everything when the session ends. Fixing that usually means wiring up a vector database, an embedding pipeline and a retrieval service — a platform, with servers to run and backends to choose. I wanted the other thing: a primitive. npm install, one file on disk, done. The SQLite of agent memory.

So I built rememori: an embedded memory engine in pure TypeScript with zero runtime dependencies. This post is about the three design decisions that turned out to matter — the scoring formula, the entity graph, and an HNSW index that taught me a lesson about benchmarks.

Three verbs

The whole API:

import { Memory } from 'rememori';
import { ollama } from 'rememori/embedders';

const mem = await Memory.open('./agent.mem', {
  embedder: ollama('nomic-embed-text'),
});

await mem.remember('User prefers dark mode', { tags: ['prefs'], importance: 0.8 });
const hits = await mem.recall('UI settings?', { limit: 5, halfLifeDays: 90 });
await mem.forget(hits[0].id);

The embedder is an interface, never bundled — local Ollama, any OpenAI-compatible endpoint, or your own (texts) => Float32Array[]. That single decision is what lets the same engine run in Node, Bun, and (with transformers.js) entirely inside a browser tab.

Similarity alone is not recall

Pure cosine similarity is a fine search primitive and a poor memory. Two problems show up immediately when you use it for an agent:

Time doesn't exist. A preference stated yesterday and one stated two years ago rank identically. Human memory doesn't work like that, and agent memory shouldn't either.

Names beat words. Ask "any updates from Giorgio?" and the embedding of that sentence may be nowhere near the embedding of "fixed the flaky deploy pipeline" — even if Giorgio is the one who fixed it. The connection is the entity, not the phrasing.

So recall in rememori scores every candidate as:

score = (cosine + entityBonus) × importance × 0.5^(age / halfLife)

• entityBonus = min(0.3, 0.1 × shared entities) — memories that mention the same entities as the query get a boost, capped so it never swamps genuine similarity.
• importance is a 0..1 weight you set at write time.
• The decay term is optional and per-query: a half-life of 90 days means a six-month-old memory needs roughly 4× the raw relevance to outrank a fresh one.

The entity term is what I'd defend hardest. At write time, every memory is linked to the named entities it mentions — a bipartite memory↔entity graph. The default extractor is a zero-dependency capitalization heuristic (pluggable if you want an LLM doing it). At recall time, entities are extracted from the query too, and any memory sharing one is guaranteed into the candidate set. A memory with zero textual similarity can still surface because the who matches. In practice this rescues exactly the queries that pure vector search fumbles: people, projects, product names.

One file on disk, structured clone in the browser

Storage is an adapter interface with three real implementations:

• Node/Bun: an append-only JSONL log. Writes are appends, deletes are tombstone lines, and a compaction pass rewrites the file. Crash-safe by construction, grep-able by accident, and you can read your agent's memory with less.
• Browser: IndexedDB, selected by path (idb://agent). A pleasant surprise here: IndexedDB stores Float32Array natively via structured clone — no base64, no serialization layer at all.
• :memory: for tests.

The browser storage is why the demo on rememori.dev works the way it does: transformers.js generates embeddings locally, IndexedDB persists them, and after the model downloads once you can watch the Network tab stay silent while you store and semantically recall memories. Pure TypeScript is the load-bearing constraint — native bindings would have stopped at the browser's edge.

The HNSW index, and the benchmark that lied to me

Exact search over contiguous Float32Arrays is genuinely fast — ~0.5ms per query at 1,000 memories. Most agents never need more. But "it doesn't scale" is a fair objection, so v0.4 added a pure-TS HNSW index: hierarchical navigable small world graph, the same algorithm behind most production vector databases. About 250 lines including a hand-rolled binary heap.

First benchmark run, 10,000 vectors:

recall@10 = 36.8%

Ouch. I spent an hour hunting the bug. Neighbor selection heuristic — textbook. Layer assignment — textbook. Beam search termination — textbook. Then I ran the same index on 32-dimensional vectors: 100% recall at every size.

The bug wasn't in the code. It was in the data. I was benchmarking on uniformly random 384-dimensional vectors, and in high dimensions random points concentrate: every pair is nearly equidistant, there's no neighborhood structure, and no graph-based index can navigate structure that doesn't exist. This is a documented pathology — the hardest datasets in ann-benchmarks are exactly this shape — and it has nothing to do with real workloads. Actual embedding models produce vectors on low-dimensional manifolds; that structure is the whole reason embeddings are useful.

Re-benchmarked with realistic data (384-dim vectors with 24 dimensions of intrinsic structure, queries as perturbations of stored points — i.e., a paraphrased question against a stored memory):

memories	exact scan	HNSW	recall@10
1,000	0.5 ms/query	(index off — scan wins)	100%
10,000	4.4 ms/query	2.3 ms/query	100%
50,000	26 ms/query	2.8 ms/query	100%

Two takeaways I'd pass on:

1. If your ANN benchmark uses uniform random high-dim vectors, it's measuring the wrong universe. Generate data with intrinsic structure or use real embeddings.
2. Below ~1,000 items, don't index at all. The exact scan wins on latency, memory and simplicity. rememori's default (index: 'auto') only builds the graph past that threshold — and tag-filtered recalls always use the exact scan, because approximate + filtered is where correctness quietly dies.

What it deliberately isn't

No multi-user server, no auth, no cloud sync, no document-ingestion pipeline. If you need those, Cognee and Mem0 are good platforms. rememori is for when you want memory inside your process, in five minutes, with nothing to operate.

One more thing that fell out of the design: since the engine is embedded, wrapping it in an MCP server took an afternoon. claude mcp add rememori -- npx -y rememori-mcp gives Claude Code — or Cursor, or any MCP client — persistent semantic memory across sessions, stored in a local file, embedded via local Ollama.

---

rememori is MIT-licensed and early (v0.4). The name is Esperanto for "to remember" — a language built to run anywhere, which felt right. Site and in-browser demo: rememori.dev. Feedback I'm most interested in: the recall formula, the storage format, and what you'd need before trusting it in a real agent.