Crystal memory, part 3. Part 1: Notes that arrive when you act. Part 2: Whole notes, not fragments.
This piece: what NodeRAG is, what it buys a frontier coding agent, and why it is the door into everything else we build.
Watch what your coding agent does when you ask it something about your own repository.
It greps. It lists a directory. It reads a file, decides that was the wrong file, reads two more. Eight tool calls later it has assembled the context it needed, and somewhere in that process it burned thirty thousand tokens of your money reconstructing something you had already written down.
Then it answers, and sometimes it answers wrongly, because the note it finally found was cut into four pieces by your retrieval and it was handed the piece that matched the words in your question. The rule was in that piece. The exception was in a different one.
NodeRAG hands the agent the whole note instead. That sentence is the entire method. The rest of this is what it costs, what it buys, and why everything else we ship hangs off it.
The number, and it is a frontier number
We measured this on a frontier coding agent, not a toy. Same model, same tasks, two arms.
In the first arm the agent had to find its own context: real navigation in a read only sandbox, grepping and reading, which is what your agent does right now. In the second arm we handed it the one right note and asked it to answer from that.
Mean 39,458 tokens down to 1,337. That is 97% fewer, about thirty times cheaper, at identical accuracy: 75% correct in both arms.
Nothing was traded away for that. The agent learned nothing from the files it read and discarded; it was paying to rediscover what somebody had already written down.
And that saving scales with how much your agent would otherwise have to read. Our test repository was small, so treat 30x as a floor and not a headline. On a large codebase the arm that navigates gets more expensive while the arm that is handed one note stays flat.
This is why we build this for the frontier tier first. A large model with a big context window is exactly the reader that benefits: the tokens are cheap enough that handing it a complete note is affordable, and the risk of retrieval having picked the wrong passage costs far more than the tokens ever will.
What a node actually is
The word node is doing work, so here is the definition.
A node is one finished knowing, written as a self contained unit. Not a document about a topic. One thing that is true, with the reason it is true and the exception that bounds it, in one file, written so a reader arriving with no context can act on it.
That authoring discipline is the part people skip, and it is the part that makes the retrieval work. If your notes are long documents covering six subjects, whole document retrieval hands the model six subjects and buries the answer. When each note is one knowing, the unit that scores well is the unit that answers.
Everything else is deliberately boring:
plain markdown files, one knowing each, with a small header
a local sqlite index, one vector per whole note
a local embedder, so the notes never leave the machine
a links table, because notes reference each other
Our own store today holds 3,038 documents, 11,798 chunks in the chunked index, and an 8,693 edge link graph, in two sqlite files. Every source note reads with cat. A memory system you cannot inspect by hand is one you have to trust, and trusting it is the failure we are engineering out.
There is no chunking step in the whole note path. Nothing decides where a passage ends. There is no reranker choosing which slice of a document answers you. The unit that goes in is the unit that comes out, which is why the note arrives with its exception still attached.
Why the unit is the whole argument
Standard retrieval is built for a world where the answer is a passage. Ask about a person, get the paragraph about that person. It works well there.
Engineering knowledge has a different shape. The things you need an agent to respect are rules, and a rule is a structure: the rule, the reason, the exception, the cost of getting it wrong. Those sentences sit together because they only mean anything together.
Chunking cuts that structure apart and ranks the pieces by how well they match your words. The piece that matches your words is usually the bare statement of the rule, because that is where the vocabulary lives. The exception is phrased differently and scores worse. So the retriever confidently hands over the half of the note that is dangerous on its own.
One caution before the numbers, because we walked into it ourselves today. This argument is about which reader. Everything above describes a reader with room to hold a whole document and the judgment to use it. Hand that same whole document to a small model and you get the opposite result, measured, twice. The next section is that measurement.
The same note is the wrong shape for a smaller reader, and we have the numbers
This is the part almost nobody publishes, and it is the part that changes how you build.
The optimal way to deliver a knowing inverts with the capability of the thing receiving it. We measured the inversion, and it means the question "how should memory reach the model" has a different right answer for each reader.
| reader | what it should receive | what we measured |
|---|---|---|
| frontier agent (Claude, Codex) | the whole note | fed the one note instead of navigating: 39,458 tokens down to 1,337, identical accuracy 75% and 75% |
| small local model (7B) | a compact one line record, not the whole note | compacted full corpus 53% at 343 context tokens, beating whole file delivery at 50% for 1,608 tokens |
| anything safety critical | always on, never retrieved at all | retrieval surfaced the safety rule 2 times in 3. Always on scored 100% |
Read the middle row again, because it is counterintuitive and it is ours. Handing a small model the entire note made it slightly worse than a one line summary of the same knowledge, at nearly five times the tokens. More context, less accuracy. The frontier reader runs the other way: the tokens are cheap at that size, and the risk that retrieval picked the wrong passage costs far more than the tokens ever will.
So the store holds one copy of the truth and the delivery splits by reader. That is the design, and it is why we say NodeRAG is built for the frontier tier first.
The bottom row is the one we would argue hardest for. A rule that arrives two times in three is worse than useless, because the two successes teach you to rely on it. Guarantees have to be guarantees, so safety rules bypass the clever part entirely and load every time.
There is a fourth number in the same family, cheap and worth stealing. Holding memory constant, changing only how the question was framed, moved a small model 39% to 56%: a gain the size of the memory effect itself, bought with instruction wording alone. Adding step by step reasoning to that same exact recall task made it worse, 56% down to 43%, because the model reasons itself off the verbatim answer. For looking up an exact value, tell the model to quote it and say if it is absent, and leave the thinking alone.
The prediction, tested again tonight
That table is from June. Tonight we ran the one variable version of the delivery question against the small tier, to see whether the June rule still held.
Same retriever, same store, same query, same ranking. The only thing that changed is what came back:
| what came back to the 7B | correct or partial | context used |
|---|---|---|
| ten heading chunks from the store | 6 of 14 | 910 tokens |
| the top hit's whole document | 4 of 14 | 1,479 tokens |
| the right file, chunked down to the answer | 9 of 14 | 328 tokens |
Serving the whole document to the small model made it worse, and cost 60% more context to do it. The smallest payload won by a clear margin. In June the same comparison read 53% at 343 tokens against 50% at 1,608 tokens, so this is the second independent construction giving the same answer.
Which bounds this piece, so we will say it plainly. The whole note claim here is a claim about the frontier reader, and the evidence for it is the fed versus unfed measurement at the top of this article. The small model table speaks to a different reader, so we leave it out of that claim. The one variable test with a frontier model doing the answering is the measurement we owe next, and we have not run it yet.
The real difference is judgment. One reader executes, the other decides.
Context length is the obvious explanation for that table, and it is the shallow one.
A compact one line record is enough for a reader that is going to execute: look up the value, apply it, move on. A whole note is for a reader that is going to decide, and a decision needs the reason and the exception, which is exactly the material a compact record throws away.
We have tested that boundary directly, from the other side. You can train a small model into the habit of working through a problem in steps, and the habit sticks: cold, on questions it has never seen, it lays out its own working and gets them right. The format bakes into the weights.
What would not bake was the disposition. Bolt a constant instruction to reason carefully onto every training example, and the model learns to recite it. On a new task it produced the ritual, looped on it, and reasoned worse than before. We got a tic, not a temperament.
That line between the two readers matters more than the context window. A small model takes facts and formats. Judgment has to arrive live, in language, to a reader capable of weighing it, every time. Which is precisely why the frontier reader gets the whole note: it is the reader that can use the sentence explaining when the rule does not apply.
What the small model is genuinely good for here
We keep the 7B in the harness for one reason: it has no room to absorb a bad retrieval, so a bad ranking falls over where you can see it. That makes it a sharp instrument for the retrieval question while being the wrong reader for the delivery question.
Here is what it showed. Running a whole note arm that chose among a small pool of candidate files, it answered ten of fourteen at partial credit or better against six for the full store search. That gap is mostly the pool rather than the unit, because those two arms search candidate sets three orders of magnitude apart. We caught that in our own harness today and it is why the one variable test above exists.
More interesting than the score is where the failures sat. Whole note retrieval picked the right file on nine of fourteen questions, and on those nine the model never once answered wrongly. Every failure in that arm was the ranker choosing the wrong note rather than the model failing to use the right one. The ceiling there is entirely in retrieval, which is a far better problem to have, and it points straight at the 8,693 link edges sitting in the store that the ranker does not yet use.
The cost, and the condition, because both are real
Whole notes are a bigger payload than a passage, so precision per token is worse. On a corpus of short documents you pay for size the answer never needed.
And the edge is conditional. On a public benchmark of short scientific abstracts, whole note retrieval and chunked retrieval tie: 0.7014 against 0.7016 on SciFact, with keyword search at 0.6644 as a control that matched the published figure and validated our harness before we read anything off it. An abstract is 250 words, so a chunk already is the whole document and there is nothing left to win. On a harder public set, NFCorpus, our retrieval scores 0.3417 against keyword search at 0.3098.
So the claim we defend is conditional. Whole note retrieval wins when your documents are long and the answer is a rule carrying its own exception. That is what an engineering knowledge base looks like and what a corpus of abstracts does not. The condition is the useful half, so the condition is what we hand you.
The door
Here is the part we have under sold in every previous piece, and it is the reason we build this at all.
NodeRAG is the door, and everything else we ship grows out of the store you create by walking through it.
One store, two ways in. Write a knowing once. It becomes answerable when somebody asks, which is NodeRAG, and it becomes deliverable at the moment of an action, which is the crystal half from part 1. The same markdown file, read two ways. That matters because the two fail in opposite directions. Retrieval answers the question you formed and is silent on everything you did not think to ask. A pushed note reaches you for a mistake you had no reason to suspect and cannot answer a question. You want both, and with one store you maintain neither twice.
It is what makes an agent cheap to run. Read the 97% figure as an operating cost rather than a retrieval statistic. Your agent stops spending its context window rediscovering your repository and spends it on the actual work. On a long session that compounds, because the tokens it does not burn navigating are the tokens it still has left when the task gets hard.
It is what makes a cheaper model viable underneath. Our gateway serves a single model identifier and decides internally which model answers, cheap by default, escalating when verification says the cheap answer will not hold. That routing only pays when the cheap model is genuinely competent on your material, and memory delivery is the lever that makes it competent. Retrieval quality is what buys the routing.
It lands where you already work. Ra is our surface: your work, your terminal, your tasks and your notes on one screen with this retrieval underneath. The index lives on your own disk and only the embedding call leaves the machine, chosen deliberately over the hosted alternative so that custody of your source and your private notes stays with you.
And the store is what the discipline attaches to. Once your knowings live as inspectable units you can check them. Every number we publish has to exist as a row in a claims ledger with a date and an artifact. Commits run gates. Notes carry the date they were last verified. None of that is possible over a pile of embedded fragments, and all of it is straightforward over a store of whole notes you can read.
The thing we will admit in public
Here is that discipline working, at our expense.
While writing this piece we went to pull a number for it and found that the retrieval our own agents query every day was the chunked arm rather than the whole note arm. We had published the method and left the fragment version running. Then, checking the comparison, we found our own benchmark had been giving the whole note arm an easier retrieval problem than the chunked one, which is why the one variable test above exists at all.
Two corrections, both ours, both found by our own rules, both before publishing. If a memory system cannot catch its owners being wrong, it will not catch you being wrong either.
Where to get it
The push half is packaged, public, and runs on Claude Code today: github.com/Tirthahq/crystal-memory. It installs a scratchpad that survives a context reset and a set of crystals that arrive at the act. It also ships twelve real verification failures from our own repository, each with the rival explanation and the discriminator that separated them, readable without installing anything.
The retrieval half described here is what we are turning on next, and the numbers above are what we owed ourselves before doing it.
Next
The piece after this one is about what happens when there is nothing to fetch, because the knowledge is already resident. We have a result where twenty one thousand tokens of somebody else's codebase, a project the model had never seen, answered in under a second after a restart that destroyed the working cache. Instant knowing, and whether it survives a cold start, is next.
If you want to argue with any of this, the comments are the right place and we read them.
Top comments (0)