Crystal memory: notes that arrive when you act, not when you go looking
Stop your agent making the same mistakes again and again, and looking up the same facts again and again. Sixty days in we still cannot prove it helped, so we started hiding it from ourselves at random to find out.
The knowing arrives before the act
A crystal is a short knowing bound to an action rather than to a topic, and it is delivered at the moment of that action.
Not after the mistake. Not when somebody remembers to search. In the second before the command runs, carrying only what applies to that command.
That is the whole idea, and as far as we can tell it is the part nobody else is building. Everything below is detail.
What it does
A crystal carries a line that says, in effect, "show me when somebody is about to run a command that looks like this." Nobody searches for it. When the agent is about to run a shell command, write a file, or make a commit, the crystals bound to that action arrive in its context right then.
We call them crystals, and the system crystal memory. The name is the most fanciful part of it. Underneath is five files of plain Python and a rule about when a crystal is allowed to speak.
Here is a real one, and it is in the free starter set because we hit it ourselves.
In a shell, npm run build 2>&1 | tail -20 reports the exit code of tail, not of npm. A build that failed comes back looking like a build that passed. We read a green exit code over a compiler error and believed it. Now, when the agent is about to run any command with a pipe into tail, the crystal arrives before the command runs. It is about forty words. It costs nothing to read and it has to be right only once to pay for itself.
Where crystals come from
The obvious question is who writes the crystals, and the answer is that three moments generate almost all of it.
An error you just paid for. Something failed, you found the cause, you fixed it. The knowing exists for about ten minutes and then the session ends and it is gone.
A lookup. If the agent goes hunting through the filing system for a stable fact, the store should have handed that over and did not. The hunt is the signal. That one is mechanically detectable, which is why we count blocked lookups at all: every one is the store confessing it failed, and a confession is a good place to mint a crystal.
A lesson the work produced. A finding, a correction, a decision. No mechanical tell for these, which is the honest gap in the detection.
The package ships a hook for the end of a session. It checks two things: did real work happen, and was anything written down. Work with nothing banked gets one interruption, once, carrying a filled-in template.
It never writes the crystal, and it never decides what is worth keeping. That limit is deliberate and it is the part we would defend hardest. We have a scar here: an earlier pipeline of ours generated notes automatically from text it had been handed, and produced a confident, specific, entirely invented statistic about a stranger's article, which we then quoted back to them in public. A machine that mints from whatever it read will fill your store with plausible fiction on day one, and a wrong crystal delivered at the moment of an act is worse than an empty store. So the hook guarantees you are asked. The judgement stays yours.
Why not just let it search
An AI coding agent forgets everything when a session ends. Next time it starts clean. It will happily make the same mistake it made last Tuesday, because for it there was no last Tuesday.
The usual fix is a search box over your notes. The agent gets stuck, the agent searches, the agent finds the note. This works, and it has one hole in it that bothered us: the agent only searches for things it already suspects. It cannot look up the mistake it does not know it is about to make. The moment you most need the note is the moment you have no reason to go looking for one.
Binding the note to the action closes that hole, and it costs you the thing search is good at, which is answering a question you actually have. The two are complements, so we built both, and the next section is the other one.
The other half: what the store is, and how anything gets pulled out of it
A crystal has to live somewhere, and the somewhere is deliberately boring. Every note is a plain markdown file with a short header on top saying what it is and when it was last checked. No database. You can read the whole store with cat, and so can any tool you already own. That matters more than it sounds: a memory system you cannot inspect by hand is one you have to trust, and trusting it is the failure we are trying to engineer out.
On top of those files sits the pull half, which we call NodeRAG. Crystals push at you when you act. NodeRAG answers a question you actually have. You need both, and they fail in opposite directions, which is why neither one replaces the other.
The part that is ours, and the part worth stealing:
We retrieve whole notes, never fragments. Standard retrieval slices documents into chunks of a few hundred words, embeds each slice, and hands back the slices that scored well. That works when the answer is a passage. It goes wrong when the answer is a rule, because a rule and the reason it exists and the one exception to it get cut into three pieces, and you are handed whichever piece matched the words in your question. Our notes are written as self-contained units in the first place, so the unit we embed and the unit we return is the whole note. You get the rule with its exception attached.
The internal number and the external number, side by side
We have two measurements of whole-note retrieval. One of them flatters us and one of them does not, and the difference between them is the most useful thing in this section.
| corpus | what we measured | can you re-run it | |
|---|---|---|---|
| external | SciFact, 5,183 short abstracts, 300 queries | whole-note 0.7014 vs chunked 0.7016, nDCG@10. A tie. | yes, it is a public set |
| external | NFCorpus, 323 queries | whole-note 0.3417 vs keyword search 0.3098 | yes |
| internal | our own store, 14 tasks | whole-note 52% vs standard snippet retrieval 27%, about 1.9x | no |
The internal number is roughly twice as good and you have no way to check it. That is worth saying out loud, because it is the shape of almost every benchmark you will read this year.
Two honest notes on the tie, because a tie is easy to wave away in either direction. First, the tie is genuine. Our own arm scored 0.7014, then 0.7019, then 0.7014 on three separate runs, so the 0.0002 gap to chunked retrieval is an order of magnitude smaller than the noise in our own measurement. Second, we validated the harness before trusting any of it, by running plain keyword search as a control and getting 0.6644 against a published figure of about 0.665. The instrument was checked before the result was read off it.
So why is the internal number twice the external one? The answer is in the corpus, and it survives inspection. SciFact is built from abstracts of about 250 words, so a chunk already is the whole document, and whole-note retrieval has nothing left to win. The public set fairly measures whether our retrieval works at all, while being structurally incapable of measuring the property that makes it ours. Our own store runs the other way: long notes with a terse rule buried inside them, surrounded by near-duplicates.
That makes the honest claim a conditional one. Whole-note retrieval helps when your documents are long and the answer is a rule carrying its own exception. On short documents it does nothing measurable. The condition is the useful half, so the condition is what we are handing you.
One more thing about the internal number, kept in the open: 14 tasks, three runs, scored by a model. An earlier scoring pass using a different judge read 62% against 33%, and we found that judge was lenient and threw its numbers out in favour of the lower ones above. The number that went in the table is the less impressive of the two we measured.
And the method underneath, which is the one we would defend: the same file reaches different readers in different shapes.
We ran this as an experiment and the result inverted our intuition. A large model with a big context window does best when you hand it the entire note, because the tokens are cheap at that size and the risk that the retrieval picked the wrong passage costs more than the tokens ever will. A small local model does worse with that same whole note, and better with a compact one-line record of the same knowledge. More context made the small model less accurate. The optimal feed inverts with the capability of the thing being fed, which means there is no single correct answer to "how should memory reach the model", only a correct answer per reader.
So the store holds one copy of the truth and the delivery splits three ways: whole files to the big reader, compact records to the small one, and a third rule that matters more than either.
Safety rules are never retrieval-gated. Anything that must always hold gets loaded every time, unconditionally, and is never subject to a similarity score. We tested the alternative and similarity retrieval missed the safety rules often enough to settle it. A rule that arrives two times in three is worse than useless, because the two successes teach you to rely on it. So those bypass the clever part entirely. The clever part is for knowledge. Guarantees do not get to be clever.
What else is in here, and why you are not running it yet
Crystals are level one. They pay on an empty store, because the first crystal you write is the first thing the store holds. Nothing has to accumulate first.
Level two arrives on its own, a few hundred notes in. Everything a crystal says has to live somewhere, and a few hundred of anything is a library with a library's problems. Two notes that contradict each other. A folder that quietly grew to ninety files. A link pointing at something since renamed. Two versions of one truth, which is worse than none, because now the agent has to pick.
So the system we run here has four more pieces. All four tend the store:
- The Librarian keeps it navigable: broken paths, dangling links, signposts that rotted.
- The Cleaner consolidates. When a folder goes over its cap it proposes what to archive, and every move is reversible.
- The Gardener grows the store from what actually got used, mining our own usage ledger. It is barred from minting out of text it merely read, for the reason described further up.
- The Corrector makes bounded edits to whatever the other three flag.
Now the part that belongs in a paper with this title.
On 2026-09-17 we pointed all four at a store from another machine, for the first time ever. Zero of the four were correct. Each broke in its own way, and the Cleaner is the one worth looking at. The list of folders it walks names three of our own directories, by hand, in the source. Aimed at a stranger's store it found nothing over cap and announced that everything was within cap: a green tick over an empty set, from a guard whose entire job is to notice. We learned this by running it. Reading the source would have left it hidden.
There was a fifth failure one layer up, and it is the one we would put on the cover. The health check for the whole package looked at that same foreign store and passed it, green, while every one of the four agents beneath it was unable to use the thing it had just approved.
So the download is crystals, and the other four stay here until they can survive a store they did not grow up in. Shipping them this week would put four agents on your machine that report success by looking at nothing.
There is a second reason, and it is the one that sets the order. All four idle against a small store. Five notes give the Cleaner nothing to consolidate and the Librarian nothing to repair, and the Gardener feeds on usage you have yet to generate. They earn their keep at a few hundred notes, which is the same point at which the problems they solve begin to exist. Level one and level two came out in the order the work arrives in.
What we can count
Over 60 days, from 2026-07-19 to 2026-09-17, on one machine doing real work:
| what | count | window |
|---|---|---|
| crystals registered | 266 | as of 2026-09-17 |
| crystals delivered at least once | 243 | 60 days |
| deliveries | 14,375 | 60 days, about 240 a day |
| budget per action, shared | 4,000 characters | fixed |
| blocked lookups, the opposite signal | 387 | 94 days, from 2026-06-15 |
| suppressed so far, the new control arm | 19 | since 2026-09-17 |
That last-but-one row is the one we watch hardest. A separate guard blocks the agent when it tries to hunt through the filing system for something the system should have handed it. Every block is a small confession that delivery failed.
What does look like it is working
We lean hard on our own failures, so a fair question is what the data says in the other direction. Three things, and we went looking for the reasons each might be wrong.
Hunting fell by two thirds. Splitting the window in half: in the first half the agent went hunting through the filing system 22 times per thousand notes delivered, and in the second half 7.5. Counting every hunt, marked and blocked alike, it is 61.9 down to 20.5.
The obvious rival: we simply learned to mark our hunts with the escape hatch, which would make the drop an accounting change. It is checkable, and the check came back against it: the blocked share of all hunts held flat at 35% and then 37%. Volume fell while the habit held still.
What we cannot rule out is the work itself. The two halves contain different projects, including a fortnight spent away from this repository, and a person also gets more familiar with a codebase over ninety days. So read it as suggestive, short of causal. It remains the closest thing we have to the system getting better at its job.
Almost the whole store is live. 237 of 264 crystals have been delivered at least once, which is 90%. Knowledge bases usually rot toward the opposite, and 27 crystals that have never fired is a short enough list to read in one sitting.
The load is spread across the store. The busiest ten crystals out of 241 account for 17% of all deliveries, and the median crystal has fired 29 times. That kills the objection we expected to face, that a number like 14,000 really means one loud crystal firing over and over.
Now the honest part
None of those numbers say the thing you actually want to know.
Counting deliveries measures how often a crystal showed up. Whether the crystal helped is a separate question, and that count is silent on it. A system that fired 14,375 useless crystals would produce exactly the same chart.
The number we want is "how many lookups did this save," and here is the trouble with it: a lookup that did not happen leaves no trace. You cannot observe a search nobody ran. Any figure we computed from our own logs would really be measuring how rarely we search on the one channel we happen to watch, dressed up as a result. We got close enough to building that number to be embarrassed about it, and an outside review took it apart before we shipped it.
We do have two small effect measurements from 2026-07-11, and we are going to describe them exactly as weakly as they deserve. On a long task where the guiding instruction scrolls out of the model's context, re-supplying it as a crystal held the model on target where a control collapsed, and a placebo crystal carrying no instruction collapsed too, which at least says the content mattered and not the interruption. Two runs, eight chunks, synthetic constraints. On a second task, injecting a fixed pattern and generating only the new part cut output by about 95% at equal correctness. Eight cases, one small local model. Both are directional. Neither is evidence that this helps you.
What we are doing about it
Since 2026-09-17, our own system withholds a random ten percent of the crystals it would otherwise deliver. The agent never sees them. Every suppression is logged, so there is a control group.
Before any data existed, we wrote down the rules:
- The unit is one crystal per session, not one delivery. A crystal that fires forty times in a day is one decision, not forty results. An hour after switching it on we had three suppressions covering one crystal, which is exactly the trap.
- The outcome is named "a lookup we can actually see," because we only watch one channel. Silence on an unwatched channel stays silence, and we refuse to read it as "they never looked."
- We stop at 100 units or on 2026-12-17, whichever comes first, and that is fixed now so it cannot be chosen later to suit the answer.
- A null result gets published. If withholding the crystals changes nothing, they were decoration and we will say so.
That last point is the whole reason for writing this down in public before we have the answer. It is easy to run a quiet experiment and mention it only if it flatters you.
What this costs, and what we are still blind to
The first thing a reader usually asks is what this does to the context window, so here is the arithmetic. The channel is capped at 4,000 characters per action, shared across every crystal that matched. That is roughly a thousand tokens on a shell command, and zero on the many commands where nothing matches. Against a coding context that is small, and the bound comes from the code, never from our good behaviour.
We are worse at our own guidance than that makes us sound. We tell people to keep a crystal under about 1,500 characters. Our median sits at 1,496, comfortably inside. Then 126 of our 264 crystals run over it. The longest is 8,401 characters, more than double the entire per-action budget, so it arrives truncated and shoves its neighbours out on the way in. Truncated and short look identical from inside the model, which is the silence problem again one level down.
The honest cost is attention, more than tokens. A crystal that arrives at the moment the agent has already chosen a tool is authority-weighted: it competes with the instruction you gave. A wrong one wastes a slot and then argues with you.
Withholding crystals from your own working system makes your own agent worse on purpose, a little. We think that is a fair price for finding out.
We are still blind in ways worth naming. This is one operator on one repository, so read it as a case study, never a population. We watch shell commands and not file reads, so our view of "did they go looking" has a hole in it. And suppressing a note frees up room in the shared budget for its neighbours, so the two groups are not perfectly separate, which is a confound we can describe and cannot remove.
What it is actually for
The mistakes worth catching are the ones that look like success.
A failed build that reports a green exit code. A test suite that passes while asserting nothing. A guard that lost the ability to fail when somebody narrowed what it checks. A deploy that reports Success because the error was swallowed by a pipe. Every one of those ends with an agent telling you it is done, in exactly the tone it uses when it is done, and you find out later.
An agent is very good at the first ninety percent and has no memory of the last time the final ten bit you. Telling it once does nothing. It has to be told at the moment, every time, and only about the thing in front of it. That is the whole design goal: a channel narrow enough that you tolerate it firing on every command, carrying knowledge specific enough to be worth reading in the second before you act.
We think we are a reasonable team to build it because we are the ones getting caught. Every crystal in the starter set is a mistake we made and paid for. The system exists because we kept making the same three or four classes of error across sixty days, watched them cost real hours, and wanted something that would interrupt the fourth time rather than the fortieth. The discipline underneath it, in one line: we treat our own agent's unqualified yes as a claim awaiting evidence, and this is the machinery for that.
That is also why this piece leads with what we cannot prove. A tool that claims to make an agent more careful should survive someone asking it for evidence. We asked ourselves, and the answer was a measurement we had skipped. So we are running it now, in public, with the rule written down first.
If you want to try it
The delivery half is five files of standard-library Python, Apache 2.0, no network, no service. It runs on your machine and talks to nothing.
It has no public repository yet, and that is on purpose. We would like it to work before we embarrass ourselves in public, and the only way we know to find out is to put it in front of someone who did not write it. Every check we own was written by the same hands that wrote the thing being checked, so our green is worth less than a stranger's red.
Ask in the comments and we will send it over. Handing the first few out by hand lets us watch where someone else gets stuck, which is the part we most need and the part a download page hides.
The most useful thing you could send back is the moment you thought "I installed this and nothing happened," because a system whose failure mode is silence is a system that can quietly do nothing for a week while you assume it is working.
We would rather hear that from you in week one than discover it ourselves in month six.
Top comments (1)
Action-bound recall is a sharp constraint: it can prevent a known failure without filling every turn with generic memory. I’d attach a confidence level and a visible “why this appeared” link to each crystal, then log whether it changed the action. That makes the intervention auditable and gives you evidence for pruning rules that are merely noisy.