Introduction
I use Claude Code for most of my projects. Invariably, a lot of documents end up accumulating over the course of a project: a list of design tenets, how to write prompts for an image-generation AI, how to write documentation for a specific part of the codebase, instructions for letting the AI see or control my laptop or phone screen. All of it real, useful information, none of it something the AI needs sitting in front of it on every task.
Having all of that inside the context window all the time is obviously inefficient. Most of it has nothing to do with whatever the current task actually is.
One way to look at this is as building a knowledge base for the project, a small encyclopedia, structured the way people actually maintain that kind of knowledge: a set of separate entries, looked up only when needed.
That's really all this knowledge is underneath, a set of documents, each with its own file path. So the fix was simple. Put a list of those paths into one index file, one line per topic, what it covers and where to find it. Claude always has access to the full list, so nothing is ever missing. But having access to the list doesn't mean reading everything on it, only the entry that's actually relevant to the task gets opened, the same way a person reaches for one entry in an encyclopedia or a dictionary instead of reading the whole thing cover to cover.
I built this before Claude Code had anything like it built in. It now ships with Skills and Memory, aimed at a similar problem, but neither existed yet when I first put this together by hand.
A few real rows from the actual list look like this:
See the user's laptop screen, take a screenshot -> automation/see-user-screen.md
Control the user's Android phone via adb -> automation/mobile-control.md
Write a prompt for AI image generation that matches the app's design tenets -> automation/image-generation-prompt.md
Turn a screen recording into a rounded-corner GIF for a README -> guidelines/ffmpeg-gif-framing.md
If the task is "what's on my screen right now," that first line matches, that one file gets opened, and only then does the agent know how to run the capture script. If the task is something else entirely, say, writing a test, none of these lines ever turn into a file read. Checking the list costs almost nothing. The full instruction file only opens when it's actually needed.
What changes once the list gets big
Right now the list is a few dozen lines. Reading the whole thing costs a few hundred tokens, worth a fraction of a page. Against a context window that can hold hundreds of pages, that's nothing, and nowhere close to worth optimizing.
At 100 lines, it's still basically nothing, somewhere around 2,000 to 3,000 tokens read on every single task, since each line runs about 20 to 30 tokens for a short description plus a file path. At 1,000 lines, it stops being nothing. Reading the whole list at that size runs into the tens of thousands of tokens, spent on every task regardless of whether the one relevant line is even in there.
The fix is something most people already do without thinking about it: folders. Nobody keeps a thousand files loose in one folder, they get split into subfolders, and if a subfolder still has too many, that gets split again. The index can be organized the same way. Instead of one flat list of 1,000 lines, split it into 10 folders of 100, and split each of those into 10 folders of 10. A task now walks down that structure instead of scanning it all at once: pick the right folder out of 10, then the right one out of the 10 inside it, then the one line that actually matches. That's roughly 30 lines read, out of 1,000, to land on the same entry a flat scan would've found by reading all 1,000.
That only works if the folders are actually balanced, roughly the same number of entries in each one. If one folder ends up holding 900 of the 1,000 lines while the other nine hold barely 10 each, checking that one lopsided folder is no better than scanning the whole flat list, the split bought nothing. The benefit comes specifically from keeping every folder roughly even, not from the folders existing.
This is what computer science already has a name for: a balanced tree. Folders are nodes, the entries inside are their children, and the well known result is that a balanced tree turns a search that would otherwise cost the whole list into one that costs roughly the depth of the tree instead, a handful of steps rather than a thousand entries. An unbalanced one loses that guarantee and degrades back toward scanning everything, the same way an unbalanced binary search tree degrades toward a plain list.
Comparison with other alternatives
The obvious alternative is RAG, retrieval-augmented generation. It's the standard approach once a document collection gets too big to read in full, and it works by cutting every document into smaller pieces, turning each piece into something like a fingerprint, and comparing a new question's fingerprint against all of them to pull back whichever pieces look closest. It's common, it's well tested, and it scales further than a hand-built list ever could.
But it works differently from mine in ways that matter here. The match RAG makes is a similarity score, not a yes or no, so a genuinely relevant piece can land just outside the cutoff and never get pulled in, quietly, with no way to notice after the fact. A line in my list either matches or it doesn't. RAG also retrieves fragments, not whole documents, so a piece cut in the wrong place can come back missing the reasoning that led to its own conclusion. And it needs infrastructure of its own to run, a model to build the fingerprints and a database to search them, both extra systems to set up and maintain. My list is plain text files, read directly by the same model doing the task, nothing else involved.
There's also a token cost difference. RAG's matching happens outside the model, the fingerprint comparison runs in a separate retrieval system, so the search itself costs no tokens at all. Checking my list does cost tokens, since the model reads it directly. But for most projects and codebases that cost stays small enough not to matter, especially once the list is split into the tree structure covered above. At that size, the two land close enough on tokens that cost isn't the reason to pick one over the other. For a usual project, I'd still pick mine. It's deterministic, a match either happens or it doesn't, and there's no embedding model or vector database underneath it to set up or maintain.
Comparing this to Claude Code's Skills and Memory is worth doing directly. Skills work the same way mine does: a short description sits in view cheaply, and the full instructions only load when something actually calls for them. Memory is Claude Code's own equivalent of remembering things about how I work, across conversations. Both are genuinely useful, and I don't see them as competing with mine, the model can keep building up its own Skills and Memory in parallel to whatever I maintain in the list.
The real difference is what mine doesn't depend on. It's plain files and a written convention, not tied to Claude Code's schema or its rules for what counts as a match. Any AI reading the project can follow it, and so can I, without opening the tool at all. If I ever stopped using Claude Code, this keeps working exactly as it is.
Skills and Memory can't do that tree-style split. The full list of skill descriptions, and the full MEMORY.md, always load, there's no folder structure underneath them to walk down instead. That's a minor point in practice, most projects never get anywhere near the number of documents where tree-based optimization actually matters, but it's still worth noting as a real difference.
Future of this system
That raises the obvious question: now that Skills, Memory, and RAG all exist, do I keep using this? I do. Comparing them properly gave me a clearer split than I had before, and it's the one I'm going to use going forward, each one matched to what it's actually good at rather than picking just one. The list is for anything meant to be read, project conventions and how-tos. Memory is for anything about how I personally like to work, feedback, preferences, things that don't belong to any one project. Skills are for anything that needs to actually run rather than just be read. I'd already been doing rough versions of the second and third without separating them out. The real change from here is keeping that boundary deliberate instead of letting the three overlap the way they had been.
Top comments (0)