Suppose retrieval had to work with no embeddings, no vector database, and no index you did not type by hand. No cosine similarity, no chunking strategy, no re-ranker. Files and a search box, nothing else. For roughly eight months that has been my whole setup for handing a language model the context it needs, and the part that keeps surprising me is how seldom it loses to the machinery it skipped.
I want to run that as a thought experiment rather than a recommendation, because the interesting question is not "is grep enough for you." It is which pieces of a retrieval system are load-bearing for one person and which are load-bearing only at a scale I do not have. When I strip my own setup down and ask what is actually doing the work, almost none of the answer is the part that shows up in RAG tutorials.
What is a RAG, stripped to the part that matters?
Retrieval-augmented generation is usually drawn as a pipeline: embed your documents into vectors, store them, embed the query, find nearest neighbours, stuff the top hits into the prompt, generate. Six moving parts before the model reads a word.
Strip it to the load-bearing sentence and only one clause survives. A RAG puts relevant text you already have into the prompt so the model answers from your world instead of its average of everyone's. Retrieval is the whole idea. Embeddings are one way to do retrieval, the way that pays off when a machine has to guess what "relevant" means across a corpus no human has read end to end.
That last condition is the one I quietly fail to meet. My corpus is a folder of dated plain-text lines I have read, written, and re-read myself. It is a few thousand lines, not a few million documents. The retrieval problem embeddings solve beautifully is a problem I mostly do not have.
What does my retrieval actually look like?
Honestly, it is this. My notes are markdown, one append-only file per month, every line stamped with a date and written in words I expect to search for later. When I sit down to work through a problem with Claude or inside Cursor, I do not summon a pipeline. I search the folder for the five to twenty lines that bear on the thing, read them, and paste the ones that survive that read into the prompt.
The search is a single command, and it is the least clever part of the system:
rg -i --sort path 'outbox|retry|idempoten' notes/
That is the entire retrieval layer. The "index" is not a data structure. It is a habit: I write each line as if a stranger with my exact question will grep for it in a year, so I lead with the noun, not the feeling. The model never touches my notes. I do. It only ever sees the handful of lines I chose to carry across.
There is no embedding step because there is no moment where I need a machine to guess which lines are relevant. I already know. The cost of being the retriever myself is a few seconds of reading; the thing I buy with those seconds is that I see exactly what goes into the context window before the model does.
The team-scale RAG and the one-person retrieval are not the same machine
The reason this feels like heresy is that most writing about RAG is built for teams and products, and it is tempting to assume the shape transfers straight down to one desk. It does not. Almost every axis that justifies the vector pipeline moves the other way when the corpus is one person's notes.
| Axis | RAG built for scale | My one-person retrieval |
|---|---|---|
| Corpus size | Millions of chunks nobody has fully read | A few thousand lines I wrote and reread |
| Who runs the query | An automated step guessing relevance | Me, already knowing what I want |
| What ranks the hits | Cosine similarity plus a re-ranker | My eyes on the search output |
| Freshness | A re-embedding job on a schedule | The line is searchable the second I save it |
| Cost of a wrong hit | Silent, buried in a long context | Obvious, I am reading each line |
| Recall of forgotten items | The main thing you are paying for | The one real hole, see below |
Read down the right column and the vector database is not being lazily omitted. It is being asked to earn its place against a corpus small enough to hold in one head and a query planner, me, who is free. On that column it mostly cannot.
Why does the dumb version keep winning?
Three properties do the work, and none of them are retrieval algorithms.
The first is scale, or rather the lack of it. Literal search over a few thousand lines returns in the time it takes to lift my hands off the keyboard. There is no recall ceiling to engineer around because I can skim every hit. Embeddings shine exactly where skimming every hit is impossible, and that boundary sits far above the size of a personal notebook.
The second is that I am the query planner, and I am a very good one for my own material. A vector search has to infer that "outbox" and "the thing that holds unsent mail" are the same concept. I do not have to infer it; I remember writing both, and I search for whichever I used. The expensive intelligence in a RAG is spent reconstructing intent that, for my own notes, never left the room.
The third is trust, and it is the one I underrated at the start. Because I read the retrieved lines before they enter the prompt, I am also auditing them. I catch the stale note, the line that reads as current but was a guess, the number I should not hand over as fact. A pipeline that injects the top eight chunks silently would rob me of the one review step that keeps the model from confidently building on something I already know is wrong. I argued in an earlier piece that the context is the product, not the prompt; doing retrieval by hand is how I keep ownership of that product instead of delegating it to a similarity score.
Where does it break?
I would be lying by omission if I stopped at the part that works. The dumb version has one real hole, and it is precisely the hole embeddings were invented to fill.
Literal search can only find notes I remember exist. If I forget I ever wrote a line, or wrote it with words I would not think to search today, grep cannot rescue it, because grep matches strings and I am feeding it the wrong string. Semantic search does not care what I remember. It would surface the six-month-old line about retry storms when I search for "the outbox keeps double-sending," even though those share not one word. That recall-on-forgotten-material case is the genuine argument for the machinery, and it gets stronger every month the folder grows.
It also breaks the instant a second person needs to query my notes, because now the free, expert query planner, me, is not in the loop, and all the intent I never had to write down has to be reconstructed by a machine after all. And it breaks softly with pure size: the day skimming the hits stops being instant is the day I have quietly crossed into the regime where the tutorials were right.
None of these have bitten hard yet. But I can see the edges, and a thought experiment that only flatters its own setup is just an ad.
What makes a note retrievable at all?
This is the part I would keep even if I bolted a vector index on tomorrow, because it is upstream of every retrieval method. A note is findable in proportion to how well the words in it match the words a future query will use. Embeddings loosen that constraint; they do not remove it. A line written as "felt off, fixed it" is nearly unfindable by any method, semantic or literal, because it encodes none of the handles a later question will reach for.
So the discipline is the same one that makes notes legible to a future human: write the line so its own subject is in it. I have written before about keeping notes a later reader and a later model can both parse, and personal RAG turns out to be that same habit wearing a fashionable acronym. The retrieval method is downstream. The writing is the index.
FAQ
Should everyone skip embeddings? No, and that is the whole point of the table. Scale flips the answer. Past the size where you can skim every hit, or the moment someone other than the author queries the corpus, the vector pipeline stops being overhead and starts being the only thing that works. The claim is narrow: for one person's few thousand notes, retrieval is not the bottleneck, and paying for it first is solving the wrong problem.
Isn't grep just worse recall wearing a straight face? Yes, exactly, and worse recall is the one weakness worth spending money on. Everything else the pipeline offers I either do not need or would rather do by eye. So the sane hybrid is obvious: keep literal search as the primary path, add semantic search only as the fallback for "I know I wrote something about this and cannot find it."
Does the model ever read the whole notebook? Never. It reads the lines I paste. Keeping the corpus out of the context window is a feature, not a limitation; the small, hand-picked set is why the answers stay grounded instead of drifting into whatever the largest nearby chunk happened to say.
The half I am going to actually run
The honest move at the end of a thought experiment is to test the part I keep half-believing, so here is the one I have avoided. I am going to add a semantic fallback, but only for the recall-miss case: literal search stays the front door, and the embedding index gets consulted only when grep returns nothing and I still believe a note exists. I will run it for a month against the same folder and count two numbers, how often the fallback fires and how often it surfaces something real, because a fallback that never earns a hit is just infrastructure I have to maintain alone.
I suspect it will fire rarely and be worth it exactly on the days it does. But I have been wrong about which half of a system carries the weight before, so I would rather measure than keep narrating.
Here is the question I actually want answered. What is the last thing a model got right because of a note you had forgotten you wrote, and when you trace it back, did you find that note by searching for it, or did the retrieval surface it without you asking? I am collecting the cases where the retrieval, not the model, quietly did the work, because those are the cases that decide whether the machinery is worth it for one person.
Simple Memo is the iOS app I make by myself; its only job is to get a line out of my head and into plain text before it evaporates. I write here every week or so about the workflow that grows around that one habit, and the curated markdown side of those notes is where the slow half lives.
Top comments (0)