Part 5 of Claude Code, Beyond the Prompt — patterns from running a live automated trading system on Claude Code. Part 4: your first MCP server.
Ask Claude "where do we handle failed payments?" and watch what it does.
It greps for payment. Too many hits. It greps for failed. Different too-many hits. It tries retry, then decline, then reads six files looking for the one function that's actually named process_declined_charge. It gets there eventually — but it burned tokens and wall-clock time spelunking, and here's the scary part: if your code used different vocabulary than your question, it might not have gotten there at all.
That's not Claude being dumb. That's grep being the wrong tool. And it's the default Claude reaches for, so you have to give it a better one.
Why grep falls short
grep matches strings. Your question is about meaning. Those are different things, and the gap between them is exactly where it fails.
You asked about "failed payments." The code says "declined charge." Same concept, zero shared keywords. Grep — which can only find text you already know how to spell — is blind to that. It's all-or-nothing: either you know the magic string, or you don't find the code.
For a human who wrote the codebase, this is a minor annoyance. For an AI that's meeting your repo fresh every session, it's a constant tax.
Semantic search: find by meaning, not by string
The fix is RAG — retrieval-augmented generation — pointed at your own code. The idea, in plain terms:
- Split your codebase into chunks (ideally function- or class-sized).
- Run each chunk through an embedding model that turns meaning into a vector — code about "handling declined charges" lands near code about "failed payments," even with no shared words.
- Store those vectors.
- At query time, embed the question the same way and return the chunks whose meaning is closest.
Now "where do we handle failed payments?" retrieves process_declined_charge on the first try — because it matched on concept, not spelling.
The simplest version
You don't build this from scratch. Several ready-made "code RAG" tools and MCP servers exist — you point one at your repository, it indexes, and Claude gains a code_search tool that returns the most semantically relevant chunks. (Some editors and agent setups now ship this built in.) The ecosystem moves fast, so pick whatever's current; the capability is what matters.
Then adopt one rule:
code_searchbeforegrep. Semantic first to find by intent; grep second, only for exact identifiers you already know.
Both tools keep a job. When you know the exact symbol — UserSession, a specific error code — grep is perfect and precise. When you're describing a concept — "the thing that rate-limits logins" — semantic wins. The mistake is using grep for the second kind, which is what Claude does by default until you give it the alternative and tell it the rule.
How I scaled it
I run a code-RAG service over my codebase, and it's the first thing Claude reaches for when it needs to find where something lives.
The one design choice that matters most: it's hybrid. It runs semantic embedding search and classic keyword (BM25) search, then merges the results. That's not belt-and-suspenders — it's because the two fail in opposite directions:
- Pure semantic misses exact matches. Ask for a specific error string or a precise variable name and embeddings will hand you things that are thematically close but not the actual line.
- Pure keyword misses concepts — the "failed payments" / "declined charge" gap.
Run both and merge, and you get the concept-matching of embeddings with the precision of keyword search. Hybrid retrieval is the robust default, and it's the single most transferable RAG lesson I can give you: don't choose between semantic and keyword, combine them.
The payoff in daily use: Claude finds the right function by description, not by me remembering what I named it two years ago — and it does it in one tool call that returns a handful of relevant chunks, instead of reading ten entire files to locate one.
The token and speed angle
That last point is bigger than it sounds. When Claude greps-and-reads its way to an answer, it pulls whole files into context — easily 6,000 tokens across five files to find 30 lines that matter. A good code_search returns just those 30 lines: ~300 tokens.
That's a 20× reduction on a single lookup, and lookups happen constantly. Retrieving the relevant chunk instead of the whole file is one of the biggest levers you have on both cost and speed. We'll add up all these levers in Part 7 — but semantic search is one of the largest.
One thing that decides quality: chunking
If you go past the off-the-shelf setup, the thing that most determines whether your code-RAG is good or useless is how you chunk. Split by function and class boundaries and each chunk is a coherent, retrievable unit. Split by arbitrary 50-line windows and you get half-functions that match poorly and read confusingly.
That's as deep as I'll go here, because it's a rabbit hole — and it's exactly the rabbit hole my separate research track goes down. If retrieval quality is your thing, that's where the real depth lives.
What this gets you — and what it doesn't
Straight version: Claude finds things faster and misses less, and you stop watching it grep-spelunk through your repo. That's the whole win — and it's plenty, because "find the relevant code" is the first step of almost every task, so making it fast and reliable speeds up everything downstream.
What it is not: Claude does not now "understand your whole codebase." It retrieves relevant pieces well. That's a different, humbler, more accurate claim — and it's enough.
Next — and a fork in the road
You've now got Claude that remembers your project (Parts 1–2), runs your workflows (Part 3), can act on your systems (Part 4), and can find code by meaning (Part 5). One piece remains: where does the source of truth about what to change — and the record of what changed — actually live?
The answer is GitHub, used as more than a place to push code. Part 6: GitHub as Claude's task queue — issues as tasks and gates, pull requests as the unit of change, and how the vibe-coding loop stays honest.
The fork: everything in this part — embeddings, hybrid retrieval, chunking — is the same machinery that, pointed at your memory instead of your code, gives an agent a real long-term memory. That's the subject of my open-source RE-call — the architecture, the eval harness, and the honest findings written up in full. If Part 5 was your favorite so far, that's where to go next.
Part 5 of a series on running real systems on Claude Code. The deeper RAG research is open source — see RE-call. Part 6 is next.
Top comments (0)