I asked my system how a request gets handled. It returned one function, which was the handler whose docstring happened to mention the word "request".
The base class it dispatches to was missing. The subclasses were missing. The whole call path was missing. The system was not broken. Vector search only matches on text, and it cannot find the connections between pieces of code.
Embeddings answer one question well, which is what text is close to the query. But most code questions are not about text. They are about how the code connects. You want to know who calls a function, where it is defined, and what inherits from it. A vector index can find a starting point, but it cannot show the relationships between the starting points.
The fix was to put a knowledge graph next to my vectors.
Why vector retrieval breaks over code
Two things break pure vector retrieval over code, and they come back to the same idea.
The answer is usually a path through the code, not a single chunk. A request travels from the network socket to a handler through several files. No single chunk holds the whole path, so the answer is the sequence of calls.
Textual similarity is also not the same as a connection. A one-line dispatcher can sit at the center of an entire subsystem while it shares zero tokens with your query. Two functions can look almost identical and still be unrelated.
But code is densely cross-referenced. Imports, calls, inheritance, and containment are explicit, and you can extract them with a program. You can build a graph that captures them and let retrieval follow it.
The two phases
Phase one localizes the query. You embed the query, run dense retrieval, and run BM25 in parallel (BM25 is the standard keyword search algorithm that scores documents by how often the query terms appear, how rare those terms are, and the length of the document), because exact token matches catch what dense retrieval blurs. You fuse the rankings and get seed chunks that carry both signals.
Phase two elaborates on the seeds. You take those seeds as entry points into the graph. You traverse outward two hops, and you map the nodes back to chunks.
Vector search alone cannot reach a file that does not resemble the query. Graph traversal alone has no anchor for where to start. The first phase localizes the query, and the second phase elaborates. The whole design rests on one join. Each node points to its chunks, and each chunk points back to its node. Build the join first, and everything else is tuning.
The mistake everyone makes
The mistake is treating every edge as equal. A "contains" edge exists between almost every file and its functions. Following it blindly floods your results with noise. A "calls" edge is selective, and it usually carries the answer.
A useful ordering, from high signal to low:
| Relation | Why |
|---|---|
| inherits | structural, small fan-out |
| calls, uses | the actual control and data flow |
| imports | locates dependencies, high fan-out |
| contains | almost universal, low signal |
The weights should respond to the query. If you ask about how something flows, boost "calls". If you ask about types or subclasses, boost "inherits". Traverse in best-first order with a priority queue, and cap the depth and the result count. Without both caps, one well-connected seed pulls in half the codebase.
When it clicked
I indexed a small Python service. It had one base class, a couple of subclasses, and a handler that calls into them.
I asked how a request gets handled. Vector search alone returned the handler and one subclass whose docstring mentioned "request". The base class and everything else were absent.
The graph search started at the same handler. It followed "calls" to the methods, then "inherits" back to the base class, and returned the whole dispatch path. Some of those chunks shared zero words with my query, which was the whole reason I built the graph.
The warning I wish I had heard
In my evaluation, most of the quality gain came from the larger context budget, not from better retrieval. My graph agent spent five times the tokens of the baseline. Some of the win was real retrieval, and some of it was just more context.
The only way to tell the difference is an ablation study. You add one component at a time, e.g., dense only, then sparse, then the unweighted graph, then the weighted graph. Without an ablation, a headline like "graphs beat grep" tells you nothing, and you end up trusting a number you cannot explain.
The graph is also a static approximation. Dynamic dispatch, reflection, and generated code are invisible to a syntax tree. A wrong seed also amplifies the error, because it pulls in its whole neighborhood.
The takeaway
Vector search finds plausible entry points. The graph adds the code that is structurally related to those entry points. You keep both signals visible so you can always tell whether the graph helped.
Build the join first. Weight the traversal second. Measure third.
If you are building RAG over code, I am curious whether you use pure vector search or a graph layer, and what broke for you.
Top comments (1)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.