We Spent Five Articles Measuring the Boundary — Now We Need a Bigger Map
If you've read straight through from Article 03, then together we've laid to rest a ghost that haunted us for five full articles — Q8.
Let's recap the long chase. Q8 is process payment and create Stripe charge, and its ground truth includes calculate_order_total. We threw every trick in the text-retrieval book at catching it: the vector baseline couldn't; three chunking strategies couldn't; encoding called_by into the embedding only nudged similarity to 0.51 and still couldn't; and finally even the industry's universally endorsed "ultimate weapon" — BM25 + vector hybrid search — crashed, not only failing to fix Q8 but dragging the total from 0.958 down to 0.931.
The five-article conclusion condensed into a single sentence:
The link between
calculate_order_totaland "Stripe payment" is purely structural. It lives on an edge in the call graph (it's called byprocess_checkout), and is written in no function's text. Therefore any pure-text approach — vector, BM25, hybrid — physically cannot reach it.
That sentence is a hard, number-backed conclusion we bought with five articles of experiments. It's valuable — but it's only one piece of the puzzle.
Because while chasing Q8, we incidentally mapped out the territory of three distinct retrieval routes: what vector is good at and what it can't reach; what graph is good at and what it costs; and a whole class of query — "where is the function named validate_jwt_token" — that needs no semantics at all, where exact matching returns in a millisecond. Stitch those three territories together and you get the complete map of codebase retrieval.
So this article shifts altitude. The previous five held a magnifying glass to the recall of a single retrieval algorithm; this one pulls back to satellite view to draw a system architecture: how the vector, graph, and symbol signals should actually be organized into a working production system.
A heads-up to keep in mind before you read: anything in this article with a concrete Recall number is a conclusion already validated by experiments in the previous five articles; anything describing "how to design the system" is an engineering design recommendation derived from those conclusions, not yet validated end-to-end on a full production system. I'll flag this repeatedly at the key points — don't mistake design recommendations for experimental conclusions.
First, Draw the Three Signals' Territories Clearly
Before designing anything, let's put the three territories the previous five articles measured out on the table. This is the bedrock of the entire architecture, and every brick in that bedrock carries an experiment number.
Route one: vector retrieval — the home turf of semantic-similarity queries.
This is the baseline established in Article 03 and repeatedly validated in the four that followed: AST function-level chunking + raw-code embedding, Recall@5 = 0.958, the strongest vector baseline among all text approaches (experimentally validated). It excels at queries of the form "describe a capability in natural language, find the implementation" — say "find the function that encrypts and securely stores passwords" and it hits hash_password, even if you never mentioned the function name.
Its boundary is measured just as clearly: the semantic gap can't be filled (Articles 06, 07). calculate_order_total and "Stripe payment" are genuinely different things in the real world, and no embedding trick can pull them closer.
Route two: graph retrieval — the home turf of structural-relationship queries.
Article 05 proved that the structural information carried by the call graph is an orthogonal truth vector can't reach (experimentally validated): Q8 was scooped back in two hops via the process_checkout → calculate_order_total call edge. But Article 05 also measured its cost — naive BFS 2-hop expansion bloats the candidate set, squeezing Q1's correctly-hit verify_password out of the top-5, one fixed and one broken, netting zero.
The conclusion is subtle: right direction, crude method. The graph signal is valuable, but "doing naive BFS expansion after retrieval" is the wrong way to use it. This lesson directly determines graph retrieval's role in the architecture below.
Route three: symbol retrieval — the home turf of exact-match queries.
The previous five articles didn't run dedicated experiments on this route, but its value is self-evident and needs no proof: when your query is itself a precise symbol — "where is the function named validate_jwt_token defined," "which files import redis.Redis" — you don't want "semantically close," you want "literal exact hit." Here vector is a sledgehammer for a nut, and easily led astray by semantic approximation; a grep or a function-name → file → line-number symbol table returns precisely in under 10 milliseconds.
Place the three territories side by side and a beautiful fact emerges — they barely overlap:
Query type Best route Example
──────────────────────────────── ────────── ─────────────────────────
NL capability description, find impl Vector "find the function that validates JWT"
Exact symbol name / import Symbol "where is validate_jwt_token"
Structural relation / call chain Graph "who calls createPayment"
──────────────────────────────── ────────── ─────────────────────────
The "four knowledge layers" framework from Article 01 maps neatly onto this: syntax layer (AST/symbol), semantic layer (vector), architecture layer (graph), plus the intent layer (Git history). The first principle of a production system is to accept that these four signals are each independent and none is dispensable — there is no single "best retrieval method" that dominates all four layers.
Principle One: Multi-Path Retrieval, Each to Its Own Job
The first principle, and the most counterintuitive: stop looking for "the one best retrieval method."
That's exactly what we did for five articles — endlessly asking "does vector work? does BM25 work? does hybrid work?", each time betting one route could dominate every query. Five straight losses later, we were forced to admit: for structural queries like Q8, the text route physically can't reach.
Flip the framing: not one all-rounder, but let each route do only what it's best at. Vector for semantics, graph for structure, symbol for exact matching — three routes retrieve in parallel, each covering its own territory.
An analogy: this is like a hospital's triage desk. You wouldn't expect one general practitioner to handle heart bypass, tooth extraction, and blood tests all at once. You triage first — fractures to orthopedics, toothaches to dental, bloodwork to the lab. Each department is an expert in its own domain and an amateur outside it. Retrieval is the same — using vector for exact symbol matching is like having your dentist do the heart bypass; sure, he technically knows what a heart is, but you really don't want him operating.
This principle flows directly from the previous five articles' lessons: we've already used five articles' worth of failures to prove what happens when you force one signal to do a job it's bad at. Vector was forced to reach Q8's structural relationship, tried for five articles, and never reached it. So let graph do that job and let vector return to its semantic home turf.
Flag: "Multi-path retrieval" as an architectural direction is a design principle derived directly from the experimental conclusions of Articles 05/07. Each route's territory is experimentally backed (vector 0.958, graph fixing Q8, symbol needing no experiment), but "the overall Recall after combining three routes into a full system" has no end-to-end experimental data yet — this is a design recommendation, not an experimental conclusion.
Principle Two: Query Routing, Activate the Right Path by Intent
With three retrieval routes, a question immediately arises: when a query comes in, which routes do we activate?
If we blindly run all three every time and then fuse, it's not just slow — the routes pollute each other. Article 07 demonstrated this graphically: vector scored a perfect 1.00 on Q7, but BM25 dragged it to 0.67 by over-matching the high-frequency generic word "execute," and once RRF fused them, the good route got pulled down with the bad. Blind fusion lets the route that's bad at a query drag down the route that's good at it.
So we need a Query Router: first judge what type the query is, then decide which routes to activate.
Query "where is the validate_jwt_token function" → Symbol route (exact match)
Query "the function that validates JWT tokens" → Vector route (semantic)
Query "what does createPayment call" → Graph route (structure)
Query "the full payment flow call chain" → Graph route + Vector route (mixed)
Implementation doesn't require reaching for an LLM up front. The vast majority of query intents can be recognized by a handful of rules:
- The query contains a precise identifier (a
snake_case/camelCasefunction name, animported module name) → prefer the symbol route. - Structural keywords appear — "who calls," "what does it depend on," "call chain," "full flow" → activate the graph route.
- Everything else, natural-language description → the vector route.
- Compound queries (both structural intent and semantic description) → multiple routes jointly.
Edge cases the rules can't classify can fall back to a small model for assistance — but that's not a necessity; rules cover the overwhelming majority of scenarios.
Flag (important): The "query routing" here is a pseudocode-level design concept, not an experimentally validated result. The "query → route" mapping table above is a set of routing rules I hand-designed based on the three territories; it aligns with the previous five articles' experimental intuition, but questions like "how accurate is the routing, how costly is a misroute" have not been experimented on in this series. Please read it as a design recommendation.
Principle Three: Graph Retrieval Is a First-Class Citizen, Not a Post-Processing Bolt-On
This is the single most important shift in the whole architecture, and a direct correction of Article 05's failure.
Recall why Article 05 crashed: its flow was vector-retrieve the top-3 seeds first, then BFS-expand the graph from the seeds. Graph traversal was a post-processing step trailing behind vector. That order was wrong — it treated graph as a patch on vector, letting the "structurally related but query-irrelevant" functions dragged in by graph expansion pollute vector's ranking and break Q1.
The right approach promotes graph retrieval to a first-class retrieval route on equal footing with vector:
- The graph index and vector index are built in parallel, both independent first-class retrieval entry points, neither depending on the other.
- The vector route takes its own top-k; the graph route executes independently — it identifies the function/module names mentioned in the query and directly traverses the graph (along
CALLS/CALLED_BYedges), returning structurally related functions. - The two routes' results are fused at the end. Note this fusion is not the context-blind RRF of Article 07, but a weighted merge by query type: the graph route weighs high for structural queries, the vector route weighs high for semantic ones.
One diagram makes the shift clear:
❌ Article 05 (graph as a post-processing patch)
Query → Vector top-k → BFS expand → rerank
↑ graph here, trailing vector, polluting the ranking
✅ Production architecture (graph as an equal first-class citizen)
Query ─┬─→ Vector route ─┐
└─→ Graph route ─┴─→ weighted fusion by query type → results
two routes in parallel, each retrieving independently
Why does this change solve Article 05's problem? Because Article 05's root cause was "graph expansion bloated vector's candidate set and diluted vector's ranking." When the graph route becomes an independent route with its own retrieval logic and trigger conditions (Article 07's ending recommended: only trigger on high-confidence functions, only 1 hop, filter neighbors by business rules), it no longer indiscriminately stuffs noise into vector's candidate pool. The graph route scoops back Q8's calculate_order_total, the vector route holds Q1's verify_password, each minds its own business, no interference.
Flag: "Graph as a first-class citizen, weighted fusion by query type" is a design-correction direction derived from the failures of Articles 05/07. Article 05's "graph post-processing pollutes ranking" is an experimentally validated failure; but "parallel first-class citizen + weighted fusion can hold both Q1 and Q8 at once" is a design inference, not yet validated by experiments in this series. The direction has experimental backing; the specific fusion weights and trigger conditions need the upcoming hands-on article to tune.
Principle Four: Incremental Updates, Never Full Rebuilds
The first three principles solve "how to query"; the fourth tackles a more brutal engineering reality: code changes every day.
The previous five articles all ran on a static 28-function toy dataset. But a real project isn't a specimen — dozens to hundreds of commits a day, functions added/removed/modified, signatures changing, call relationships rewiring. If the indexing strategy is "full rebuild every time," then for a codebase of hundreds of thousands of lines, building the vector index alone takes tens of minutes, and the graph and symbol table have to be fully recomputed along with it. A developer waiting ten minutes for re-indexing after changing one line — no one will use that system.
So the fourth principle: Git diff-driven incremental updates, never full rebuilds.
Three specifics:
1. Git diff-driven. When a commit comes in, first compute exactly which files and functions it changed, and re-index only those change points. For untouched functions, not a single byte of their embedding, graph nodes, or symbol-table entries needs recomputing.
2. Change propagation. This one is easily overlooked but crucial. A function change isn't isolated — if create_payment_intent's signature changes, then all functions that call it need their call-graph edges updated too. So incremental updates aren't just "recompute changed functions" — they must also propagate the change to affected neighbors along the call graph. This is yet another benefit of treating graph as a first-class citizen: the graph structure itself is the propagation path.
3. Version snapshots. Support querying historical versions by commit hash — which connects to Article 01's fourth layer, the intent layer. "What did this function look like three months ago," "which commit introduced this line and what problem was it solving" — these queries want not the current code but the code's evolution history, and the answer lives in Git.
Code Change (git commit)
│
┌─────▼──────┐
│ Git Diff │ ← find only changed files / functions
└─────┬──────┘
│
┌─────▼──────┐
│ AST Parser │ ← re-parse only changed files
└──┬──────┬──┘
│ │
┌───▼──┐ ┌─▼──────┐
│Embed │ │ Graph │ ← two routes update incrementally in parallel
│Update│ │ Update │ Graph also propagates changes
└──────┘ └────────┘ to neighbors along call edges
Flag: Incremental updating is a pure design recommendation — all five prior articles experimented on a static dataset and ran no incremental-update experiments whatsoever. But this direction is uncontroversial — it's standard equipment for any production-grade indexing system, and a direct response to the "dynamism is the biggest challenge" point Article 01 already made.
The Complete System Architecture
Assemble the four principles and you get a complete system architecture. View it from two angles: what happens at query time, and what happens at index time.
Query flow (online, when a user issues a query):
Query
│
┌──────▼──────┐
│Query Router │ ← recognize query intent (rules first)
└──┬───┬───┬──┘
│ │ │
┌───────▼┐ ┌▼──────┐ ┌▼────────┐
│ Vector │ │ Graph │ │ Symbol │
│ Index │ │ Index │ │ Index │
│(AST + │ │(CALLS/│ │(grep / │
│embedding│ │CALLED_│ │AST │
│) │ │BY) │ │symbols) │
└───┬────┘ └──┬────┘ └────┬────┘
│ │ │
┌───▼─────────▼───────────▼────┐
│ Result Merger │
│ (weighted merge by query │
│ type, deduplicate) │
└──────────────┬────────────────┘
│
Top-k Results
Three index routes retrieve in parallel (the Router decides which to activate), then the Result Merger merges by query-type weighting, deduplicates, and produces the final results. Note the Merger isn't a blind RRF — it knows whether this is a structural or a semantic query and adjusts each route's weight accordingly. This is exactly what Article 07 taught us: fusion must be context-aware, or the good route gets dragged down by the bad.
Index flow (offline, Git hook triggered):
That's the Git diff-driven incremental-update diagram from the previous section. The three index routes update incrementally in parallel on commit, with the graph index additionally handling change propagation.
Put the two diagrams together and you have the full picture of this architecture: at query time, three routes retrieve in parallel and fuse with intent-aware weighting; at index time, Git diff drives three routes' parallel incremental updates. Every design decision traces to an experimental lesson from the previous five articles.
Implementation Complexity of Each Index Layer
Design is design; when it comes to landing it, you need to know each route's engineering cost. The table below lays out the three routes (plus the intent layer): build cost, update cost, query latency, applicable scenarios.
| Index type | Build cost | Update cost | Query latency | Applicable queries |
|---|---|---|---|---|
| Symbol index (grep / AST symbols) | Low (seconds) | Very low (incremental) | < 10ms | Exact symbol queries |
| Vector index (AST + embedding) | Medium (minutes) | Medium (changed funcs only) | ~100ms | Semantic queries |
| Call-graph index (AST parse) | Low (seconds) | Low (changed files only) | < 50ms | Structural queries |
| Git history index | High (initial full scan) | Low (incremental commit) | Varies | Intent / history queries |
A few things worth noting:
- The symbol index is the value king. Seconds to build, 10ms to query, near-negligible cost, yet it cleanly covers an entire class of exact queries. Any system should ship it first.
- The vector index is the most expensive route. Embedding runs a model, build is measured in minutes, and query latency is highest (~100ms). This is exactly why incremental updates matter most for it — you absolutely don't want to re-run full embedding on every commit.
- The call-graph index is surprisingly cheap. It's just AST parsing plus an edge table, seconds to build, fast to query. Article 05 already proved its value, and its cost is so low there's no reason not to ship it. Graph retrieval has been neglected not because it's expensive, but because people never figured out how to use it (Article 05's lesson).
- The Git history index is priciest on the first run. A full scan of commit history isn't cheap, but every new commit afterward is incremental, with low marginal cost.
Flag: The costs and latencies in the table are order-of-magnitude estimates (seconds/minutes/milliseconds), drawn from the actual run experience of the previous five demos and ordinary engineering experience, not precise benchmarks on production-scale codebases. Use it to judge relative priority (ship the cheap symbol and graph routes first, the expensive vector route later), don't treat it as an SLA.
Implementation Path: Small to Large, Three Phases
No matter how pretty the architecture, shipping it all at once will choke you. The right posture is a phased rollout where each phase delivers standalone value; get one working before moving to the next.
Phase 1: Vector + Symbol (minimum viable system)
Ship the two cheapest, least controversial routes first:
- Symbol index: ripgrep + an AST symbol table (function name → file → line-number dictionary). Seconds to build, covers all exact queries.
- Vector index: AST function-level chunking + raw-code embedding. This is the strongest vector baseline validated in Article 03 (Recall@5 = 0.958, experimentally validated), copy it directly.
- Query routing: start with the simplest rule — precise identifier in the query goes to symbol, otherwise vector.
This phase already covers the two big classes of "exact symbol query" and "semantic description query," a system usable immediately. Don't underestimate it — the vast majority of daily code retrieval falls into these two classes.
Phase 2: Add the call graph (cover structural queries)
Once Phase 1 runs smoothly, add the third route:
- Call-graph index: AST parsing to build
CALLS/CALLED_BYedge tables. Low cost (seconds), high value (Article 05 validated it fixes Q8). - Wire the graph route in as an independent first-class retrieval route — identify function names from the query, traverse the graph independently, absolutely no Article-05-style "BFS trailing behind vector" post-processing.
- Upgrade the query router: recognize structural keywords like "who calls," "call chain," and activate the graph route.
- Upgrade the Result Merger: from "single-route passthrough" to "weighted fusion by query type."
This phase fills in the Q8 class of structural queries — the piece of the map the text route physically can't reach.
Phase 3: Add Git history + incremental updates (productionize)
The first two phases are "can query accurately"; this phase is "can survive production":
- Git history index: connect to Article 01's intent layer, support queries like "why is this line written this way," "the version three months ago."
- Incremental updates: refactor all three routes' index building to be Git diff-driven. This is the critical leap from "demo" to "production" — without incremental updates, the architecture above won't run on a real project.
- Change propagation: when a function signature changes, update affected neighbors' edges along the call graph.
The point of phasing is that each phase is a complete system that can ship independently and create value independently. Phase 1 solves most retrieval needs; Phase 2 covers the structural blind spot; Phase 3 lets it survive a real project's daily evolution. You needn't wait for all three phases to deliver — on the contrary, you should collect real queries after Phase 1 ships and use them to guide the priorities of Phases 2 and 3.
Tool Selection Recommendations
Finally, down to the actual screws. This part is conventional engineering selection advice, not experimental conclusions.
Vector storage: pgvector (a PostgreSQL extension) or Qdrant. The key reason to pick them is support for metadata filtering — you can attach conditions like "only in the payment module," "only in these few files" during vector retrieval, which is extremely useful for code retrieval (many queries naturally carry a module scope).
Graph storage: no Neo4j needed. This deserves emphasis, because the moment you say "knowledge graph," many people reflexively reach for a graph database. Every demo's call graph in this series is a Python dict ({function_name: [called functions]}) held in memory and serialized to a file, and that's entirely sufficient (validated in Article 05). For a large project where memory won't hold it, NetworkX or simply an edge table in SQLite is still far lighter than Neo4j. The scale of a code call graph can't justify the operational cost of a dedicated graph database.
Symbol index: ripgrep (version 15.1.0 supports PCRE2 + JIT, fast enough) for full-text exact matching, plus an AST-parsed symbol table (function name → file → line number) for definition-level queries. The two together cover all exact-symbol needs.
Incremental-update triggers: a Git pre-push hook, or an index-update step in the CI/CD pipeline. The former updates instantly and locally; the latter is centralized, suited to a team-shared index.
Query router implementation: rules first (keyword recognition + identifier pattern matching), LLM assistance second (and non-essential). Don't reach for LLM classification up front — rules cover the overwhelming majority, and they're fast, free, and explainable.
A Ready-Made Landing Case: codebase-memory-mcp
After all this design talk, you might ask: is there anything running that implements this architecture? Yes.
The codebase-memory-mcp MCP Server introduced in Article 02, held up against this article's architecture design, is essentially a complete landing case:
- It already implements vector retrieval + call graph + symbol index, three-route retrieval — exactly this article's Principle One, multi-path retrieval.
- It's exposed directly to Claude Code via the MCP protocol — within Claude Code you can directly call its
search_graph(symbol/semantic search),trace_path(call-chain tracing),query_graph(graph queries), and other tools. - It organizes the three signals into a unified knowledge-graph interface — exactly this article's idea of "combining three routes into one system."
In other words, the architecture diagram in this article isn't armchair theory — it has an implementation that already works and is already wired into the Claude Code workflow. In the next hands-on article, we'll take codebase-memory-mcp and run an end-to-end demo on a real project, to see how this multi-path retrieval architecture actually performs on a real codebase — and that's when all the experimental intuition accumulated over the previous five articles finally faces the test of a real project.
Summary
- Five experiments measured the three signals' territories, and they barely overlap. Vector for semantic-similarity queries (0.958, validated), graph for structural queries (fixing Q8, validated), symbol for exact-match queries (no experiment needed). The first principle is accepting there's no "best method" that dominates all four layers.
- Principle one: multi-path retrieval, each to its own job. Stop looking for an all-rounder, let each route do only what it's best at — a design direction derived from five articles of consecutive failure at "forcing one signal to dominate."
- Principle two: query routing, activate the right path by intent. Precise identifiers to symbol, structural keywords to graph, natural language to vector, rules first and LLM second. (Pseudocode-level design concept, not experimentally validated.)
- Principle three: graph retrieval is a first-class citizen, not a post-processing bolt-on. Directly correcting Article 05's failure — graph and vector retrieve independently in parallel, fused by query-type weighting, rather than letting graph trail behind vector doing BFS and polluting the ranking. (Direction experimentally backed, fusion details to be validated hands-on.)
- Principle four: Git diff-driven incremental updates, never full rebuilds. Recompute only changed functions, propagate changes along the call graph, support commit version snapshots connecting to the intent layer. (Pure design recommendation, responding to Article 01's "dynamism is the biggest challenge.")
- Land it in three phases, each delivering standalone value. Phase 1 vector+symbol (minimum viable) → Phase 2 add call graph (cover structural blind spot) → Phase 3 add Git history + incremental updates (productionize).
- Tool selection: pgvector/Qdrant for vectors (need metadata filtering), no Neo4j for graph (a Python dict / SQLite edge table suffices), ripgrep + AST symbol table for exact matching, Git hook or CI to trigger incremental updates.
-
codebase-memory-mcpis a ready-made landing case for this architecture, already implementing three-route retrieval and wired into Claude Code. The next article uses it for an end-to-end hands-on demo on a real project.
From Article 03's single vector baseline to this article's three-route architecture, we've completed the shift in vantage point from "retrieval algorithm" to "system architecture." The sentence Q8's story taught us is the very bedrock of this architecture:
Some links are inherently not in the text — so what we've always needed isn't a smarter single route, but multiple routes each doing its own job.
References
- codebase-memory-mcp tool introduction: see Article 02 of this series
- The four knowledge layers framework: see Article 01 of this series
- The graph-retrieval double-edged-sword experiment: see Article 05 of this series
- The text-route boundary experiment: see Article 07 of this series
Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.
Find more useful knowledge and interesting products on my Homepage
Top comments (0)