DEV Community

Cover image for Code knowledge graph vs RAG: why structure beats similarity
Syed Fahad for Graphify Labs Inc

Posted on Edited on Originally published at graphify.com

Code knowledge graph vs RAG: why structure beats similarity

Retrieval-augmented generation keeps the text and drops the structure. It splits a corpus into chunks, embeds each chunk, and pulls back the few that sit nearest the question in vector space. Lewis et al. (2020) showed that this works for lookup: a question whose answer is sitting in one passage comes back. A question about how two parts of a codebase connect does not, because that connection was never stored.

A code knowledge graph stores the connection. Functions, classes, tables, and config are nodes. Calls, imports, and references are typed edges. The assistant answers by walking those edges, so the result is a path through the repo rather than a pile of passages that happen to share words. Graphify builds that graph from the code. The side-by-side lives at Graphify vs RAG.

How it works

RAG's retrieval step is a similarity search. The question and the chunks become vectors, and the closest chunks are pasted into the prompt. Closeness means "these strings embed near each other." It does not mean "this function calls that one." Two files can sit next to each other in vector space because they share vocabulary, and a real call edge can sit far apart because the names do not match.

The multi-hop version is worse, and it is not fixed by retrieving more chunks. A question that needs two steps has to land the first answer before the second step can be asked. Benchmarks such as HotpotQA exist because that second hop falls over. On a graph each hop is one edge, so the third is the same kind of step as the first. GraphRAG builds a knowledge graph over a document corpus for this reason. Graphify starts from the graph a parser already extracted.

On a repo, that extraction is local. Tree-sitter reads the code. Call and import edges come from the syntax tree, with no model in that step. Docs and other non-code files can be folded in by a model, and that model can stay on the machine. The map lands in graphify-out/ as graph.html, GRAPH_REPORT.md, and graph.json. Asking it something is a traversal:

graphify query "what calls this handler"
graphify path "webhook" "database"
Enter fullscreen mode Exit fullscreen mode

What comes back is a handful of nodes and edges, not the full text of every chunk that scored well.

The limit

Similarity is the right tool when the question is fuzzy and the corpus is prose. "Find the paragraph about this idea" is a nearest-neighbour problem. A graph will not beat an embedding index at that, and Graphify does not try. Keep a vector index if that is the job. Some teams run both: vectors for semantic recall over writing, the graph for anything that is a relationship.

Graphify can store an embedding as one signal on a node. Retrieval is still a walk. The embedding does not choose which passage gets pasted into the prompt.

Code edges come from a parser, so a changed file is re-parsed. graphify . --update re-scans what changed, and the rest of the graph stays. A RAG index is only as fresh as the last embedding run, and a stale chunk goes back through a model.

The open-source engine runs on your machine. Code parsing is local. There is no telemetry. Graphify Cloud is the same map, hosted and kept current. This post is about what gets stored. Cloud is a different product.

This also does not replace grep. A question that is really "find this string" should still be a search.

Proof

You can check an answer. Every edge carries a provenance tag: EXTRACTED from the syntax tree, INFERRED by the model, or AMBIGUOUS when the evidence did not resolve. A similarity score does not say why a chunk matched. A path does. The tags are explained in concepts.

The cost follows from the same difference, and the number belongs to the person who measured it. Steve Scargall, at MemVerge, reported 79× fewer tokens on a 496K-token codebase, with no vector database in the stack. That is his measurement, not a Graphify benchmark. A traversal sends the path. A retrieval pipeline sends the chunks.

Install

uv tool install graphifyy
graphify install
Enter fullscreen mode Exit fullscreen mode

Then, in the project:

/graphify .
Enter fullscreen mode Exit fullscreen mode

The package on PyPI is graphifyy (two y's). First graph: docs.graphify.com/guides/first-graph. The full comparison: graphify.com/vs/rag.

Top comments (12)

Collapse
 
ahmetozel profile image
Ahmet Özel •

The AMBIGUOUS provenance tag matters most for dynamic dispatch, dependency injection and callback registration. A syntax tree can identify a call expression without establishing which implementation will receive it at runtime, so an extracted edge should not automatically imply a complete execution path.

I would show a small example where one interface call has several candidate targets and preserve that uncertainty through a multi-hop traversal. If an early hop is ambiguous, later deterministic edges cannot make the full path certain. That would make the graph's relationship evidence useful without overstating what local parsing can establish.

Collapse
 
syedfahads profile image
Syed Fahad Graphify Labs Inc •

Spot on, Ahmet. That's why AMBIGUOUS is a first-class tag and not a footnote. It travels with the path, so the agent can see exactly which hop needs a closer look before it trusts the chain. An interface call with several candidate implementations is a great small example. We'll walk through one in a follow-up.

Collapse
 
ahmetozel profile image
Ahmet Özel •

Carrying AMBIGUOUS on the path also needs a rule for aggregation. If two independently discovered paths reach the same symbol and one contains an unresolved dispatch hop, merging their endpoints should not erase that uncertainty.

For the interface example, showing the candidate implementations alongside the unresolved hop would make the tag actionable. A trace could then resolve that particular edge without implying that every possible call through the interface has been established.

Thread Thread
 
syedfahads profile image
Syed Fahad Graphify Labs Inc •

Really good point on the merge rule, honestly hadn't thought about it that way. Right now for C# and Swift, a call through an interface links to every implementation, explain lists them with file and line, and those hops come out INFERRED with a lower score the more candidates there are. Carrying that across a whole path and not losing it when paths merge isn't there yet, so I've opened issues for both (github.com/Graphify-Labs/graphify/... and github.com/Graphify-Labs/graphify/...). Writing up a follow-up with a real two-implementation example, will share it here.

Thread Thread
 
ahmetozel profile image
Ahmet Özel •

Marking those C# and Swift hops INFERRED makes the current limitation visible. For the two-implementation follow-up, I would retain the dispatch call-site ID and the candidate-set ID along the path, rather than carrying only a reduced score.

Then two paths sharing the same unresolved dispatch cannot accidentally count as independent support when they merge. The explanation can show that shared ambiguity once, with both possible targets and their locations. Candidate count is useful context, but I would avoid presenting its inverse as a calibrated confidence probability.

Thread Thread
 
syedfahads profile image
Syed Fahad Graphify Labs Inc •

Love this framing, Ahmet. Two paths through the same unresolved call should count as one open question, not two votes. I'll show exactly that in the follow-up. Thanks for pushing on it!

Collapse
 
alexshev profile image
Alex Shev •

The provenance tags on graph edges are an important practical distinction: they give a reviewer a way to separate parser-extracted relationships from model-inferred ones. A useful next example would be a mixed query that starts with semantic retrieval for a concept, then presents the graph traversal and provenance for the code-level claim it makes.

Collapse
 
syedfahads profile image
Syed Fahad Graphify Labs Inc •

Thanks Alex, that's exactly the split we care about. Reviewers should be able to see at a glance which edges came straight from the parser and which were inferred. Love the mixed-query idea: semantic search to find the concept, then the traversal with a provenance tag on every hop. Adding it to the list for a follow-up post.

Collapse
 
alexshev profile image
Alex Shev •

A follow-up that shows one mixed query end-to-end would be valuable: retrieval finds a concept, traversal returns the dependency path, and every inferred hop is visibly distinct from parser-backed edges. That makes the graph’s value testable rather than just architectural.

Thread Thread
 
syedfahads profile image
Syed Fahad Graphify Labs Inc •

Took your suggestion and actually built it out. One real query on itsdangerous, traced hop by hop, with each hop marked as either read straight from the code or inferred. Funny enough, the inferred one turned out to be a catch-and-re-raise when I checked the source, which kind of proves your point about keeping them separate. Would love your take: dev.to/graphify/why-vector-search-...

Thread Thread
 
alexshev profile image
Alex Shev •

I read the walkthrough—nice to see the inferred hop called out and then checked against source. The next useful artifact would be a confidence label per hop plus a counterexample query, so readers can see where graph traversal helps and where it should yield to inspection.

Thread Thread
 
syedfahads profile image
Syed Fahad Graphify Labs Inc •

Quick update since you asked about per-hop labels: that bit already works. Every hop in graphify path shows EXTRACTED or INFERRED, and explain gives you the file and line for each connection. There's a numeric score stored too but it isn't printed yet, so I've opened an issue for that on our own OSS repo haha (github.com/Graphify-Labs/graphify/...), plus one for a path-level label so a single weak hop flags the whole path (github.com/Graphify-Labs/graphify/...). Doing a follow-up post with a real run, will drop the link here once it's up.