Code Knowledge Graphs: How to Evaluate Graphify, GitNexus, and CodeGraph

#mcp #ai #architecture #opensource

Short version: Graphify, GitNexus, and CodeGraph all parse your repo with tree-sitter and serve a graph to an AI coding agent over MCP. They occupy the same square of the market, so raw capability will not separate them. Three things will: the license, how long the graph stays out of date, and how much of the call graph the tool can actually resolve.

What is a code knowledge graph, and what is it not?

A code knowledge graph is a pre-built index of every symbol in a repo and every edge between them: calls, imports, extends, implements. An agent queries that index instead of running grep and read until it reconstructs your architecture. In CodeGraph's benchmark across seven open source repos, the agent's file reads dropped to roughly zero on all seven (README).

What it is not is ground truth. Every edge was either read from source or guessed by a heuristic, and some edges never get built at all. Treat the output as a lower bound on impact, never a guarantee. That reframe is the difference between using a graph to explore a system and using it to approve a change.

Where do these tools sit in the market?

Two axes matter: how deep the analysis goes, and where it runs.

Text match. grep. Free, exhaustive, no structure.

Symbol index. Serena runs real language servers and answers at the symbol level on your machine, and it also edits by symbol path rather than line number. Sourcegraph indexes with SCIP and returns every reference rather than a ranked list, at multi-repo scale, on a server (Sourcegraph).

Call graph. All three tools in this article, plus FalkorDB's code-graph, which needs a graph database running.

Data flow. Joern builds a code property graph (syntax tree plus control flow plus program dependence) for questions like whether user input reaches a query unsanitized. GitNexus reaches into this tier with an opt-in --pdg mode, currently TypeScript and JavaScript only.

The three land in the same cell. Evaluate them on operations and risk, not on feature lists.

How stale does the graph get?

This is the failure mode nobody demos. Between your edit and the next index, the tool answers from a graph that describes code you no longer have.

CodeGraph watches the filesystem with native OS events and re-indexes after a 2 second debounce (tunable via CODEGRAPH_WATCH_DEBOUNCE_MS). Inside that window its responses prepend a banner naming the pending file and telling the agent to read it directly. It also reconciles on reconnect, so edits made while no server was running get absorbed on the next query.

Graphify ties refresh to commits. Between commits, the graph drifts from your working tree.

GitNexus needs gitnexus analyze re-run. Incremental indexing is still listed under actively building, though a post-commit hook can prompt a reindex in Claude Code and Codex.

If your team points agents at a repo they are actively editing, measure this first. An index that lags by a working session is worse than no index, because the agent stops hedging.

Can you trust the blast radius?

Every one of these tools reports impact analysis. None of them can report the edges it failed to build, so read the provenance model before you trust a number.

Graphify tags each edge EXTRACTED, INFERRED, or AMBIGUOUS, so you can tell what was read from source and what was reasoned (graphify.com). GitNexus attaches a confidence score and, when several symbols share a name, returns a ranked candidate list instead of guessing. CodeGraph publishes measured cross-file coverage per language against a named benchmark repo and does not hide the weak results: 95.8% on TypeScript, 86.7% on Rust, 73.8% on Liquid.

The residual is a static analysis ceiling, not a bug to file. Dynamic dispatch, reflection, dependency injection containers, and framework conventions resolve at runtime, so no AST parser will ever see them.

Which gate eliminates a tool first?

	Graphify	GitNexus	CodeGraph
License	MIT	PolyForm Noncommercial	MIT
Telemetry	none	none	anonymous, on by default
Languages	36	14	30+
Query language	none	Cypher	none
Human-readable output	`graph.html`	browser UI	none
Non-code input	docs, PDFs, SQL, Terraform	no	no

Run the license gate first. GitNexus ships under PolyForm Noncommercial, so commercial use needs a license from Akon Labs (README). That is a procurement conversation, not an install command, and it should happen before anyone builds a proof of concept on it. CodeGraph's telemetry is anonymous and documented, and it is on until you run codegraph telemetry off.

What to test on your own repo

Edit a file, then immediately ask about it. This measures the staleness window, which is the only number that reflects daily use.
Pick a symbol whose blast radius you already know. Compare what the tool returns against what you know breaks. The gap is your resolution ceiling.
Check your real language mix, including infrastructure code. Terraform, SQL, and legacy modules are where coverage varies most.
Decide who reads the output. If only agents do, visualization is dead weight. If a lead needs to review architecture, its absence is a blocker.
Time a cold index on your largest repo, then a warm one. Adoption dies on the first number, not the second.

Do not adopt any of these on a small repo. On a few hundred files an agent's own search is fine and the index is overhead. The category earns its keep on large, tangled, polyglot codebases, which is exactly where it is hardest to evaluate cheaply.

FAQ

Is a code knowledge graph better than vector RAG for code?
For structural questions, yes. Vector search returns chunks ranked by similarity and hopes the model reconnects them, while a graph returns an explicit path with file and line citations. That makes the answer auditable rather than merely plausible.

Does my code leave my machine?
Not for source parsing. All three parse locally with tree-sitter and zero model calls on code. Graphify's optional pass over docs and PDFs uses a model API you configure, which can point at a local Ollama backend.

Can I gate a pull request on graph impact analysis?
Only as a signal, not as a merge condition. The graph cannot enumerate the edges it failed to resolve, so a clean impact report is weak evidence of safety.

Which one supports the most languages?
Graphify lists 36 grammars, CodeGraph more than 30 in its language table, GitNexus 14. CodeGraph reaches the unusual ones: COBOL, Solidity, Terraform, Nix, Erlang, VB.NET, Delphi.

Do I still need an LSP tool like Serena?
Often yes. Graph tools find the blast radius, Serena makes the edit at symbol level. They solve different halves of the problem and run side by side.