DEV Community: Nikita Rybalchenko

Your coding agent is burning tokens grepping your repo. Here's a one-command fix.

Nikita Rybalchenko — Sun, 28 Jun 2026 21:58:17 +0000

graphlens-mcp gives Claude Code, Cursor, and compatible clients a typed graph of your code, so they ask "who calls create_order?" and get one small answer instead of reading half the codebase. Below: how the engine works, what a 936-run benchmark says about when it actually pays off, and the five-minute install.

I've been building graphlens in the open. The full story is three posts on Habr — the engine, a benchmark, the product — but this version stands on its own; I've folded in what matters. Links at the end.

The loop everyone knows

Picture a big project. A few hundred thousand lines, Python on the backend, TypeScript on the front, a legacy corner nobody wants to touch. You point a coding agent at it and ask something ordinary: "how does auth work here?" or "what breaks if I change this method's signature?"

The agent can't see the whole repo at once. So it does the only thing it can: grep a name, open a file, read it, follow an import, grep again. It reads a dozen files, every one of them lands in the context window, and every one gets re-billed on the next turn.

This isn't hypothetical overhead. Anthropic's own engineering blog notes that tool definitions and intermediate results can eat "50,000+ tokens before an agent reads a request" — the window fills before the agent has even started on your question.

A code graph attacks exactly this. Instead of "read the file and eyeball it," the agent asks a precise question — who calls create_order — and gets back a small structured answer: resolved edges, not a text search and a prayer.

That's the pitch. The rest of this post is whether it holds up, and what it took to make it usable.

What the engine actually does

The hard part of "code → graph" isn't drawing boxes. It's the edges. Most lightweight tools resolve references by name: they see a call to save() and draw an edge to everything named save. Fast, and wrong — a real codebase has a dozen saves.

graphlens, the engine under the MCP server, splits the work in two:

Tree-sitter parses each file into a concrete syntax tree: exact structure, precise 1-based span positions. Every use-site gets recorded as an occurrence with a role — call, read, write, annotation, base class.
A type-aware resolver, specific to the language, answers definition_at(file, line, col) for each occurrence. The resolved definition becomes a real edge to the real declaration.

The resolvers are the same machinery your IDE runs: ty (Astral's Rust type-checker) for Python, the TypeScript Compiler API for TS, gopls for Go, rust-analyzer for Rust. So a CALLS edge points at the actual function, HAS_TYPE at the actual class, INHERITS_FROM at the actual base class. It's the difference between "probably related" and "related." The engine knows the process_order in services.py is the one called from api.py, not the namesake in tests/.

That handles the first wall — name ambiguity. The second wall is that most code-intel tools are monolingual. They understand Python beautifully and go blind the moment a TypeScript frontend calls a FastAPI route. Real systems are polyglot; the tools around them usually aren't.

graphlens emits language-neutral BOUNDARY nodes for the interfaces a service exposes or consumes: HTTP routes, queue topics, gRPC methods. The boundary ID carries no project and no language, and HTTP paths get normalized so /users/1, /users/{user_id} (FastAPI), <int:id> (Flask), and :id (Express) all collapse to the same key. A FastAPI route and a TypeScript fetch to that endpoint therefore produce the same boundary ID. Merge the two graphs, link them, and you get edges crossing the language border — which lets the agent answer "which frontend calls hit this endpoint?", a question a single-language tool can't even phrase.

Two more choices matter for trust. IDs are deterministic: a node's ID is a SHA-256 of project::kind::qualified_name, so the same scan yields the same IDs on any machine, which is what makes diffing and incremental updates work. And the graph never lies about being incomplete: if a toolchain is missing or a file fails type-checking, the resolver records a status (ok / degraded / unavailable) instead of quietly handing back a half-resolved graph. In CI, anything but ok fails the build.

Does it actually pay off? 936 runs

Here's the part most "my tool is faster" posts skip. I wrote that first piece, claimed agents burn tokens on grep, and put zero numbers behind it. So I built a benchmark to find out, and the answer surprised me.

The setup is one controlled variable. Same agent (Claude Code), same prompts, same tasks. The only thing that changes is which MCP server feeds the agent context. Four "hands": filesystem (grep + read), graphlens (the structural graph), serena (an LSP), and codegraph (a competing graph tool). Three models (Haiku, Sonnet, Opus), three seeds, 26 tasks on apache/superset (~400k lines, Python + TS). That's 936 runs.

A few things I locked down so the numbers mean something. The built-in Claude Code tools (Read, Grep, Bash) are disabled — otherwise the agent ignores the MCP server and the test measures nothing. Reference answers are hand-verified against a fixed tag, and crucially not generated by any tool under test. temperature=0 doesn't make these models deterministic, so three seeds and I report the median, not the mean. A run that hits the turn ceiling without answering counts as accuracy 0: "the tool didn't finish in budget," not "no data."

The headline finding: the ranking inverts depending on the task.

On simple point lookups — "where is class X defined," "what does it inherit from" — all four tools tie on accuracy. The only difference is price, a spread of roughly 3×, and graphlens sits unremarkable in the middle. If I'd measured only these, I'd have written "the graph isn't worth it, grep is fine." That would have been half the truth.

On the work that actually matters — blast-radius questions, finding every override, resolving an ambiguous name — the tools diverge hard:

Tool	Accuracy	Tokens	Tool calls	$/task
filesystem (grep)	0.71	12,596	27	$0.424
graphlens	0.84	748	1	$0.018
serena (LSP)	0.85	1,368	5	$0.065
codegraph	0.93	1,114	2	$0.036

grep collapses. Lowest accuracy, and it only reaches an answer in 83% of runs — the rest burn through the 50-turn ceiling. The runs that do finish cost 10–23× more and take 10–18× longer. Text search drowns in noise when the question is "every call to this" or "which of ten identically-named methods."

The same graphlens that looked dull on point lookups is now the cheapest ($0.018) and fastest option, answering in a single tool call instead of twenty-seven. That's roughly a 94% token cut against grep on these tasks. codegraph is the most accurate (0.93); serena holds its own.

There's a second twist I didn't predict: the best tool depends on which model you run. graphlens returns token-heavy results — graph neighborhoods, reference lists. On a cheap model that verbosity is nearly free, so on Haiku graphlens is the cheapest of all four. On Opus, which prices those same tokens far higher, graphlens becomes the most expensive of the structural tools (still cheaper than grep). serena and codegraph return tight, pointed results and stay cheap on any model.

Which leads to the one takeaway I'd bet money on: a cheap model on a structural tool beats an expensive model on grep. codegraph + Haiku (~$0.023, ~0.99 accuracy) beats filesystem + Opus (~$0.087, 0.93) on every axis at once.

One of my predictions flat-out failed, and it's worth reporting. I'd planted cross-language tasks (a TS call resolving to a Python handler across the /api/v1/... boundary) as a stress test, sure the single-language tools would trip. They didn't — every hand, grep included, solved both. The agent steps across the boundary on its own, whatever feeds it context. A benchmark that only confirms what you hoped isn't a benchmark.

The honest fine print: one repo, one harness, 26 tasks (20 simple, 6 hard). The cost difference is statistically solid; the accuracy gap on hard tasks is a strong signal but not proven at n=6. cost_usd is an API-equivalent, not your subscription bill. This is a reproducible measurement on one case, not a universal ranking — and the whole harness plus raw data is open if you want to run it on your own code.

The gap nobody mentions: an engine isn't a product

So the engine works on the tasks where a graph should work. But there's a hole I glossed over in both Habr posts: the engine is not something you can hand to an agent as-is.

graphlens, by design, stops at producing the graph. It doesn't own a database, doesn't watch the filesystem, doesn't reindex itself, doesn't raise a long-running service. For an engine that's the right call — a small core is trivial to test, cache, and compose. But to actually wire it into an agent, someone has to write the layer on top: graph storage, invalidation (which files to reindex when one changes), a filesystem watcher, an MCP server with tools the agent can call, registration in each client's config format, and a navigation skill so the agent knows how to use any of it.

That layer is the work everyone ends up redoing by hand. I wrote it once and packaged it as graphlens-mcp: a thin runtime over the engine that owns the storage, the freshness model, and everything the agent sees.

One command

uv tool install graphlens-mcp        # or: pipx install graphlens-mcp
cd your-project && graphlens-mcp init

init detects the project's languages, runs a toolchain "doctor," indexes the code into a local graph, writes the MCP server into your agents' configs (it knows Claude Code, Cursor, Windsurf, VS Code/Copilot, Codex CLI, and writes idempotently without clobbering your other servers), and installs the navigation skill. The agent starts the server itself from that config — you never run serve by hand. Restart the agent and ask it something like "what breaks if I change the signature of create_order?"

Requirements: Python ≥ 3.13 (inherited from the engine). MIT licensed. Current version 0.1.2, and it's early — more on that below.

What the agent gets

Eight tools, each cut to a specific question about code:

Tool	What it answers
`search_symbols`	Full-text search over symbol names — the entry point
`get_node_info`	Source snippet + signature + docstring + position
`get_file_structure`	A file's symbol outline
`get_callees`	What a function calls (outgoing, to `max_depth`)
`get_callers`	Who calls a function — the core of impact analysis
`get_neighbors`	Nodes within N hops, any direction
`find_references`	Non-calls: type annotations, assignments
`get_cross_language_calls`	Links across service boundaries (HTTP/gRPC/queues)

Every response carries a graph-quality status (ok or degraded), so the agent never mistakes a partial answer for a complete one. Lists are capped and flagged truncated rather than silently cut.

The pattern the navigation skill teaches: start at search_symbols, fan out with get_callers / find_references, and only pull get_node_info for the spots that actually need the source — instead of reading every calling file end to end.

Why the graph doesn't go stale while you type

What separates a product from "the engine plus a script" is that the graph stays current on its own while you edit.

A filesystem watcher starts with the server. When a file changes on disk, the server reindexes the connected set — the changed file, the files that import it, and the files it imports — in one full pass, so cross-file edges rebuild correctly instead of half-breaking. Deleting a file purges its symbols and updates its importers. The case everyone forgets — edits made while the server was off — is handled by a one-shot reconcile on startup: scan the project, index what's new, drop what vanished, refresh what changed, then hand control to the watcher.

The graph lives in .graphlens/graph.db (SQLite). It's a regenerable cache, safe to delete; reindex rebuilds it. Add .graphlens/ to your VCS ignore.

What it deliberately isn't

Status is early, and I'd rather say so than dress it up. The navigation core works; the rest is in progress.

The watcher reindexes the connected set of a change, not the whole project — a refactor that ripples through many layers of indirection may need a full reindex for a perfectly accurate graph. Cross-language COMMUNICATES_WITH edges rebuild on a full reindex and can erode on incremental edits. Languages other than Python need their toolchains present (Python works out of the box; ty ships with it); without them a language reports as degraded — structure parsed, calls and types not fully resolved — but init never blocks on it, and status tells you exactly what's missing.

And the one boundary that's structural, not a roadmap item: graphlens-mcp does not do embeddings or semantic "find me something like this" search. The graph is structural and type-aware, not a vector index. If you need "find code conceptually similar to rate limiting, whatever it's called," that's a vector tool's job. This answers the structural questions: who calls this, what it depends on, what breaks.

Try it, and tell me where it breaks

Install it if you run Claude Code, Cursor, or a compatible client on a big polyglot project and you're tired of watching the agent grep its way through the repo — especially if you do impact analysis before refactors, which is exactly the mode where the graph earns its keep. Skip it if your project is small (grep is instant on a few dozen files) or you mainly need semantic search.

Zero barrier to bail: everything's local, nothing leaves your machine, MIT, and it uninstalls in one command (graphlens-mcp remove --purge-db). Point it at your main project, confirm the MCP server is live in your agent, and compare the tool-call count on the same architectural question with the graph and without.

What I need most right now is independent runs on codebases that aren't superset. Issues, numbers from your projects, complaints about tool granularity — all welcome in the repo. The more measurements on different code, the closer this gets to an answer you can carry over, instead of "works on superset."

graphlens-mcp: github.com/Neko1313/graphlens-mcp
The engine: github.com/Neko1313/graphlens
The benchmark: github.com/Neko1313/agent-context-bench

How much does context cost an AI coding agent? grep vs graph vs LSP, measured across 936 runs

Nikita Rybalchenko — Wed, 24 Jun 2026 23:10:19 +0000

In my last post I described graphlens — what it does, how it works — and along the way I casually claimed that an agent "burns tokens grepping around a repo." I gave exactly zero numbers to back that up.

This post fixes that. Here are the measurements, the data, and a reproducible harness. Spoiler: the conclusion is not the one I expected going in, and that's the interesting part.

TL;DR

I took one agent (Claude Code), changed exactly one thing — which MCP server feeds it code context — and ran it over 26 tasks on apache/superset. Four "arms": filesystem (grep + read), graphlens (structural graph), serena (LSP), and codegraph. Three models (haiku / sonnet / opus), three seeds — 936 runs.

The headline: the answer flips depending on the kind of task.

On simple "where is X defined / what does X inherit from" lookups, all four tools are tied on accuracy. The only difference is cost (~3×). graphlens is unremarkable here.
On "estimate the blast radius / find every override / disambiguate an overloaded name" tasks, the tools separate hard: grep collapses (0.71 accuracy, only 83% of runs even finish, and the ones that do cost 6–24× more), while the structural tools stay cheap and accurate.

If I'd only measured the easy tasks, I'd have written "you don't need a graph, grep is fine." If only the hard ones, "you don't need grep, get a graph." The truth sits in the middle, and it's about what work you hand the agent.

The business case we're actually measuring

Picture a familiar situation. You have a large project: hundreds of thousands of lines, a Python backend, a TypeScript front end, legacy code you're scared to touch. You wire an AI agent into it — for review, refactoring, answering questions like "what breaks if I change this method's signature?"

The agent can't see the whole repo at once. Something has to feed it context: which functions live where, who calls whom, what inherits from what. And here's an architectural decision with a price tag: what exactly do you feed it?

There are basically four classes of answer:

Hand it grep + read — let it search by text and open files. Zero infrastructure, works everywhere.
Build a structural code graph (graphlens) — entity nodes, typed edges, exact answers to "who calls this."
Stand up an LSP (serena over a language server) — what your IDE already runs on.
Use an off-the-shelf code-graph product (codegraph).

Each option costs money (tokens), time (latency), and risk (the agent gives up and hits a turn cap). apache/superset is an almost perfect stand-in for this case: ~400k LOC, Python + TypeScript, an /api/v1/... boundary between front and back. A big polyglot project — exactly when this question is worth asking.

So how much does each option cost? Let's measure.

Experiment design: change one variable

The whole methodology rests on one principle: fix everything except one thing. Model, system prompt, settings, task set — constants. Only the context-providing MCP server changes. Then any difference in the numbers is the contribution of that tool, not a config accident.

No tool is designated "the baseline to beat." All four are measured on equal footing, and the numbers rank them.

The four arms

Arm	Context provider (MCP server)	Indexing step
`filesystem`	`@modelcontextprotocol/server-filesystem` (read_file + grep)	none
`graphlens`	graphlens graph over MCP	`graphlens analyze`
`serena`	Serena (LSP)	LSP workspace warm-up
`codegraph`	a graph-based competitor	`codegraph init`

One detail that matters for fairness: Claude Code's built-in tools (Read / Grep / Bash, etc.) are disabled. If you don't take them away, the agent ignores the MCP server and falls back to its usual path — and you'd be measuring the wrong thing. So the harness runs claude -p in a clean room: a fresh CLAUDE_CONFIG_DIR with only subscription credentials (no hooks, plugins, skills, memory), --strict-mcp-config (only this arm's server is visible), --disallowedTools on every built-in (an explicit deny, because in headless mode an allow-list alone forbids nothing), and --allowedTools mcp__<server> to auto-approve the one server.

The second axis: models

In parallel I varied the model answering the question:

Key	model id
`haiku`	`claude-haiku-4-5`
`sonnet`	`claude-sonnet-4-6`
`opus`	`claude-opus-4-8`

Why a second axis becomes clear near the end: the optimal tool depends on which model you picked. That's probably the least obvious finding in the whole thing.

Total: 4 arms × 3 models × 26 tasks × 3 seeds = 936 runs (on Claude Code 2.1.187).

What counts as an honest measurement

Benchmarks are easy to bend toward the conclusion you want. So the rules are fixed up front — without them the numbers aren't trustworthy:

Gold answers are hand-verified against source at tag 6.0.0 (every task carries a file:line reference). Crucially, gold is not generated by any tool under test (not ty, not pyright, not graphlens itself) — otherwise the comparison is biased toward whoever's output you labelled with. Set-task gold is checked with an independent oracle: Python's ast.
The "naive" arm has hands. filesystem is grep + read, not "an agent with no tools." Naive ≠ toolless.
Index cost is measured separately, once. grep pays nothing to index; a graph amortizes. You can't mix those currencies.
There is no determinism. temperature=0 does not make these models deterministic. So 3 seeds, and the report shows the median, not the mean.
Versions are recorded — models and every MCP server — plus a price snapshot and date.
cost_usd is an API-equivalent, not your bill. The subscription is flat-rate, so cost_usd (emitted by the CLI) is what the same tokens would cost via the API. It's not your actual invoice, but it is a correct relative $/task metric for comparing arms.
Use a tool, or it doesn't count. The system prompt forbids answering from memory; a run with zero tool calls is retried (and a stubborn refusal is tagged __NO_TOOLS__). Answering "from memory" about a well-known repo wouldn't measure the context provider.

And separately: failure counts as accuracy 0. If grep hits the 50-turn cap and never produces an answer, that's not "no data" — it's "the tool didn't get there within budget." That's how it's scored.

The tasks: two regimes, and why you can't blend them

26 tasks split into two classes.

SIMPLE — 20 pinpoint lookups ("where is X defined / what does X inherit from"). One-point answers, checked by substring:

Kind	#	What it probes
`where_defined`	7	Python class → defining file
`inherits_from`	5	Python class → base class
`abstract_methods`	1	ABC → its abstract methods
`ts_where_defined`	1	TS hook → defining file
`ts_route_call`	4	`/api/v1/...` route → the TS hook that calls it
`xlang_link`	2	TS consumer → Python handler across the API boundary

HARD — 6 blast-radius and disambiguation tasks. This is the regime where structure and semantics should beat text search — and which pinpoint lookups simply can't measure:

Kind	#	What it probes	Scoring
`disambiguate`	2	an ambiguous bare method name (e.g. `cache_key`, defined on many classes) → the right class	substring
`overrides_count`	2	the full set of subclasses overriding a base method	set F1
`impact_set`	2	every file calling a given method (the blast radius)	set F1

Set tasks are scored by F1: reward for recall (find them all), penalty for precision (text search loves to dump every occurrence of .get_indexes(). Gold sets are kept small (3–5 elements, one ≈17) so they can be exhaustively checked by hand.

Why I stratify instead of averaging

The set is deliberately unbalanced — 20 simple vs 6 hard. A single blended average would be entirely dictated by the easy tasks and would hide exactly the difference the hard ones expose. So I report each regime separately, and never mix them.

And no, I deliberately don't "balance to 50/50" by dropping simple tasks. That would throw away data and statistical power, and open the door to cherry-picking. Stratification neutralizes the skew without discarding data. (General principle: if regimes give different answers, it's more honest to show both than to bury the conflict under an average.)

Results

SIMPLE — 20 pinpoint lookups

Tool	accuracy	complete	tokens	calls	$/task	sec
filesystem	0.97	100%	1780	10	$0.063	43
graphlens	0.98	100%	690	3	$0.038	13
serena	0.99	100%	402	3	$0.031	20
codegraph	0.99	100%	372	1	$0.022	10

Accuracy is a tie (formally: Friedman χ²=0.40, not significant). The tools differ only on cost — a ~3× spread — and the terse ones win. graphlens is unremarkable here — a solid mid-pack.

This is exactly the story a benchmark that only measured pinpoint lookups would tell: "structural tools are nice, but grep nearly keeps up, and codegraph gives the cheapest answer." And it would be an incomplete truth.

HARD — 6 blast-radius and disambiguation tasks

Tool	accuracy	complete	tokens	calls	$/task	sec
filesystem	0.71	83%	12596	27	$0.424	165
graphlens	0.84	100%	748	1	$0.018	9
serena	0.85	98%	1368	5	$0.065	29
codegraph	0.93	100%	1114	2	$0.036	16

Now the tools separate.

grep collapses. Lowest accuracy (0.71), only 83% of runs finish (the rest hit the 50-turn cap), and the ones that finish cost 6–24× more ($0.42 vs $0.018–0.065) and take 6–18× longer (~165s vs 9–29s). Text search drowns in noise when the question is "every call to this" or "which of a dozen identically-named methods."

And the key bit: graphlens — the mid-pack tool on easy tasks — is here the cheapest ($0.018) and fastest (9s). Its semantic graph finally pays off: one call instead of twenty-seven. The most accurate tool is codegraph (0.93). serena is competitive (0.85).

So the same graphlens that looked unremarkable on pinpoint lookups becomes the most economical the moment the work is real — blast radius, refactoring. The ranking inverts between regimes.

Fairness note. MCP resources are disabled for all arms. graphlens was the only server exposing resources, and in an early run the agent wandered into enumerating them and inflated cost ~24% until I denied them. All numbers above are from the clean re-run.

Where the money goes: the mechanism is round-trips

The cost difference is mostly how many times the agent calls the tool, which follows from how a server slices its primitives.

On a simple "symbol → file" (where_defined), one call is enough for everyone. The gap opens on relationship queries — inheritance, route → handler, cross-language links. There graphlens chains fine-grained primitives (find → neighbors → references), while codegraph packs "source + call paths in one shot" (explore / node).

This isn't a difference in what the graph knows — graphs know roughly the same things. It's a difference in API granularity: fewer round-trips → cheaper and faster. That's why codegraph has the efficiency edge on simple tasks, and why grep bankrupts itself on hard ones — it makes 27 round-trips where the graph needs one or two.

Model × tool interaction: the ranking drifts with model price

This is the least obvious part. Take median $/task (across both regimes) broken down by model:

Tool	haiku	sonnet	opus
filesystem	$0.053	$0.080	$0.087
graphlens	$0.020	$0.041	$0.046
serena	$0.026	$0.033	$0.042
codegraph	$0.023	$0.041	$0.031

Cheapest-first ranking within each model:

haiku: graphlens \$0.020 < codegraph \$0.023 < serena \$0.026 < filesystem \$0.053
sonnet: serena \$0.033 < graphlens \$0.041 < codegraph \$0.041 < filesystem \$0.080
opus: codegraph \$0.031 < serena \$0.042 < graphlens \$0.046 < filesystem \$0.087

Watch what happens to graphlens. On haiku it's the cheapest of all. On opus it becomes the most expensive of the structural tools (still cheaper than grep, though).

The mechanism: graphlens results are token-heavy — graph neighborhoods, reference lists. On a cheap model that verbose context is nearly free; on an expensive one, opus prices the same tokens far higher, and verbosity hits the wallet. serena and codegraph stay cheap on any model because they return pinpoint results — they're robust to model choice; graphlens isn't.

Which gives the most valuable takeaway of the lot: a cheap model on a structural tool beats an expensive model on grep. codegraph + haiku (~$0.023, accuracy ~0.99) beats filesystem + opus (~$0.087, accuracy 0.93) on every axis at once.

The hypothesis that didn't hold

I planted the two xlang_link tasks as a stress test: a TS call resolves to a Python handler across the /api/v1/... boundary, and I was sure single-language tools would trip on it.

They didn't. Every arm, grep included, solved both cross-language tasks. The agent steps across the boundary itself, regardless of the context provider. On this set the hypothesis failed, and I report that as loudly as the findings that held. A benchmark that only reports what it hoped to see isn't a benchmark.

Statistics, honestly

Friedman test across the four tools, over task blocks, within each regime (df=3; critical values: 0.05 → 7.82, 0.01 → 11.34):

SIMPLE:
  accuracy  n=20  χ²= 0.40  (n.s.)    — tie
  cost      n=20  χ²=18.42  (p<.01)   — serena < codegraph < graphlens < filesystem

HARD:
  accuracy  n= 6  χ²= 3.50  (n.s.)    — underpowered
  cost      n= 6  χ²=11.80  (p<.01)   — graphlens < codegraph < serena < filesystem

What's honest to claim from this:

The cost difference is significant in both regimes (p<.01). On HARD, graphlens is reliably the cheapest and grep reliably the most expensive. That's a solid result.
The accuracy gap on HARD is large but not statistically significant at n=6 (χ²=3.50). It's a strong descriptive signal, not a proven one. Six tasks is few.
To firm up the accuracy claim you'd add hard tasks, not cut simple ones. Trimming the simple regime gives the hard one zero extra power — it just throws away good data.

I'm leaving this in the article on purpose. The temptation to write "graphlens/codegraph are more accurate than grep, proven" is real, but n=6 doesn't carry it, and pretending otherwise would be dishonest.

Index amortization: different currencies

The structural tools build an index once — pure static work, zero LLM tokens, wall-clock only:

Tool	one-time index
filesystem	0s
codegraph	48s
graphlens	84s
serena	94s

grep pays nothing up front but pays more per query. These are different currencies (seconds vs $/tokens), so I draw no single "break-even point" — that'd be a stretch. The picture is simple: the index is a one-time time cost with not a single token spent, while the $/task savings drip on every task. Over a long session the structural tools amortize; on a couple of one-off queries, grep's zero setup can win on time-to-first-answer.

Takeaways for the business case

Back to the original question: what do you feed the agent on a large project?

There is no "this tool is always best" answer. There's a "depends on what work you hand it" answer:

One-off pinpoint lookups ("where is this class defined," "what does it inherit from"): use anything. grep keeps up, accuracy is the same, zero setup. You pay at most a small token overhead.
Sustained blast-radius work — refactoring, impact analysis, disambiguation on a large base: structural tools cut cost 6–24× and latency 6–18× vs grep — and, just as important, they don't hit the turn cap. grep on these tasks isn't just expensive; 17% of the time it never reaches an answer at all.
Model choice interacts with tool choice. A verbose graph is cheap on a small model and pricey on a big one. Running opus? Pick a tool with pinpoint output (codegraph, serena). Running haiku? graphlens is suddenly the cheapest.
The cheapest combo isn't "expensive model + simple tool" — it's "cheap model + structural tool."

And the honest caveats, without which you can't transfer the conclusions to your project:

One repository (apache/superset @ 6.0.0), one harness, 26 tasks (20 simple / 6 hard). Regimes are reported separately and never blended. cost_usd is an API-equivalent, not a subscription bill. Failure = accuracy 0. This is not a universal ranking — it's a reproducible measurement on one concrete case.

Where graphlens fits

Since this is a follow-up to the graphlens post, let me say it straight. This benchmark does not prove graphlens is "the best." It shows the specific regime where its structural graph pays off (impact analysis, cheap and fast on cheaper models), and just as plainly shows where it lags (on opus its verbose output costs more than codegraph and serena; codegraph is more accurate on hard tasks).

For me that's more useful than any victory lap. graphlens was built as an engine and a precise polyglot graph model, not a turnkey app — and the benchmark confirms exactly that: on structural questions the graph beats text search by a wide margin, and there's clear room to grow — MCP tool granularity (fewer round-trips, like codegraph) and output compactness (so it doesn't bankrupt itself on expensive models). That's my next work item, now backed by numbers instead of intuition.

Reproduce it

The whole harness and the raw data are open. A run reassembles deterministically from data/.

Benchmark repo: https://github.com/Neko1313/agent-context-bench
See metrics.ipynb (all charts and per-section stats) and README.md (methodology).
uv run main.py runs the full pipeline (clone superset → build indices → 936 runs, resumable within subscription limits), then open metrics.ipynb.

If you've got a large project of your own and the itch to run the harness on it — issues and results welcome. The more independent runs across different codebases, the closer we get to an answer that transfers, rather than "works on superset."

(More on the tool itself: graphlens · docs.)

graphlens: a polyglot code-analysis framework that turns your repo into a typed graph

Nikita Rybalchenko — Mon, 22 Jun 2026 01:19:45 +0000

graphlens: turn any repo into one typed graph — across Python, TypeScript, Go and Rust

Every code-intelligence tool I've ever used falls into one of two traps.

The first is the grep-and-read loop: you (or your AI agent) search for a name, open ten files, read around the matches, follow an import, search again. It works, but it's slow, it burns tokens, and it has no idea that the process_order you found in services.py is the same process_order that gets called from api.py — versus the unrelated one in tests/.

The second is the single-language silo: tools that understand Python beautifully but go blind the moment your TypeScript front end calls a Python FastAPI route. Real systems are polyglot. Your tooling usually isn't.

graphlens is an open-source (MIT) framework built to escape both traps. It parses a source project, normalizes its structure into a shared graph IR, and hands you that graph to do whatever you want with — dependency analysis, navigation, dead-code detection, or feeding an LLM agent precise answers instead of file dumps.

Repository → Language Adapter → GraphLens (IR) → Graph Backend

Layer	Responsibility
Language Adapter	Parses source files, produces a `GraphLens`
GraphLens	Typed nodes + directed relations — the intermediate representation
Graph Backend	Persists or queries the graph (Neo4j, in-memory, your own)

The key design decision: adapters are pure data producers. They never write to a database, never touch the filesystem after reading, never run a server. The graph is the only output. That makes the whole pipeline trivially testable, cacheable, and serializable.

30 seconds to your first graph

pip install "graphlens-cli[python]"
graphlens analyze ./my-project

graphlens · my-project
  nodes:      1240
  relations:  3981
  resolver:   ok

nodes by kind        relations by kind
  FUNCTION    410       CONTAINS    980
  METHOD      265       DECLARES    870
  CLASS        98       CALLS       640
  MODULE       54       REFERENCES  410

Or from Python:

from pathlib import Path
from graphlens import adapter_registry

adapter = adapter_registry.load("python")()
graph = adapter.analyze(Path("./my-project"))

print(len(graph.nodes), "nodes,", len(graph.relations), "relations")

fn = graph.nodes_by_name("process_order")[0]
print("called by:", [n.name for n in graph.callers(fn.id)])

What makes the edges real (and not name-matching guesses)

Most lightweight code-graph tools resolve references by name: see a call to save(), draw an edge to anything called save. That's fast and wrong — there are usually a dozen saves in a codebase.

graphlens splits the work in two:

Tree-sitter parses every file into a concrete syntax tree, giving exact structure and 1-based span positions. It records every use-site as an occurrence with a role (call / read / write / annotation / base).
A language-specific, type-aware resolver then answers definition_at(file, line, col) for each occurrence. The resolved definition becomes a real edge to the actual declaration node.

Language	Resolver	Engine
Python	`TyResolver`	`ty` (Astral, Rust-based) via LSP
TypeScript	`TsResolver`	the TypeScript Compiler API (Node subprocess)
Go	`GoplsResolver`	`gopls`
Rust	`RustAnalyzerResolver`	`rust-analyzer`

So a CALLS edge points at the real function, a HAS_TYPE edge at the real class, an INHERITS_FROM edge at the real base. This is the difference between "probably related" and "is related".

Honesty about partial failures

Type analysis can degrade — a toolchain is missing, a file doesn't type-check. Instead of silently producing a half-resolved graph, graphlens records the outcome:

from graphlens import RESOLVER_STATUS_KEY
graph.metadata[RESOLVER_STATUS_KEY]   # 'ok' | 'degraded' | 'unavailable'

In CI you flip on --strict and a non-ok status fails the build, so an agent or dashboard never consumes a graph that's quietly incomplete.

The graph model

Nodes (PROJECT, MODULE, FILE, CLASS, METHOD, FUNCTION, PARAMETER, VARIABLE, ATTRIBUTE, TYPE_ALIAS, IMPORT, DEPENDENCY, EXTERNAL_SYMBOL, BOUNDARY) are frozen dataclasses with an id, kind, qualified name, file path, span, and free-form metadata.

Relations are directed, typed edges:

Kind	Meaning
`CONTAINS` / `DECLARES`	structural containment & declaration
`IMPORTS` / `RESOLVES_TO`	import statements and where they resolve
`CALLS` / `REFERENCES` / `INHERITS_FROM` / `HAS_TYPE`	resolved, type-aware edges
`DEPENDS_ON`	declared package dependency
`EXPOSES` / `CONSUMES` / `COMMUNICATES_WITH`	cross-language boundaries

Deterministic IDs

A node's ID is a SHA-256 hash of project::kind::qualified_name:

from graphlens import make_node_id
make_node_id("my-project", "my.module.func", "FUNCTION")
# → the same id every scan, on every machine

Because the ID depends only on identity, not file position, re-scanning yields the same IDs. That's what makes graph.diff(other) and incremental updates work — and what makes a graph cacheable in CI.

The feature single-language tools can't have: cross-language boundaries

This is my favorite part. Adapters emit language-agnostic BOUNDARY nodes for the interfaces a service exposes or consumes — HTTP routes, queue topics, gRPC methods, Temporal activities — with an EXPOSES edge (provider) or CONSUMES edge (consumer).

A boundary's ID is make_boundary_id(mechanism, key) — no project or language in it. HTTP paths are normalized so that /users/1, /users/{user_id} (FastAPI), <int:id> (Flask), and :id (Express) all collapse to GET /users/{}.

The payoff: a Python FastAPI route and a TypeScript fetch to the same endpoint produce the same boundary ID. Merge the two graphs, run graphlens-link, and you get COMMUNICATES_WITH edges spanning the language gap:

from graphlens import adapter_registry
from graphlens_link import link_graph

py = adapter_registry.load("python")().analyze(python_project)
ts = adapter_registry.load("typescript")().analyze(typescript_project)

merged = py
merged.merge(ts, allow_shared=True)   # identical BOUNDARY nodes coincide
result = link_graph(merged)           # adds consumer → provider edges

print(result.relations_added, "COMMUNICATES_WITH edges added")

Now you can answer "which front-end calls hit this endpoint?" — a question no single-language tool can even represent.

Five ways to use it

As a library — load an adapter, get a GraphLens, query it: callers, callees, references, neighborhoods, diffs, JSON round-trips, multi-language merges.

From the CLI — five subcommands cover the common workflows:

graphlens analyze ./repo --output graph.json   # index
graphlens query process_order -g graph.json --op callers
graphlens visualize ./repo                      # interactive vis.js HTML
graphlens neo4j ./repo --uri bolt://localhost:7687
graphlens mcp --graph graph.json                # serve to agents

In CI — --strict plus a Docker image (ghcr.io/neko1313/graphlens) with every adapter and toolchain pre-installed. Index on every push, publish the graph as an artifact, fail on a degraded graph.

To LLM agents over MCP — graphlens mcp exposes a saved graph as Model Context Protocol query tools (stats, find, callers, callees, references, neighbors, boundaries, communicates_with). Instead of dumping a codebase into the prompt, the agent asks precise questions and gets small structured answers — resolved edges, not best-effort text search.

As a Neo4j export — straight into a graph database with UNWIND … MERGE Cypher (no APOC required), then query it however you like.

Plugin architecture: the SQLAlchemy-dialect pattern

The core never imports an adapter. Each language is a separate package that registers itself via Python entry points:

[project.entry-points."graphlens.adapters"]
python = "graphlens_python:PythonAdapter"

Callers resolve adapters through a registry, by name string:

adapter_registry.available()        # ['python', 'typescript', ...]
adapter = adapter_registry.load("python")()

Adding a new language means writing one package against the LanguageAdapter contract — no changes to the core.

What graphlens is not

The scope is deliberately narrow, and the docs spell it out. graphlens produces a graph IR and stops there. It does not:

persist state or own a database (backends are a separate consuming layer);
watch the filesystem or re-index incrementally on its own (scans are pure functions; deterministic IDs enable incremental updates, but the caller drives them);
compute embeddings, semantic search, or relevance ranking (the graph is structural and type-aware, not a vector index);
provide a UI or an agent runtime (visualize emits static HTML, mcp exposes query tools — neither hosts a long-running service).

Those belong to tools built on top of graphlens. Keeping the core minimal is what keeps it composable.

Benchmarks

Throughput on real-world projects, refreshed on every release inside the published Docker image (single cold run, indicative):

Project	Lang	LOC	Nodes	Time	Resolved
apache/superset	python	399 519	156 251	148.7s	84%
colinhacks/zod	typescript	74 194	8 741	19.0s	91%
gin-gonic/gin	go	23 672	7 227	13.9s	100%
gohugoio/hugo	go	224 821	34 809	112.7s	99%
BurntSushi/ripgrep	rust	50 275	9 612	113.1s	99%

Try it

pip install "graphlens-cli[python]"
graphlens analyze . --output graph.json
graphlens visualize .

Repo: https://github.com/Neko1313/graphlens
Docs: https://Neko1313.github.io/graphlens/
Requirements: Python 3.13+. Python (ty) and TypeScript (Node) toolchains install on demand; Go and Rust adapters come via the Docker image.

If you've ever wanted a single, accurate, language-agnostic model of "how does this codebase actually fit together" — that's exactly what graphlens hands you. I'd love feedback, issues, and adapter contributions.