How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms

Jiangang Chen — Thu, 20 Aug 2026 18:51:52 +0000

How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms

If your coding agent has ever stalled for tens of seconds on "what's in this repo?" — or burned hundreds of tokens re-reading a file after a failed edit — this is the story of why that happens and how we fixed it.

TL;DR — we rebuilt code tooling for agents that have no hands, no eyes, and no memory:

repo_map in 98ms (was tens of seconds): Rust tree-sitter parse daemon + SQLite index + incremental self-heal
Every write is transactional, with an undo journal that survives kill -9 — no more silently lost work
Quality gates are deterministic (zero LLM calls) and honestly scoped
44 tools across read / analyze / edit / gate / verify / system — MIT, zero-build deploy

LiuHe (https://github.com/wulun811/LiuHe) is a code-operation toolchain designed for LLMs rather than humans: Node orchestration, a Rust tree-sitter parse daemon, a SQLite symbol index, and transactional journal-backed writes. MIT, v0.4.6, zero-build deploy (no cargo, no npm install).

This post is the architecture story: what was slow, what we changed, and the numbers we measured while doing it.

The 44 tools, at a glance

Six families, all deterministic, all reproducible from benchmarks/:

Read & index — read_symbol (version-anchored), symbol_search, code_search, repo_map (98ms, paginated skeleton), reindex, dep_graph, references
Analyze — impact_analysis, call_chain, trace_symbol (constant tracing), inspect, sweep_dead_code, config_drift
Edit — edit_batch (4-level tolerant matching), edit_transaction (atomic + undo journal), rename_symbol, git_worktree, edit_sandbox, edit_collision_guard, diff_facts
Gate — code_review, security_review, code_quality, style_sniffer, guard_patterns, naming_consistency, dependency_gatekeeper, fix_imports, mock_sync — zero LLM calls
Verify — test_bridge, find_tests, verify_pipeline, debug_runner, tsc_check, patch_parser, spec_gen
System — health (self-healing), gc, feedback

If you've watched an agent burn thousands of tokens re-reading a file because a sed didn't match, or lose work to a silently botched write — these are the failure modes this toolkit exists for. Everything below is about making those tools fast, safe, and cheap.

The problem: every query re-parsed the world

Our first version answered "what symbols exist in this repo?" by walking the tree and parsing every file on every request. On a 347-file project that was tens of seconds. On a real Ansible repo of 1,482 files it was worse — and agents ask for repo maps constantly (every tool-call needs file → symbol → reference context).

The fix came in three layers:

A Rust parse daemon — all CPU-bound AST work lives in Rust (tree-sitter + tokio + rayon), talking to Node over a Unix socket. Zero-copy source slicing, no per-node N-API boundary crossings, true parallelism.
A SQLite index — parse results go straight into SQLite (WAL mode, per-workspace). Every subsequent query becomes a point lookup instead of a re-parse.
Incremental self-heal — mtime + dirty flags re-extract only changed files; if the Rust binary's sha256 changes, the whole DB is marked dirty and rebuilt automatically. "Forgot to rebuild the index" is prevented by construction.

Result: full index of 1,482 files in 9.7s (153 files/s); repo map afterwards: 98ms.

Why parsing lives outside Node

The original tree-sitter binding inside Node had two failure modes: a parse exception killed the entire MCP process (on average every 2–4 hours of use), and GC pauses + per-node JS↔C crossings made batch indexing stall.

The Rust daemon fixes both:

Crash isolation via catch_unwind — a panic in the parser becomes a PARSE_PANIC error code; the MCP server keeps running. Users forgive slow, never dead.
No GC pauses in the parse path; rayon gives real parallelism without worker_threads startup costs.

Writing code without hands

Human tools assume you have hands, eyes, and memory. An LLM has none. Three compensations:

No hands → atomic operations. edit_transaction is all-or-nothing; every write produces an undo journal. We tested kill -9 mid-write: the half-written transaction rolls back, source files untouched.
No eyes → structured output. Every tool returns machine-consumable JSON, never prose the model has to parse. Errors carry suggestion and next_action — an executable recovery call the model reissues verbatim instead of guessing.
No memory → self-contained calls. Every call carries workspace_dir; writes are version-anchored (optimistic concurrency), so even if the model forgets the version it read, the write fails loudly instead of silently corrupting.

On the "silent corruption" point: while building with a default agent tool stack, one overwrite write silently lost 400+ lines — surfaced ~40 turns later, by luck. We stopped betting on "models will get better" and moved the safety into the tool layer.

Errors aren't a dead end — they're an interface. Every failure carries a stable code, a human-readable suggestion, and a next_action that is executable, not advice:

{ "error": { "code": "VERSION_CONFLICT", "message": "base_version mismatch: FILE_CHANGED", "suggestion": "Re-read the file and regenerate the batch.", "next_action": { "tool": "read_symbol", "params": { "locator": { "file_path": "src/api.js" } } } } }

The model doesn't parse the suggestion and decide what to do — it reissues next_action verbatim and recovers. Successful calls carry a next_step the same way. Errors become signposts with navigation instead of dead ends.

`edit_batch`: tolerant matching, paranoid writing

LLMs generate old_string anchors with mistakes humans rarely make: collapsed double spaces, truncated line ends, curly quotes where the code has straight ones. A bare no_match sends the model off to re-read the whole file — thousands of tokens per failed match.

Matching degrades in four stages: exact → trailing-whitespace-stripped → edit-distance candidates (similarity ≥ 0.5) → diagnostics (whitespace visualized, 17 Unicode confusable pairs listed). A typical failure reads: "candidate at line 42, similarity 0.87 — you used curly quotes, the code has straight ones." Usually one retry fixes it.

The write side is paranoid: symlink guards, unique temp file names, TOCTOU check between match and commit, rename retries for Windows AV file locks, post-write syntax check (node --check / py_compile / JSON.parse).

Deterministic quality gates, honestly scoped

security_review, code_review, sweep_dead_code are pure regex/AST — zero LLM calls. Same input, same output; CI-safe and auditable.

We state the boundary explicitly: these tools do not cover control flow, data flow, or cross-module semantics. Zero findings ≠ safe; a high score ≠ healthy. A deterministic pattern scanner that admits its scope beats a "comprehensive security" claim every time.

The tool audits itself

30+ rounds of "LiuHe reviews LiuHe" — every bug found becomes a regression test. Real fixes from those rounds: a scope filter scanning outside its target directory, dead-code false positives on registration patterns, constant tracing missing read sites, SQL parameterization cleanup. Assertions grew every round: 2,013 JS + 92 Rust, full chain green.

Why the name: 六合 (six harmonies)

LiuHe (六合, "six harmonies") names the six design constraints applied to every tool in the toolkit:

Contract — parameters are self-describing; ambiguity returns candidates instead of guessing
Guard — dry-run, collision, and syntax checks run before anything executes
Persist — every operation is atomic, idempotent, and undoable
Frugal — incremental returns, batching, and trimming keep token spend down
Observable — trace ids, pipeline steps, and a recovery path on every failure
Trace-back — misuse data feeds back into thresholds

The three compensations earlier (hands / eyes / memory) are the user-facing summary; these six are the per-tool checklist behind them. The AST layer that enforces them is called Malong.

Measured numbers (not paper benchmarks)

All under a real docker --memory=512m cgroup:

Metric	Value
`repo_map`	98ms (was tens of seconds)
Full index	1,482 files in 9.7s
Concurrency	128 concurrent / 256 in-flight, zero OOM
Peak RSS	134MB (~26% of limit)
Throughput	~588 calls/s (60–600× realistic agent demand)
Hot-file storm	32-way read/write mix, zero torn writes, `integrity_check` PASS, 95/95 conflicts rejected as `FILE_LOCKED`
Token savings	↓65.3% (7,673 → 2,662 est. on same task)

Honest boundary: throughput doesn't scale with concurrency — better-sqlite3's synchronous queries serialize on the Node event loop. We evaluated worker_threads and decided the risk wasn't worth the gain. 588 calls/s is already overkill.

Where the token savings come from: tiered tool-description compression (44 tools ≈ 1.33k tokens — core tools keep full descriptions, low-frequency ones shrink to ≤70 chars, verbose ones ≤230, with the detail deferred to next_step hints), incremental returns with explicit pagination instead of dumping everything, and batch endpoints (read_symbols, write_symbols) that cut round-trips. Same task: 7,673 → 2,662 estimated tokens (↓65.3%) and 6 calls → 3 (↓50%).

All benchmarks are reproducible from benchmarks/ and tests/ in the repo (concurrency correctness: tests/test-mvp-concurrency.js).

Day-one DeepSeek Harness support

We also shipped first-day support for DeepSeek Harness (dsh web) — one line to register, all 44 tools exposed as malong__* with workspace_dir auto-filled from the conversation's workspace:

dsh plugin --profile web add @jieai/dsh-malong-bridge

Full guide in the repo: malong/dsh/DSH-INTEGRATION.md.

Try it

Zero-build deploy (no cargo, no npm install):

git clone https://github.com/wulun811/LiuHe liuhe && cd liuhe/malong
mkdir -p ~/.local/bin
tar -xzf ../releases/malong-liuhe-0.4.6-linux-x86_64.tar.gz
cp malong-parse/target/release/malong-parse ~/.local/bin
malong-parse &   # start the parse daemon
node --max-old-space-size=512 --expose-gc mcp-server.js --workspace /path/to/project

If better-sqlite3 is unavailable, it falls back to vendored sql.js WASM — no install, no compile, no network.

GitHub: https://github.com/wulun811/LiuHe
Docs site: https://wulun811.github.io/LiuHe
Changelog: https://github.com/wulun811/LiuHe/blob/master/CHANGELOG.md

Skepticism welcome — all numbers above are self-measured and reproducible. If this resonates with an agent failure you've had, try it, break it, and tell us where we're wrong — the repo is wulun811/LiuHe.

DEV Community: Jiangang Chen

How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms