How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms

How we cut repo-wide symbol indexing for LLM agents from 30s to 98ms

If your coding agent has ever stalled for tens of seconds on "what's in this repo?" — or burned hundreds of tokens re-reading a file after a failed edit — this is the story of why that happens and how we fixed it.

TL;DR — we rebuilt code tooling for agents that have no hands, no eyes, and no memory:

• repo_map in 98ms (was tens of seconds): Rust tree-sitter parse daemon + SQLite index + incremental self-heal
• Every write is transactional, with an undo journal that survives kill -9 — no more silently lost work
• Quality gates are deterministic (zero LLM calls) and honestly scoped
• 44 tools across read / analyze / edit / gate / verify / system — MIT, zero-build deploy

LiuHe (https://github.com/wulun811/LiuHe) is a code-operation toolchain designed for LLMs rather than humans: Node orchestration, a Rust tree-sitter parse daemon, a SQLite symbol index, and transactional journal-backed writes. MIT, v0.4.6, zero-build deploy (no cargo, no npm install).

This post is the architecture story: what was slow, what we changed, and the numbers we measured while doing it.

The 44 tools, at a glance

Six families, all deterministic, all reproducible from benchmarks/:

• Read & index — read_symbol (version-anchored), symbol_search, code_search, repo_map (98ms, paginated skeleton), reindex, dep_graph, references
• Analyze — impact_analysis, call_chain, trace_symbol (constant tracing), inspect, sweep_dead_code, config_drift
• Edit — edit_batch (4-level tolerant matching), edit_transaction (atomic + undo journal), rename_symbol, git_worktree, edit_sandbox, edit_collision_guard, diff_facts
• Gate — code_review, security_review, code_quality, style_sniffer, guard_patterns, naming_consistency, dependency_gatekeeper, fix_imports, mock_sync — zero LLM calls
• Verify — test_bridge, find_tests, verify_pipeline, debug_runner, tsc_check, patch_parser, spec_gen
• System — health (self-healing), gc, feedback

If you've watched an agent burn thousands of tokens re-reading a file because a sed didn't match, or lose work to a silently botched write — these are the failure modes this toolkit exists for. Everything below is about making those tools fast, safe, and cheap.

The problem: every query re-parsed the world

Our first version answered "what symbols exist in this repo?" by walking the tree and parsing every file on every request. On a 347-file project that was tens of seconds. On a real Ansible repo of 1,482 files it was worse — and agents ask for repo maps constantly (every tool-call needs file → symbol → reference context).

The fix came in three layers:

1. A Rust parse daemon — all CPU-bound AST work lives in Rust (tree-sitter + tokio + rayon), talking to Node over a Unix socket. Zero-copy source slicing, no per-node N-API boundary crossings, true parallelism.
2. A SQLite index — parse results go straight into SQLite (WAL mode, per-workspace). Every subsequent query becomes a point lookup instead of a re-parse.
3. Incremental self-heal — mtime + dirty flags re-extract only changed files; if the Rust binary's sha256 changes, the whole DB is marked dirty and rebuilt automatically. "Forgot to rebuild the index" is prevented by construction.

Result: full index of 1,482 files in 9.7s (153 files/s); repo map afterwards: 98ms.

Why parsing lives outside Node

The original tree-sitter binding inside Node had two failure modes: a parse exception killed the entire MCP process (on average every 2–4 hours of use), and GC pauses + per-node JS↔C crossings made batch indexing stall.

The Rust daemon fixes both:

• Crash isolation via catch_unwind — a panic in the parser becomes a PARSE_PANIC error code; the MCP server keeps running. Users forgive slow, never dead.
• No GC pauses in the parse path; rayon gives real parallelism without worker_threads startup costs.

Writing code without hands

Human tools assume you have hands, eyes, and memory. An LLM has none. Three compensations:

• No hands → atomic operations. edit_transaction is all-or-nothing; every write produces an undo journal. We tested kill -9 mid-write: the half-written transaction rolls back, source files untouched.
• No eyes → structured output. Every tool returns machine-consumable JSON, never prose the model has to parse. Errors carry suggestion and next_action — an executable recovery call the model reissues verbatim instead of guessing.
• No memory → self-contained calls. Every call carries workspace_dir; writes are version-anchored (optimistic concurrency), so even if the model forgets the version it read, the write fails loudly instead of silently corrupting.

On the "silent corruption" point: while building with a default agent tool stack, one overwrite write silently lost 400+ lines — surfaced ~40 turns later, by luck. We stopped betting on "models will get better" and moved the safety into the tool layer.

Errors aren't a dead end — they're an interface. Every failure carries a stable code, a human-readable suggestion, and a next_action that is executable, not advice:

{ "error": { "code": "VERSION_CONFLICT", "message": "base_version mismatch: FILE_CHANGED", "suggestion": "Re-read the file and regenerate the batch.", "next_action": { "tool": "read_symbol", "params": { "locator": { "file_path": "src/api.js" } } } } }

The model doesn't parse the suggestion and decide what to do — it reissues next_action verbatim and recovers. Successful calls carry a next_step the same way. Errors become signposts with navigation instead of dead ends.

edit_batch: tolerant matching, paranoid writing

LLMs generate old_string anchors with mistakes humans rarely make: collapsed double spaces, truncated line ends, curly quotes where the code has straight ones. A bare no_match sends the model off to re-read the whole file — thousands of tokens per failed match.

Matching degrades in four stages: exact → trailing-whitespace-stripped → edit-distance candidates (similarity ≥ 0.5) → diagnostics (whitespace visualized, 17 Unicode confusable pairs listed). A typical failure reads: "candidate at line 42, similarity 0.87 — you used curly quotes, the code has straight ones." Usually one retry fixes it.

The write side is paranoid: symlink guards, unique temp file names, TOCTOU check between match and commit, rename retries for Windows AV file locks, post-write syntax check (node --check / py_compile / JSON.parse).

Deterministic quality gates, honestly scoped

security_review, code_review, sweep_dead_code are pure regex/AST — zero LLM calls. Same input, same output; CI-safe and auditable.

We state the boundary explicitly: these tools do not cover control flow, data flow, or cross-module semantics. Zero findings ≠ safe; a high score ≠ healthy. A deterministic pattern scanner that admits its scope beats a "comprehensive security" claim every time.

The tool audits itself

30+ rounds of "LiuHe reviews LiuHe" — every bug found becomes a regression test. Real fixes from those rounds: a scope filter scanning outside its target directory, dead-code false positives on registration patterns, constant tracing missing read sites, SQL parameterization cleanup. Assertions grew every round: 2,013 JS + 92 Rust, full chain green.

Why the name: 六合 (six harmonies)

LiuHe (六合, "six harmonies") names the six design constraints applied to every tool in the toolkit:

• Contract — parameters are self-describing; ambiguity returns candidates instead of guessing
• Guard — dry-run, collision, and syntax checks run before anything executes
• Persist — every operation is atomic, idempotent, and undoable
• Frugal — incremental returns, batching, and trimming keep token spend down
• Observable — trace ids, pipeline steps, and a recovery path on every failure
• Trace-back — misuse data feeds back into thresholds

The three compensations earlier (hands / eyes / memory) are the user-facing summary; these six are the per-tool checklist behind them. The AST layer that enforces them is called Malong.

Measured numbers (not paper benchmarks)

All under a real docker --memory=512m cgroup:

Metric	Value
repo_map	98ms (was tens of seconds)
Full index	1,482 files in 9.7s
Concurrency	128 concurrent / 256 in-flight, zero OOM
Peak RSS	134MB (~26% of limit)
Throughput	~588 calls/s (60–600× realistic agent demand)
Hot-file storm	32-way read/write mix, zero torn writes, integrity_check PASS, 95/95 conflicts rejected as FILE_LOCKED
Token savings	↓65.3% (7,673 → 2,662 est. on same task)

Honest boundary: throughput doesn't scale with concurrency — better-sqlite3's synchronous queries serialize on the Node event loop. We evaluated worker_threads and decided the risk wasn't worth the gain. 588 calls/s is already overkill.

Where the token savings come from: tiered tool-description compression (44 tools ≈ 1.33k tokens — core tools keep full descriptions, low-frequency ones shrink to ≤70 chars, verbose ones ≤230, with the detail deferred to next_step hints), incremental returns with explicit pagination instead of dumping everything, and batch endpoints (read_symbols, write_symbols) that cut round-trips. Same task: 7,673 → 2,662 estimated tokens (↓65.3%) and 6 calls → 3 (↓50%).

All benchmarks are reproducible from benchmarks/ and tests/ in the repo (concurrency correctness: tests/test-mvp-concurrency.js).

Day-one DeepSeek Harness support

We also shipped first-day support for DeepSeek Harness (dsh web) — one line to register, all 44 tools exposed as malong__* with workspace_dir auto-filled from the conversation's workspace:

dsh plugin --profile web add @jieai/dsh-malong-bridge

Full guide in the repo: malong/dsh/DSH-INTEGRATION.md.

Try it

Zero-build deploy (no cargo, no npm install):

git clone https://github.com/wulun811/LiuHe liuhe && cd liuhe/malong
mkdir -p ~/.local/bin
tar -xzf ../releases/malong-liuhe-0.4.6-linux-x86_64.tar.gz
cp malong-parse/target/release/malong-parse ~/.local/bin
malong-parse &   # start the parse daemon
node --max-old-space-size=512 --expose-gc mcp-server.js --workspace /path/to/project

If better-sqlite3 is unavailable, it falls back to vendored sql.js WASM — no install, no compile, no network.

---

• GitHub: https://github.com/wulun811/LiuHe
• Docs site: https://wulun811.github.io/LiuHe
• Changelog: https://github.com/wulun811/LiuHe/blob/master/CHANGELOG.md

Skepticism welcome — all numbers above are self-measured and reproducible. If this resonates with an agent failure you've had, try it, break it, and tell us where we're wrong — the repo is wulun811/LiuHe.

Comments

[Reid Marlow]

The recovery contract is the part I care about most here. A structured next_action after a version conflict lets an agent repair the turn without burning half the context rereading the repo. That boring interface detail is where a lot of agent tools still fall over.

[Jiangang Chen]

Glad this is what you're zeroing in on — it's the detail we dogfooded the most, and I'd sharpen it slightly:

The recovery isn't a "go read the file" push. After a write the tool returns the diagnosis as part of the success payload, not a bare "ok". Every write_symbol response carries a structured diff (a unified hunk with every removed line prefixed, plus a lines_changed count), a post-write validation result (bracket balance + syntax check), and any warnings. So when an agent intended to tweak one function and the diff comes back showing -400 lines — or the validation fails, or a warning flags "new_string omitted, old_string will be removed" — it can see the damage in the same turn and hit the undo token that's riding in the same response.

That's the design bet: the agent detects its own mistake from its own write's return value, instead of being sent off to re-read the repo and hope it notices. And when a version conflict does reject a write, that error is equally structured — stable code plus an executable next_action pointing at a symbol-level read, not a full re-scan — so repair targets the smallest correct unit and the context bill stays small. Errors and successes share the same self-describing shape; that's what makes either path recoverable.

[Mads Hansen]

The warm lookup result is impressive. I’d make the freshness contract as prominent as the 98ms number, because mtime plus dirty flags can miss same-timestamp rewrites, branch/worktree switches, renames, deletions, generated files, and editors that replace via temp files.

A useful response watermark would include index generation, repository HEAD, dirty-file digest, parser binary hash, language-extractor versions, and last completed scan. Then every map tells the agent what snapshot it describes. Periodic content-hash reconciliation can repair anything filesystem events or mtimes missed.

The per-call workspace_dir also deserves a hard boundary: canonicalize it, bind it to an allowed workspace identity, reject symlink escapes, and partition SQLite/journals by that resolved identity—not raw caller text. Otherwise a self-contained tool argument becomes a filesystem capability.

For benchmarks, I’d report cold build, incremental one-file update, rename/delete repair, and warm map p50/p95 separately, with output-equivalence fixtures against a clean full reindex. Fast stale context is worse than slow correct context for an editing agent.

[Jiangang Chen]

Fair points — answering each precisely:

1) Freshness is layered, not just mtime+dirty. An fs.watch on the workspace re-extracts on change events, so renames, writes, and same-second rewrites are caught event-driven, independent of mtime precision. Checkout/branch switches are covered because staleness compares with != (not >) — a checkout moves mtimes backwards and still triggers re-extract. Reads reconcile per file (stat + content_hash), and a changed parser binary re-stamps the whole index on open. Deletions are flagged on read (file_missing_on_disk) and cleaned on reindex. And when a repo_map is behind the disk, it says so — index_stale_note with both timestamps, so a stale map never looks fresh. What we don't have is the full watermark (generation/HEAD/dirty digest) or a periodic background reconcile — fair add; we chose event-driven watch + access-time verification.

2) workspace_dir is already hard: shared realpath guard rejects symlink escapes and anything outside the workspace on reads and writes (incl. journal backups), randomized tmp names, index+journals partitioned per resolved workspace, escape tests in the suite.

3) Benchmarks: cold/hot p50-p95, full-reindex p50, 20k-file walker, index-repair/rename tests exist. Missing is your packaging — separate cold/incremental/rename-delete/warm p50-p95 with an output-equivalence check vs clean full reindex. That's how we'll frame the next numbers.