DEV Community

Cover image for Static Repo Packs Give the Map. Autonomous Coding Agents Need the Body.
Mai Chí Kha
Mai Chí Kha

Posted on Originally published at github.com Fully Autonomous

Static Repo Packs Give the Map. Autonomous Coding Agents Need the Body.

#mcp #ai #python #opensource

Over the past few months, we've seen fantastic innovations in static repository packing tools like ContextCraft (contextcraft-ai), repomix, and gitingest. If your workflow is copy-pasting a repository snapshot into the web interfaces of ChatGPT or Claude for an architectural high-level review, static budget packing (using Tree-sitter + PageRank + a knapsack solver) is genuinely great.

However, when using autonomous multi-step coding agents (Claude Code, Cursor, Google Antigravity, OpenAI Codex), developers run into a fundamental architectural bottleneck: the "Function Body Blind Spot".

Core Architectural Pillars & Key Metrics

The Structural Dilemma: Pre-Packing vs. On-Demand Pulling

  1. Static Packing freezes the context before the agent starts investigating. To compress a 500k-token repository into an 8k or 32k token snapshot, a static packer must prune almost every file down to signatures only (SIGS) or directory outlines (TREE).
  2. You cannot fix a bug inside a function body that was pruned down to a one-line signature. In real debugging, an agent reads Function A, realizes the root cause is inside callee Function B, and needs Function B's 100% complete implementation body plus its surrounding callers/callees.

Instead of pre-packing the whole repository upfront, we built token-context-mcp (v0.3.5) -- a 100% local, zero-cloud-API Model Context Protocol (MCP) server that lets your coding agent progressively pull exact AST code context on demand.

Context Supply Paradigms: Flat Dump vs. Static Pack vs. Interactive MCP Pull

How Progressive MCP Pull Works (The 5-Step Zoom-In)

Instead of cat-ing entire 5,000-line files or relying on a static pre-computed snapshot:

Architecture & 5-Step Progressive Pull Pipeline

  1. Global Orientation (get_repo_map): Builds a PageRank-weighted symbol map of the entire repository in ~682-703 tokens (tiktoken o200k_base) vs ~507k-539k tokens of raw source (~0.13% of repo size).
  2. Hybrid Localization (search_source + find_symbols): Combines SQLite FTS5 keyword search inside function bodies with AST symbol globbing.
  3. Structural Skeletons (get_file_skeleton): Uses Tree-sitter byte spans to elide function bodies into # ... while preserving 100% of imports, class hierarchies, and type signatures (saves ~79%-82% tokens).
  4. On-Demand Full Body Pull (inspect_symbol / get_symbol_context): Pulls the 100% complete source body of the exact target symbol + docstrings + 1-hop caller/callee signatures in a single call (~500-850 tokens, saving ~91.5% vs reading full files).
  5. Blast-Radius Analysis (get_impact_slice / get_module_dependents): Traces multi-hop callers/callees and module imports before the agent edits a shared interface.

Empirical Benchmarks: 59-Task Head-to-Head & M10 Suite

Every token count below is verified with OpenAI's real tiktoken (o200k_base) tokenizer across two real-world codebases: rich (Python, 213 files, 507k tokens) and fastify (JS/TS, 291 files, 538k tokens). All scripts and raw JSON outputs are public in the repository.

59-Task Head-to-Head & M10 Benchmarks

Experiment 1: Does the target function's FULL BODY survive into context? (59 Tasks)

A task counts as a Full Body Hit if >=90% of the ground-truth target function's non-empty lines appear in the delivered context:

Method / Configuration Median Real Tokens (o200k_base) Full Function Body (>=90%) Partial Body (50%-89%) Signature Only (SIGS) Missed
token-context-mcp (search_source + top-3 symbol_context) 7,652 tok 33 / 59 (55.9%) 2 / 59 (3.4%) 11 / 59 (18.6%) 13 / 59 (22.0%)
ContextCraft v0.3.2 (32k budget + --focus "<task>") (4x budget) 29,773 tok 6 / 59 (10.2%) 4 / 59 (6.8%) 40 / 59 (67.8%) 9 / 59 (15.3%)
ContextCraft v0.3.2 (8k budget + --focus "<task>") (Equal budget) 7,469 tok 2 / 59 (3.4%) 0 / 59 (0.0%) 28 / 59 (47.5%) 29 / 59 (49.2%)
ContextCraft v0.3.2 (8k budget, default generic snapshot) 7,450 tok 1 / 59 (1.7%) 0 / 59 (0.0%) 12 / 59 (20.3%) 46 / 59 (78.0%)

Key Takeaways:

  • At a 32k budget with --focus, ContextCraft includes at least the signature for 50/59 (84.7%) of tasks -- proving its PageRank + BM25F ranker is genuinely effective at building a high-level map.
  • But when an agent needs the actual implementation body to write a code patch, on-demand MCP pull delivers 33/59 (55.9%) full bodies at ~7.6k tokens vs 2/59 (3.4%) for equal-budget static packing (16.5x more full bodies).
  • For the 13/59 (22%) tasks where the zero-LLM deterministic 2-step pull missed, our standardized prompt protocol instructs the agent to fall back gracefully to direct file reading.

Experiment 2: Equal-Budget Retrieval vs. Truncated Grep (M10 Suite)

On 30 pre-registered tasks in rich at an identical ~1,900-token budget:

  • File Recall: 0.90 (token-context-mcp) vs 0.57 (budget-truncated grep) -- +58% improvement
  • Symbol Recall: 0.54 vs 0.10 -- 5.4x improvement

7-Dimension Technical Comparison Matrix

7-Dimension Technical Comparison Matrix

  • 10 Core Read-Only Tools (Zero Schema Bloat): Exposes only 10 read-only retrieval tools by default to keep initial context small. Optional extensions unlock multi-agent SQLite memory with mutex locks, tool discovery, and local Ollama summarization.
  • 3-Layer Governance Microkernel: Enforces clean AST boundary caps, per-tool execution timeouts (10s/5s), and a 150,000 cumulative token circuit breaker to prevent runaway billing loops.
  • Local & Private: No arbitrary shell command execution, zero external telemetry, with automatic secret redaction ([REDACTED]).

Getting Started in 10 Seconds

token-context-mcp is open-source (MIT), indexed on the official Model Context Protocol Registry, and live on PyPI.

Run it directly without installation via uvx:

uvx token-context-mcp quickstart .
Enter fullscreen mode Exit fullscreen mode

Or connect it to Claude Code:

claude mcp add --transport stdio --scope user token-context -- uvx token-context-mcp serve
Enter fullscreen mode Exit fullscreen mode

Or add to Cursor Settings > Features > MCP Servers:

  • Name: token-context
  • Type: command
  • Command: uvx token-context-mcp serve

Huge respect to the author of ContextCraft for advancing open-source context compression -- static packers and interactive MCP servers solve complementary parts of the AI developer workflow!

Top comments (0)