Why the codebase is your biggest context bill, and why output compression doesn't fix it

foldwork — Sun, 12 Jul 2026 15:30:47 +0000

When people think about reducing AI coding costs, they usually think about the model's output: fewer words, shorter explanations, tighter responses. It's intuitive. The AI talks a lot. Make it talk less.

The problem is that output is the wrong target.

Paul Kinlan analysed OpenRouter's programming traffic using their public usage rankings and a reproducible script against the OpenRouter API. What he found:

"Real-world data from OpenRouter's programming category shows 93.4% input tokens, 2.5% reasoning tokens, and just 4.0% output tokens. It's almost entirely input."

That stat surfaced in a 471-point Hacker News thread discussing output-compression techniques. Even if you silenced the model completely, you'd be optimising 4% of the problem.

The other 93% is what you put in: your codebase, loaded into context on every message.

Why input is so heavy

When a coding agent works on a real project, every conversation turn is expensive on the input side:

The agent reads files to understand the current state
Build output, test results, and error logs get appended
Tool call results feed back into context
The previous turn's entire response rolls forward as history

A typical agentic session on a mid-sized codebase can consume tens of thousands of input tokens per task, sometimes hundreds of thousands. The AI's response is a small fraction of that. The ratio is not 1:1. It's more like 50:1.

Output compression approaches like CLAUDE.md instruction files are addressing 4% of a 50:1 problem.

What other tools in this space do

Several tools target different slices of the input problem:

RTK (70.4k GitHub stars) is a Rust CLI proxy that intercepts shell command output before it reaches the model. When the agent runs git status or cargo test, RTK rewrites the output into a compact form, stripping boilerplate, grouping related lines, truncating long logs. It claims 60-90% reduction on supported commands and covers 100+ commands across git, test runners, Docker, AWS, and more.

MemStack is a structured skill framework for Claude Code with 125+ skills across free and Pro tiers. Skills load on demand and include session memory and project handoffs. It addresses a different problem: reloading the same project context from scratch at the start of every session.

claude-mem captures everything an agent does during a session, compresses it with AI, and injects relevant context back into future sessions. Persistent cross-session memory.

Lumen by ORY applies local semantic search so the agent retrieves only the code fragments most relevant to the current query, rather than entire files.

Headroom is an API proxy that applies general-purpose compression to the entire request payload before it leaves your machine.

These tools are largely complementary. None of them address the core structural problem the same way: the codebase itself, loaded as raw text.

The structural problem: AST folding vs. loading raw files

When an agent needs to understand a function, it typically reads the entire file containing that function. For a 500-line Go file with 12 functions, it might only need the signature, parameters, and return type of 11 of them, but it loads all 500 lines anyway.

This is not a tooling problem. It's a representation problem. The agent has no way to ask for "just the structure" and can only read files.

AST body-folding changes the representation. Instead of loading the raw file, the codebase is indexed using its abstract syntax tree. Function bodies are replaced with fold markers, preserving:

Every function and method signature
All type definitions and interfaces
All imports and package structure
All comments and docstrings

The result is a structural view that lets the agent understand the architecture of a file at a fraction of the token cost. When the agent needs a specific function body to actually edit it, it requests that file uncompressed. Everything else stays folded.

The actual numbers, on a real codebase

The Next.js monorepo is a useful benchmark because it's large, well-known, and representative of the TypeScript projects that most Cursor and Claude Code users work on.

Benchmark: vercel/next.js

Representation	Tokens	vs. raw
Raw (all source files uncompressed)	23,963,330	baseline
Compressed (mcp-injector default)	10,212,684	-57.4%

That's 13.75 million tokens saved per full codebase load.

At current Claude Sonnet 5 pricing ($2.00/M for input tokens, with a 10% cache-hit multiplier for repeated context on the compressed representation), loading the raw Next.js codebase costs approximately $47.93. With compression: approximately $2.04. Difference: $45.88 saved on a single full load.

The benchmark data across other repositories (Django, Spring, Tokio, Gin, Redis, cURL, and more) is available at foldwork.dev/benchmarks.

The objection you're already thinking of

"But mcp-injector is itself an MCP server. Doesn't installing it add to the same token overhead it claims to reduce?"

Yes, and it's worth being precise about this.

An MCP server adds its tool definitions to every message. mcp-injector has 5 tools. Those definitions cost a few thousand tokens of fixed overhead per message.

From the Next.js benchmark: the tool saves 13,750,646 tokens per full codebase load. The fixed overhead of 5 tool definitions is in the low thousands.

The overhead is real but it's roughly 1,000-4,000x smaller than the saving. It's not a close call.

The concern about MCP server overhead is legitimate in the abstract. A commenter in the Hacker News thread raised exactly this point, noting that six MCP servers with 15-20 tools each can add 22,000+ tokens per message. That's a real problem. The answer is to be selective about which MCP servers you install, and to understand the fixed-overhead cost of each one relative to what it returns.

What this is and isn't

AST folding is not a replacement for RTK, Lumen, Headroom, MemStack, or claude-mem. Shell output compression, semantic search, session memory, and API-level compression solve real problems that body folding doesn't touch. A well-configured setup could reasonably stack several of them.

What AST folding is: a structural solution to a structural problem. The codebase is your biggest input source. Loading it as raw text is a choice, not a requirement.

mcp-injector is available at foldwork.dev. It works with Claude Desktop, Cursor, VS Code, Devin Desktop, and Antigravity. The Pro tier is $12/month or $99/year.