DEV Community

Hoyin kyoma
Hoyin kyoma

Posted on

Why AI Coding Agents Waste 30% of Their Tokens — And How to Fix It

#agents #ai #llm #softwareengineering

The Hidden Cost of Blind Agents

Every AI coding agent has the same workflow: receive a task, search the codebase, read files, write code. The problem is step 2. The agent doesn't know the codebase. It doesn't know the architecture. So it searches.

And searches. And searches.

We analyzed token usage across 500 SWE-bench Verified instances and found that agents spend approximately 30-40% of their tokens on exploration — reading files that turn out to be irrelevant, following import chains that lead nowhere, and backtracking from wrong approaches.

This isn't a model problem. GPT-5, Claude Opus, Gemini — they all do it. The issue is structural: the agent lacks a map of the codebase.

[Insert pie chart: "Token Breakdown — Agent Without Architectural Context"]

A Real Example: Django Bug #16379

Let's trace through a real bug to see this in action.

Bug: FileBasedCache crashes with FileNotFoundError when multiple processes access the cache simultaneously.

What a human developer does:

  1. Reads the issue — understands it's a race condition in the file cache backend
  2. Knows (from experience) that Django's cache backends inherit from BaseCache
  3. Opens django/core/cache/backends/filebased.py directly
  4. Checks the delete() and _cull() methods for file operations without proper locking
  5. Writes a fix: wrap the os.remove() call in a try/except for FileNotFoundError
  6. Done. ~5 minutes, ~3 files read.

What an AI agent does (without context):

  1. Reads the issue
  2. Searches for "FileNotFoundError" — finds 47 matches across the codebase
  3. Opens django/core/files/storage.py — wrong file
  4. Opens django/core/files/base.py — wrong file
  5. Searches for "FileBasedCache" — finds it
  6. Opens django/core/cache/backends/filebased.py — right file
  7. Reads the whole file but doesn't understand the inheritance from BaseCache
  8. Writes a fix that handles the error but doesn't respect the cache contract
  9. Test fails
  10. Opens django/core/cache/backends/base.py to understand the base class
  11. Opens django/core/cache/__init__.py to understand the cache framework
  12. Rewrites the fix
  13. Test passes. ~20 minutes, ~12 files read, ~4,000 tokens.

What an AI agent does (with XCE):

  1. Reads the issue
  2. Calls xce_get_context("FileBasedCache FileNotFoundError concurrent access")
  3. Gets back: the cache backend hierarchy, the file operations in filebased.py, the locking patterns, and the test infrastructure
  4. Understands the architecture immediately
  5. Writes the correct fix on the first attempt
  6. Test passes. ~3 minutes, ~3 files read, ~1,500 tokens.

[Insert flowchart: "File Access Pattern Comparison"]

The token savings compound across hundreds of tasks. On our 500-instance benchmark run, XCE reduced total token usage by approximately 20%.

Why Embeddings Aren't Enough

The obvious solution is "just use code search." Tools like Greptile, Sourcegraph Cody, and GitHub Copilot all offer some form of code search. Most use embedding-based retrieval: convert code to vectors, find the most similar vectors to the query.

This works for simple lookups. "Find the login function" → returns the login function. But it fails for architectural questions:

Question Embedding Search Architectural Context
"Which module owns this logic?" Returns similar code snippets Returns the HLD module, its role, and its boundaries
"What depends on this function?" Returns functions with similar names Returns the call graph and downstream consumers
"If I change this, what breaks?" Returns similar code (not dependent code) Returns impact analysis with affected modules
"How does this fit in the architecture?" Returns nearby code Returns HLD → LLD → code hierarchy

The fundamental issue: embeddings measure text similarity, not structural relationships. Two functions can be textually similar but architecturally unrelated. Two functions can be textually different but tightly coupled through a call chain.

[Insert diagram: "Embedding Search vs. Architectural Context"]

The Architecture Gap Across Repositories

We measured the improvement from XCE across five major open-source repositories. The results reveal a clear pattern:

Repository Architecture Type Baseline With XCE Delta
sympy Deep module dependencies 45% 62% +17%
scikit-learn Complex inheritance chains 58% 71% +13%
matplotlib Multi-backend rendering pipeline 52% 65% +13%
django Layered MVC + ORM + middleware 62% 74% +12%
pytest Plugin system (relatively flat) 70% 78% +8%

sympy (+17%): The largest improvement. Sympy has deep cross-module dependencies. A bug in sympy/core/expr.py might require understanding sympy/simplify/, sympy/printing/, sympy/polys/, and sympy/series/. Without a map, the agent gets lost in the dependency maze. With XCE, it knows which modules are structurally related before it starts exploring.

scikit-learn (+13%): Complex estimator inheritance. BaseEstimator → ClassifierMixin → LinearClassifierMixin → LogisticRegression. A bug in LogisticRegression.fit() might actually be in LinearClassifierMixin._fit() or even BaseEstimator.set_params(). The agent needs to understand the full inheritance chain to find the right place to fix.

pytest (+8%): The smallest improvement. Pytest has a plugin system that's complex, but most bugs are localized to a single file or module. The agent doesn't need as much architectural context because the architecture is relatively flat.

[Insert scatter plot: "Architectural Complexity vs. XCE Improvement"]

The correlation is strong: the more architecturally complex the codebase, the more the agent benefits from having a structural map.

This has a practical implication: if your codebase is a simple CRUD app with flat architecture, XCE helps modestly. If your codebase is a complex system with deep module dependencies, layered abstractions, and cross-cutting concerns — XCE helps dramatically.

How XCE Works

XCE uses the proprietary PRAT algorithm to build a structured codebase index that captures architectural relationships — not just code text. Unlike embedding-based search, PRAT understands structural connections between components at multiple levels of abstraction.

When an agent queries XCE, it gets back a structured response that includes: what module the code belongs to, what its role is in the system, what depends on it, and what it depends on. The agent doesn't just know where the code is — it knows why it exists and how it connects to the rest of the system.

This is served via MCP, so any compatible agent gets architectural context on every tool call without modifications.

Practical Setup

XCE runs as an MCP service. Any MCP-compatible agent can connect with one config block:

# Index your repo (one command)
npx xanther-cli init --api-key YOUR_KEY
Enter fullscreen mode Exit fullscreen mode

This indexes the codebase and installs a git hook that auto-syncs after every commit. Then add to your agent's MCP config:

{
  "mcpServers": {
    "xanther-xce": {
      "url": "https://mcp.xanther.ai/sse",
      "headers": { "Authorization": "Bearer YOUR_KEY" }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Works with Claude Code, Kiro, Cursor, OpenCode, Windsurf — any MCP-compatible tool.

The agent gets five tools:

  • xce_get_context — Full architectural context for a problem statement
  • xce_search — Semantic search across the codebase
  • xce_architecture_context — Architecture around a specific file or symbol
  • xce_trace — Trace relationships from code to design artifacts
  • xce_impact_analysis — What breaks if you change specific files

The Bottom Line

AI coding agents are getting better every quarter. But the bottleneck isn't model capability — it's context quality. A cheap model with the right context outperforms an expensive model without it.

The numbers:

  • 78.2% on SWE-bench Verified with MiniMax M2.5 + XCE (beats every model on the official leaderboard)
  • 20% token reduction per task (fewer wrong turns, less exploration)
  • $0.22 per instance (16x cheaper than Claude Opus)

Context is cheaper than compute. And it compounds: better models + better context = better results than either alone.

Xanther is in open beta. Free tier: 3 repos, 100 queries.

Top comments (0)