Julian Brown

Posted on Sep 7

How We Cut AI Agent Token Usage by 85% with Local MCP

#ai #mcp #python #productivity

In Part 1, we gave our coding agent infinite memory. Here is how we used that retrieval engine to eliminate 3,000-turn marathon sessions and crush our token bill.

The Problem

In How to Give Your AI Coding Agent Infinite Memory, we showed how to index session tapes into SQLite FTS5 for sub-10ms recall.

Yet even with local search tools available, we fell into the exact operational trap every developer encounters: The Marathon Session.

Our main Antigravity session reached 3,223 turns—accounting for 46% of all operational steps ever recorded across our entire studio history.

Why did we let the thread grow so large? Context-loss fear. We hesitated to close the session because we didn't want the agent to lose working agreements, file paths, and architectural nuances.

The hidden cost was catastrophic:

Context Compounding: In an agentic IDE, every turn and tool call re-transmits the active conversation history. At Turn 3,200, each inference step sent 80,000 to 120,000 tokens. A simple 4-step tool routine burned 400,000 input tokens on a single prompt.
Subagent Memory Flooding: Spawning 11 background audit subagents ran 532 steps, generating ~225,000 tokens in logs and dumping 37,000 words of raw diagnostic output directly into the parent thread's prompt window.
Full-File Ingestion: Viewing large draft files (73k+ characters) repeatedly parked massive payloads in active working memory.

The Fix

Having an MCP memory server is only half the battle; you must operationalize it to eliminate session hoarding.

Instead of keeping one giant conversation on life support, we instituted a Zero-Loss Context Protocol that rotates threads frequently and uses search-antigravity as an on-demand retrieval bridge.

┌─────────────────────────────────────────────────────────────┐
│                    The 85% Reduction Loop                   │
└─────────────────────────────────────────────────────────────┘
                               │
            ┌──────────────────┴──────────────────┐
            ▼                                     ▼
┌──────────────────────────────┐    ┌─────────────────────────────┐
│ 1. Milestone Thread Rotation │    │ 2. Silent Worker Protocol   │
│ • Retire sessions at ~40 turns│   │ • Subagents write to disk   │
│ • Context: 100k+ ➔ 5k tokens │    │ • 2-sentence summary in chat│
└──────────────┬───────────────┘    └─────────────┬───────────────┘
               │                                  │
               └──────────────────┬───────────────┘
                                  ▼
┌─────────────────────────────────────────────────────────────┐
│                 search-antigravity Engine                   │
│   (Fresh session restores exact historical context in 8ms)  │
└─────────────────────────────────────────────────────────────┘

The 3 Core Operational Shifts

1. Milestone Session Rotation (Zero-Loss Handoff)

Because our agent can call search_antigravity_conversations() to pull prior decisions in sub-10ms, there is zero risk in closing a thread.

We now retire sessions at distinct operational milestones (every 25–40 turns). A fresh session drops input context from ~110,000 tokens back down to ~6,000 tokens—an instant 90%+ drop in per-turn burn.

2. The Silent Worker Protocol

Subagents should never dump raw analysis into the parent context.

We updated our subagent orchestrator: background workers write their complete diagnostic reports and diffs directly to disk (e.g., Audits/Continuity_Report.md). They return only a two-sentence summary and clickable file paths to the main thread. This prevents 40,000-token summaries from polluting parent working memory.

3. Surgical File Slicing

We replaced monolithic file reads with ripgrep and targeted line slicing (StartLine / EndLine). Instead of ingesting a 75,000-character manuscript to check one dialogue line, the agent searches the target pattern and reads only the exact 20-line window.

The Result in Practice

When starting a clean session for a new milestone, the agent restores context on demand:

search_antigravity_conversations(query: "Book 2 continuity audit findings")

Within 8 milliseconds, SQLite returns the exact file location and rationale from the previous session for ~110 tokens:

[Match 1 | Session: f40f-22... | Step #3248]
"All 11 Book 2 continuity audits written to disk in The Armor We Keep/Audits/..."

Per-turn input footprint: Slashed from ~105,000 tokens to ~7,800 tokens (85%+ reduction).
Parent thread bloat: Eliminated (subagents write to disk).
Context loss: 0% (exact tape recall via MCP).

Grab the Code

The complete implementation and parser scripts are open source on GitHub:

👉 github.com/kingjulian24/search-antigravity (Includes setup instructions, indexer script, and MCP configuration).

Stop letting context fear trap you in 3,000-turn marathons. Give your agent local recall, enforce silent subagents, and keep your active context lean.

DEV Community