DEV Community

tappers
tappers

Posted on Originally published at tappers.hashnode.dev

How Save-Game Context Compaction Fixes Spec Drift in Multi-Step Coding Agents (-91.9% Input Tokens)

#ai #llm #contextengineering #agents

💡 AI Labo Benchmark Series — Experiment E12 (Ranked #6 out of 41 Patterns)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

Metadata & Experiment Identity

  • Experiment ID: E12 (Backlog Candidate C008)
  • Run ID: e12-20261028-vm216-01
  • Sandbox Environment: Proxmox Isolated VM VM216 (ai-e12-216, 0 NICs, virtio-serial QGA relay, systemd UID separation)
  • Model: gemini-3.8-flash (thinkingLevel: LOW)
  • Pattern Under Test: Save-Game Context Compaction (/compact at Milestones) — "Unbounded History Accumulation vs. Milestone JSON State Compaction"

1. Executive Summary (TL;DR)

When an AI coding agent tackles a multi-step engineering task across several modules, keeping every prior file dump, telemetry log, and intermediate code output inside a single growing messages[] array (Unbounded Context Accumulation) causes two compounding problems:

  1. Quadratic Token Inflation: Every past turn's tool output is re-billed on every subsequent turn (6,556 ➔ 11,769 ➔ 17,544 input tokens).
  2. Mid-Task Spec Drift (Lost-in-the-Middle): By Turn 2 or Turn 3, fine-grained string normalization rules and boundary conditions buried in the middle of the history get drowned out by verbose telemetry logs—causing subtle bugs that break end-to-end integration tests.

To test the solution—Milestone Context Compaction (the "Save-Game" Pattern)—we benchmarked a 3-milestone power-grid telemetry and carbon-credit clearing pipeline (Milestone 1: collector.py ➔ Milestone 2: clearing.py ➔ Milestone 3: dispatch.py) inside isolated Proxmox VM VM216:

  • Mode A (AccumulatedHistoryRunner): Appends all 3 milestone specifications, code outputs, and verbose telemetry audit dumps (VERBOSE_TELEMETRY_AUDIT_DUMP) into one continuous chat thread (0 compactions).
  • Mode B (MilestoneCompactionRunner): Immediately after each milestone passes its deterministic verification gate, executes a /compact reset—discarding raw logs and replacing the chat history with a 120-token structured JSON checkpoint (SAVE_GAME_COMPACTION_V1) plus only the next milestone's specification.

Key Result

Mode A's accumulated history (35,869 cumulative input tokens) suffered a classic Lost-in-the-Middle failure on Milestone 2 (clearing.py), forgetting to apply .strip().upper() to contract_tier and failing 2 out of 20 hidden E2E tests (18 / 20, 90.0%). Mode B's Save-Game Milestone Compaction kept every turn under 1,400 input tokens (2,922 cumulative input tokens, -91.9% reduction), cut API cost by -72.2% ($0.0340 ➔ $0.0095), and achieved a flawless 20 / 20 (100%) pass.

Metric Mode A: Accumulated History (AccumulatedHistory) Mode B: Milestone State Compaction (MilestoneCompaction) Delta / Improvement
Hidden Grading Suite Score (20 Cases) 18 / 20 (90.0% — failed 2 E2E cases) 20 / 20 (100% Perfect Pass) +10.0 pt (Zero spec drift)
Context Compactions (compactions) 0 compactions 2 compactions (After M1 & M2) Reset history at each milestone gate
Turn 1 Input Tokens (M1: collector.py) 6,556 tokens ($0.00725) 650 tokens ($0.00290) -90.1% reduction
Turn 2 Input Tokens (M2: clearing.py) 11,769 tokens ($0.01247) 1,388 tokens ($0.00461) -88.2% reduction
Turn 3 Input Tokens (M3: dispatch.py) 17,544 tokens ($0.01429) 884 tokens ($0.00195) -95.0% reduction on final turn
Cumulative Input Tokens (promptTokenCount) 35,869 tokens 2,922 tokens -91.9% (12.28x token efficiency)
Cumulative Output Tokens (candidatesTokenCount) 1,894 tokens 1,937 tokens Equivalent code generation volume
LLM Calls (model_calls) 3 calls 3 calls (1 per milestone) Identical call count
Elapsed Execution Time (elapsed_seconds) 24.81 sec 22.52 sec -9.2% (1.10x faster)
Estimated API Cost (estimated_micro_usd) $0.0340 (34,005 micro-USD) $0.0095 (9,457 micro-USD) -72.2% (3.60x cheaper)

2. Experiment Architecture: Unbounded History vs. Save-Game Compaction 

+-------------------------------------------------------------------------+
| Proxmox Isolated VM216 (0 NICs, virtio-serial QGA, systemd UID split)   |
|  Target Task: 3-Milestone Power Grid & Carbon Clearing Pipeline         |
|               (M1: collector.py --> M2: clearing.py --> M3: dispatch.py)|
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: AccumulatedHistoryRunner (No Compaction)                  |  |
|  |  - Turn 1 (M1): 6,556 in  --> collector.py PASS                   |  |
|  |  - Turn 2 (M2): 11,769 in --> Audit logs drown out M2 spec!       |  |
|  |                               Forgets contract_tier.strip().upper()|  |
|  |  - Turn 3 (M3): 17,544 in --> Cumulative 35,869 input tokens      |  |
|  |  --> Final Score: 18 / 20 (Crashes on lowercase/padded tiers!)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: MilestoneCompactionRunner (Save-Game JSON Compaction)     |  |
|  |  - Turn 1 (M1): 650 in   --> collector.py PASS                    |  |
|  |  - [Compaction #1]: Discards raw logs; writes 120-tok JSON state  |  |
|  |  - Turn 2 (M2): 1,388 in --> Fresh context nails all M2 rules!    |  |
|  |  - [Compaction #2]: Updates verified_contracts in JSON checkpoint |  |
|  |  - Turn 3 (M3): 884 in   --> Cumulative 2,922 input tokens (-91.9%)| |
|  |  --> Final Score: 20 / 20 (100% Perfect Pass)                     |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
Enter fullscreen mode Exit fullscreen mode

3. Code-Level Postmortem: How Unbounded History Caused a Lost-in-the-Middle Bug

3.1 Mode A (AccumulatedHistory): The Missing .strip().upper() on Turn 2

Inspecting accum-bundle.json (Mode A's final submitted code) reveals a textbook example of Attention Dilution:

  • On Turn 1 (collector.py), Mode A properly implemented .strip().upper() normalization for station_id and grid_zone.
  • By Turn 2 (clearing.py), however, 45 lines of VERBOSE_TELEMETRY_AUDIT_DUMP_M1 plus the full Turn 1 code had bloated the prompt to 11,769 tokens. Buried in the middle of the prompt was the requirement that clearing.py must also normalize contract_tier via .strip().upper().
  • Mode A completely missed that normalization rule in clearing.py: 
# ❌ Mode A (accum-bundle.json — clearing.py generated at 11,769 input tokens)
def settle_grid(readings, contract_tier="BASE", credit_offset=0):
    if not isinstance(readings, list) or len(readings) == 0:
        raise ValueError("readings must be a non-empty list")
    # BUG: Forgot isinstance(contract_tier, str) and contract_tier.strip().upper()!
    if contract_tier not in ALLOWED_TIERS:
        raise ValueError(f"contract_tier must be one of {ALLOWED_TIERS}")
Enter fullscreen mode Exit fullscreen mode

Because of that single omitted .strip().upper(), Mode A crashed with a ValueError on two hidden E2E integration tests that passed padded/lowercase tier strings (contract_tier=" green " in e2e_01 and contract_tier="hydro" in e2e_02).

3.2 Mode B (MilestoneCompaction): The Exact Save-Game JSON Checkpoint

In Mode B, as soon as Milestone 1 (collector.py) passed verification, the harness wiped the raw conversation history and started Turn 2 with this compact SAVE_GAME_COMPACTION_V1 JSON checkpoint:

{
  "checkpoint_type": "SAVE_GAME_COMPACTION_V1",
  "completed_modules": ["collector.py"],
  "history_compacted": true,
  "milestone_completed": 1,
  "next_target_module": "clearing.py",
  "verified_contracts": {
    "collector.collect_readings": "PASS (returns list of dicts with station_id, grid_zone in NORTH/SOUTH/WEST, mwh_units>0, rate_cents>=0, 0<=carbon_bps<=1500, non-mutating)"
  }
}
Enter fullscreen mode Exit fullscreen mode

Because Turn 2 ran in a fresh 1,388-token context containing only the verified upstream contract and Milestone 2's specification, Mode B had zero attention dilution—properly implementing clean_tier = contract_tier.strip().upper() and passing all 20 / 20 test cases:

Test Case Group Verification Target Mode A (Accumulated) Mode B (Save-Game Compact)
collector_01 – collector_06 Milestone 1: Non-mutation, type checks, bounds (0..1500 bps) 6 / 6 PASS 6 / 6 PASS
clearing_01 – clearing_08 Milestone 2: First-seen order, 400 vs 401 bps levy, 12000 tier boundary 8 / 8 PASS 8 / 8 PASS
e2e_01_green_below_12000_minus_zero_and_at_separator E2E: Normalize contract_tier=" green " + @ delimiter + -0 ❌ FAIL (ValueError) ✅ PASS
e2e_02_hydro_levy_and_credit_dispatch E2E: Normalize contract_tier="hydro" + 20% discount + credit offset ❌ FAIL (ValueError) ✅ PASS
e2e_03 – e2e_06 E2E: Zero rate cents, line order preservation, cross-stage error propagation 4 / 4 PASS 4 / 4 PASS
Total 20 Hidden Cases 18 / 20 (90.0%) 20 / 20 (100%)

4. Practical Takeaway: Compact to Structured State, Not Prose Summaries

When building multi-step agent workflows (or using /compact in interactive coding agents):

  1. Don't wait until the 200k context window is 95% full to compact: Attention degradation (Lost-in-the-Middle) starts happening well above 10,000–15,000 tokens of noisy tool logs. Compact at every milestone boundary as soon as a module's tests turn green.
  2. Use deterministic JSON checkpoints instead of free-form prose summaries: As we saw in E05 (Subagent Folding), natural-language summaries often drop exact type contracts. Instead, record a structured JSON object containing completed_modules, verified_contracts (exact function signatures & invariants), and next_target_module.

🔗 Continue Reading the Series

Top comments (0)