DEV Community

tappers
tappers

Posted on Originally published at tappers.hashnode.dev

Benchmarking the Claude Code /advisor Pattern: Solo Executor Loops vs. Upfront Advisor Blueprint Routing

#agents #ai #architecture #llm

💡 AI Labo Benchmark Series — Experiment E18 (Ranked #7 out of 41 Patterns)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

Metadata & Experiment Identity

  • Experiment ID: E18 (Backlog Candidate C014)
  • Run ID: e18-20261028-vm222-01
  • Sandbox Environment: Proxmox Isolated VM VM222 (ai-e18-222, 0 NICs, virtio-serial QGA relay, systemd UID separation)
  • Model: gemini-3.8-flash (thinkingLevel: LOW)
  • Pattern Under Test: Claude Code /advisor & Upfront Architectural Routing — "Solo Executor Trial-and-Error Loop vs. Upfront Advisor Blueprint + Executor One-Shot"

1. Executive Summary (TL;DR)

When an AI coding agent is tasked with fixing a bug that spans multiple coupled modules, running a single worker loop without upfront architectural guidance (Solo Executor Loop) often leads to myopic Whack-a-Mole patching: the agent edits Module 1, runs a trial, discovers a downstream contract mismatch in Module 2, patches Module 2, and repeats—accumulating trial logs and regenerating code across multiple turns.

Anthropic's Claude Code /advisor pattern (pairing a high-level architectural Advisor memo with a fast Executor) proposes a cleaner division of labor: consult the Advisor once at the start of the task to produce a concise cross-module Architectural Blueprint Memo, then let the Executor implement all coupled files in a single pass.

To test this pattern empirically, we benchmarked our 3-module payment settlement pipeline (validator.py, settlement.py, statement.py) inside isolated Proxmox VM VM222 across two modes:

  • Mode A (SoloExecutorLoopRunner): No Advisor consultation (advisor_calls = 0). The Executor runs an initial trial (run_solo_trial), applies piecemeal patches (stage_executor_patch on Turn 2 and Turn 3), and finally submits all 3 modules on Turn 4 (4 LLM calls, 5 file edits).
  • Mode B (AdvisorExecutorRoutingRunner): Calls consult_advisor on Turn 1 (advisor_calls = 1) to retrieve a 1,143-character 5-point cross-module execution blueprint, allowing the Executor to implement and submit all 3 modules in one shot on Turn 2 (2 LLM calls, 3 file edits).

Key Result

Injecting a 1,143-character upfront Advisor blueprint eliminated all unplanned partial-patch retries (2 ➔ 0), halved the number of LLM calls (4 ➔ 2), cut cumulative input tokens by -84.9% (27,699 ➔ 4,172 tokens, 6.64x efficiency), reduced cumulative output tokens by -45.9% (2,897 ➔ 1,568 tokens), lowered API cost by -71.5% ($0.0316 ➔ $0.0090), and sped up execution by 1.78x (33.45s ➔ 18.78s)—with both modes passing 20 / 20 (100%) on the hidden grading suite.

Metric Mode A: Solo Executor Loop (SoloExecutorLoop) Mode B: Advisor-Executor Routing (AdvisorExecutorRouting) Delta / Improvement
Hidden Grading Suite Score (20 Cases) 20 / 20 (100%) 20 / 20 (100%) Both passed all 20 cases
Unplanned Partial-Patch Retries (unplanned_retries) 2 retries (Turn 2, Turn 3) 0 retries (One-shot implementation) -100% (Zero rework)
Total File Edits (file_edits) 5 edits 3 edits (1 per module) -40.0% reduction
LLM Calls (model_calls) 4 calls 2 calls (1 Advisor + 1 Executor) -50.0% fewer calls
Cumulative Tool Output (tool_output_chars) 22,720 chars 1,245 chars -94.5% (18.25x less log noise)
Turn 1 Input / Output Tokens 1,909 in / 10 out 1,846 in / 8 out Equivalent initial invocation
Turn 2 Input / Output Tokens 5,000 in / 488 out (validator.py patch) 2,326 in / 1,560 out (All 3 modules done!) Mode B finishes on Turn 2
Turn 3 Input / Output Tokens 8,527 in / 795 out (settlement.py patch) — (Completed on Turn 2) Avoided intermediate retry turn
Turn 4 Input / Output Tokens 12,263 in / 1,604 out (Final submit) — (Completed on Turn 2) Avoided 12,263-token final prompt
Cumulative Input Tokens (promptTokenCount) 27,699 tokens 4,172 tokens -84.9% (6.64x token efficiency)
Cumulative Output Tokens (candidatesTokenCount) 2,897 tokens 1,568 tokens -45.9% (No duplicate code output)
Elapsed Execution Time (elapsed_seconds) 33.45 sec 18.78 sec -43.9% (1.78x faster)
Estimated API Cost (estimated_micro_usd) $0.0316 (31,640 micro-USD) $0.0090 (9,010 micro-USD) -71.5% (3.51x cheaper)

2. Experiment Architecture: Solo Trial Loop vs. Advisor Blueprint Routing 

+-------------------------------------------------------------------------+
| Proxmox Isolated VM222 (0 NICs, virtio-serial QGA, systemd UID split)   |
|  Target Task: 3-Module Settlement Pipeline Bugfix                       |
|               (validator.py / settlement.py / statement.py)             |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: SoloExecutorLoopRunner (No Upfront Design Blueprint)      |  |
|  |  - Turn 1 (1,909 in):  run_solo_trial (+7,500 chars trial log)    |  |
|  |  - Turn 2 (5,000 in):  stage_executor_patch (validator.py only)   |  |
|  |  - Turn 3 (8,527 in):  stage_executor_patch (settlement.py only)  |  |
|  |  - Turn 4 (12,263 in): submit_settlement_bundle (All 3 modules)   |  |
|  |  --> Total: 27,699 in / 2,897 out / 5 file edits / 33.45 sec      |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: AdvisorExecutorRoutingRunner (Claude Code /advisor Style) |  |
|  |  - Turn 1 (1,846 in / 8 out):                                     |  |
|  |      Calls `consult_advisor` --> Receives 1,143-char 5-point      |  |
|  |      cross-module `one_shot_execution_blueprint`                  |  |
|  |  - Turn 2 (2,326 in / 1,560 out):                                 |  |
|  |      Executor implements & submits all 3 modules in 1 shot!       |  |
|  |  --> Total: 4,172 in / 1,568 out / 3 file edits / 18.78 sec       |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
Enter fullscreen mode Exit fullscreen mode

3. Why Advisor-Executor Routing Beats Both Solo Loops and Role Subagents

3.1 The Dual Token Penalty of Solo Trial-and-Error (Mode A)

Analyzing usage.json and execution-traces.json shows why piecemeal patching without an upfront architectural plan is so expensive:

  1. Input Token Snowballing: Every time Mode A staged a partial single-file patch (validator.py on Turn 2, settlement.py on Turn 3), the harness ran a trial diagnostic that appended roughly 7,500 characters of output to the chat history (1,909 ➔ 5,000 ➔ 8,527 ➔ 12,263 input tokens).
  2. Duplicate Output Token Generation (+84.8% extra output): Because Mode A generated validator.py on Turn 2 (488 output tokens), settlement.py on Turn 3 (795 output tokens), and then re-emitted all 3 modules in the final Turn 4 bundle (1,604 output tokens), it generated 2,897 output tokens compared to Mode B's 1,568 output tokens. Since LLM output tokens cost 3x to 5x more per token than input tokens, avoiding duplicate code generation directly slashes API cost and latency.

3.2 How E18 (Advisor-Executor) Differs from E05 (Role Subagents)

A natural question is: "In E05 (Subagent Folding), you showed that splitting a task across Architect ➔ Coder ➔ QA subagents was 2.66x slower. Why did E18 (Advisor-Executor) succeed?"

The difference lies in who holds the context and how the handoff works:

  • In E05 (Role Subagent Handoff — Anti-Pattern), the Architect subagent ran in an isolated context and passed a lossy natural-language summary to the Coder, hiding the original codebase and dropping fine-grained rules (like signed zero -0).
  • In E18 (Advisor Blueprint Injection — Golden Pattern), the Executor retains direct visibility into the actual source files, and the Advisor injects a concise 1,143-character checklist of cross-module invariants (one_shot_execution_blueprint) directly into Turn 1 before the Executor writes code. Nothing is lost in translation, and the Executor completes all 3 files in a single pass.

4. All 20 Hidden Grading Suite Results (labjudge18)

Both modes passed all 20 hidden unit and E2E test cases verified by labjudge18:

# Test Case ID Invariant / Boundary Verified Mode A (Solo Loop) Mode B (Advisor-Executor)
1–6 validator_01 to validator_06 Trim/uppercase, non-mutation, JPY, bool rejection, 0 cents & 1000 bps bounds 6 / 6 PASS 6 / 6 PASS
7–15 settlement_01 to settlement_09 First-seen order, conflict check, 300 vs 301 bps, 14999 vs 15000 tier, rebate cap 9 / 9 PASS 9 / 9 PASS
16–20 e2e_01 to e2e_05 # separator, signed zero -0, $0 auth statement, line order, error propagation 5 / 5 PASS 5 / 5 PASS
Total 20 Hidden Cases — 20 / 20 (100%) 20 / 20 (100%)

5. Practical Takeaway: Inject an Architectural Blueprint Before First Edit

When using Claude Code (/advisor), Antigravity (/plan), or your own custom agent harness on multi-file tasks:

  1. Never let an Executor start editing File 1 before mapping File 2 and File 3's contracts: Piecemeal file-by-file trial loops double your output token generation and bloat input history with intermediate failure logs.
  2. Generate a 5-bullet Cross-Module Blueprint on Turn 1: Have your Advisor model (or planning step) output a compact <1,500-character contract checklist specifying the exact data types, boundary inequalities (> vs. >=), and formatting invariants across all affected files before unlocking file edit tools.

🔗 Continue Reading the Series

Top comments (0)