Iteratively refining execution pipelines delivers measurable gains even when model parameters stay frozen. Recent work shows that a few plug‑and‑play edits to an agent’s harness can lift benchmark scores by double digits, challenging the assumption that performance upgrades must come from costly weight updates.
Until now most performance jumps were achieved through fine‑tuning, prompt‑engineering sweeps, or adding external tools that required re‑training or heavy hyper‑parameter searches. The prevailing view treated the LLM’s internal weights as the primary lever for improvement, while the surrounding execution harness was regarded as a static wrapper.
A plug‑and‑play harness revision lifts GPT‑5.4‑nano’s SpreadsheetBench accuracy to 77.58 %, exceeding GPT‑5.5 with SkillOpt’s 76.20 % and improving all four tested models by 5.69–11.39 percentage points [1]. The authors describe this as “Harness self‑evolution revises an agent’s software scaffolding, including prompts, memory, workflows, and executable components, while keeping model weights fixed,” and each retained revision feeds back into the next update cycle.
A single recursive harness edit adds up to 11.5 points on WorkBuddy‑Bench for GLM‑5, with similar gains of 4.5 points on JobBench and an 11.8 % lift in pass rate on LAB, all without touching model weights [2]. The paper frames the process as “Let be a model with frozen parameters… A harness consists of editable components… It composes with to form an agent without changing its weights,” demonstrating that orchestration alone can rival traditional fine‑tuning.
The results leave open how well these methods scale beyond curated professional suites and whether the high‑probability admission rule based on fresh, independent validation data will hold in truly open‑world deployments [1]. Moreover, the authors note that Harness² uses a contrastive Propose‑Probe‑Compose step designed to guide edits without ground‑truth verifiers, though its performance in scenarios lacking any reliable verification signal remains an open question.
If these findings generalize, benchmark pipelines should incorporate systematic harness optimisation as a first‑order hyperparameter, rerunning evaluations after each approved revision. Treating the execution scaffold as an editable artifact unlocks low‑cost performance boosts and reduces reliance on expensive retraining cycles.
Top comments (0)