Population‑based prompt and tool evolution now delivers state‑of‑the‑art agent performance while the underlying LLM stays frozen.
Until recently, most gains came from scaling model parameters or fine‑tuning on task‑specific data; harnesses—prompts, tools, and skill graphs—were hand‑crafted or iteratively refined in a single lineage, often overfitting to narrow benchmarks.
DarwinX pushes verified Terminal‑Bench 2.1 accuracy to 84.7% on a frozen GPT‑5.6 Sol at medium effort. “On a frozen GPT-5.6 Sol at medium effort, DarwinX scores 84.7%, at the frontier of the verified Terminal-Bench 2.1 leaderboard” [1].
DarwinX also lifts a baseline agent from 75.5% to 83.2% on GPT‑5.5 under a strict zero‑tolerance protocol, proving that pure harness upgrades can eclipse model scaling. “On GPT-5.5, DarwinX lifts base Monet from 75.5% to 83.2% ( points) under the strict leaderboard protocol, in which every errored trial scores zero (§4.2).” [1].
AutoDesign reaches a PosterBench score of 78.32, surpassing the proprietary Claude Design system by 7.45 points. “under the matched Claude Code and Claude 4.8 configuration, AutoDesign scores 78.32, exceeding Claude Design by 7.45 points” [2].
Integrating the learned DesignHarness consistently raises average PosterBench performance from 54.99 to 67.39, a +12.4% jump across seven model‑agent configurations. “integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%)” [2].
SkillZip outperforms the strongest baseline by up to 12.2 points while compressing skill graphs 3.46×, preserving 99.2% of dependencies and 98.7% of verifier reachability. “SkillZip consistently outperforms the strongest baseline by up to 12.2 points, while achieving a 3.46 compression ratio with 99.2% dependency preservation and 98.7% verifier reachability.” [3].
DarwinX’s preserve‑and‑extend contract restricts candidate harnesses to those that do not regress on any benchmark, which curtails aggressive exploration and makes the method dependent on reliable task verifiers. “DarwinX makes that obligation explicit: it searches the harness rather than whole‑agent code, admits a child only under a preserve‑and‑extend contract that bounds what a win may cost elsewhere.” [1].
AutoDesign’s evaluation is confined to paper‑to‑poster synthesis; its benefits on unrelated domains remain unproven. “...meta‑harness optimizer guides a code agent to recursively improve harness based on rollout feedback.” [2].
SkillZip’s compression relies on reversible macros, and the reported gains are limited to technical and embodied benchmarks; scalability to libraries of millions of skills has not been demonstrated. “...SkillZip rewrites recurring contract‑valid motifs into reversible ported macros while preserving boundary signatures...” [3].
If evolutionary harness selection can rival model scaling on frozen LLMs, the default development pipeline should prioritize population‑based prompt, tool, and skill evolution before investing in larger models or extensive fine‑tuning, and existing agent benchmarks ought to be re‑run with evolved harnesses as a new baseline.
Top comments (0)