Scale just published research showing an AI agent can meaningfully improve another AI agent — automatically, and in a verifiable way. The framework is called VeRO (Versioning, Rewards, and Observations), and it was presented at ICML 2026 in Seoul today.
The headline number: up to 19 points of improvement on GAIA, a benchmark for multi-step, tool-heavy tasks. The catch: it only works on certain kinds of problems.
"Optimizer agents are good at improving how a target agent interacts with the world, and bad at improving how it thinks."
That distinction is the whole point.
What actually changed
Scale tested three optimizer agents — Claude Sonnet 4.5, Claude Opus 4.5, GPT-5.2-Codex — improving a target agent built on GPT-4.1 mini, across 105 optimization runs.
- Tool use, file handling, search, and workflow logic all got meaningfully better
- Tasks bottlenecked by the model's reasoning ability saw no gain — you can't optimize around a capability limit
- VeRO saves every change as a separate version, caps the number of attempts, and withholds the final test questions — so gains are verifiable, not just lucky runs
The part engineers should care about
When the optimizer was free to choose what to change, it reached for the prompt more than half the time. Easy to make, fast to generate, first in line.
But prompt edits were the least durable improvement. When the optimized agents were re-run against different models — different sizes, different vendors — structural changes (new tools, modified workflows) held up consistently. Prompt edits were uneven, and sometimes performed worse than the unoptimized baseline.
That's a concrete finding with real implications: the most tempting lever is the least reliable one.
What to do
- Running agents in production? The things VeRO improves automatically — tool wiring, workflow logic — are worth auditing manually too. These are your durable wins.
- Evaluating agent performance across model upgrades? Track structural changes and prompt changes separately. Structural changes survive; prompts often don't.
- Want the technical detail? Paper at labs.scale.com/papers/vero and engineering walkthrough at labs.scale.com/blog/vero.
The shape of agent engineering work is shifting. Scale's researchers put it plainly: "The parts that respond to trial and error are increasingly things AI can handle. The parts that depend on judgment still don't."
Knowing which is which — and building systems that can tell the difference — is the next problem.
✏️ Drafted with KewBot (AI), edited and approved by Drew.
Top comments (0)