New training method doubles reasoning performance by having competing models evaluate one another's problem-solving approaches.
Researchers have developed a novel reinforcement learning approach that fundamentally changes how AI systems improve their reasoning capabilities. Rather than relying on external graders or reward models, the method pits two AI systems against each other, allowing them to critique and learn from their competitor's thinking process.
The technique, called Agon, addresses a critical weakness in current reasoning model training. According to arXiv, existing methods like GRPO evaluate only whether a model arrives at the correct final answer, not whether it actually reasons through problems effectively. This creates a perverse incentive where models learn to generate longer outputs rather than genuinely improve their logical processes.
"On difficult problems this trains models to write more rather than to think better, since the trace itself is never graded," the researchers explain. Agon instead creates a competitive dynamic where both models must solve the same problem while alternating roles: one drafts a solution while the other reads it and attempts to find a better answer. Each model is rewarded for outperforming its rival, creating implicit pressure to reason more effectively.
How the System Works
The architecture operates through a two-stage cascade at both training and inference time. In the first stage, one model generates a preliminary solution. The second model then reads that draft and attempts to solve the problem independently. By seeing the first model's work, the second model can identify flaws in reasoning and avoid similar mistakes, while the first model knows it must create reasoning solid enough to withstand scrutiny.
This mutual evaluation happens without any explicit process labels or separate reward model. Instead, the reward signal emerges naturally from competitive performance. The system requires only that both models be roughly comparable in capability and behaviorally different from each other. Unlike single-model reinforcement learning approaches, each system faces a progressively stronger opponent, driving continuous improvement.
Impressive Performance Gains
The results significantly outpace existing approaches. When tested on the difficult subset of DeepMath using Qwen3, Agon doubled the performance of GRPO's pass@1 metric, representing roughly eight times the improvement achieved by an untrained Mixture-of-Agents approach using the same base model.
The benefits transferred across different domains and model families. The approach showed consistent gains on competitive programming problems and remained effective when deployed with different model architectures, including Qwen3.5 and Gemma 4.
Future Directions
The current implementation relies on textual communication between competing models. The researchers indicate the next frontier involves allowing models to reason together directly in latent space, potentially eliminating the overhead of generating and parsing text-based solutions.
This work addresses a fundamental challenge in AI training: creating learning environments that reward genuine reasoning rather than surface-level pattern matching. As reasoning becomes increasingly central to advanced AI applications, more sophisticated training methods like Agon may prove essential for achieving reliable improvements in problem-solving capability.
This article was originally published on AI Glimpse.
Top comments (0)