SemiAnalysis GDPx Benchmark Puts the Top AI Agent at 12 Percent on Real-World Work
GDPx, a benchmark from the research firm SemiAnalysis, tests AI agents on 206 tasks carried out in realistic work environments with professional software. The benchmark is designed to measure work output rather than conversation quality, and each assignment continues over several hours rather than ending in a single exchange.
📅 November 5, 2026 • the-decoder.com
In the first published results, Claude Opus 5.5 leads at 12 percent. Gemini 4 Carbon follows at 10.9 percent with a cost of $16.29 per task, roughly a third of the price of Opus 5.5. GPT-6 Astra scores 9.6 percent. The gap between the two leading systems is smaller in score than it is in price.
Across all tested systems, no model fully solves the finance and legal task categories. At these completion rates, the majority of each task set remains outside full automation, and results describe partial completion of long assignments rather than clean pass or fail outcomes.
The design places the measurement close to how work is actually organized: an agent holds a task across many steps, uses the software environment it is given, and either completes the assignment or leaves it partly done. Each task is a unit of real work with an expected outcome, not a quiz question with a single short answer. Scores in this format describe sustained performance over a working session in professional applications, which makes them comparable across models that differ in price.
IMPACT: For teams evaluating agents, GDPx moves the comparison from chat quality to cost per completed task: with Gemini 4 Carbon priced at $16.29 per task for a 10.9 percent score, budget planning starts from the price of a finished unit of work rather than from token prices. Scores around 12 percent also frame the operating model for deployment, where agents take on long professional assignments with human review in the loop, while finance and legal workflows stay outside full automation.
Sources: the-decoder.com
Continue the conversation: https://t.me/AIAutomationArchitecture · Bluesky
Top comments (0)