What just happened? SemiAnalysis, a chip and infrastructure research firm, launched a new benchmark called GDPx on 2 November. It scores AI agents on 206 long, realistic tasks across 11 professional domains — chip design, cybersecurity, biology, finance, legal work, quantitative trading, hardware design, materials science, software engineering, alignment and recursive self-improvement. The best model in the world scored 12 percent.
Who leads? Anthropic’s Claude Opus 5.5 leads at 12.0 percent, ahead of Google’s Gemini 4 Carbon at 10.9 percent and OpenAI’s GPT-6 Astra at 9.6 percent. Opus 5.5 wins in quantitative trading, hardware design and chip design. The leaderboard publishes 13 of the 20 models tested; the rest are not yet public.
Why is 12 percent good news for the benchmark? Because most AI benchmarks are saturated — models score in the nineties on tests designed years ago, and the numbers stop telling you anything. GDPx is deliberately built from tasks that take skilled humans hours and that are graded pass-or-fail, averaged over three attempts with an eight-hour limit per task. A 12 percent top score means the test still has headroom. SemiAnalysis calls it “a true frontier-level benchmark” for exactly that reason.
What does the low score actually mean? Less than the headlines suggest, and more. It does not mean AI can only do 12 percent of the economy’s work — GDPx measures performance on its own selected tasks, not the share of jobs AI can automate. But the pattern inside the score is revealing: under binary grading, almost every model scored zero on finance and legal tasks, where a half-right answer is a wrong answer. And the cost is sobering. Opus 5.5 costs $51.41 per attempt, which works out at roughly $428 per successful outcome at a 12 percent pass rate. Gemini 4 Carbon, at $16.29 per attempt, works out far cheaper per success — though it is not yet generally available.
What are the caveats? Several, and SemiAnalysis states some of them itself. These are one firm’s tasks and one firm’s grading, with no independent reproduction yet. Safety classifiers held back Opus 5.5 and GPT-6 Astra in cybersecurity, biology and alignment tasks, so their headline scores are, in the firm’s word, “sandbagged”. The benchmark is a living one, due to refresh every three to six months. Treat GDPx as a new yardstick, not a final verdict.
Why this matters: the AI industry has spent two years arguing about whether agents can replace professional work, usually with cherry-picked demos. GDPx is the first widely watched attempt to answer with a hard, shared test — and its answer is that the best systems in the world still fail roughly seven out of eight long professional tasks. That context belongs alongside Anthropic’s filing revealed an $895 billion compute bill: enormous money is being spent to close a gap this benchmark has just measured in public. And while AI’s godfathers cannot agree on how dangerous AI is, GDPx gives the rest of us a plainer question to ask: not how smart the models sound, but how often they actually finish the job.
