SemiAnalysis launches GDPx, an agent benchmark where the best model scores 12%

SemiAnalysis published GDPx on November 2: 206 tasks in 11 domains, including chip design, cybersecurity, biology, finance, legal, quant trading, hardware design, materials science, alignment, software engineering and what it calls recursive self-improvement. Fourteen specialized data startups built the tasks, and each one sits inside an environment (a deal room of about 4,000 files for finance, live services for cyber, a SolidWorks or Siemens NX session for hardware).

Each model ran in its native harness where one exists: Claude Code for Opus 5.5, Codex for GPT-6 Astra, Antigravity for Gemini 4 Carbon, and OpenCode for the rest. The default time limit is 8 hours. The headline number is pass@1 across three attempts, with domains weighted equally.

The top of the leaderboard: Claude Opus 5.5 12.0, Gemini 4 Carbon 10.9, GPT-6 Astra 9.6, Muse Wonder Full 9.0, Claude Fable 5.1 8.5, GPT-6.1 Sol 8.4, Claude Sonnet 5.5 8.2. DeepSeek 4.1 Flash 1017 is the best open-weight entry at 5.4. SemiAnalysis says safety classifiers pull down Opus and Astra on cyber, bio and alignment, and that finance and legal were so hard that basically every model scored 0 with binary grading, so those domains report partial scores.

The software engineering tasks (from Datacurve) carry the finding we'd look at first: 76% of failed runs missed requirements around the main request, such as keeping existing behavior and edge cases, and models tended to write tests that confirmed their own fix. The leaderboard and public tasks are on gdpx.semianalysis.com.