ChatGPT hits a wall! It can crack millennium math problems, but the paper reproduction rate is as low as 13.98%?

What happened: UniPat AI released PaperBenchX, a benchmark of 93 end-to-end paper reproduction tasks across 12 research directions. The strongest tested configuration, GPT-6 Astra, fully reproduced only 13.98% of tasks. Key numbers: 93 reproduction tasks, 12 research directions, 10 scientific software environments, 3168 expert-checked scoring items, 13.98% Why it matters: The results indicate models can produce plausible partial results but rarely complete a verifiable end-to-end scientific workflow. The benchmark aims to measure the gap between current agents and a general AI scientist. Coverage: 3 outlets in 24h

UniPat AI released PaperBenchX, a benchmark of 93 end-to-end paper reproduction tasks across 12 research directions. The strongest tested configuration, GPT-6 Astra, fully reproduced only 13.98% of tasks.

The results indicate models can produce plausible partial results but rarely complete a verifiable end-to-end scientific workflow. The benchmark aims to measure the gap between current agents and a general AI scientist.