Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers

What happened: A preprint introduces SciSlopBench, a benchmark of 390 AI-generated scientific papers paired with human-written papers, and SciSlopHarness, a framework that revises AI papers only where experiment records support the change. Key numbers: 390 AI-generated papers. Detection accuracy 85.9% versus 68.7% for Binoculars. Why it matters: The authors state that existing token-based AI detectors miss reasoning-level failures, and that direct slop-aware prompting triggers reward hacking. They plan to run the check on 60K ICLR submissions.

A preprint introduces SciSlopBench, a benchmark of 390 AI-generated scientific papers paired with human-written papers, and SciSlopHarness, a framework that revises AI papers only where experiment records support the change.

The authors state that existing token-based AI detectors miss reasoning-level failures, and that direct slop-aware prompting triggers reward hacking. They plan to run the check on 60K ICLR submissions.