SignalTech/AI
Can AI Reviewers See Your Efforts to Improve Your Paper? A Large-Scale Study of Paper Revision Trajectories
Large-scale study examines whether AI reviewers detect paper revision trajectories.
Context
What happened: A study using PaperTrajectory-2026 examined whether AI reviewers reflect manuscript improvements in their scores, testing eight AI reviewers on annotated paper revisions.
Key numbers: 6,959 paper trajectories across five AI conferences, 14 revision dimensions, eight AI reviewers.
Why it matters: Most revisions annotated as major improvements received unchanged scores, and a brief negative user opinion could erase or reverse a scoring advantage, raising questions about how AI reviewers judge manuscripts.
What changed
A study using PaperTrajectory-2026 examined whether AI reviewers reflect manuscript improvements in their scores, testing eight AI reviewers on annotated paper revisions.
Why it matters
Most revisions annotated as major improvements received unchanged scores, and a brief negative user opinion could erase or reverse a scoring advantage, raising questions about how AI reviewers judge manuscripts.