
The importance of this paper cannot be overstated. The authors compared AI reviews with 134 reports written by carefully selected, paid human experts. The results are impressive. Not only AI received higher ratings across all seven dimensions assessed by an AI judge. But even three former journal editors preferred AI reviews in 90% of the cases. The potential benefits of AI reviews for scientific advance are getting enormous: faster and higher quality feedback, less reviewing work, more time for research. We could even imagine journals run entirely by AI editors and reviewers. This could radically disrupt a publishing system that costs researchers considerable time and money. Let me say: some disruption would be welcome. But there are also major concerns. What happens when a few AI systems become gatekeepers across sciences? They might increase average review while narrowing the range of ideas that survive. This is particularly critical at the scientific frontier, where an important contribution may challenge the assumptions that existing research takes for granted. And even worse: let’s not forget that, near the frontier, AI systems become more likely to hallucinate. So they might embed errors at the very core of human knowledge. A tsunami is coming. And we’re not ready. * Paper in the first reply





















