←
Max Steinhardt489 posts
Max Steinhardt avatar

Max Steinhardt

@steinhardt_max

Professor of Economics, Freie Universität Berlin.

◷ Joined December 2021▧ 25 Photos and videos
Joachim Voth avatar
⟳ Max Steinhardt retweeted
Joachim Voth@joachim_voth·Nov 17

been saying this for a while; without an LLM first line of defence, the journals cannot survive...

I am sharing a new paper today with Dario Caldara: “Can Peer Review Survive AI-Generated Text?” sas.upenn.edu/~jesusfv/Peer_… Dario and I, as editors, are increasingly getting papers submitted to our journals that are 99 percent AI-generated. Even worse, we are also receiving many AI-generated referee reports, both as editors and, more frustratingly, as authors. And we cannot emphasize this enough: the real problem is not that the papers and reports were AI-generated; it is that no one had done the thinking behind them, and you can so easily tell. The papers were eminently forgettable, and the referee reports were disappointingly uninsightful. We have seen too many reports of the form: “The paper is well-written and addresses an important problem. However, the authors should consider additional baselines.” This deluge of AI-generated text puts peer review in jeopardy: journals will be flooded with low-quality submissions, economists will decline requests to referee papers or routinely use AI to referee them, and which papers get published will become less and less linked to their quality. We do not want to defend the status quo entirely. Peer review is slow, many reports are poor, and editors (including us!) make bad calls all the time. Still, well-executed peer review remains the best system we have for assessing each other’s work and advancing the profession. AI could improve peer review if used deliberately for that purpose, for instance, to check proofs and code. That is not what we are seeing. The difficulty is that readers, including editors and referees, cannot see how AI was used. As a result, in practice, the burden of verification shifts from the author to the reader. Since we reject this shift, Dario and I did what economists always do when something bothers us: we wrote a paper. So, what do we do in it? First, we check whether AI detectors such as Pangram get this right: is a referee report flagged as AI-generated indeed AI-generated? Or is Pangram just flagging proofreading and editing? No. We run an experiment with 25 referee reports from the pre-AI era. Proofreading and editing do not trigger AI-generated flags. Only when we ask the LLMs to substantially rewrite the referee report or to generate a new report do we get the AI-generated flags. Pretty compelling evidence, if I may say so. Second, we build a theoretical model to explore who should bear the burden of verification: the author or the reader? (Yours truly always enjoyed the courses on the rules of evidence in law school; yes, as Billy Wilder would say, nobody is perfect.) The answer is that it is much more efficient for the author to do it, because the author can verify at a much lower cost and only needs to do it once (rather than each reader repeatedly). Before AI, the authors more or less had to verify the results, even if only because they had to type them in. This was not perfect (and all papers ever published in economics probably have at least one mistake). But the problem is that, with AI, the author does not even need to read the text. Or can just skim it. Writing is thinking, and without writing there is usually not much thinking. So, a paper flagged as AI-generated is prima facie evidence that the author has not verified it. Should this lead to an automatic rejection? No. But there should be a zero-tolerance policy for serious mistakes. The first serious mistake we find in the paper, and it is out! In comparison, for a non-flagged paper, a serious mistake is something that the editor might determine it can be fixed in a revision. We are stricter with referee reports. There is absolutely no point in the referee generating a report using an LLM. As an editor, I can do that directly and save myself the trouble of asking you for it. Now, of course, you can use the LLM to help you with the report (if the journal allows it! Many journals do not allow you to upload the paper to an LLM, and you need to follow the rules). For example, you can use the LLM to search for existing results in the literature. But peer review is not about finding one sentence that is not entirely precise or a slightly sloppy mathematical proof. Referees are asked: “Is this idea important despite its flaws?” And, relatedly: “Is the paper a good fit for the journal in terms of quality and focus?” That judgment is the task for which you are selected as a subject expert. We also propose that journals require that, before a paper is submitted, the author submit it to an LLM using an approved prompt and include the LLM report and a discussion of what was and was not fixed. That way, the referee would not need to check (or at least not check so carefully) every technical detail, and they could focus on the big picture: Is this a good paper? Is this a good fit for this journal? What is fundamental for this paper to become publishable at this journal? Anyway. The paper has many other arguments, and we try to be nuanced. So, before ferociously typing the first complaint that crosses your mind, would you consider reading it? Chances are that we have already discussed your objection. We have been thinking about this for a long time. But by all means, leave good comments!

Post attachment
EAYE avatar
⟳ Max Steinhardt retweeted
EAYE@EAYEconomists·Oct 7

📣 EAYE 27 CFP is open! We invite early-career economists from all fields of economics to submit their work for the next Annual Meeting 📍 IMT School for Advanced Studies, Lucca, IT 📅 22–24 May 27 📄 Paper/extended abstract 📅 Deadline: 15 Nov 26 Spread the word! #EAYEAM27

Post attachment
Toman Barsbai avatar
⟳ Max Steinhardt retweeted
Toman Barsbai@TomanBarsbai·Oct 5

Join us for the 2027 Bristol Applied Economics Meetings (BÆM)! 💥 Economics of Crime, Violence, and Conflict, 4-5 May Keynotes: Emily Nix & Paolo Pinotti 🌍 Development Economics, 6-7 May Keynotes: Benjamin Marx & Diana van Patten Call for papers👉 baem.info

Post attachment
Alex Veremeyenko avatar
⟳ Max Steinhardt retweeted
Alex Veremeyenko@alex_verem·Sep 12

MIT published a brutally honest report on what AI is doing to students. A committee of professors and students spent five months studying how AI changed learning on campus, and the findings read like a warning to every university on the planet. Study groups are disappearing. Office hours are emptying out. Problem sets and take-home exams no longer prove anything, because AI can produce credible solutions to almost any written assignment in the undergraduate curriculum. Students who lean on chatbots lose mastery and confidence, and some slip into what the report calls cognitive surrender, reaching for AI at the first hint of struggle. The numbers are rough. 46 percent of surveyed MIT undergrads use LLMs daily. 90 percent worry about their own overreliance. Undergrads who feel AI makes them replaceable now outnumber those who feel it makes them capable. The committee's answer surprised me. They refused to fight AI with surveillance. The report calls AI detectors unreliable, says lockdown browsers feel like spying, and warns that policing students builds a classroom atmosphere of mutual distrust. Instead, MIT wants to rebuild education around the things AI can't replace. That means oral exams, semester portfolios, in-person project work, and a required social component in every subject. The report even floats the idea of rethinking grades entirely, since without a GPA to optimize, much of the incentive to cheat with AI evaporates. The committee warns professors against replacing undergrad research assistants with AI agents just because they're cheaper, because a university exists to grow people, not output. The most famous tech school on earth admitted the machines broke its way of teaching. Its answer is more humans, not more software.

Post attachment
Rudi Bachmann avatar
⟳ Max Steinhardt retweeted
Rudi Bachmann@BachmannRudi·Sep 4

Es gibt eine offizielle Stellungnahme des @VfS_econ , verfasst von Regina Riphahn, Kerstin Schneider und mir (unter Mithilfe einiger KollegInnen) zum drohenden Verlust von Daten in Deutschland. Bitte RT und mit Euren Kontakten in der Politik sprechen. socialpolitik.de/de/kommissi…

Andy Ferrara avatar
⟳ Max Steinhardt retweeted
Andy Ferrara@Andreas_Ferrara·Mar 19

My research team (w/ Sam Bazzi, Eric Chyn, Martin Fiszbein, Thomas Pearson, and Pat Testa) is hiring a full-time economics pre-doc starting in May! If you know someone who might be interested, see the link below: workforcenow.adp.com/mascsr/…

Arianna Ornaghi avatar
⟳ Max Steinhardt retweeted
Arianna Ornaghi@ariannaornaghi·Jan 7

📢 Call for papers 9th Economics of Media Bias Workshop Keynotes: Ruben Durante (NUS) and Pinar Yildrim (Wharton) 🗓️ May 28th-29th 2026 📍 Berlin, Germany ⏰ Deadline: January 31st 2026 Full CfP and information on how to submit: tinyurl.com/2s3f6tar

Post attachment
Luca Repetto avatar
⟳ Max Steinhardt retweeted
Luca Repetto@luca_repetto_c·1 Dec 2025

Fourth edition of our workshop, with the great Sascha Becker as keynote. May 28-29, if you have a paper on institutions/culture/political econ and more with historical flavour please feel free to apply. Help us spread the word!

Announcing the 4th edition of the Workshop in Economic History in Uppsala, with Sascha Becker as keynote! Workshop takes places on May 28-29, 2026. Apply by January 31, 2026.    Call for papers: dropbox.com/scl/fi/6xdmpjp3i… Sign-up form: docs.google.com/forms/d/e/1F…

Post attachment
Max Steinhardt avatar
Max Steinhardt@steinhardt_max·21 Nov 2025

Just arrived in Tampa for the 95th Meeting of the Southern Economic Association! Looking forward to great discussions and new ideas. If you’ll be here and want to meet up, feel free to send me a DM. Always happy to offer advice to early-career scholars and students. #SEA2025

Post attachment