Professor of behavioural genetics with my own twins. I mix behaviour and genetics @unitartu and @mcgillu to understand obesity. I also mix global music as a DJ.
1/ New in @CellMetabolism: In an RCT, 160 adults drank a branded, sugary or unsweetened, calorie-free drink daily for 3 weeks. The sugar group got faster at approaching their drink’s logo. How much they liked the drink? Unchanged.
cell.com/cell-metabolism/ful…
I am sharing a new paper today with Dario Caldara: “Can Peer Review Survive AI-Generated Text?”
sas.upenn.edu/~jesusfv/Peer_…
Dario and I, as editors, are increasingly getting papers submitted to our journals that are 99 percent AI-generated. Even worse, we are also receiving many AI-generated referee reports, both as editors and, more frustratingly, as authors.
And we cannot emphasize this enough: the real problem is not that the papers and reports were AI-generated; it is that no one had done the thinking behind them, and you can so easily tell. The papers were eminently forgettable, and the referee reports were disappointingly uninsightful. We have seen too many reports of the form: “The paper is well-written and addresses an important problem. However, the authors should consider additional baselines.”
This deluge of AI-generated text puts peer review in jeopardy: journals will be flooded with low-quality submissions, economists will decline requests to referee papers or routinely use AI to referee them, and which papers get published will become less and less linked to their quality.
We do not want to defend the status quo entirely. Peer review is slow, many reports are poor, and editors (including us!) make bad calls all the time. Still, well-executed peer review remains the best system we have for assessing each other’s work and advancing the profession. AI could improve peer review if used deliberately for that purpose, for instance, to check proofs and code. That is not what we are seeing.
The difficulty is that readers, including editors and referees, cannot see how AI was used. As a result, in practice, the burden of verification shifts from the author to the reader.
Since we reject this shift, Dario and I did what economists always do when something bothers us: we wrote a paper.
So, what do we do in it? First, we check whether AI detectors such as Pangram get this right: is a referee report flagged as AI-generated indeed AI-generated? Or is Pangram just flagging proofreading and editing? No. We run an experiment with 25 referee reports from the pre-AI era. Proofreading and editing do not trigger AI-generated flags. Only when we ask the LLMs to substantially rewrite the referee report or to generate a new report do we get the AI-generated flags. Pretty compelling evidence, if I may say so.
Second, we build a theoretical model to explore who should bear the burden of verification: the author or the reader? (Yours truly always enjoyed the courses on the rules of evidence in law school; yes, as Billy Wilder would say, nobody is perfect.)
The answer is that it is much more efficient for the author to do it, because the author can verify at a much lower cost and only needs to do it once (rather than each reader repeatedly). Before AI, the authors more or less had to verify the results, even if only because they had to type them in. This was not perfect (and all papers ever published in economics probably have at least one mistake). But the problem is that, with AI, the author does not even need to read the text. Or can just skim it. Writing is thinking, and without writing there is usually not much thinking.
So, a paper flagged as AI-generated is prima facie evidence that the author has not verified it. Should this lead to an automatic rejection? No. But there should be a zero-tolerance policy for serious mistakes. The first serious mistake we find in the paper, and it is out! In comparison, for a non-flagged paper, a serious mistake is something that the editor might determine it can be fixed in a revision.
We are stricter with referee reports. There is absolutely no point in the referee generating a report using an LLM. As an editor, I can do that directly and save myself the trouble of asking you for it. Now, of course, you can use the LLM to help you with the report (if the journal allows it! Many journals do not allow you to upload the paper to an LLM, and you need to follow the rules). For example, you can use the LLM to search for existing results in the literature.
But peer review is not about finding one sentence that is not entirely precise or a slightly sloppy mathematical proof. Referees are asked: “Is this idea important despite its flaws?” And, relatedly: “Is the paper a good fit for the journal in terms of quality and focus?” That judgment is the task for which you are selected as a subject expert.
We also propose that journals require that, before a paper is submitted, the author submit it to an LLM using an approved prompt and include the LLM report and a discussion of what was and was not fixed. That way, the referee would not need to check (or at least not check so carefully) every technical detail, and they could focus on the big picture: Is this a good paper? Is this a good fit for this journal? What is fundamental for this paper to become publishable at this journal?
Anyway. The paper has many other arguments, and we try to be nuanced. So, before ferociously typing the first complaint that crosses your mind, would you consider reading it? Chances are that we have already discussed your objection. We have been thinking about this for a long time.
But by all means, leave good comments!
In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics. We indexed about 150,000 papers, audited 50,000 for issues, and did progressively more thorough checks on thousands more. We found nearly 1000 major errors that got cited many times, most are still uncorrected. This survey enabled me to estimate the efficacy of peer review for correcting errors (not very good) and to identify dozens of open questions that anyone with spare Claude Max tokens could answer today (.md context files included).
Bottom line: AI can speed up quantitative science by >1000x with no further model improvement required. But I'm seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs.
Among my fellow reformed academic high agency founders, however, several are speed running entire disciplines.
This survey has implications for how science will be run in the very near future. Hint: Academic publishing is dead. It was already a very stupid way to record and promulgate knowledge, and had been so for decades, even before all that publicly funded knowledge was locked behind paywall cartels. Now AIs can generate thousands of novel, correct papers per day, it is no longer meaningful for humans to consider a research paper a "unit of science" or to even bother much with the 99.9% of science which, up to this point, was necessary drudgery. Time to discover something like General Relativity or Evolution every week. LFG!!
Enjoy: science.caseyhandmer.com/phy…
In Estonia, certain food producers are making a massive lobby against Nutri-Score. Citizens took the matter in their own hands & over the weekend TWO Nutri-Score portals were born!
AMAZE! AMAZE! AMAZE!
@HercbergS
Apologies for neglecting you the last fortnight, but I've been exceptionally busy & haven't posted as much as I would have liked. Here is a fascinating comparison shot, seperated by 113 years - a colour Paget Plate photo showing the English village of Whitchurch-on-Thames in Oxfordshire in 1913 - a year before the Great War erupted - and as it stands today, almost untouched by the passing century. The lefthand image was taken using an early colour glass-plate process and isn't colourised.
I spent the last three months trying to start a scientific journal inside of Asimov Press. Everybody says journals suck, that they’ll become outdated with AI, and that we need to adapt and run more publishing experiments. I agree!
Journals are also not fun to read; PDFs are uninspiring, and we should be making more interactive versions of these texts. After looking into this, though, I think most journal editors would like to run more experiments, but they’re effectively stuck in a valley of death because of the publishers and indexers.
For a journal to be considered “real,” in a sense, and attract submissions, it often needs to be indexed by Clarivate or another major indexer. Clarivate also calculates the Journal Impact Factor. But these indexers have many dumb requirements. Clarivate, for example, requires access to full-text PDFs. So you can’t, as a journal seeking inclusion in Web of Science, just get rid of PDFs and move to a digital, interactive model; you would need to do all the digital stuff and THEN ALSO generate static PDFs to satisfy the indexer. This is one of the reasons journals don’t run more experiments; it’s a logistical headache. And if a journal steps too far out of line, it can lose indexing coverage or its impact factor.
This is what happened to eLife. It ran an aggressive experiment with peer review, lost its impact factor, and saw submissions fall 55 percent between 2024 and 2025. Much of its content is still indexed, and it has substantial funding from big institutional donors to help it survive. Many journals don’t have this luxury; they depend on publishing fees or subscriptions, and if that revenue dries up, they risk going out of business.
If you want to build a journal that is super creative and unique without worrying about these constraints, then, you need philanthropic or institutional funding, or you need to convince scientists not to care about impact factors and submit to you anyway. (But the latter is hard because many universities still weigh the perceived “quality” of the journals scientists publish in when deciding whom to promote.) It seems like one of the only ways forward, then, is to completely abolish the indexers or pass some kind of federal policy that prohibits universities from promoting people on the basis of impact factors.
TL;DR: Don’t misdirect your anger at the journals. The problem is with the people above the individual journals, especially the publishers and indexers. Academic publishing is filled with perverse incentives.
Anthropic lanzó una guía de 37 minutos para construir Agentes de IA que automatizan una empresa entera.
Gratis. De los ingenieros que construyeron Claude.
Agentes que trabajan, se reparten tareas y ejecutan todo solos.
Subtitulado al español.
We're launching a Creative Fellowship for an artist or scientific illustrator at @AsimovPress. This is a paid, 20hr/wk, three-month program (with negotiable, but competitive, pay.) You'd help revamp our website and make all the art for our upcoming Issue, plus work on your own projects.
If you are interested, please send three samples of prior work to my email, niko[at]asimov[dot]com, with a few sentences about who you are and why you are interested. We are especially looking for artists or illustrators who have an understanding of biology, yet maintain a unique style. (Art below by Ella Watkins-Dulaney.)
Kallas on Witkoff/Kushner: "We have institutions in place, so we can't just send any person who has no institutional role, like the Americans are sending. Clearly, we should not copy the same patterns."
The ability of this woman to grill people in the sweetest voice 🤌
New explanation for the missing heritability just dropped
(It's probably not THE explanation for iq related stuff since snph2 and twin h2 differs even when estiamted in the same sample with the same measures)
link.springer.com/article/10…
i finally built the AI scribe i always wanted, fully open and on-device!! sound on 🔊
it listens to doctor and patient, writes every word while they're still talking, and the moment they hang up it drafts the note and maps the symptoms and risk factors to a possible stroke
How should PhD students leverage AI in their research? First, you should write papers where you leverage AI to analyze something at scale. There are many low-hanging fruit because gen AI is essentially still a new thing. Exploit it!🧵
Terrific @TheAtlantic piece on archaeologists' and anthropologists' slowly dawning realization that now-submerged areas on the coastal shelf--swathes of land drowned when the glaciers melted--may be where much of our species' early history took place. (Link in reply)
🗣️Postdoctoral Research position at the Montreal Neurological Institute: at least two years of secured funded for working on machine-learning analysis of biological-contextual data and body-brain interactions👉neuropm-lab.com/open-positio…
Being a parent of a teenager in the modern digital environment means begging your kid to play Minecraft or do *something* even slightly less brain-rotting than watching short video content from Spotify, which is what they're started doing because you've blocked all YouTube content from the XBox devices at the router level and turned off the WiFi and limited the carrier data throughput after 9PM for their smart phones, which they legitimately need because that is how their coaches communicate with them about cheer routines or band practice
And if you dare to complain about how annoying it is to have this second job as an IT admin just to keep your kids from guzzling down the worlds shittiest content, half the internet calls you a bad parent for not locking them down more and the other half calls you a bad parent for being a content Nazi