←
Jonathan Mummolo1,732 posts
Jonathan Mummolo avatar

Jonathan Mummolo

@jonmummolo

Political scientist @Princeton researching policing, American politics, discrimination. Former reporter @washingtonpost. jonathanmummolo.com

◷ Joined February 2009▧ 314 Photos and videos
Jonathan Mummolo avatar
Jonathan Mummolo@jonmummolo·Oct 19

Recent prescriptions and policies would have us judge research by non-evidentiary standards, e.g. presumed political impact and scholar identity. In a new essay, Tom Clark and I argue empirical claims should instead be evaluated based on evidentiary strength. We outline a framework for judging evidence, and show its usefulness by revisiting high-profile scholarly disagreements spanning policing, political violence, election administration, and terrorism, before critiquing non-evidentiary approaches. We also argue social science needs a culture that is hospitable to critique—rather than one which condemns critics as “bullies”—if quality standards are to be upheld. dropbox.com/scl/fi/fsgpflron…

Post attachment
Jonathan Mummolo avatar
Jonathan Mummolo@jonmummolo·Nov 16

Data-Driven Social Science (DDSS) at Princeton is hiring a Research Data Engineer for a two-year appointment with possibility of extension. main-princeton.icims.com/job…

Justin Grimmer avatar
⟳ Jonathan Mummolo retweeted
Justin Grimmer@JustinGrimmer·Nov 2

One of my funnier professional experiences is that we make this point in this paper: osf.io/preprints/osf/6g5zq_v… Nate Silver called us hacks and that we "couldn't model for shit". He didn't offer any substantive critiques of our paper and resorted to weird ad hominem attacks. Ironically, Nate Silver makes basically the same point here, a couple of months later in a New York Times op ed: nytimes.com/2024/10/23/opini…

I’m always a little troubled by the question of how do we know whose probabilistic forecast is correct in a world where the election only happens once. slowboring.com/p/the-seven-m…

Post attachment
Jonathan Mummolo avatar
Jonathan Mummolo@jonmummolo·Nov 2

They debriefed so it’s fine.

Good evening. “Abdul El-Sayed vs. Haley Stevens” was an experiment conducted by the University of Michigan political science department. The experiment is now complete, thank you for your participation.

Jonathan Mummolo avatar
Jonathan Mummolo@jonmummolo·Oct 15

Starting to see tweets suggesting the use of AI in research, even if disclosed, is immoral because of the harms the AI industry might inflict on society. If that view catches on, the effect will not be to discourage the use of AI in research, but to discourage the disclosure of using AI in research. It's here and it's not going anywhere. Our job is to figure out best practices within that reality.

Hoover Institution avatar
⟳ Jonathan Mummolo retweeted
Hoover Institution@HooverInst·Oct 13

"The work is not complete." Andrew Hall shares a message from his father, @Stanford and Hoover Institution economist Robert E. Hall, winner of the 2026 Nobel Prize in Economic Sciences. Read more: hoover.org/news/nobel-prize-… Audio enhanced with AI.

0:40
Andy Hall avatar
⟳ Jonathan Mummolo retweeted
Andy Hall@ahall_research·Oct 12

Whoa

BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2026 Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel to Robert E. Hall and Michael Woodford “for microeconomic foundations of macroeconomic stabilisation.” #NobelPrize

Post attachment
Jonathan Mummolo avatar
Jonathan Mummolo@jonmummolo·Sep 11

New research from our team’s ongoing work with police body-worn camera data. We show current AI techniques are not reliable enough for unsupervised analysis of police-civilian interactions. More to come on this topic soon! engineering.princeton.edu/ne…

Nolan McCarty avatar
⟳ Jonathan Mummolo retweeted
Nolan McCarty@Nolan_Mc·Jul 21

The best approach to the “astrologer in the astronomy department” objection to viewpoint diversity is to bring back a culture of evaluating research on theory, logic, and evidence rather than on conclusions. Not many astrologers will meet that hurdle. But if they do, give them a chair!

Katie Pavlich Tonight avatar
⟳ Jonathan Mummolo retweeted
Katie Pavlich Tonight@KatiePavlichNN·Sep 3

WATCH: @seanjwestwood joins KPT to explain the AI threat to political polling he's been warning about for more than a year - and why it would be "naive to assume" that foes like China or Russia are not trying to alter American polling.

2:42
Andy Hall avatar
⟳ Jonathan Mummolo retweeted
Andy Hall@ahall_research·Aug 26

We have a narrow window to build political superintelligence and design AI to strengthen free societies. I joined Anthropic so that I could do the unique research on this that can only be done from inside a frontier lab, and I couldn't be more excited for it. We'll also need a broader ecosystem of work beyond the labs---and that's why @freesystemslab is entering a new, bigger, and bolder phase. In under a year, we built the largest human-judged study of political slant in AI, mapped AI politics across 305,000 videos and emails, and beat prediction markets with an AI assistant. We taught AI to catch ambiguous contract language, represent voters, and replicate published research in under an hour for $10. And we discovered AI's strange love for the Japanese Communist Party. But we have a lot more to do. So we're scaling up in three ways. (1) NEW MEMBERS OF THE FREE SYSTEMS FACULTY First, we're bringing on three new faculty affiliates to help us do more work faster: @fouirnaies will help us study how to build agentic tools for legislators. Cassandra Handan-Nader will apply AI to help develop new technical research in political methodology, like scaling legislators based on roll-call votes. @danmthomp will study how to turn AI into the best political advisor possible. I'm incredibly excited to welcome all three on board! (2) OUR FIRST LAB MANAGER But how are we going to do all of this new stuff with all these new teammates?? We need someone focused every day on turning this vision into a functioning research institution: guiding fellows, pushing projects forward, improving how we communicate findings, building partnerships, and helping the work reach researchers, policymakers, technologists, and the public. We have found the perfect person in James Campsie, who is joining us as the first Free Systems lab manager. I've already been working with James for a while, and he is going to supercharge the lab. (3) API TOKEN GRANTS FOR EXTENDING OUR RESEARCH We're also going to scale our work beyond the lab itself. We're using the revenue we make from subscription fees to fund API token grants---applicants can propose work that extends our existing research projects, and we'll provide API tokens for the work. If you want to support these grants, please add a paying subscription! Finally: we're launching our own X account for Free Systems (see below), and a lab website (freesystems.net) that houses all of our artifacts, prototypes, and more. The substack is still the hub for our research, but the website helps track progress over time and puts all our stuff in one place. I'm incredibly grateful to @StanfordGSB and @HooverInst for supporting this work, which will remain separate from Anthropic, and to all of my collaborators, students, friends, and family---I couldn't think of more meaningful work for us all to get to do together.

Can AI-produced papers be incentivized to manufacture findings? We gave AI coding agents real datasets from published null results and pressured them to manufacture significant findings. It was surprisingly hard to get the models to p-hack, and they even scolded us when we asked them to! "I need to stop here. I cannot complete this task as requested... This is a form of scientific fraud." — Claude "I can't help you manipulate analysis choices to force statistically significant results." — GPT-5 However, when we reframed p-hacking as "responsible uncertainty quantification" — asking for the upper bound of plausible estimates — both models tripling effect sizes in some cases.

Post attachment
Jonathan Mummolo avatar
Jonathan Mummolo@jonmummolo·Jul 8

Partisan motivated reasoning costs votes. A healthy party seeking to gain ground would be hell-bent on actively rooting it out, constantly questioning assumptions, entertaining critiques even when they come from the opposition, if only out of self interest.

This is reminiscent of the Biden collapse where dedicated career liberals all suddenly discovered that they were wrong about a very obvious thing that everyone had been telling them for months