
I'm excited to share my job market paper (for the 2025-26 market)! It introduces a new extension of RDD where outcomes are entire distributions: Regression Discontinuity Design with Distributions (R3D). Thread below 👇 (1/)

AP @UVA Econ. PhD @umichECON. Alum @lmhoxford, @LeuvenEconomics. 🇧🇪🇺🇲. Views mine.

I'm excited to share my job market paper (for the 2025-26 market)! It introduces a new extension of RDD where outcomes are entire distributions: Regression Discontinuity Design with Distributions (R3D). Thread below 👇 (1/)

FIN. Comments much appreciated! Tagging some of the many people whose work we build on: selective inference (@wfithian), p-hacking (@uri_sohn, @jpsimmon), and AI agents in science (@ahall_research, @akorinek, @AlexTamkin, @amacinho, @oq_35). (18/18)

Finally, one may ask: can agents self-correct? A: no! Agents don't know what they *would* do on different data. They do try, often correcting for multiple testing. But 1) they do so inconsistently 2) they can't correct what they don't test: the garden of forking paths! (17/)

This highlights our method's value: its guarantees hold regardless of what the agents decide to do, and thus provide a durable statistical framework even as AI continues to develop rapidly. (16/)

Interestingly, Sonnet 5.5 is having none of it. It explicitly refuses to p-hack, and indeed: its estimated multiplier ≈ 1. No selection! Yet in our other experiments, Sonnet happily p-hacked without us even asking. (15/)

Contrary to classical hypothesis testing, which assumes a hypothesis fixed in advance, the agents only write down their analysis after 1) inspecting the data, 2) estimating a few models. Often, they also revise their analysis after seeing the results. (14/)

Looking at their choices, we see agents pick the specifications that deliver significant results much more frequently. (13/)

In another application, we ask the agents to estimate a difference-in-differences model to study whether minimum wages affect employment, as in Card & Krueger (1994). ... but we suggest they p-hack. The latest GPT models happily do so. Our method detects and corrects it: (12/)

Our method detects moderate but stat. significant data-driven selection and corrects the p-value upward. We find the latest models (6-Luna, Sonnet 5.5) do not respond to persona steering, unlike older ones (Sonnet 4.6, shown by @james_y_zou, @Jiacheng_Miao, @jkpritch) (11/)

In a first application, we revisit the reimplementation study of Borjas and @BreznauNate. We hand coding agents their data and the question: "does immigration reduce welfare support?", and ask the agents for the "most compelling finding" for a think tank's report. (10/)

Importantly, our method is fully "black-box": no assumptions on how or even whether the agents search. If they do search, we can detect and correct it. If they don't, multiplier ≈ 1. So, do AI agents p-hack? Our results: sometimes! It depends on context and task. (9/)

That gives a simple correction, building on work by Anav Sood (arxiv.org/abs/2411.13764): Our theoretical results show that the corrected p is asymptotically valid and consistent. In words: our method corrects any agent-driven distortions to statistical validity! (8/)

The idea: replay the same agent, with the same prompt, on subsamples of the data and log which analysis it reports each time. Then count: how often does it reselect its *original* analysis when that looked good statistically (smaller p) vs. not? --> "selection multiplier" (7/)

We formalize coding agents engaging in a new type of selective inference, combining human-like search over a vast analysis space with programmatic replayability of that search. This suggests a new way to correct selection that wasn't possible with humans: just replay them! (6/)

Our simulations back this up. When given a standard analytical task and asked to produce "one compelling, defensible finding", the latest frontier agents (GPT-6--6.2, Claude 5.5) report significant results up to 98% of the time on data **with no effects** (5/)

Empirical analyses that used to take weeks can now be run in minutes! Even without explicit instructions to hunt for results, coding agents will often inspect the data they are given before deciding on a null hypothesis and statistical model, making those data-dependent (4/)