←
Stephen Bates279 posts
Stephen Bates avatar

Stephen Bates

@stats_stephen

Assistant Professor, MIT EECS. Rigorous stats & ML methods for data-driven science and reliable AI systems. My research group is hiring postdocs & PhDs!

◷ Joined April 2018▧ 8 Photos and videos
Stephen Bates avatar
Stephen Bates@stats_stephen·15 Oct 2021

📰 Excited to share our new work on risk control in prediction! Multiple testing leads to practical calibration algorithms with PAC guarantees for any statistical error rate. Works with any model + data distribution! arxiv.org/abs/2110.01052 #Statistics #MachineLearning

Thrilled to share Learn then Test, a tool to calibrate any model to control risk (eg. IOU, recall in object detection). No assns on model/data. See arXiv arxiv.org/abs/2110.01052 + Colab colab.research.google.com/gi… ✍️w/@stats_stephen, E.J. Candes, M.I. Jordan, @lihua_lei_stat! 🧵1/n

0:54
gokul gowri avatar
⟳ Stephen Bates retweeted
gokul gowri@gokl_g·Nov 6

the transcriptional state of a cell is not generally sufficient to predict its past or future. in our new preprint we show that conditioning on lineage context (sisters, cousins, etc) can help a lot. this one was v fun :) arxiv.org/abs/2610.40447

New preprint on generative modeling for mouse development! Rather than single cell trajectories, we set out to model embryos as trees. Led by @gokl_g, @njwfish, @WilliamNColgan, w/ Philippe Rigollet, @stats_stephen, Zack Smith, @marinkazitnik, @lukekoblan and a wonderful team

0:05
Stephen Bates avatar
Stephen Bates@stats_stephen·Apr 3

Announcing the Statistical Frameworks for Uncertainty in Agentic Systems workshop at ICML '26!

Excited that our ICML 2026 workshop Statistical Frameworks for Uncertainty in Agentic Systems got accepted 🎉 @icmlconf #icml2026 We want to bring together people thinking about uncertainty and agentic systems.

Post attachment
Cai Zhou @ COLM2026 avatar
⟳ Stephen Bates retweeted
Cai Zhou @ COLM2026@zhuci19·Apr 2

(1/5) Modern reasoning systems rely on test-time scaling: CoT, self-consistency, MCTS... But two challenges remain: 1️⃣ Confidence signals shift across tasks/prompts 2️⃣ Stopping decisions are typically static and heuristic We ask: Can we adapt confidence within each reasoning trajectory — while still preserving statistical guarantees? Calibrating LLM reasoning in test-time scaling is not new. But what if calibration itself could adapt online — at test time — to the specific reasoning trajectory of each instance? Our new paper proposes a Test-Time Training framework for calibrating generalizable LLM reasoning, enabling instance-level adaptation with distribution-level robustness. Paper: arxiv.org/abs/2604.01170

Anastasios Nikolas Angelopoulos avatar
⟳ Stephen Bates retweeted
Anastasios Nikolas Angelopoulos@ml_angelopoulos·Feb 24

Today I'm sharing a preprint on conformal risk control for non-monotonic losses, a paper three years in the making. The key idea: validity of conformal can be reframed as a consequence of algorithmic stability. Therefore, any stable algorithm inherits a conformal guarantee. 🧵

Post attachment
Edgar Dobriban avatar
⟳ Stephen Bates retweeted
Edgar Dobriban@EdgarDobriban·10 Dec 2025

I wrote a review paper about statistical methods in generative AI; specifically, about using statistical tools along with genAI models for making AI more reliable, for evaluation, etc. See here: arxiv.org/abs/2509.07054! I have identified four main areas where statistical thinking can be helpful. These are just a subset of what is out there; other topics have been well-covered in other reviews. 1. Designing "statistical wrappers" around a model, for instance, changing behavior of a trained model (e.g., abstaining), where a score, e.g., an "unsafety score" is too high. The key connection to statistics is to use the quantiles of the loss (on a calibration set) to set the critical threshold, thus enabling conformal-type high probability guarantees. 2. Closely related, methods for uncertainty quantification, which enable the model to express uncertainty in an answer. A crucial component here is "calibration", whereby the uncertainty is required to reflect reality. 3. Statistical methods for AI evaluation: Specifically, tools for statistical inference (e.g., confidence intervals) on model performance. Exciting recent work proposes careful statistical models for leveraging a very small high-quality dataset, possibly combined with much larger low-quality datasets, for accurate evaluation. 4. Experiment design and interventions. Careful AI experiments to understand and steer models may require interventions such as modifying experimental settings in a controlled manner. This brings up connections to classical experimental design in statistics. This connection has largely remained implicit so far, and my review aims to make it more explicit; hoping that experimental design principles will become useful here. This review references the work of many, including @HamedSHassani @obastani @tatsu_hashimoto @yuekai_sun @CsabaSzepesvari @ml_angelopoulos @stats_stephen @yaniv_romano @yaringal @KilianQW @_onionesque +their teams, and some work that I was also involved in. Hopefully, my review will be helpful to orient yourself in this exciting area. Nonetheless, since the area is rapidly expanding, it is possible that I missed important references. Please feel free to let me know of anything that I should add/change!

Post attachmentPost attachmentPost attachmentPost attachment
Aaron Roth avatar
⟳ Stephen Bates retweeted
Aaron Roth@Aaroth·8 Nov 2025

If you work at the intersection of CS and economics (or think your work is of interest to those who do!) consider submitting to the ESIF Economics and AI+ML meeting this summer at Cornell: econometricsociety.org/regio…

Cai Zhou @ COLM2026 avatar
⟳ Stephen Bates retweeted
Cai Zhou @ COLM2026@zhuci19·13 Oct 2025

(1/5) Beyond Next-Token Prediction, introducing Next Semantic Scale Prediction! Our @NeurIPSConf NeurIPS 2025 paper HDLM is out! Check out the new language modeling paradigm: Next Semantic Scale Prediction via Hierarchical Diffusion Language Models. It largely generalizes Masked Diffusion Models (MDM), and provides the progressively denoising capability for each token in the semantic level. Minimal computation overheads, much better results! arxiv: arxiv.org/abs/2510.08632 code: github.com/zhouc20/HDLM

Post attachment
Sherrie Wang avatar
⟳ Stephen Bates retweeted
Sherrie Wang@sherwang·8 Sep 2025

Happy to share that our paper on how to obtain reliable statistical inferences from satellite-based maps is now published in Remote Sensing of Environment!

Post attachment
U.S. National Science Foundation avatar
⟳ Stephen Bates retweeted
U.S. National Science Foundation@NSF·13 Jun 2025

Today, NSF announced an add’l 500 NSF Graduate Research Fellowship Program awardees for the 2025-2026 cohort, bringing the total to approx 1,500. #NSFGRFP supports grad students as they pursue their dreams, build STEM skills, & become the next generation of innovators & leaders.

Post attachment