
6/ In the future, we envision models that use fully observed data from simulators as priors before leveraging numerous “real-world” experimental observables, spanning length and timescales, to accurately model the real world.

Assistant Professor at UC Berkeley

6/ In the future, we envision models that use fully observed data from simulators as priors before leveraging numerous “real-world” experimental observables, spanning length and timescales, to accurately model the real world.

5/ Prior works often consider only expectation alignment (EA), matching only the means of observables of interest. But matching finitely many expectations does not recover the correct distribution. ADA instead aligns to the full distribution of experimental outcomes. Even in a simple Gaussian mixture setting, EA fails to correct the energy landscape, while ADA successfully captures the true energy landscape.

4/ ADA can handle high-dimensional, noisy observables. In this example, we start from a classical force field (inducing a Boltzmann distribution) and shift the distribution toward experimentally observed protein structures (from the PDB)—using cryo-EM image observables Despite observing only indirect, noisy images, ADA recovers meaningful structural corrections in the underlying distribution

3/ We satisfy the observable constraint using Wasserstein critics and apply an entropic regularizer to stay close to the simulator reference. This provides a principled framework for matching the distribution of partial observables in the absence of data from the full joint distribution Note that this is different from conditional generative modeling, as this framework can naturally handle multiple observables!

2/ To use both sources of data, we frame experimental observable alignment as a distributional constraint. If a model is well aligned, it should match the full distribution of real-world observables

1/ A grand challenge in science: the simulation-to-experiment gap Simulators model complex systems but introduce model errors; experiments give real-world data but are costly and noisy Can we leverage both? Yes! We introduce Adversarial Distribution Alignment (ADA), at #NeurIPS2026 ADA aligns generative models—initially trained on simulations (e.g., molecular dynamics with classical force fields)—to full real-world distributions (e.g., protein structures) using only partial, noisy observations (like cryo-EM images) ADA goes beyond just matching the expectation of observables: it can match full distributions, even in the presence of multiple correlated, high-dimensional observations Check out our website: kaityrusnelson.com/ada/ And our paper: arxiv.org/abs/2510.02259 In collaboration with @KaiTyrusNelson, @tobykreiman, and @svlevine

Code for our new world model planner is live! github.com/michael-psenka/gr… Includes our implementation on dino-wm, as well as implementations on jepa-wm and le-wm, and minimal pseudocode for anyone to re-implement themselves.
tl;dr New planner for world models! GRASP: gradient-based, stochastic, parallelized. Long range planning for world models has always been an issue. 0th order methods like CEM/MPPI dominate, but have degrading performance at longer contexts or higher-dimensional actions. We wanted to address this from the ground up. w/ Michael Rabbat, @ask1729 , @ylecun*, @_amirbar* (equally advised)

3/ This project was lead by Yiheng Du, with: Paper: openreview.net/forum?id=bla5… Code: github.com/ASK-Berkeley/Eddy…

2/ EddyFormer can resolve solutions to fluid dynamics problems that other ML models fail to converge on (taken from the Well). It can also generalize to larger spatiotemporal domains than what it was trained on.

1/ How can we improve our models of the physical world? We develop EddyFormer: integrating spectral methods w/ the Transformer architecture. We can accelerate 3D turbulence simulation by up to 30x compared to top numerical solvers, at the same level of accuracy! At #NeurIPS2025 on Dec 4 at 4:30 PM, Exhibit Hall C, D, E: #2316 Project page: mrlazy1708.github.io/eddyfor…

(Inspired by ongoing thoughts about the state of the field, including discussions w/ @Andrew_S_Rosen, @SamMBlau, @bwood_m + many others)

At this point the AI for Science community should stop focusing on achieving "state-of-the-art” on datasets like QM9 & MD17: chasing small improvements on these outdated datasets is scientifically meaningless. It's like telling vision researchers to ditch internet-scale and go back to benchmarking on MNIST/CIFAR10

I value approaches that work by subtraction: stripping away the unnecessary until the essential insight remains. Doing less often demands more: a deeper understanding of data and method to reveal the simplest formulation at the core. This is an attempt of ours. Hope it resonates.
1/ Can molecular AI move past hard-coded Graph Neural Networks and embrace scalable Transformers that discover molecular structure on their own? We demonstrate that you can train a 1B parameter Transformer model without any graph priors or physical inductive biases. And surprisingly, not only can it maintain competitive performance under equal compute on the Open Molecules 2025 dataset… it’s faster than a 6M parameter equivariant GNN, and exhibits scaling laws that don’t saturate. We use this as a starting point to investigate emergent internal representations, and find that it adaptively discovers molecular structure! Check out the interactive demo on our website: tkreiman.github.io/projects/… And our paper: arxiv.org/abs/2510.02259 In collaboration with @tobykreiman, @YutongBAI1002, Fadi, Elizabeth, and @EricQuCal. Here’s a video showing how the Transformer learns distance-aware attention patterns (purple gradient) that adapt to atomic environments 👇

6/ Our results demonstrate that many favorable properties of GNNs can emerge adaptively and more flexibly in Transformers, challenging the necessity of hard-coded graph inductive biases and pointing toward standardized, scalable architectures for molecular modeling. This has been a hot topic in the community, and we hope that this adds more to the discussion!

5/ We really do mean an unmodified Transformer: no explicit calculation of pairwise distances, no graph-based features, no rotational equivariance, etc. Leveraging modern software and hardware, a 1B parameter Transformer trains and runs inference faster than a 6M parameter equivariant GNN.

4/ The model learns attention scores that decay inversely with interatomic distance in early layers, and in later layers learns attention that remains roughly constant with distance. This suggests that the Transformer naturally first learns local feature extraction before performing global aggregation in later layers. The more isolated an atom is, the farther out attention adaptively expands.