Genomes are long but redundant. Not great for pretraining.
To fix that, we've developed a lossless tokenization strategy that shortens sentence length by >2000x.
For pretraining, we achieve a 0.55 validation CE on All of Us.
Public replication on 1KG linked below.
Can flow language models scale to billions of parameters and handle real generative tasks?
Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B.
Co-led with @ArnavChavan6@UdbhavBamba.
Joint work with @wognsfjq96@hongseu33@AdtRaghunathan@jw9730
Excited to share what we’ve been building with Chaperone!
In our benchmark, Chaperone reduced token usage by 86%. We also went from hypothesis to sequences ready for wet-lab testing in just two weeks—yielding a 1.32 nM cortisol binder in the first experimental round!
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org
Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
Public preview of BioEmu2! @MSFTResearch AI for Science’s new foundation model for biomolecular structure, dynamics and function. Open weights, MIT license.
github.com/microsoft/bioemu2
Movie: BioEmu2-simulated binding of intrinsically disordered peptide PUMA to MCL-1 receptor. 🧵⬇️
This vial contains a new drug called PAC-832, which I recently invented to treat Alzheimer’s disease. It is the world’s first selective GalR1 antagonist.
I designed and synthesized PAC-832 in a chemistry lab I built in my garage. (1/16)
Agree with the oversimplification of AI in DD. "Vertically integrated target-discovery company" is basically @Xaira_Thera . Super hard but the only way for flywheel value generation in this space. Need more bets like Xaira.
AI for Bio is hot again. Given that, I wrote a primer on why this field is so hard. tl;dr it's because the APIs are fuzzier than you might think.
ankitg.me/blog/2026/05/04/fu…
New paper! Presenting Discrete Flow Maps:
paper: arxiv.org/abs/2604.09784
blog: malbergo.me/discrete-flow-ma…
A laughable problem for me these days is that @nmboffi and I share a research brain, and we have had, time and again, a conversation that ends with “ha so I guess we’re writing the same paper.” Soon we will return to just doing it together :). Here we are doing it again with discrete flow maps and flow language models! A complete and thorough paper led by @PPotaptchik@json_yim@adhisarav@peholderrieth. We took a bit of time to post it to ensure we understood a few more things about the stability of the loss functions.
Like @osclsd , @FEijkelboom, and @nmboffi , we think this could be a very helpful paradigm for thinking about fast inference and even better alignment!
Here’s our version of the story, and I hope it makes clear how green field this research direction is — we provide a comprehensive picture of the KL losses you can write from the properties of the flow map, some nice geometric proofs about the mean denoiser and the simplex, and find that at this time, the ESD can actually be the most performant, with some caveats. Excited for everyone to work together and push this class of models to their limit!
We release Diamond Maps💎 unlocking accurate and efficient guidance for diffusion models. Our experiments show that our methods scale incredibly well. Excited to see what people will build with this!
Accurate guidance has been a notoriously hard problem, but in this work, we’re bringing TWO (!) solutions to the table. The recipe for success:
1️⃣ Speed: Use distilled models (flow maps, mean flows, consistency models).
2️⃣ Exploration: Inject stochasticity to properly explore your search space.
Because this fundamentally improves anything using flow matching and diffusion, we see a lot of potential for applications across audio, robotics, molecules, and beyond.
Paper: arxiv.org/abs/2602.05993
Code: github.com/PeterHolderrieth/…
Huge thanks to an amazing team: Douglas Chen, @LucaEyring, @ishin_shah, Giri Anantharaman, @electronickale, @zeynepakata, Tommi Jaakkola, @nmboffi, and @max_simchowitz. It was awesome bringing this to life together!
Love seeing the recent parade of breakthroughs in Diffusion LMs! 🎉 Here is what our lab is bringing to the party: Meet LangFlow 🌊: The first continuous diffusion language model that rivals discrete DLM. LangFlow achieves strong PPL and Gen PPL on LM1B and OpenWebText. It also outperforms the best discrete diffusion models on 3 out of 7 zero-shot transfer benchmarks, and beats autoregressive baselines on 4 out of 7!
Check out our paper below:
📄 Arxiv: arxiv.org/abs/2604.11748📝 Blog: caradryanl.github.io/blog/20…💻 GitHub: github.com/nealchen2003/Lang…
What if AI could invent enzymes that nature hasn’t seen? 👩🔬🧑🔬
Introducing 🪩 DISCO: Diffusion for Sequence-structure CO-design
14 rounds of directed evolution and over a year of wet lab work. That's what it took to engineer an enzyme for selective C(sp³)–H insertion, one of the most challenging transformations in organic chemistry.
DISCO surpasses this with a single plate. No pre-specified catalytic residues, no template, no theozyme, no inverse folding, just joint diffusion over protein sequence and structure.
📝 Blog: disco-design.github.io/
📄 Paper: arxiv.org/abs/2604.05181
💻 Code: github.com/DISCO-design/DISC…
Protenix-v2: A Biomolecular Modeling System for Structure Prediction and Zero-Shot Antibody Design @ai4s_protenix
1. Protenix-v2 achieves massive gains in antibody-antigen structure prediction, with up to 13-point improvements over Protenix-v1 at DockQ >0.23 and comparable gains at the stricter DockQ >0.8 threshold. Most remarkably, its 5-seed performance surpasses previous 1000-seed results, representing a dramatic leap in sampling efficiency.
2. The system demonstrates 100% target-level success rate in zero-shot VHH antibody design across novelty-controlled targets, with BLI-confirmed hit rates ranging from 2% to 48%. The resulting hits show exceptional developability with 100% thermostability pass rate, 98% self-interaction pass rate, and 93% polyreactivity pass rate.
3. On challenging GPCR targets with small and flexible exposed epitopes, Protenix-v2 achieves hit rates of 16%-88% in VHH-Fc format and up to 50% in mAb format, despite testing only 16-30 designs per target. This demonstrates effective sample efficiency on difficult membrane proteins.
4. The model introduces training-free guidance (TFG) variants that significantly improve ligand-related plausibility, reaching 60.46% success rate on recent protein-ligand benchmarks under a revised stricter validity criterion that checks planarity around sp2 centers and non-planarity at sp3 centers.
5. Protenix-v2 successfully designs dual-specific binders against both prototype and Omicron SARS-CoV-2 RBD variants with nanomolar-scale KD, showing potential compensatory mechanisms at the structural level to accommodate sequence differences.
6. The system supports flexible, target-conditioned generation with granular control over CDR loop lengths and integration of predefined frameworks, spanning diverse formats from miniproteins to VHH and full-length antibodies.
💻Code: github.com/bytedance/Proteni…
📜Paper: github.com/bytedance/Proteni…#ProtenixV2#AntibodyDesign#StructurePrediction#AlphaFold3#BiomolecularModeling#DrugDiscovery#GPCR#MachineLearning#ComputationalBiology#ZeroShotDesign
🤯 big update to our flow map language models paper!
we believe this is the future of non-autoregressive text generation.
read about it in the blog: one-step-lm.github.io/blog/
full details in the paper: arxiv.org/abs/2602.16813
we introduce a new class of continuous flow-based language models and distill them into their corresponding flow map for one-step text generation.
we beat all discrete diffusion baselines at ~8x speed!
v2 gives a complete theory of the flow map over discrete data, with three equivalent ways to learn it (semigroup, lagrangian, eulerian). it turns out you can train these with cross-entropy objectives that look very similar to standard discrete diffusion — but without the factorization error that kills discrete methods at few steps.
beyond improving results across the board, we showcase properties that are unique to continuous flows. in particular, inference-time steering and guidance become straightforward. autoguidance brings generative perplexity down to 51.6 on LM1B, while discrete baselines completely collapse at the same guidance scale.
we also show reward-guided generation for steering topic, sentiment, grammaticality, and safety at inference time — and it works even at 1-2 steps with our flow map model. simple, well-understood techniques from continuous flows just work incredibly well in practice for language.
we’re extremely excited about the future of this class of models.
stay tuned for results on scaling, reasoning, and reinforcement learning-based fine-tuning. 🚀
We are also releasing self-contained lecture notes that explain flow matching and diffusion models from scratch. This goes from "zero" to the state-of-the-art in modern Generative AI.
📖 Read the notes here: arxiv.org/abs/2506.02070
Joint work with @EErives40101.
🚀MIT Flow Matching and Diffusion Lecture 2026 Released (diffusion.csail.mit.edu/)!
We just released our new MIT 2026 course on flow matching and diffusion models! We teach the full stack of modern AI image, video, protein generators - theory and practice. We include:
📺 Videos: Step-by-step derivations.
📝 Notes: Mathematically self-contained lecture notes
💻 Coding: Hands-on exercises for every component
We fully improved last years’ iteration and added new topics: latent spaces, diffusion transformers, building language models with discrete diffusion models.
Everything is available here: diffusion.csail.mit.edu/
A huge thanks to Tommi Jaakkola for his support in making this class possible and Ashay Athalye (MIT SOUL) for the incredible production! Was fun to do this with @RShprints!
#MachineLearning#GenerativeAI#MIT#DiffusionModels#AI