Continuous diffusion dominates image & video generation, but people used to believe that it inherently lags behind its discrete counterparts in language modeling.
Today, we challenge this belief with LangFlow: the first continuous diffusion language model that rivals—and even beats—discrete diffusion. (1/7)
Blog: caradryanl.github.io/blog/20…
GitHub: github.com/nealchen2003/Lang…
Arxiv: arxiv.org/abs/2604.11748
Heading to SF for #COLM2026, Oct 7–10!
I work on forecasting with LLMs and RL for language agents — in particular, how models can predict the future and leverage such predictions for better RL algorithms for long-horizon decision-making.
I’m looking for internship and full-time research opportunities around:
• RL for language agents
• Forecasting and simulation with LLMs
If your team is working on these problems (or hiring!), I’d love to chat in SF.
I’ll also have two posters at COLM. We swapped our original poster slots, so please note the updated schedule:
📍 SocialVeil: Probing Social Intelligence of Language Agents under Communication Barriers
Oct 7, 11:00 AM–1:00 PM · Imperial Ballroom · Poster #62
arxiv.org/abs/2602.05115
📍 ArtifactLinker: Linking Scientific Artifacts for Automatic State-of-the-Art Discovery
Oct 8, 11:00 AM–1:00 PM · Grand Ballroom · Poster #140
arxiv.org/abs/2605.16902
Come by or DM me if you’d like to chat!
A language is only as useful as the community speak it!
"Beyond the code, we have a community goal. We want to build the language of statistics in Lean together with statisticians and lean experts: shared design decisions, open discussion of architecture"
AI evaluation is entering an interactive benchmark era.
Across tool-use agents, web/OS benchmarks, multi-agent systems, and reliability evaluations, interaction is becoming central to how modern AI systems are tested.
But the field risks adding interaction faster than it develops the scientific principles for evaluating interaction.
Our position:
Interactive evaluation is not just longer tasks, tool use, or multi-turn interaction.
It requires a design science for mapping trajectories to valid evaluative claims.
📄 arxiv.org/abs/2605.17829
💻 github.com/keyangds/interact…
👋Excited to share our new work:
Taming Outlier Tokens in Diffusion Transformers
We found that outlier tokens are not only a ViT recognition problem — they also appear in modern diffusion pipelines and hurt generation quality.
📜Project-page: a-little-hoof.github.io/dit-…
The potential of continuous DLMs extends far beyond just performance. They open the door for all continuous diffusion techniques to be introduced into language modeling:
- One-step generation, such as Consistency Models
- Guided generation, such as CFG
- Unified multimodal generation, such as protein structure-sequence co-design
LangFlow suggests: continuous diffusion is NOW a viable and promising paradigm for language modeling. (7/7)
Using ODE sampling, LangFlow can naturally derive a novel ODE-based likelihood bound from Flow Matching to evaluate the likelihood of language data, which empowers LangFlow to be compared with discrete counterparts by perplexity, the core language modeling metric. (6/7)
Another new finding is about self-conditioning. We find that it significantly improves likelihood and sample quality, with effects substantially different from discrete diffusion. Disabling it during comparison with discrete DLMs is unfair. The protocol of training and evaluating continuous DLMs should be rectified! (5/7)
However, training such continuous diffusion on language has long been a struggle. We thereby reveal that the noise schedule is the key, and that it should make the information gain per unit time uniform. Under this principle, the optimal noise scheduler for language is a Gumbel distribution, greatly different from that for images. (4/7)
LangFlow connects embedding-space DLMs to Flow Matching. The model predicts clean token probabilities from noisy token embeddings, and then derives the embedding-space flow in a closed form. Endorsed by Bregman divergence, we train the model via the cross-entropy loss. (3/7)
LangFlow shows for the first time that continuous diffusion can rival discrete counterparts in language modeling:
On LM1B and OpenWebText, both our perplexity (PPL) and generative perplexity (Gen. PPL) match or surpass the best discrete diffusion models.
On zero-shot transfer, we outperform the best discrete diffusion on 3 out of 7 and autoregressive baselines on 4 out of 7 benchmarks. (2/7)
🚨 RL for LLMs is finally accessible.
Introducing OpenTinker: The first community-driven, open-source framework designed to democratize Reinforcement Learning for LLMs.
github.com/open-tinker/OpenT…
Inspired by @thinkymachines's amazing Tinker, we realize the biggest bottleneck in agentic LLM research isn’t the math—it’s the setup. Current RL pipelines are messy. Configuring VeRL for every single experiment is a productivity killer.
OpenTinker fixed it.
🛠 How OpenTinker Works: Decoupled Design of Server and Client
- Setup Once, Run Forever: Configure the OpenTinker backend on your GPU cluster once.
- Develop Locally: Define your RL environments directly on your laptop.
- Train on the Cloud: Simply point your local client to the backend. The cluster handles the compute; you handle the science.
📉 The 10x Development Efficiency
Thanks to our elegant architectural decomposition, OpenTinker reduces the time to develop a new RL training pipeline by at least an order of magnitude.
⚡ Turn Idle GPU Compute into Gold
Small labs often have underutilized hardware. OpenTinker turns your idle GPUs into an internal/external API service for
- RL Training
- SFT
- Inference
🎯 Who needs OpenTinker?
- Researchers tired of infrastructure hell.
- Labs needing to standardize workflows.
- Teams wanting to maximize hardware ROI.
Thanks my amazing PhD student @realagi25 for leading the project. We are building the future of open RL infra. Be the first to build with us.
👇 Start Building with OpenTinker Now
🚀 Repo: github.com/open-tinker/OpenT…
🌐 Blog: open-tinker.github.io/openti…
If you believe RL should be accessible to everyone, give us a star, repost this 🔄 post, and let us know what agents you plan to build!
(1/n) Tiny-A2D: An Open Recipe to Turn Any AR LM into a Diffusion LM
Code (dLLM): github.com/ZHZisZZ/dllm
Checkpoints: huggingface.co/collections/d…
With dLLM, you can turn ANY autoregressive LM into a diffusion LM (parallel generation + infilling) with minimal compute. Using this recipe, we built a 🤗collection of the smallest diffusion LMs that work well in practice.
Key takeaways:
1. Finetuned on Qwen3-0.6B, we obtain the strongest small (~0.5/0.6B) diffusion LMs to date.
2. The base AR LM matters: Investing compute in improving the base AR model is potentially more efficient than scaling compute during adaptation.
3. Block diffusion (BD3LM) generally outperforms vanilla masked diffusion (MDLM), especially on math-reasoning and coding tasks.
Multi-Agent Evolve is now fully open-source 🚀
With our codebase, you can pick your favorite LLM checkpoint and let it self-evolve, WITHOUT external supervision
💻Code:
github.com/ulab-uiuc/Multi-a…
🤗Model Checkpoints:
huggingface.co/collections/u…
Feedback and contributions are welcome!
Introducing Multi-Agent Evolve 🧠
A new paradigm beyond RLHF and RLVR:
More compute → closer to AGI
No need for expensive data or handcrafted rewards
We show that an LLM can self-evolve — improving itself through co-evolution among roles (Proposer, Solver, Judge) via RL — all without external supervision.
On Qwen2.5-3B-Instruct, Multi-Agent Evolve boosts average accuracy from 55% → 58% across 22 benchmarks.
Remarkably, the model automatically learns balance among roles:
- The Proposer first generates easy tasks.
- The Judge refines the difficulty metric.
- The Proposer then raises the challenge, forcing the Solver to improve.
- The system co-evolves until reaching equilibrium.
Multi-Agent Evolve: LLMs self-improve through co-evolution.
📄 Paper: arxiv.org/abs/2510.23595
💻 Code (coming soon): github.com/ulab-uiuc/Multi-a…