←
Discrete Diffusion Reading Group307 posts
Discrete Diffusion Reading Group avatar

Discrete Diffusion Reading Group

@diffusion_llms

📚 Journal club on discrete diffusion models 🎥 Replays available on YouTube! Contact: diffusion.llms@gmail.com Hosted by @ssahoo_, @jdeschena, @zhihanyang_

◷ Joined August 2025▧ 90 Photos and videos
Justin Deschenaux avatar
⟳ Discrete Diffusion Reading Group retweeted
Justin Deschenaux@jdeschena·Nov 17

Missed the talk? Check out the recording on YouTube: youtube.com/watch?v=Hb5mCdy0…

This Monday, Nov 16, Hermann Kumbong (@KumbongHermann) and Ishan S. Khare (@ishanskhare ) will present their paper: Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

Post attachment
Discrete Diffusion Reading Group avatar
Discrete Diffusion Reading Group@diffusion_llms·Nov 17

Missed the talk? Check out the recording on YouTube: youtube.com/watch?v=Hb5mCdy0…

This Monday, Nov 16, Hermann Kumbong (@KumbongHermann) and Ishan S. Khare (@ishanskhare ) will present their paper: Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

Post attachment
Morteza Mardani avatar
⟳ Discrete Diffusion Reading Group retweeted
Morteza Mardani@MardaniMorteza·Nov 16

📣 NVIDIA is releasing a collection of open model weights: Nemotron-Labs-Sigma — 14 checkpoints at 3B/8B. Language models have stayed discrete. We bring continuous diffusion to language as in vision. Smooth ODE trajectories (16-D) unlock distillation and steering: 8 TPFs for math, 4 TPFs for code, with accuracy and diversity preserved. 🧵 1/

Post attachment
Wei Guo avatar
⟳ Discrete Diffusion Reading Group retweeted
Wei Guo@WeiGuo01·Nov 16

We have done a major update on distillation and speculative decoding, and release our Nemotron-Labs-Sigma model family on Huggingface. Check it out at huggingface.co/collections/n… Kudos to all collaborators and looking forward to the future of continuous diffusion LLMs!

📣 NVIDIA is releasing a collection of open model weights: Nemotron-Labs-Sigma — 14 checkpoints at 3B/8B. Language models have stayed discrete. We bring continuous diffusion to language as in vision. Smooth ODE trajectories (16-D) unlock distillation and steering: 8 TPFs for math, 4 TPFs for code, with accuracy and diversity preserved. 🧵 1/

Post attachment
Zhihan Yang avatar
⟳ Discrete Diffusion Reading Group retweeted
Zhihan Yang@zhihanyang_·Nov 16

We've released the 3B/8B checkpoints for Sigma on HuggingFace 🤗! huggingface.co/collections/n…

💫 Large Language Continuous Diffusion Models Presenting Sigma, the first large-scale (3B/8B) continuous diffusion LM (dLM) trained with likelihood. 📈 Comparable with discrete dLMs and AR on math and coding 🎛️ Enables meaningful study of inference-time interventions Jointly led with Wei Guo (@WeiGuo01), Jean-Marie Lemercier (@jm_lemercier), Simon Welker (@_cobalamin) Team: @YongganFu @mmkamani7 Sajad Norouzi @julberner @tomasgeffner @karsten_kreis @YongxinChen1 @MoleiTaoMath @jwthickstun @PavloMolchanov Ante Jukic @ArashVahdat @MardaniMorteza

0:33
Discrete Diffusion Reading Group avatar
Discrete Diffusion Reading Group@diffusion_llms·Nov 11

📅 Friday Nov 13 ⏲️ 10-11am PT Zoom: luma.com/fli4wjlr

Giving a talk on Diffusion augmented LLMs (Uno) @askalphaxiv Some new results include: > A newer version of Uno which beats DFlash2 > Scaling to 400B scale > Speeding up RL post-training 📅 Friday Nov 13 ⏲️ 10-11am PT Zoom: luma.com/fli4wjlr

Subham Sahoo avatar
⟳ Discrete Diffusion Reading Group retweeted
Subham Sahoo@ssahoo_·Nov 11

Giving a talk on Diffusion augmented LLMs (Uno) @askalphaxiv Some new results include: > A newer version of Uno which beats DFlash2 > Scaling to 400B scale > Speeding up RL post-training 📅 Friday Nov 13 ⏲️ 10-11am PT Zoom: luma.com/fli4wjlr

Making LLM inference up to 3x faster using discrete diffusion without sacrificing accuracy? Come join us for this AI4Science talk: Unlocking Lossless Speedups in LLMs via Discrete Diffusion. In this session, the author Subham Sahoo will introduce Uno, a new approach from IFM that combines autoregressive and diffusion modeling to accelerate LLM inference. While parallel generation methods often sacrifice accuracy for speed, Uno uses lightweight diffusion adapters to generate multiple tokens in parallel, then relies on the original model to verify them, preserving its output distribution without requiring a separate draft model. So not only is it 3x faster, it also outperforms top speculative decoding methods, with speedups even works at large batch sizes. If you're working or interested in LLM inferencing, this is the talk to check out. :microscope: AI4Science on alphaXiv :calendar_spiral: Friday November 13th 2026 · 10 AM PT :microphone2: Featuring Subham Sahoo :speech_balloon: Casual Talk + Open Discussion Check out the paper here: alphaxiv.org/abs/2609.04010 Sign up for the talk here: luma.com/fli4wjlr

Post attachment
Subham Sahoo avatar
⟳ Discrete Diffusion Reading Group retweeted
Subham Sahoo@ssahoo_·Nov 10

Great blog post by @JonathanGeuter and @KempnerInst on our recent Uno paper. Autoregressive LLMs + Diffusion adapters >>> Training Diffusion LLMs from scratch or Speculative Decoding with small drafters Stay tuned for a follow-up to the Uno paper with comparisons against DFlash2 and DSpark. kempnerinstitute.harvard.edu…

Post attachment

Diffusion LLMs have two limitations relative to AR models: (1) Lower quality, and (2) Slower inference at large batch sizes. We address this "Uno" > Retains the AR architecture of LLMs > Each layer has two sets of weights: AR weights and Diffusion weights > Diffusion weights enable parallel sampling from the AR distribution losslessly Results: 💥Faster than all speculative decoding methods: DFlash and EAGLE-3 🔥 Beats ALL diffusion LLMs: Mercury 2, Diffusion Gemma, Llada Links to the paper, models, code below

Post attachment