←
Yi Pan106 posts
Yi Pan avatar

Yi Pan

@conlesspan

PhD Student @BerkeleySky | Systems & AI | conless.dev Prev. BS @sjtu1896, RA @UWSyFI

◷ Joined August 2019▧ 8 Photos and videos
Linzhang Li avatar
⟳ Yi Pan retweeted
Linzhang Li@Yuchuan2025·Nov 16

Coding agents now write a growing share of ML compiler code. Which language should they write it in: Rust, Python, or C++? Each language has different strengths, so why pick just one? Meet TIRx Rust 🚀: a Rust-native, cross-language ML compiler foundation for agents, with: 🔗 Fine-grained access to shared IR objects: Rust natively uses objects from any language, no bindings. 🦀 Ergonomic Rust APIs: write passes natively in Rust, as easily as in C++ or Python. 🧩 Dynamic extension: add new objects and functions as standalone modules, with zero changes to the core. How does it work under the hood? Full post 👇 blog link: tvm.apache.org/2026/11/16/ru…

Post attachment
Kunjun Li avatar
⟳ Yi Pan retweeted
Kunjun Li@li_kunjun·Nov 16

Reasoning models can now drive robots. But the robot sits still while the model thinks. SpecPilot now speeds up agentic robotics. On PEAR (Qwen3.8-27B as a robot policy), the same LIBERO task finishes in 49 s instead of 189 s. 3.84× faster task completion 71 → 241 tok/s decode No training 🤖

0:53

RSI starts with infra 🔁 mlsys is in a golden age. agents are great researchers now. with tokens, "low-hanging fruit" is everywhere. agents optimize speculative decoding for parallel drafters (DFlash2, DSpark) without training or quality loss. their 12-bit format beats cuBLAS.

Post attachment
Yichuan Wang avatar
⟳ Yi Pan retweeted
Yichuan Wang@YichuanM·Nov 16

Beyond Score Centering: what else can we fix in async RL? 🧵 Score centering (SC) is the hot new trick for off-policy LLM RL. We stress-tested it under real async staleness (every batch up to 127 steps old) and found what it fixes, what it doesn't, and two cheap wins on top 🔍 What SC actually fixes • Without SC, a trajectory's gradient norm grows ~linearly with its age: at age 64 it's 3.2× an on-policy one. With SC: 1.5× • The culprit is a hidden drift term, an uncontrolled −A·KL(μ‖π) baked into the policy gradient. SC removes it ⚠️ What SC + IS gets wrong • SC then IS → adds a new drift • IS then SC → a no-op without truncation; with truncation it centers on a distribution with no guarantee of being closer to π • So "IS and SC are orthogonal" (from the SC paper) doesn't hold Two wins on top of SC 1️⃣ State-level KL mask: drop states where KL(μ‖π) > τ, instead of masking by token-level ratios. It depends only on the state, so SC's baseline property survives. At age 127: 3/3 seeds climb steadily, while TIS+SC and MIS+SC spike and crash 2️⃣ Random buffer sampling, a free lunch from the system: same buffer, sample at random instead of FIFO. Mean age 127 → 52, better for every method at equal wall-clock Recipe = SC + KL penalty + KL mask. At lr 4e-2 (w/ SGD), MIS+SC and DPPO collapse and TIS+SC swings, while every recipe seed trains and learns faster early 🤔 So far, we’ve only tested on Countdown with Qwen3-0.6B (wants to scale it up!). When staleness is low, all the fixes perform similarly, but the differences become clearer as staleness increases. We also haven’t compared against sequence-level masking (which might already be enough!) or isolated kernel mismatch from async lag. Both are worth looking into next. #LLM #RL #AsyncRL #ReinforcementLearning

Hao Kang ✈️ Neurips avatar
⟳ Yi Pan retweeted
Hao Kang ✈️ Neurips@Mlsys_HaoKang·Nov 16

We breathed life into LLM system. Introducing Evolving Infra⚙️: an inference framework that adapts diverse and dynamic agent workload purely online and retires fixed PD disaggregation. vs. NVIDIA Dynamo on up to 256 GB300 GPUs: ➡️ Up to 4.1× lower latency ➡️ Up to 14× lower P95 TTFT ➡️Up to 2.36× more revenue on real agent traces(Computer Use, Coding and math agents) and opensource models. 🌐 Website: inference-rsi.com 📄 Tech report: alphaxiv.org/en/abs/2611.123… 💻 Code: github.com/mit-han-lab/Evolv… Which agent(e.g. Pi/Openhands) do you want us to accelerate next? Tell us below 👇

Post attachment
2:05
Jiaming Tang avatar
⟳ Yi Pan retweeted
Jiaming Tang@jmtang42·Nov 13

Everyone's been showing Fable and GPT-6 Astra driving robot arms. We trained an open 27B Qwen model to do it, purely from its own attempts in simulation, with zero demonstrations. And it can run locally at 220+ tok/s on an RTX 5090. On a real robot it had never touched, it now succeeds in 18 of 30 tests, up from 0 before training. It also copies a painting and designs its own chemistry experiment. It generalizes and follows instructions better than π0.5, and on LIBERO it comes close to Astra max. All without losing its general capabilities, like math and coding. Built on Qwen3.8-27B. We've open-sourced the model weights, code, and full training infra! Check out our blog at mit-han-lab.github.io/pear-b….

0:31
Lequn Chen avatar
⟳ Yi Pan retweeted
Lequn Chen@abcdabcd987·Nov 14

So glad to have the opportunity to share our team's recent work of rebuilding a training framework at #SGLangSummit. We'll share blog posts and open source the project very soon. Stay tuned 😜

Post attachment
Baris Kasikci avatar
⟳ Yi Pan retweeted
Baris Kasikci@bariskasikci·Nov 12

SyFI is hiring a postdoc or research engineer (~$100K/year) to lead our work on the NSF AI Efficiency Challenge. Build AI infrastructure with us! Reach out to me or @thepadawang. Please repost or tag a colleague!

Mayank Mishra avatar
⟳ Yi Pan retweeted
Mayank Mishra@MayankMish98·Nov 11

Checkout LeapQuant from @conlesspan and @HaochengXiUCB! Training free quantization for recurrent states of linear attention 🚀🚀🚀

🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token. Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work. 📈 Across Qwen, Kimi, and GLM models: • Matches FP32 accuracy • 4.25× faster linear attention kernels • Up to 2× higher inference throughput • Cuts memory by up to 56%

0:34
Haocheng Xi avatar
⟳ Yi Pan retweeted
Haocheng Xi@HaochengXiUCB·Nov 11

For Linear Attention models, the recurrent state itself becomes the next bottleneck. LeapQuant makes 8/4-bit linear states work in a training-free way, incurring no accuracy degradation, and improves throughput by 2×🚀

🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token. Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work. 📈 Across Qwen, Kimi, and GLM models: • Matches FP32 accuracy • 4.25× faster linear attention kernels • Up to 2× higher inference throughput • Cuts memory by up to 56%

0:34
Xingyang Li avatar
⟳ Yi Pan retweeted
Xingyang Li@XYLi_Bruce·Nov 11

Amazing work! So fortunate to have watched this grow from a neat observation into a full paper. Congrats to the whole team!

Amazing collaboration with @HaochengXiUCB @KanZhu854772 @XYLi_Bruce @BetaGamma20 @MayankMish98 @zhangho72245693 @WilliamXinze @songhan_mit @KurtKeutzer @rishiyer @istoica05 ! Huge thanks to @shulynnliu @yhfchitanda @YichuanM @melissapan for insightful discussions and helpful feedback! Feel free to check out the paper, code, and demo. We'd love to hear your feedback! 🙌 (7/7)

Ion Stoica avatar
⟳ Yi Pan retweeted
Ion Stoica@istoica05·Nov 11

Great performance improvements for attention with almost no downside: matching the FP32 baseline using 8-bit or even 4-bit quantization! Check out our latest work.

🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token. Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work. 📈 Across Qwen, Kimi, and GLM models: • Matches FP32 accuracy • 4.25× faster linear attention kernels • Up to 2× higher inference throughput • Cuts memory by up to 56%

0:34
Yi Pan avatar
Yi Pan@conlesspan·Nov 11

📈 Across Qwen, Kimi, and GLM models, LeapQuant matches FP32 accuracy, speeds up linear attention kernels by up to 4.25×, boosts inference throughput by up to 2×, and cuts memory by up to 56%. 📄 arxiv.org/abs/2609.38166 💻 github.com/leapquant/leapqua… 🌐 leapquant.github.io (6/7)

Yi Pan avatar
Yi Pan@conlesspan·Nov 11

Fix #3: Residual smoothing 📉 Balance the residual's row magnitudes before quantizing; scales are fixed within a window and refreshed at each boundary. In one GDN head, outliers shrink from 176× the median to just 5.3×. (5/7)

Post attachment
Yi Pan avatar
Yi Pan@conlesspan·Nov 11

Fix #2: Compensator Tokens 🎯 We pull the largest outliers out of the state as a few rank-one outer products, kept in high precision and folded into the window's update path. What's left is a residual that's far easier to quantize. (4/7)

Post attachment