Coding agents now write a growing share of ML compiler code. Which language should they write it in: Rust, Python, or C++?
Each language has different strengths, so why pick just one?
Meet TIRx Rust 🚀: a Rust-native, cross-language ML compiler foundation for agents, with:
🔗 Fine-grained access to shared IR objects: Rust natively uses objects from any language, no bindings.
🦀 Ergonomic Rust APIs: write passes natively in Rust, as easily as in C++ or Python.
🧩 Dynamic extension: add new objects and functions as standalone modules, with zero changes to the core.
How does it work under the hood? Full post 👇
blog link: tvm.apache.org/2026/11/16/ru…
Reasoning models can now drive robots. But the robot sits still while the model thinks.
SpecPilot now speeds up agentic robotics. On PEAR (Qwen3.8-27B as a robot policy), the same LIBERO task finishes in 49 s instead of 189 s.
3.84× faster task completion
71 → 241 tok/s decode
No training 🤖
RSI starts with infra 🔁 mlsys is in a golden age. agents are great researchers now. with tokens, "low-hanging fruit" is everywhere.
agents optimize speculative decoding for parallel drafters (DFlash2, DSpark) without training or quality loss. their 12-bit format beats cuBLAS.
Beyond Score Centering: what else can we fix in async RL? 🧵
Score centering (SC) is the hot new trick for off-policy LLM RL. We stress-tested it under real async staleness (every batch up to 127 steps old) and found what it fixes, what it doesn't, and two cheap wins on top
🔍 What SC actually fixes
• Without SC, a trajectory's gradient norm grows ~linearly with its age: at age 64 it's 3.2× an on-policy one. With SC: 1.5×
• The culprit is a hidden drift term, an uncontrolled −A·KL(μ‖π) baked into the policy gradient. SC removes it
⚠️ What SC + IS gets wrong
• SC then IS → adds a new drift
• IS then SC → a no-op without truncation; with truncation it centers on a distribution with no guarantee of being closer to π
• So "IS and SC are orthogonal" (from the SC paper) doesn't hold
Two wins on top of SC
1️⃣ State-level KL mask: drop states where KL(μ‖π) > τ, instead of masking by token-level ratios. It depends only on the state, so SC's baseline property survives. At age 127: 3/3 seeds climb steadily, while TIS+SC and MIS+SC spike and crash
2️⃣ Random buffer sampling, a free lunch from the system: same buffer, sample at random instead of FIFO. Mean age 127 → 52, better for every method at equal wall-clock
Recipe = SC + KL penalty + KL mask. At lr 4e-2 (w/ SGD), MIS+SC and DPPO collapse and TIS+SC swings, while every recipe seed trains and learns faster early
🤔 So far, we’ve only tested on Countdown with Qwen3-0.6B (wants to scale it up!). When staleness is low, all the fixes perform similarly, but the differences become clearer as staleness increases. We also haven’t compared against sequence-level masking (which might already be enough!) or isolated kernel mismatch from async lag. Both are worth looking into next.
#LLM#RL#AsyncRL#ReinforcementLearning
We breathed life into LLM system.
Introducing Evolving Infra⚙️: an inference framework that adapts diverse and dynamic agent workload purely online and retires fixed PD disaggregation.
vs. NVIDIA Dynamo on up to 256 GB300 GPUs:
➡️ Up to 4.1× lower latency
➡️ Up to 14× lower P95 TTFT
➡️Up to 2.36× more revenue
on real agent traces(Computer Use, Coding and math agents) and opensource models.
🌐 Website: inference-rsi.com
📄 Tech report: alphaxiv.org/en/abs/2611.123…
💻 Code: github.com/mit-han-lab/Evolv…
Which agent(e.g. Pi/Openhands) do you want us to accelerate next? Tell us below 👇
Everyone's been showing Fable and GPT-6 Astra driving robot arms.
We trained an open 27B Qwen model to do it, purely from its own attempts in simulation, with zero demonstrations.
And it can run locally at 220+ tok/s on an RTX 5090.
On a real robot it had never touched, it now succeeds in 18 of 30 tests, up from 0 before training. It also copies a painting and designs its own chemistry experiment.
It generalizes and follows instructions better than π0.5, and on LIBERO it comes close to Astra max.
All without losing its general capabilities, like math and coding.
Built on Qwen3.8-27B. We've open-sourced the model weights, code, and full training infra! Check out our blog at mit-han-lab.github.io/pear-b….
So glad to have the opportunity to share our team's recent work of rebuilding a training framework at #SGLangSummit. We'll share blog posts and open source the project very soon. Stay tuned 😜
SyFI is hiring a postdoc or research engineer (~$100K/year) to lead our work on the NSF AI Efficiency Challenge. Build AI infrastructure with us!
Reach out to me or @thepadawang. Please repost or tag a colleague!
🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.
Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.
📈 Across Qwen, Kimi, and GLM models:
• Matches FP32 accuracy
• 4.25× faster linear attention kernels
• Up to 2× higher inference throughput
• Cuts memory by up to 56%
For Linear Attention models, the recurrent state itself becomes the next bottleneck.
LeapQuant makes 8/4-bit linear states work in a training-free way, incurring no accuracy degradation, and improves throughput by 2×🚀
🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.
Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.
📈 Across Qwen, Kimi, and GLM models:
• Matches FP32 accuracy
• 4.25× faster linear attention kernels
• Up to 2× higher inference throughput
• Cuts memory by up to 56%
Great performance improvements for attention with almost no downside: matching the FP32 baseline using 8-bit or even 4-bit quantization! Check out our latest work.
🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.
Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.
📈 Across Qwen, Kimi, and GLM models:
• Matches FP32 accuracy
• 4.25× faster linear attention kernels
• Up to 2× higher inference throughput
• Cuts memory by up to 56%
Fix #3: Residual smoothing 📉
Balance the residual's row magnitudes before quantizing; scales are fixed within a window and refreshed at each boundary.
In one GDN head, outliers shrink from 176× the median to just 5.3×.
(5/7)
Fix #2: Compensator Tokens 🎯
We pull the largest outliers out of the state as a few rank-one outer products, kept in high precision and folded into the window's update path.
What's left is a residual that's far easier to quantize.
(4/7)