←
Berkeley AI Research1,760 posts
Berkeley AI Research avatar

Berkeley AI Research

@berkeley_ai

We're graduate students, postdocs, faculty and scientists at the cutting edge of artificial intelligence research.

◷ Joined July 2017▧ 48 Photos and videos
Yichuan Wang avatar
⟳ Berkeley AI Research retweeted
Yichuan Wang@YichuanM·Nov 16

Beyond Score Centering: what else can we fix in async RL? 🧵 Score centering (SC) is the hot new trick for off-policy LLM RL. We stress-tested it under real async staleness (every batch up to 127 steps old) and found what it fixes, what it doesn't, and two cheap wins on top 🔍 What SC actually fixes • Without SC, a trajectory's gradient norm grows ~linearly with its age: at age 64 it's 3.2× an on-policy one. With SC: 1.5× • The culprit is a hidden drift term, an uncontrolled −A·KL(μ‖π) baked into the policy gradient. SC removes it ⚠️ What SC + IS gets wrong • SC then IS → adds a new drift • IS then SC → a no-op without truncation; with truncation it centers on a distribution with no guarantee of being closer to π • So "IS and SC are orthogonal" (from the SC paper) doesn't hold Two wins on top of SC 1️⃣ State-level KL mask: drop states where KL(μ‖π) > τ, instead of masking by token-level ratios. It depends only on the state, so SC's baseline property survives. At age 127: 3/3 seeds climb steadily, while TIS+SC and MIS+SC spike and crash 2️⃣ Random buffer sampling, a free lunch from the system: same buffer, sample at random instead of FIFO. Mean age 127 → 52, better for every method at equal wall-clock Recipe = SC + KL penalty + KL mask. At lr 4e-2 (w/ SGD), MIS+SC and DPPO collapse and TIS+SC swings, while every recipe seed trains and learns faster early 🤔 So far, we’ve only tested on Countdown with Qwen3-0.6B (wants to scale it up!). When staleness is low, all the fixes perform similarly, but the differences become clearer as staleness increases. We also haven’t compared against sequence-level masking (which might already be enough!) or isolated kernel mismatch from async lag. Both are worth looking into next. #LLM #RL #AsyncRL #ReinforcementLearning

Jitendra MALIK avatar
⟳ Berkeley AI Research retweeted
Jitendra MALIK@JitendraMalikCV·Nov 14

Sometime ago I had proposed an antidote to the hype in robotics demo tapes released on X - a live challenge where a child competes with a robot to finish a task, and we measure performance by speed of completion, weighted by success. I would like to congratulate @roeiherzig and collaborators for making this happen at CoRL 2026. Two robot policies competed and as expected they were trounced by the child. But this is fine! Progress will come and one day robots will exceed humans across the board. In computer vision the famous PASCAL object detection challenge started in 2005 with performance around 5%, and look where we are now.

“Stuck” in sunny San Diego due to the FAA, so I can finally wrap my head around what an amazing #CoRL2026 it was! Thank you @corl_conf and everyone who made this year’s conference so special. 🧒🤖 We ran our LIVE Child–Robot Challenge for the first time at the conference, and the room was buzzing with curiosity! Huge thanks to @JitendraMalikCV @DJiafei @micoolcho @_amirbar @ZoharRimon @YiruHelenWang James Ni, Frank Hu, @k7agar @hq_fang & @SharpaRobotics for bringing it to life, and to our youngest competitor (my little Adam), the true star of the day. 🎤 And thank you Georgia Chalvatzaki @GeorgiaChal, Jeannette Bohg @leto__jean, Yunzhu Li @YunzhuLiYZ & Mike Rabbat @michaelrabbat for brilliant, inspiring talks. Watch the challenge 👇 spin-workshop.github.io

0:51
Roei Herzig avatar
⟳ Berkeley AI Research retweeted
Roei Herzig@roeiherzig·Nov 14

“Stuck” in sunny San Diego due to the FAA, so I can finally wrap my head around what an amazing #CoRL2026 it was! Thank you @corl_conf and everyone who made this year’s conference so special. 🧒🤖 We ran our LIVE Child–Robot Challenge for the first time at the conference, and the room was buzzing with curiosity! Huge thanks to @JitendraMalikCV @DJiafei @micoolcho @_amirbar @ZoharRimon @YiruHelenWang James Ni, Frank Hu, @k7agar @hq_fang & @SharpaRobotics for bringing it to life, and to our youngest competitor (my little Adam), the true star of the day. 🎤 And thank you Georgia Chalvatzaki @GeorgiaChal, Jeannette Bohg @leto__jean, Yunzhu Li @YunzhuLiYZ & Mike Rabbat @michaelrabbat for brilliant, inspiring talks. Watch the challenge 👇 spin-workshop.github.io

0:51
Berkeley AI Research avatar
Berkeley AI Research@berkeley_ai·Nov 12

Congratulations to BAIR Researchers from Roberto Horowitz, @pabbeel and @KoushilSreenath labs whose paper "SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model," was awarded the Best Conference Paper at @corl_conf yesterday in Austin! #CoRL2026 corl.org/program/awards

SuperMimic has won the CORL’s best paper award. Thanks for everyone’s support!

Post attachment
Jivat Kaur avatar
⟳ Berkeley AI Research retweeted
Jivat Kaur@jivatneet·Nov 12

Can machine learning help when algorithms allocating billions of dollars in healthcare spending are gamed? We introduce strategic feature selection, a principled, practical framework for mitigating the effects of strategic behavior in algorithmic decision-making systems.

0:06
Alex Dimakis avatar
⟳ Berkeley AI Research retweeted
Alex Dimakis@AlexGDimakis·Nov 12

You can watermark text generated from an LLM. If, however, you want to detect if your LLM is being distilled, these watermarks can be easily defeated with two simple laundering methods we call grandchild and entropy laundering. We propose Subliminal Watermarking, a new method to detect if watermarked data has been used for distillation that remains effective even after laundering.

Excited to present our new work on distillation detection: Subliminal Watermarks (SubWM). SubWM is much more robust compared to existing distillation detectors, is able to distinguish different watermarking keys, and also works reliably for out-of-domain prompts! Blog: subliminal-watermarks.github… 🧵

0:24
Ion Stoica avatar
⟳ Berkeley AI Research retweeted
Ion Stoica@istoica05·Nov 11

Great performance improvements for attention with almost no downside: matching the FP32 baseline using 8-bit or even 4-bit quantization! Check out our latest work.

🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token. Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work. 📈 Across Qwen, Kimi, and GLM models: • Matches FP32 accuracy • 4.25× faster linear attention kernels • Up to 2× higher inference throughput • Cuts memory by up to 56%

0:34
Yi Pan avatar
⟳ Berkeley AI Research retweeted
Yi Pan@conlesspan·Nov 11

🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token. Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work. 📈 Across Qwen, Kimi, and GLM models: • Matches FP32 accuracy • 4.25× faster linear attention kernels • Up to 2× higher inference throughput • Cuts memory by up to 56%

0:34
Dawn Song avatar
⟳ Berkeley AI Research retweeted
Dawn Song@dawnsongtweets·Nov 11

🚨 A security patch protects every system that installs it, but it can also give attackers a clue to the vulnerability, putting every unpatched system at risk. That's the 1-day window. Turning a patch into a working exploit used to take expert reverse engineers days or weeks. Can AI agents now do it from compiled binaries alone, no source code? Introducing CyberBinGym 🧵👇

Post attachment
Akshat Gupta avatar
⟳ Berkeley AI Research retweeted
Akshat Gupta@akshatgupta57·Nov 12

Inference compute is the largest untapped training resource in AI. Can we use this ever-increasing resource to train models while they are being served? TL;DR: Yes! We call it Population Serving: a new inference paradigm where, instead of serving everyone the exact same weights, we serve slightly perturbed copies of a model, score each on the traffic it gets, and merge them periodically. No backprop and only one inference call per user query. Zeroth-order methods do their magic! Work done at @berkeley_ai with Marquita Ellis, @GopalaSpeech and @LChoshen. Blog: akshat57.github.io/blog/popu… 🧵

0:10
Ken Goldberg avatar
⟳ Berkeley AI Research retweeted
Ken Goldberg@Ken_Goldberg·Nov 11

Great summary by @oitz @therobotreport : therobotreport.com/can-ai-ag… @NVIDIARobotics @BoschGlobal @Siemens @AmbiRobotics @UCBerkeley @kaiyuan_c @Shuangyu0_0Xie @UksangYoo @XinyueZhuXY

Paul Zhou 🛫 CoRL avatar
⟳ Berkeley AI Research retweeted
Paul Zhou 🛫 CoRL@zhiyuan_zhou_·Nov 11

Going to present a few things at CoRL on Thursday! 🧵👇 (1) Early in the morning I'll be giving a talk at the R2RL workshop on my perspectives on the challenges of efficient real world RL. Slight spoiler below 🤫 🕒 9:20 AM, Marriot 213

Post attachment
Sewon Min avatar
⟳ Berkeley AI Research retweeted
Sewon Min@sewon__min·Nov 11

Model competition is heating up, and distillation is 🔥🔥 Distillation detection is hard, so watermarking can help make it detectable. But are existing watermarks robust? We find two ways to break them: distill a model, then distill that model again; or distill only selected tokens that matter. Then what do we do? We introduce SubWM, which outperforms prior methods and is more robust to laundering.

Excited to present our new work on distillation detection: Subliminal Watermarks (SubWM). SubWM is much more robust compared to existing distillation detectors, is able to distinguish different watermarking keys, and also works reliably for out-of-domain prompts! Blog: subliminal-watermarks.github… 🧵

0:24
Harshit Varma avatar
⟳ Berkeley AI Research retweeted
Harshit Varma@hrshtv·Nov 10

Excited to present our new work on distillation detection: Subliminal Watermarks (SubWM). SubWM is much more robust compared to existing distillation detectors, is able to distinguish different watermarking keys, and also works reliably for out-of-domain prompts! Blog: subliminal-watermarks.github… 🧵

0:24