←
Furong Huang2,468 posts
Furong Huang avatar✓

Furong Huang ✓

@furongh

Assoc. prof. of CS at University of Maryland. Researcher in #PhysicalAI, #TrustworthyML, #EthicalAI, AI #Alignment, #AI for ALL.

◷ Joined September 2010▧ 575 Photos and videos
Furong Huang avatar
Furong Huang@furongh·Jun 14

Hot take: robots should not dream in pixels. Pixels are too low-level. Latents are too opaque. μ₀ predicts a third thing: 3D motion traces. On real robots, it beats π₀.₅ — with ~1/100 the data scale and no action labels for world-model pretraining. 🧵 mu0-wm.github.io/ (this video features voiceover narration)

2:49
Furong Huang avatar
⟳ Furong Huang retweeted
Furong Huang@furongh·29 Jul 2025

NeurIPS rebuttal deadline is around the corner 😬 I’m not an expert, but thought I’d drop my two cents on how to write a good rebuttal, especially for folks writing their first few. Hope this helps someone! 🧵👇 (And please chime in with your own tips; let’s crowdsource the wisdom 🧠💬) 1. Start with gratitude + remind them you’re awesome Begin by echoing the reviewer’s positive feedback: > “We appreciate that Reviewer X finds our method novel, our experiments concrete, and the topic timely.” That’s right--remind them how much they liked you. 🥹✨ 2. State the overall message early Something like: > “The concerns raised are mostly due to misunderstandings. We have addressed all concerns below.” 💡 Be clear, be calm, be confident. (But don’t be a jerk 😇) 3. Formatting matters more than you think Yes, I’m talking Markdown! 🧹 • OpenReview supports it • Number your responses like a civilized human • Use hackmd.io (or something alike) to collaborate with co-authors like a team of rebuttal ninjas 🥷📄 4. Use the Bullet-point BLUF strategy 💥 Bullet Points + Bottom Line Up Front (BLUF) = Reviewer brain says thank you Each point starts with a bolded TL;DR. Then follow up with: • Clarifications ✏️ • New experiments 📊 • References 📚 • Logic & reasoning 🧠 5. Why BLUF works: • One idea per bullet → no logic spaghetti 🍝 • Bolded summaries → busy reviewers (and ACs!) can skim and still get the point 🏃‍♂️💨 6. Think from multiple angles 🔍 A good rebuttal isn’t just “we disagree.” Try: • Clarifying • Reframing • Adding results • Acknowledging edge cases 💡 Each bullet can be a different strategy. Keep it classy, keep it constructive. Rebuttals won’t fix everything, but they can build trust, show thoughtfulness, and make your case stronger. 💪 If you’re an early-career researcher writing your first few: I see you. You got this. Keep going. 🔥❤️ These are just my thoughts. Would love to hear yours! Drop your best rebuttal tips below 👇 Let’s help each other survive and thrive 💫 #NeurIPS2025 #PhDLife #RebuttalTips #MachineLearning #PeerReviewPain #AcademicTwitter

Xirui Li avatar
⟳ Furong Huang retweeted
Xirui Li@xiruili7_li·Nov 15

血泪的教训。我们是最早一批开始做Agent-as-Policy的团队(arxiv.org/abs/2606.18363),但是今年5月把文章投到了CORL 2026,三个reviewer都只关心VLA,WAM (1*weak accept,1*strong reject,1*weak reject)。很多文章都是各种VLA + heuristics,WAM + heuristics,大道至简是data + infrastructure。

Furong Huang avatar
Furong Huang@furongh·Nov 14

180B+ images and videos carry SynthID watermarks. Watermarking is becoming part of the infrastructure for trust at internet scale. As AI makes convincing media cheaper to create and remix, the burden of verification grows: who created this, how was it changed, and what evidence supports that history? Provenance helps us answer those questions. Its value depends on signals that survive editing and sharing, verification that works across platforms, and clear limits on what we can conclude. Building on our CVPR tutorial, our second edition at ACM MM 2026 connects these questions to watermarking foundations, robustness evaluation, and deployed provenance systems—with live demos and hands-on notebooks. Tutorial 📅Saturday, November 14, 2026, 2:00 – 6:00 PM · BRT (UTC−3) 📍Room Bora-Bora I, Windsor Convention & Expo Center, Rio de Janeiro Materials 👉vishal3477.github.io/waterma…

Post attachment
Yoonkyo Jung avatar
⟳ Furong Huang retweeted
Yoonkyo Jung@YoonkyoJung·Nov 10

I'm at #CoRL2026 in Austin this week! 🤠 Come say hi at: 1️⃣ μ0 (mu-zero): a 3D trace world model. Pretrained on video only, it predicts 3D traces (not pixels or actions) and transfers across embodiments. 📍 Wed 11/11, 10:30 AM · Poster Session III 🔗 mu0-wm.github.io

Jia-Bin Huang avatar
⟳ Furong Huang retweeted
Jia-Bin Huang@jbhuang0604·Nov 9

Excited for #CoRL2026 in Austin this week! 🤠 We will present mu-0, a 3D interaction-trace world model. Key idea: Learn a "physical language” for robots purely from video. Zero action labels required to build a WM! Code & dataset: mu0-wm.github.io/ and find us Wed 11 Nov, 10:30 AM | Poster Session III

2:49
Furong Huang avatar
Furong Huang@furongh·Nov 8

I want robots that can watch a person perform a task, remember what matters, and adapt when the situation changes. That’s the vision connecting our lab’s work at #CoRL2026. A human demonstration contains much more than motion. It shows how hands and objects interact and how a task unfolds. We’re studying how to make that experience useful for robot learning—from large-scale human video in EgoScale (research.nvidia.com/labs/gea…), to minutes of egocentric video in HumanEgo (humanego-ai.github.io/), to a single demonstration in Dex-One2Many (dex-one2many.github.io/). To use that experience, a robot needs to represent and remember the right things. μ₀ (mu0-wm.github.io/) builds a world model around 3D interaction traces. What to Attend, What to Keep (what-to-attend-what-to-keep.…) studies how skills can guide attention to vision and touch, and which events to retain as a task progresses. Then comes acting—and responding to what happens. Fly-by-Code (fly-by-code.github.io/) brings visual and physical feedback into embodied coding agents. Guava (guava-harness.github.io/) explores how to transfer the capabilities of frontier vision-language models into a compact robot agent. These are complementary steps toward robots that need less task-specific supervision, transfer knowledge across settings, and use feedback to act more reliably. I won’t be in Austin this year, but I’m proud of the students and collaborators advancing this agenda. Ruijie Zheng is also an invited speaker at the Scaling Human2Robot Workshop! If you’re attending CoRL, please meet the team at their talks and posters. I’d love to hear your thoughts on this direction. Presentation details are in the posters below. #RobotLearning #EmbodiedAI

Post attachmentPost attachment
Jia-Bin Huang avatar
⟳ Furong Huang retweeted
Jia-Bin Huang@jbhuang0604·Oct 27

Thanks a lot! Yup, these are all animated from manim (manim.community). It’s fully programmable so you have full control of the animation.

Furong Huang avatar
⟳ Furong Huang retweeted
Furong Huang@furongh·Oct 23

Precompute and reuse. I keep coming back to this idea, and there’s a 4 a.m. slide-making story behind it. Back at ICLR 2023 in Kigali, Rwanda, a student of mine was helping organize the Reincarnating Reinforcement Learning workshop. A speaker couldn’t make it, and they asked whether I could give a talk. I said yes of course, it was my student’s workshop! So I found myself making slides at 4 a.m. in my hotel room for the talk the next day. 😂 My talk was “Adaptable Reinforcement Learning in an Ever-Changing World: Reuse to Learn and Adapt Efficiently.” I also joined the panel, “Challenges & Open Problems in Reusing Prior Computation.” It was absolutely a great workshop. What stayed with me was a simple question: why keep starting over when we’ve already spent so much compute learning something useful? Today, that sounds almost too obvious. Isn’t that a big part of what foundation models are about? And really, RL post-training follows the same logic: start with a pretrained model, reuse what it has learned, and build on it with RL. But back then, training a specialist RL agent from scratch for each task was still very common. That workshop helped me put words to something that has become a recurring theme in our research. Once you take reuse seriously, the interesting question becomes: what should we learn ahead of time, and what should we leave for adaptation when the actual task arrives? You can see that question across our work, just to name a few: • SMART: pretrain control representations that transfer across tasks. • DynaFLIP: learn dynamics-aware perception that different robot policies can build on. • μ₀: learn a world model from video and reuse it across robot embodiments. • FlowBank: precompute useful agentic workflows, then choose one for each query. And now, Meta²: precompute and reuse knowledge about how to improve. An agent’s harness includes the prompts, tools, memory, hooks etc around the model. Often, you only discover what needs changing once the agent is already working. A missing tool. An instruction that keeps being forgotten. A check that needs to happen repeatedly. We want the agent to adapt as those needs appear. But reasoning about every possible change from scratch is expensive, and making changes doesn’t automatically mean making progress. So we borrow a core idea from meta-learning: learn how to adapt. Offline, Meta² learns a textual Meta-Skill that guides an evolver on when and how to change the harness, including when to leave it alone. We learn this guidance by comparing runs with no evolution, runs with the current evolver, and training runs where the evolver gets privileged verifier feedback. During a new run, that learned guidance stays fixed. At checkpoints, the evolver combines it with evidence from the current run to decide whether to keep or change the harness. The underlying model stays frozen. The guidance gets reused. The actual edits depend on what happens. One result I especially like: carrying one task’s harness edits into future tasks actually hurt performance in our experiments. Knowing how to adapt transferred better than keeping the adaptations themselves. That distinction matters for RSI. A system that keeps improving needs to figure out what to preserve from experience, what to revise, and what was only useful in one particular situation. Meta² gives us one concrete piece of that: reusable guidance for improvement, paired with adaptation while the agent is still working. Longer term, I see RSI as potentially the co-evolution of models, harnesses, and the environments they learn in. Progress in each could help the others improve. For now, we’re nailing down the harness part! I’m really proud of my students and collaborators for finding an efficient and effective way to make this work. That 4 a.m. slide-making session stayed with me longer than I expected. 🧬

2:58
Lingzhi Yuan avatar
⟳ Furong Huang retweeted
Lingzhi Yuan@lingzhi_yuan·Oct 23

Is your agent's harness frozen before the run even starts? In the push toward recursive self-improvement (RSI), a system that can't adjust to new evidence mid-run isn't truly adaptive. To keep evolving online, it needs guidance on how and when to evolve. Introducing Meta²: It learns this evolution guidance offline and applies it dynamically within every single run. 🧬👇 alphaxiv.org/abs/2610.meta2-… meta-square.github.io/paper.…

2:58
Feijiang Han😈 avatar
⟳ Furong Huang retweeted
Feijiang Han😈@feijianghan·Oct 22

check out this intuitive video on why interpretable directions in llms look non-gaussian, how ica finds them in minutes per layer, and the evaluation bug that made ica look weak for 3 years

Post attachment
6:07