←
Peter Chen164 posts
Peter Chen avatar

Peter Chen

@peterxichen

Building general purpose robotics at Amazon FAR. Previously Covariant CEO and Co-Founder, @OpenAI, @UCBerkeley PhD.

◷ Joined December 2017▧ 5 Photos and videos
zeji yi avatar
⟳ Peter Chen retweeted
zeji yi@ZejiYi·Oct 20

Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞. ❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings. Here's why 🧵

1:44
Peter Chen avatar
Peter Chen@peterxichen·Oct 20

This is one of the most surprising projects FAR took on recently: (naive) discrete actions can be more precise than continuous actions for robot control. Check out this cool work by @ZejiYi et al

4/12 So, 𝐰𝐡𝐲 𝐝𝐨 𝐝𝐢𝐬𝐜𝐫𝐞𝐭𝐞 𝐚𝐜𝐭𝐢𝐨𝐧𝐬 𝐰𝐨𝐫𝐤? 𝐒𝐡𝐨𝐮𝐥𝐝𝐧'𝐭 𝐭𝐡𝐞𝐲 𝐬𝐮𝐟𝐟𝐞𝐫 𝐟𝐫𝐨𝐦 𝐩𝐫𝐞𝐜𝐢𝐬𝐢𝐨𝐧 𝐢𝐬𝐬𝐮𝐞𝐬? Surprisingly, 𝐧𝐨. Overall action error is about the same for all four heads. On fine motions (expert moves under 5 mm per step), discrete is 𝐞𝐯𝐞𝐧 𝐦𝐨𝐫𝐞 𝐩𝐫𝐞𝐜𝐢𝐬𝐞: 3.2 vs. 5.4 mm single-step, 3.7 vs. 4.6 mm with flow. So binning is 𝐧𝐨𝐭 𝐚 𝐩𝐫𝐞𝐜𝐢𝐬𝐢𝐨𝐧 𝐛𝐨𝐭𝐭𝐥𝐞𝐧𝐞𝐜𝐤. Then what makes the difference?

Post attachment
Xinhao Liu avatar
⟳ Peter Chen retweeted
Xinhao Liu@xinhao6iu·Oct 19

Your navigation model already found the goal. It just doesn't know when to stop. Introducing StopWatch, a navigation agent that grounds the goal explicitly before every action. It sets a new state of the art on six object-goal navigation benchmarks. Watch the flower vase episode below. Both agents reach the goal. StopWatch stops there. LightNav-0 turns away and keeps going. Code, models and data are open source 👉 stopwatch-nav.github.io/ Work done at Amazon FAR w/ @yunshengtian @QiuYuhengQiu @pabbeel @peterxichen @GuanyaShi @simonforrest 🧵 [1/n]

0:35
Siting Li avatar
⟳ Peter Chen retweeted
Siting Li@SitingLi627·Sep 14

6/9 ↳Possibly because I2T loss reflects overall joint training progress. And since every model predicts within the same text token space, I2T loss stays comparable across tokenizers, whereas T2I loss does not. I2T loss remains informative after SFT, correlating with both image generation quality and general VQA performance.

Post attachment
Siting Li avatar
⟳ Peter Chen retweeted
Siting Li@SitingLi627·Sep 14

1/9 Image Tokenizers define Visual Languages. 🎨🔤 Image tokenizers do more than compress and reconstruct pixels. In unified multimodal models, they define the visual language a single model must learn, align with text, and use for both understanding and generation — and the best compressor may not produce the easiest language to learn. We study image tokenizers through the lens of unified AR multimodal training. 🧵

Post attachment
Max Simchowitz avatar
⟳ Peter Chen retweeted
Max Simchowitz@max_simchowitz·Aug 11

Personal Update: I'll be splitting my responsibilities between CMU and Amazon FAR over the next year. I'm convinced we need a unified understanding of how the axes of scale, data, optimization and control interact for robot learning, and am excited for my lab + FAR to push towards uniting the "science of robot learning" with the "science of deep learning" at scale. Please reach out if you are looking for a full-time position building powerful multimodal AI models for deployment on hardware, and if you are intern looking to tinker and do good science on architectures, optimizers, data, tokenization, VLAs, WAMs and more!

Utkarsh Mishra avatar
⟳ Peter Chen retweeted
Utkarsh Mishra@utkarshm0410·Jul 10

WAMs are popular because of their promise of better generalization. Is that true? We started playing with Video-Action-Model (VAMs) and realized a gap: video model backbones can compositionally generalize but VAMs often do not. We coin this the Video-Action-Generalization (VAG) gap and present a study on how to explain and improve it. More details: umishra.me/temporal-ratio/ 🧵 below

0:32
Jiaman Li avatar
⟳ Peter Chen retweeted
Jiaman Li@jiaman01·Jul 1

🤖 How can we scale up humanoid robot learning? Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed! Website: vision-language-kinematics.g…

0:24
Sirui Chen ✈️ CoRL2026 avatar
⟳ Peter Chen retweeted
Sirui Chen ✈️ CoRL2026@eric_srchen·Jun 29

Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction! By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like carrying a box upstairs. Code & data open-sourcing soon! 📦 🪜 Paper: arxiv.org/abs/2606.27581 Website: ericcsr.github.io/scenebot/

0:38
David McAllister avatar
⟳ Peter Chen retweeted
David McAllister@davidrmcall·Jun 17

A huge team effort! Fully open data, infrastructure and eval for the community to build on

0:06

Introducing ABC: open data, training, and infrastructure for robotics. We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques. @arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcall

2:52
Siheng Zhao avatar
⟳ Peter Chen retweeted
Siheng Zhao@SihengZhao·Jun 8

🪜 What if humanoids could climb ladders and work on them straight out of simulation? Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation. Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇

0:32
Tommie Kerssies avatar
⟳ Peter Chen retweeted
Tommie Kerssies@tommiekerssies·Apr 9

World models are heavy. They don't need to be. Each frame is encoded as 1024 spatial tokens. What if it were just 1? In our #CVPR2026 Highlight from Amazon FAR, we compress frames into "delta" tokens for efficient generative world modeling. Paper, code & models below ↓ (1/7)

Post attachment
Chen Feng avatar
⟳ Peter Chen retweeted
Chen Feng@simbaforrest·Mar 10

🤖We are hiring multiple Summer'26 Research Interns at @amazon FAR to work on open-world navigation and robot foundation models, especially in neural rendering & simulation/predictive world models/reasoning & agency/real-world evaluation/long-term autonomy!

Post attachmentPost attachmentPost attachment
Zhen Wu avatar
⟳ Peter Chen retweeted
Zhen Wu@zhenkirito123·Feb 17

Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world? We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal. 1/6

0:30