Finally, we can release a bit of the work we have been doing at the DexHands team in Amazon FAR ❤️🔥
Pure RL, zero teleop, PICK ANYTHING
Come to Amazon Booth at CORL
(Sound on.)
Robots that learn should generalize. Locomotion should show generalization across terrain; manipulation across object categories. Here is a video of “Pick Anything”, demonstrating human like grasps for 66 different objects, ranging in size and shape from air pods to dumbbells to pipe wrenches, using a *single* policy. The system uses the stereo camera head for vision (no wrist cameras) & Sharpa Wave hands for manipulation. It was trained using video and RL (zero teleoperation data). This is research from our Dex-Hands team at Amazon FAR and we will be doing live demos of the system at CoRL 2026 ( Demo 12, Nov. 9 &10). Please come by!
Harping on some of the learnings:
1. It is actually quite easy to have a RL tracking policy for multiple references. In fact it scales much better. We started from multiple teachers I.e., teacher policies per reference in Isaac to finally a single policy trained in newton physics for all references.
2. Data: quality of reference data for retargeting is important. We iterated a lot on getting higher quality retargeting as well as source data from iPhone videos of humans picking objects and generated videos of picks too.
3. It’s important to worry about mixing modes from multiple trajectories, partial observability from point cloud / depth when distilling from privileged teachers.
4. Scaling is exponential, going from 1 object to 20 was difficult, but 20 to 200 and later was easier.
An amazing work lead by @aahoussaini !!An all-in-one framework to benchmark dexterous manipulation
Including environment, data and a simple interface for cross-embodiment robotics!
We are using it in our incoming works 🔥
Dexterous manipulation deserves better tools 🖐️
Excited to share DexSuite: an all-in-one, open-source simulation framework for dexterous manipulation, built on Genesis 🚀
Nobody said research paper videos need to be boring...
(put volume up)
🌐 dexsuiteorg.github.io/
Ed & Vitalis are super great people and researchers and the idea of Instant Robot Learning is soooo elegant and useful for robotics. Congratulations on the release!!
Anybody interested in doing cutting-edge research should consider applying ❤️🔥
On Tuesday, we announced @Machenta_ and Instant Robot Learning.
Today, we’re looking for the next people to join us all in London!
machenta.com/team
Building Machenta with everyone here has been such an adventure. And above all, so much fun.
Partly I think this is because we’ve been fortunate enough to find a group of people who are just extraordinarily passionate, about both the research and the startup experience. But partly, it’s the magic of instant learning. Every morning when you walk into the lab, you can dream up an entirely new task, teach the robot, and just see what happens next. Sometimes it works, sometimes it doesn’t. Either way, there’s something incredible about all huddling around together to watch.
All these little moments become memories that stick. I’ll never forget the first time @vitalisvos19 showed me Instant Robot Learning on a bimanual robot, tidying up Christmas crackers last December. I’ll never forget the day an investor surprised us by turning up with a collection of their own objects to test the model on (it worked — phew!). And I’ll never forget the first day each new member of the team walked through the door and instantly became part of the gang.
We’re moving quickly here, we work very hard, and we collaborate closely in a flat structure. Everyone brings their own expertise to a different part of Machenta, but there’s a real sense that we’re all building this together. We’re a very close team, working in person every day and sharing in all the little moments along the way.
And we’re now going to go out and find the right people to join us for this next phase of the journey. We’ll give you real ownership and responsibility for your part of Machenta, and an opportunity to shape where we all take this together next.
So if you’re excited to be part of this with us all, today we're hiring for five roles:
• Member of Technical Staff — Robot Learning
• Member of Technical Staff — Robot Control
• Member of Technical Staff — Robot Simulation
• Member of Technical Staff — 3D Machine Learning
• Community Lead
Come and say hi at machenta.com/team.
See you there!
Can you believe these are open-loop rollouts?
Excited to introduce CARD 🃏 — Contact-Anchored Retargeting for Dexterous Manipulation.
Almost every method today is limited to kinematics alone as the 𝐢𝐧𝐭𝐞𝐫𝐟𝐚𝐜𝐞 between the human and the robot. With CARD, we track real world contact and forces alongside kinematics to perform unprecedented dexterous tasks through tactile reactive replay!
CARD is scalable:
📹 Multiple input sources
🤖 Multiple robot hands
⚡ needs no object models, no simulator, 10 Hz
How should we approach research, when the very field we work on is going through rapid and potentially fundamental changes?
I recently gave a talk at the Stanford Vision Seminar, reflecting on my PhD journey and how AI is changing the way I think about research. I adapted part of it into a new blog post, organized around three questions:
1. What should we work on?
2. How should we work with AI?
3. What is a PhD for?
I hope some of my reflections are helpful. The talk ended with an open discussion, and I’d love to continue it here; curious how others are thinking about these questions.
Personally -- as much as there's FOMO, I also think life is very long, and we have time.
toruowo.github.io/blog/posts…
A first glimpse of the wildest robotics project of my PhD.
🛹 The Blind Skateboarder 🛹
No vision. No markers.
Paper, code, weights — coming soon. And we have more. 😉
Presenting PTLD: an approach to learning tactile dexterous policies without ever simulating the tactile sensor.
Tactile is essential for performing highly dexterous manipulation. However, collecting tactile observations reliably has been a fundamental bottleneck: (1) teleoperating a multi fingered hand for dynamic tasks is challenging, making sim-to-real imperative, (2) one can’t realistically simulate tactile today
We are excited to announce the 2026 cohort of RSS Pioneers! This year’s cohort brings together an outstanding group of early-career researchers whose work spans the breadth of robotics. A heartfelt thank you to all the organizers who made this year’s program possible.
1/ World models are getting popular in robotics 🤖✨
But there’s a big problem: most are slow and break physical consistency over long horizons.
2/ Today we’re releasing Interactive World Simulator:
An action-conditioned world model that supports stable long-horizon interaction.
3/ Key result:
✅ 10+ minutes of interactive prediction
✅ 15 FPS
✅ on a single RTX 4090🔥
4/ Why this matters: it unlocks two critical robotics applications:
🚀 Scalable data generation for policy training
🧪 Faithful policy evaluation
5/ You can play with our world model NOW at yixuanwang.me/interactive_wo…. NO git clone, NO pip install, NO python. Just click and play!
NOTE ⚠️
ALL videos here are generated purely by our model in pixel space! They are **NOT** from a real camera
More details coming 👇 (1/9)
#Robotics#AI#MachineLearning#WorldModels#RobotLearning#ImitationLearning
Pretraining with dynamics models of motor behavior (aka world models) from video will be much more central to robotics than VLMs. There are multiple choices of representations (e.g. 3D? JEPA?) but we will figure this out by and by. Exciting times!
Fully agreed with the sentiment that much of computer vision research (concretely, those not for “human consumption”) should be grounded in robotics. But as a robotics researcher, I think the more nuanced question is: how can we *rethink* these intermediate representations for embodied intelligence rather than discarding them?
Why?
The challenge, as also pointed out in Vincent’s article, is precisely the lack of perception-action data at scale. This is why intermediate representations IMO are *preferable rather than obsolete* because they open up training from scalable data sources. This can include even the vision/language encoders people love and use in robot learning — it’s hard to imagine training low-level visual representation or high-level language understanding purely from limited robot data. The same goes for intermediate representations at the structure level — world modeling, learning from Internet videos, learning from humans, and simulation — many of which still rely on 3D representations too.
In my recent blog post, I argue that "vision" is only well-defined as part of perception-action loops, and that the conventional view of computer vision - mapping imagery to intermediate representations (3D, flow, segmentation...) is about to go away.
vincentsitzmann.com/blog/bit…
At the RI seminar at CMU yesterday, I presented a 3 level analysis of robot skills & discussed the pros and cons of teleoperation, simulation, and learning from videos, before presenting our research. Enjoy! youtube.com/watch?v=ry8itipz…
As video world models become increasingly powerful, do we still need explicit 3D?
A commonly misunderstood point is this: video world models are not “just 2D.” Their ability to maintain multi-view consistency, temporal stability, and realistic interaction necessarily implies that their latent knowledge encodes 3D world structure. Without some notion of 3D, consistency itself would not be possible.
The real distinction, therefore, is not whether a model has 3D but whether that 3D exists implicitly or explicitly.
Implicit 3D lives inside latent spaces and network weights. It supports generation, but it is difficult to localize, edit, constrain, or reason about.
It allows the world to exist, but not to be used.
Explicit 3D, in contrast, exists as structure and state: it is addressable, editable, composable, and transferable.
Its purpose is not better visual fidelity, but operability to allow the world to be manipulated, controlled, and executed.
From this perspective, video and 3D are not competing paradigms but a layered system:
2D/video is the interface to human perception; 3D is the interface to the physical world.
They can reinforce each other, but neither forms a closed loop on its own.
In practice, data not model architecture sets the upper bound of world models.
Explicit 3D may not be the final user-facing representation, but it is likely the most effective pathway toward scalable, high-quality, and controllable data.
Through explicit 3D/4D representations, worlds can be constructed systematically:
interactions can be programmatically sampled, states and actions can be composed, rendered into images and videos, and fed back to train video world models.
Seen this way, 3D is not the destination it is the starting point for scaling.
What truly drives progress forward is never the model itself.
Whether we capture the world or imagine new ones,
whether data comes from observation or intent,
whether we model what is or what should be the direction of the world is ultimately determined by human choice and purpose.
Models may extend the world,
but humans decide where it goes.
#Genie3#worldmodel
There has been a clear trend in the last months moving from VLA-type approaches to Video Generative Models + Inverse Dynamics Models (VAM).
While the probable main reason of this recent growth is the latest improvements in video generative models, I believe this shift is relevant for robotics.
While the VLA's distill the foundation models knowledge through some latent representations that intertwine semantic and spatial information, VAM distill this knowledge in a more explicit way, representing it spatially.
I believe this spatial grounding of VAM might lead to way larger generalization capabilities wrt. VLA and I am optimistic in even more 3D spatially grounded foundation models in the direction of the recent @wenlong_huangpoint-world.github.io/
While I really liked the article, it feels to me that this physical commonsense can be better capture by predicting next observations (i.e. world models) and planning on it, rather than training a policy on predicting next action (i.e. behavioral cloning)
Andy Zeng @andyzengineer Jan 29 Article The Dark Matter of Robotics: Physical Commonsense It should exist. It’s everywhere. And yet — hard to pin down. The next time you grab an object tightly packed on a shelf — like a book or a box — watch what your hands do. Do you nudge it sideways to