←
Jeff Cui114 posts
Jeff Cui avatar

Jeff Cui

@jeffacce

PhD CS @ NYU Courant, Research @ Meta

◷ Joined December 2014▧ 28 Photos and videos
Jeff Cui avatar
Jeff Cui@jeffacce·Jul 22

Your policy doesn't need 7B params. It simply needs dense features. Introducing Patch Policy: pretrained ViT + small transformer beats OpenVLA-OFT with 0.7% of its params, and trains on a 5090. Here it inserts a cable (~2mm tol), and does it again as we unplug mid-rollout. 🧵

0:22
Mahi Shafiullah 🏠🤖 avatar
⟳ Jeff Cui retweeted
Mahi Shafiullah 🏠🤖@notmahi·Nov 8

In behavior cloning, we mastered how to learn robotics from "robot-complete" data – but what happens to data that is not so perfect? Intelligence is often about learning from incomplete references; to come up with a "story" given disparate datapoints. Our Pick Anything demo shows an ability of this sort: the policy learned human-like dexterity from absolutely no human-robot interactions, no teleop or handheld tool data, just letting humans be humans and robots be robots. How did we do this? Learning from humans without actions is often reduced to a semantics problem, a fancy way to prompt the robot. What, if anything, can robots learn about the dynamics from them? RL for manipulation has the opposite problem – often we are stuck with "technically correct" solution that are nevertheless aesthetically unsatisfying. In Pick Anything, we found a way to unite the two in a way for dexterous manipulation that I personally find quite beautiful (and I can't wait to share the paper with you!) I believe what we have here is the key to human level dexterity for robots, and all of that without needing any real robot data whatsoever. For this Monday/Tuesday, you can visit our demo booth at CoRL (Griffin hall, demo 12) and ask me more questions! It's a labor of love from my past year of work, and I'm very excited to be finally able to talk about it.

1:01
MuJoCo avatar
⟳ Jeff Cui retweeted
MuJoCo@mujoco·Nov 7

MuJoCo 4.0 is out! • Fast or thin objects no longer pass through each other • Soft bodies can now deform permanently and rods can twist • MuJoCo Studio, the successor to simulate, with physically based rendering via Filament Try it live: mujoco.org

0:45
Irmak Guzey avatar
⟳ Jeff Cui retweeted
Irmak Guzey@irmakkguzey·Nov 9

As we get closer to a world where we live among robots, it’s becoming more and more valuable for robots to learn quickly with and from people. I’m very excited to share Steertuning, a novel approach that adapts robots to new tasks from 5 human demonstrations in under 5 minutes, from demonstration to deployment. Website: steertuning.github.io/ Code: github.com/billy-yibo-yan/st…

2:08
Binglin Andre Wang avatar
⟳ Jeff Cui retweeted
Binglin Andre Wang@AndreBinglin·Nov 3

After about a year of building and experimenting, I’m excited to share PanoTac, my first research project! 🤖 We built a compliant tactile fingertip that costs under $50 and takes ~30 min to assemble. By combining piezoresistive and magnetic sensing, it captures contact geometry and provides normal and shear feedback in a dense 12×32×3 tactile field. I’m especially excited about how these sensing modalities complement each other: even simple fusion broadens what either can do alone. We use touch every day to guide our actions, often without thinking about it. Seeing the robot use tactile feedback to decide how to move and when to change actions has been incredibly rewarding. This project has also shaped how I think about hardware–learning co-design: what should we build into the hardware to give a policy the information it needs? I see tactile sensor fusion as a promising direction for exploring this connection.💡 Grateful to my coauthors, advisors, and everyone who shared ideas and feedback along the way! Check out the demos below 👇 panotac.github.io/

❓I keep coming back to this: what tactile representation does manipulation need? We introduce PanoTac: A dense three-axis Tactile Sensor. In the demo, the robot reorients and inserts a tube, then wiggles the stopper free while the tube sits loosely in the rack. The right side show the tactile signals that helps robot detect when the stopper releases and when to switch from wiggling to lifting. (1/7)

0:29
Nicklas Hansen avatar
⟳ Jeff Cui retweeted
Nicklas Hansen@ncklashansen·Nov 4

Really excited to share Catch It All!, my first paper as last author! In this work led by @Kehlani_Fay @arth_shukla we develop a system for catching diverse objects using a single egocentric camera 🧵👇

0:25
Gaoyue Zhou avatar
⟳ Jeff Cui retweeted
Gaoyue Zhou@GaoyueZhou·Nov 4

Introducing ID-WM: a hierarchical world model that decouples "where to go" from "how to get there." Finally, we can plan over hundreds to thousands of steps with WMs. It beats the strongest baseline by 38.5%, single-level WMs by 135%. #NeurIPS2026 🧵 id-wm.github.io

0:07
Binghao Huang ✈️ CoRL avatar
⟳ Jeff Cui retweeted
Binghao Huang ✈️ CoRL@binghao_huang·Nov 3

❓I keep coming back to this: what tactile representation does manipulation need? We introduce PanoTac: A dense three-axis Tactile Sensor. In the demo, the robot reorients and inserts a tube, then wiggles the stopper free while the tube sits loosely in the rack. The right side show the tactile signals that helps robot detect when the stopper releases and when to switch from wiggling to lifting. (1/7)

0:29
Jeff Cui avatar
Jeff Cui@jeffacce·Nov 4

Rich force sensing is important for robotic manipulation. Ablating away force, vision-only and normal-grid-only policies fail to release the stopper properly:

0:10
0:17
Jeff Cui avatar
Jeff Cui@jeffacce·Nov 4

Together, they give a dense 3D tactile field at up to 100 Hz:

0:09
Jeff Cui avatar
Jeff Cui@jeffacce·Nov 4

PanoTac captures the contact geometry with a dense piezoelectric array, and tracks the contact force with magnets and magnetometers embedded in a deformable fingertip.

0:07
Jeff Cui avatar
Jeff Cui@jeffacce·Nov 4

Robots need to feel where they touch and how forces change. Introducing PanoTac: an open-source, dense 3D tactile sensor for <$10 USD. Watch as the robot racks a test tube and pulls the stopper: PanoTac tells the tube orientation, and feels when the stopper comes free. 🧵

0:08
Amir Bar avatar
⟳ Jeff Cui retweeted
Amir Bar@_amirbar·Oct 8

NAVA-WAM: Can we learn latent actions for World Action Models? We revisit a classic idea: jointly learning inverse and forward dynamics models while avoiding the identity shortcut (e.g., @SeonghyeonYe, @garridoq_, @jeffacce, @pulkitology, and others). The twist: Both models are implemented within a single WAM architecture, using attention masks! To prevent the identity shortcut, the latent actions can only sees a noisy version of the next frame, forcing the latent action to capture useful dynamics rather than simply copying the future. This simple pretraining strategy improves both performance and efficiency across robotics benchmarks! 🧵👇

🤖 Abundant videos capture how the world works, but robot policies still rely on costly action labels to learn. 🔥 Excited to share our work at @Meta: 📢 Native Action-Prior Learning from Videos for World Action Models 📢 📄arxiv.org/pdf/2610.03391 🌐zhaochongan.github.io/NAVA-W… 🧵

1:07