←
harpreet7,220 posts
harpreet avatar

harpreet

@DataScienceHarp

hacker-in-residence/ principal devrel engineer. data curation for physical and visual ai systems. in my physical ai era

◷ Joined April 2020▧ 1,258 Photos and videos
harpreet avatar
harpreet@DataScienceHarp·Jun 29

a robot can recognize every object in a room and still not know how to open a single drawer knowing there's a cabinet is scene understanding. knowing which handle to grab, which direction to pull, and that it's a drawer not a door is affordance understanding almost no dataset has it in 3D SceneFun3D: 14,800+ functional elements across 710 laser-scanned indoor scenes. every handle, knob, button, and switch labeled with its affordance, motion axis, motion type, and a natural language task description each scene's 3D point cloud with its iPad video recordings in fiftyone. the affordance annotations live in 3D and project into the video frames check it out here huggingface.co/datasets/Voxe…

0:30
harpreet avatar
harpreet@DataScienceHarp·21s

very few Mars images come with high-quality pixel labels. S5Mars labels only the pixels its annotators were confident about and leaves the rest unlabeled. it has 6,000 Curiosity rover images at 1200 x 1200 pixels, 3,000 easy and 3,000 hard, with 9 terrain classes including sand, bedrock, and wheel traces. i parsed it into FiftyOne so you can filter easy against hard and overlay the masks. huggingface.co/datasets/Voxe… #PhysicalAI #Robotics

0:26
harpreet avatar
harpreet@DataScienceHarp·2h

thanks for sharing! for those interested the embeddings are using qwen3vl-embeddings on 5 sec windows of the video. we can search by segment to segment as well as text to segment

Wow. This fixes one of the most annoying parts of training robots. Until now, finding one failed grasp meant rewatching hours of recordings. @voxel51 now searches MCAP and LeRobot data in plain English and returns the exact seconds. - One bad grasp → every similar one - Tells you if it's a one-off or a failure mode - All sensor streams on one timeline And it's free to use. This saves a lot of training runs.

0:59
harpreet avatar
harpreet@DataScienceHarp·3h

oh hey, it's me!

Wow. This fixes one of the most annoying parts of training robots. Until now, finding one failed grasp meant rewatching hours of recordings. @voxel51 now searches MCAP and LeRobot data in plain English and returns the exact seconds. - One bad grasp → every similar one - Tells you if it's a one-off or a failure mode - All sensor streams on one timeline And it's free to use. This saves a lot of training runs.

0:59
harpreet avatar
harpreet@DataScienceHarp·8h

the action at frame t is the joint state at frame t + 1, and this dataset card spells that out. Piper Dual Stack Cups (ROS 2 Shifted LeRobot v2.1) has 100 episodes and 61758 frames at 30 FPS, as 14D joint vectors in radians. three RGB cameras are letterboxed to 224 x 224. grippers come from the master arms, unshifted. load it with the LeRobot API to train imitation policies on dual-arm cup stacking. huggingface.co/datasets/HITd… #PhysicalAI #Robotics

Post attachment
harpreet avatar
harpreet@DataScienceHarp·Nov 17

a gaussian splat scene of a playground, with a tornado spinning through it. PhysStudio is a collection of physics-based animation projects for 3D Gaussian Splatting scenes, from solvers to rendered demos. it covers lightning, hurricanes, lava, snow, rain, wind, ice melting, and a robot pouring demo, and builds on NVIDIA Warp 1.6. run the solvers and demos on your own splat scenes with each project's run guide. github.com/zju3dv/PhysStudio #ComputerVision #AI

Post attachment
harpreet avatar
harpreet@DataScienceHarp·Nov 16

Zip-NeRF's official JAX pipeline leaves approximately 6-7x of training speed on the table. fused-zipnerf replaces it with hand-written CUDA kernels and analytic gradients, without tiny-cuda-nn or PyTorch autograd. quality matches the official implementation, and rendering is approximately 10x faster. it needs an NVIDIA GPU and installs as an extension to the NeRFICG framework. github.com/fhahlbohm/fused-z… #PhysicalAI #ComputerVision

Post attachment
harpreet avatar
harpreet@DataScienceHarp·Nov 16

if you have 10,000 teleop episodes and no plan to watch any of them welcome, this account is for you if you came for prompts, wrong door

harpreet avatar
harpreet@DataScienceHarp·Nov 15

8 of the 50 episodes in the original OpenArm cable tie recording are gone here, and the repo says which ones. OpenArm Cable Tie Threading (openarm_umeow LeRobot v3.0) removes episodes 13, 14, 15, 40, 46, 47, 48, and 49. the remaining 42 episodes (47,515 frames) are re-indexed 0 to 41 in their original order. state and action are unchanged, and videos are re-encoded in AV1. use it to train or evaluate bimanual threading policies with LeRobot. huggingface.co/datasets/etha… #PhysicalAI #Robotics

Post attachment
harpreet avatar
harpreet@DataScienceHarp·Nov 14

remove an object from a Gaussian splat and the patch can look right from the camera you edited in and wrong from the side. ComfyUI-SpEdE finds the object in every capture camera with SAM 3.1, deletes its Gaussians, and fits the fill to all cameras at once. it ships 33 ComfyUI nodes and 10 workflows, runs on your own GPU with no cloud calls, and stops when a gate check fails. you can also extract the object to its own PLY or replace it. github.com/Burgstall-labs/Co… #ComputerVision #AI

Post attachment
harpreet avatar
harpreet@DataScienceHarp·Nov 14

a driving model can predict a good trajectory for the wrong reason. KITScenes-LongTail is built to score that mismatch. each rare driving scenario has six-camera surround video, a 5 s future trajectory of 25 waypoints at 5 Hz, and expert reasoning traces in English, Spanish, and Chinese. i parsed it into FiftyOne so you can play all six cameras in sync and rank clips by Qwen3-VL embedding uniqueness to find the rarest. huggingface.co/datasets/Voxe… #PhysicalAI #AutonomousDriving

0:27
harpreet avatar
harpreet@DataScienceHarp·Nov 13

smell models are trained on small lab datasets, then asked to work in parks, gyms, and dining halls. in New York Smells, 7,000 smell-image pairs cover 3,500 distinct objects across 60 recording sessions, about 70x more objects than existing olfactory datasets. a Cyranose 320 e-nose with 32 sensors sits on a custom 3D-printed rig next to an iPhone and a RealSense depth sensor. i parsed it into FiftyOne so you can flip between the iPhone image, the depth overlay, and the olfaction heatmap of one sample. huggingface.co/datasets/Voxe… #PhysicalAI #ComputerVision

0:30
harpreet avatar
harpreet@DataScienceHarp·Nov 13

when there's no spoon, humans scoop with something else. ToolDex has scoop demonstrated with 7 distinct tools, stir with 6, and crush with 5, on egg, dough, sand, and sponge. each trial has 8 synchronized RGB-D cameras, and a large fraction ships with hand-optimized 6DoF object poses, per-frame masks, and hand trajectories. use it for policy learning, affordance discovery, and tool-use generalization studies. tooluse-dataset.github.io/ #PhysicalAI #Robotics

0:06
harpreet avatar
harpreet@DataScienceHarp·Nov 13

'shake the boba' comes with a motion description that reads 'move to the right and to the left, repeating this sequence 3 times'. MotIF-1K has 1,022 robot demonstrations across 13 task categories, each with video, trajectory overlays, task instructions, and motion descriptions. it was built to benchmark and fine-tune VLMs on motion-based success detection. i parsed it into FiftyOne so you can compare the raw video, trajectory overlay, and 16-keyframe storyboard for one demo. huggingface.co/datasets/Voxe… #PhysicalAI #Robotics

0:32
harpreet avatar
harpreet@DataScienceHarp·Nov 12

a dual-arm robot needs 72.5 to 86.1 seconds to drop loose change into a piggy bank. AgiBot World 2026 is real-world data from the AgiBot G2 dual-arm humanoid. this 10-episode subset is HG-DAgger collection, with 7 camera streams and per-frame reward, intervened, and succeed signals. i parsed it into FiftyOne so you can scrub the cameras next to the state and action vectors. huggingface.co/datasets/Voxe… #PhysicalAI #Robotics