←
Gordon Wetzstein294 posts
Gordon Wetzstein avatar

Gordon Wetzstein

@GordonWetzstein

Professor at Stanford University & Co-founder at Rhoda AI

◷ Joined January 2010▧ 109 Photos and videos
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·3h

Our new work asks: does a diffusion transformer need a separate decoder? DWD reads pixels directly from its final features while preserving the denoising trajectory. The surprising part: this separation also improves quality for direct pixel generators.

Latent diffusion needs no decoder😵 New paper: DWD, a linear head on DiT predicts pixels directly • Same or better quality on every tested latent model, less flicker in Wan video • Even pixel models improve: ImageNet FID 1.55 ➡️ 1.43 (pixel SOTA), photorealistic texture in T2I

1:17
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Nov 16

What should a flow model predict? Let it learn. Our new work, AnisoFlow, learns a direction-dependent prediction target while keeping the architecture, loss, and sampler unchanged. Better image quality and remarkably clean pixel-space generation—even in 8-bit.

What should a pixel flow model predict? Data, velocity, or others? New paper: AnisoFlow🔥 • A fully learnable anisotropic target • Theory that explains what it learns Results: • ImageNet: FID 1.90 → 1.55 over JiT, no REPA • Pixel T2I without patch artifacts, even in 8-bit

Post attachment
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Nov 10

Really exciting new work: pi-H3 uses data-free distillation to reach 2–4 sampling steps. Amazing video quality generated faster than you can watch!

Introducing pi-H3⚡, the fastest MiniMax H3 yet. 10 s of 768p video with audio generated in 9.2 s on a single B200. How we did it: Direct pixel output in 2 or 4 steps, no video VAE decoder. Blog: hanshengchen.com/blog/pi-h3/ App: huggingface.co/spaces/Lakoni… @StanfordAILab @LumaLabsAI

1:39
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Nov 3

What excites me most about mobile DVA is how little additional robot data it takes to learn robust mobility and manipulation. A powerful example of how web-scale video pretraining can make new robot capabilities easy to learn.

We taught a robot how to move around in only 15 minutes. That’s all the mobility data our Direct Video-Action-powered robot needed to accomplish a task that requires mobility and manipulation.

2:22
Jitendra MALIK avatar
⟳ Gordon Wetzstein retweeted
Jitendra MALIK@JitendraMalikCV·Nov 1

I'd like to advertise this paper in advance of CoRL 2026 where you have the opportunity to check it out. Yet another example of the power of human videos as a data source, and the cool thing is that one can augment real videos with synthesized variants, produced with the help of a video generatuon model. The real2sim2real paradigm rules!

Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real. Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts. CoRL’26 🚀 Website for details 👉 prism-real2sim2real.github.i… Work done at Amazon FAR w/ @zhenkirito123 @pabbeel @rocky_duan @JitendraMalikCV @carlo_sferrazza @ckarenliu @GuanyaShi @akanazawa

1:00
Rhoda AI avatar
⟳ Gordon Wetzstein retweeted
Rhoda AI@RhodaAI·Oct 29

Robots rarely fail on the clean runs. They fail on the edge cases. On Field Trips, @schrep and @gigascale watched our robot hit one live on camera and work through it on its own, with no human stepping in. That's what out DVA model is built for. 🎥 youtu.be/VnzfHhovGDk

0:30
Jagdeep Singh avatar
⟳ Gordon Wetzstein retweeted
Jagdeep Singh@startupjag·Oct 29

A robot that only works when everything goes to plan isn't autonomous. It just hasn't met the real world yet. During @schrep visit, our robot ended up with a bag upside down on the wrong side, a case it never trained on. It figured it out with the power of @RhodaAI's DVA model.

0:53
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 19

Excited about this new direction! Tokenizers typically optimize reconstruction (VAEs) or discrimination (DINO), creating a mismatch with downstream generation. RePR introduces a principled, scalable approach to learning representations specifically as tokenizers for generation.

1/ The tokenizer is a new scaling axis! We introduce an approach for diffusable representation learning that's principled, scalable, and works when DINO/LeJEPA collapse. We get joint tokenizer+diffusion scaling laws and it's compute-optimal vs raw diffusion w/ @drfeifei @GordonWetzstein @jcjohnss @ericryanchan @vincesitzmann @KyleSargentAI Thread, Paper, and Blogpost below

0:29
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 14

A secure grasp needs continuous feedback on contact forces. At Rhoda, we’re giving robots tactile sensing and teaching our models to predict what they’ll feel next—a step toward more efficient, reliable manipulation, even when vision is available.

What can a robot do without vision? We gave our robot a sense of touch and showed it can recognize and retrieve objects with its cameras blindfolded. Introducing Rhoda’s new tactile sensor—it feels pressure, texture, vibration, slip, and temperature, just like your fingertips.

2:53
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 8

🕹️ INSERT COIN. Spend your tokens wisely. Our Level-of-Token Diffusion gives image & video generation finer tokens where detail matters, coarser tokens elsewhere. Fewer tokens, faster generation. Enjoy the ’90s arcade vibes 👾 georgenakayama.github.io/lot…

1:06
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 6

Real-world robotics is where things get interesting: contacts are messy, objects move unexpectedly, grasps fail, and two hands often need to coordinate in very different ways. This is why I’m excited about the direction we’re taking at Rhoda: moving toward general robot models that don’t just execute learned motions, but perceive the consequences of their actions, adapt, and keep making progress toward the task.

Continuing our RhodaAtWork series with another look at Direct Video-Action (DVA) in action. This time, a single model handles multiple tasks across two home appliances: refrigerator EPS foam installation and washing machine drain hose routing.

0:50
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 6

Work led by @GeorgeNaka40190, together with @BrianCChao, @jan_on_x, @HanshengCh, @fedassa, Leonidas Guibas, @YarivLior. 📄 Paper: arxiv.org/abs/2610.05816 🌐 Project page: georgenakayama.github.io/lot…

Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 6

Generation adapts to the layout: fine tokens go where the prompt needs detail, and the speedup is set by the layout's token budget. Here LoT (right) uses 6,632 tokens instead of 14,336 and generates 2.5× faster than full resolution (left), and up to 4.6× when detail is more concentrated. 8/9

0:12
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 6

Going further, an agent can plan detail over time. Here it builds a rough 3D scene of a toy train in Blender. LoT turns the plan into a per-frame token layout (overlaid) and generates the video with 1.5× fewer tokens. 7/9

0:14
Gordon Wetzstein avatar
Gordon Wetzstein@GordonWetzstein·Oct 6

An agent can also author the layout directly. Here the agent paints an importance map on a canvas (left), marking where detail matters for “a girl riding a corgi”. LoT turns it into a token layout with 2.6× fewer tokens and generates the image (right). 6/9

0:16