Excited to share Gigapixel: a high-throughput driving simulator for scalable self-play training of E2E driving models: montrealrobotics.ca/gigapixe…. To make E2E training feasible, we propose self-play DAgger, which trains robust policies in ~3000x fewer steps than self-play RL, achieving SOTA on closed-loop driving benchmarks without human trajectory supervision!
New blog post - Chasing the Karpathy Peak (and Falling Off Slop Cliff): Confessions from Slopaholics Anonymous
My story of becoming a slopaholic, who’s most at risk, and how to avoid it (or recover)
thomasjiralerspong.substack.…
The most revealing part of this conversation between Jon Stewart and Jacob Coxon wasn't Coxon's warning about A.I. doom, but his description of A.I. utopia, and Stewart's incredulous response. I wrote about it — and Francis Fukuyama's "Last Man," and A.I. easily solving the problems that bedeviled my father's field of mathematics for decades — here: nytimes.com/2026/10/18/opini…
We’re bringing the magic of Waymo to Toronto! 🇨🇦
In the coming days, you’ll begin seeing Waymo’s all-electric vehicles driving through Toronto’s streets as we lay the groundwork for our ride-hailing service.
Our goal is to begin serving riders in 2028.
Sign up for future updates: waymo.com/waymo-in-canada
ScenarioControl (ECCV 2026) enables end-to-end controllable generative simulation 🚗✨for autonomous driving, allowing users to generate diverse traffic scenarios with fine-grained actor control while preserving realistic scene dynamics.
We have released the code, and we’re looking forward to seeing what the community builds with it!
🔗 Check out our project page for the paper & code:
princeton-computational-imag…
👏 Thanks to the first authors @Lili_Gao_ , @Yanbo_Xu_ , @WilliamKoc61157
A super long overdue (3+ years?) post on scaling laws.
Compute is expensive. Scaling laws are a way to help us reason about the optimal compute allocation between data and model size before committing to a large run.
The post covers what scaling laws predict, how compute-optimal allocation works, why Kaplan et al. and Chinchilla disagree, and how data limits + fitting details make extrapolation tricky.
lilianweng.github.io/posts/2…
New Paper: Human-like Autonomy Emerges from Self-Play and a Pinch of Human Data.
We trained self-play RL on 60 years of simulation on 1 GPU in ~15 hours. Regularizing with 30 minutes of demonstration data produces much more human-like driving policies!
We’ve just released the #Alpamayo Chain-of-Causation (CoC) Autolabeling Pipeline — a feature that has been highly requested by the community!
The pipeline automatically derives:
🔹 Meta-actions: high-level categorical descriptions of ego motion
🔹 Chain-of-causation labels: causal links between scene factors and the ego vehicle’s intended behavior
Autolabeling pipeline: github.com/NVlabs/alpamayo-c…
Learn more about the Alpamayo open platform: huggingface.co/blog/drmapavo…
We’re excited to see what the community builds with it, and we hope this tool will help accelerate research in the rapidly growing area of #reasoning models for #Physical#AI.
@NVIDIADRIVE@NVIDIAAI
Excited to introduce #TruckDrive 🚛 at #CVPR2026: a new long-range driving dataset built specifically for long-range truck autonomy, where safe braking and anticipatory planning demand perception hundreds of meters ahead, far beyond existing robotaxi datasets.
📦 TruckDrive includes:
🔹 475K samples, with 165K densely annotated frames
🔹 Benchmarks for end-to-end driving, tracking, planning, depth estimation, and up to 1,000m for 2D detection and 400m for 3D detection 📏🎯
🛰️ A purpose-built long-range sensor suite:
🔸 7 long-range FMCW LiDARs (range + radial velocity)
🔸 3 high-res short-range LiDARs
🔸 11× 8MP surround cameras for short and long-range📷
🔸 10× 4D FMCW radars 📡
⚠️ Key finding: current state-of-the-art models break down at long range
📉 with 31% to 99% drops on 3D perception tasks beyond 150m. TruckDrive exposes a long-range generalization gap that current architectures and training signals are not closing yet - a benchmark for the next generation of long-range highway autonomy research 🚚
🔗 Project and Data: torc-ai.github.io/TruckDrive…
Fun work together with @torc_robotics led by Filippo Ghilotti, Edoardo Palladin, Samuel Brucker, Adam Sigal, and Mario Bijelic.
Are we done with object detection? What about tiny objects beyond 200 meters? 🔎
Telescope 🔭 addresses long-range perception by explicitly tackling extreme scale imbalance ⚖️ in images. It hinges on a learnable hyperbolic foveation transform from a low-resolution image, magnifying distant regions 🔍 while compressing nearby ones - effectively normalizing object scales with minimal computational overhead. Objects are detected in the transformed (Riemannian) space using a novel bounding box parameterization and are then mapped back to the original image.
Project: light.princeton.edu/telescop…
New preprint led by @saeedrmd! “Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Traffic."
Drawing on 200+ papers, it offers a snapshot of where the field stands and outlines promising directions ahead.
ScenarioControl 🚗🛣️ - Scenario Generation from a single Dashcam Image 📸 or Text Prompt 💬!! Excited to introduce a new vision-language control mechanism for learned driving scenario generation. Given a single dashcam image or a scene prompt or an image, we generate a full scene layout 🧩, temporally consistent rollouts, including map 🗺️, agents 🚗, and ego video🛣️
ScenarioControl enables direct, fine-grained control over layout and traffic while preserving realism. It operates in a vectorized latent space with a new cross-global control mechanism to fuse vision-language inputs with scene structure while preserving realism. Interfaces seamlessly with generative video models!
Project: light.princeton.edu/Scenario…
Super fun project by Lili Gao, @Yanbo_Xu_ , William Koch, Samuele Ruffino, @Luke22R , Behdad Chalaki, Dmitriy Rivkin, Julian Ost, @rogg1111, Mario Bijelic.
Reasoning VLAs can think. They just can't think fast. Until now.
Introducing FlashDrive⚡
🚀 716 ms → 159 ms on RTX PRO 6000 (up to 5.7×)
✅ Zero accuracy loss
FlashDrive = streaming inference + DFlash speculative reasoning + ParoQuant W4A8
Real-time reasoning for autonomous driving is here!
z-lab.ai/projects/flashdrive…
Chop the gradients ✂️! We found that truncating decoder gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉.
Project: light.princeton.edu/chopgrad…
Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.