𝗢𝗻𝗲 𝗺𝗲𝗺𝗼𝗿𝘆 𝗰𝗮𝗻’𝘁 𝗿𝘂𝗹𝗲 𝘁𝗵𝗲𝗺 𝗮𝗹𝗹.
We present 𝗟𝗼𝗚𝗲𝗥, a new 𝗵𝘆𝗯𝗿𝗶𝗱 𝗺𝗲𝗺𝗼𝗿𝘆 architecture for long-context geometric reconstruction.
LoGeR enables stable reconstruction over up to 𝟭𝟬𝗸 𝗳𝗿𝗮𝗺𝗲𝘀 / 𝗸𝗶𝗹𝗼𝗺𝗲𝘁𝗲𝗿 𝘀𝗰𝗮𝗹𝗲, with 𝗹𝗶𝗻𝗲𝗮𝗿-𝘁𝗶𝗺𝗲 𝘀𝗰𝗮𝗹𝗶𝗻𝗴 in sequence length, 𝗳𝘂𝗹𝗹𝘆 𝗳𝗲𝗲𝗱𝗳𝗼𝗿𝘄𝗮𝗿𝗱 inference, and 𝗻𝗼 𝗽𝗼𝘀𝘁-𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻.
Yet it matches or surpasses strong optimization-based pipelines. (1/5)
@GoogleDeepMind@Berkeley_AI
how should multimodal models encode visual input?
→ pretrained encoder?
→ encoder-free?
→ before the LLM?
published models bundle these choices together. we trained 200+ models from scratch to see which of these choices matters
only one of them does
🧵 [1/n]
When an agent meets a program: 𝗣𝗹𝗮𝘆 𝗶𝘁. 𝗥𝗲𝗯𝘂𝗶𝗹𝗱 𝗶𝘁.
𝗣𝗿𝗼𝗴𝗿𝗮𝗺𝗟𝗮𝗯 gives agents 1,261 real-world programs to practice on.
𝗧𝗿𝗮𝗶𝗻 𝗼𝗻𝗲 𝗮𝗴𝗲𝗻𝘁, 𝗼𝗿 𝗮 𝘄𝗵𝗼𝗹𝗲 𝘁𝗲𝗮𝗺.
🧵0/n
We trained an agent with native visual world modeling ability.
It imagines counterfactual outcomes to plan ahead. SFT teaches it to imagine; agentic RL makes it even more adaptable.
We’ve released everything: agima.net
We trained an imaginative agent! ✨
Agima: an omni agent that plans by imagining futures.
𝗦𝗙𝗧: Directly generalizes to harder tasks! 𝟮%→𝟰𝟬%, beats GPT-5.6 Sol (𝟮𝟬%).
𝗥𝗟: New ways to imagine emerge! Imagines a second plan only if the first one fails (𝟯𝟮%→𝟭𝟬𝟬%).
Can diffusion models generate high-quality images without CFG?
Yes! We introduce ✨Correction Training for Diffusion Models✨, which significantly improves sample quality without relying on classifier-free guidance.
Project page: sunovivid.github.io/correcti…
🧵 (1/n)
Now autoregressive video models can remember. 🧠
Introducing MosaiChunk: persistent visual memory for long-horizon autoregressive video generation🧩.
Instead of remembering everything, it remembers the right things. ✍️
1/6
Generation 𝗖𝗔𝗡 improve understanding!
With the right recipe, we show that training on generation data can help understanding tasks.
Introducing OmniTaskonomy: our year-long study of when and how generation helps understanding.
Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real.
Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts.
CoRL’26 🚀 Website for details 👉 prism-real2sim2real.github.i…
Work done at Amazon FAR w/ @zhenkirito123@pabbeel@rocky_duan@JitendraMalikCV@carlo_sferrazza@ckarenliu@GuanyaShi@akanazawa
Our paper RATs has been selected as CoRL spotlight!
We apply "sleep time scaling" to agentic robotics learning. The agent aquires skills during play and then uses these skills at test time. Check out our project page at : playful-rats.github.io/@junyi42
Excited to release our new work: Playful Agentic Robot Learning w/ @junyi42!
Instead of relying on test-time scaling, we found that a "pretraining stage" through curious play enables robots to discover general skills before any tasks are assigned.
🌐 playful-rats.github.io
Our paper VisGym has been accepted to NeurIPS 2026!
It’s been crazy to see how the field has developed since we have released this paper. Hope that VisGym can continue to serve as a diagnostic playground to identify failure pattern for new frontier multimodal agents, and serve for the development of this field.
We release VisGym: 17 environments for training & evaluating VLM interaction! ✨
Interaction is still the bottleneck of VLMs:
Success rates: Gemini-3 26.0% | GPT-5 12.6%.
Kudos to co-lead @zwcolin@junyi42 and the amazing team! @berkeley_ai
I didn't get what GPT-6 meant for robotics until I actually tried it.
GPT-6 Astra just does physical ICL out of the box.
we drop a recording of a human doing a novel task into 𝗰𝗼𝗱𝗲𝘅 app. Prompt it to drive a robot arm the same way.
It just works on the first pass!
Open-source video generation is now faster than playback without compromising quality.
Introducing Video Delta Net (VDN): hybrid attention for live text-to-video with near-lossless quality.
VDN accelerates Minimax-H3 by 75 - 90 x, generating 14 seconds of 768p video in 11 seconds on 8× NVIDIA B200 GPUs.
Checkpoints + training/inference code + Technical Blog ⬇️
(1/6)
"The latent world representation is a program, not a vector"
Finding the right representation to be inverse-graphics is really important. Really cool work!
Today, we’re introducing [schema]: a harness reaching 99% RHAE with Opus 4.8 + Fable 5 and 95.35% with GPT-5.6 Sol on ARC-AGI-3 Public set.
[schema] makes an LLM think like a physicist. 🧵
The term "continual learning" has become overloaded if you see it as an ML problem.
One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much.
One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions.
These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget.
In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting.
Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory.
Key ideas:
- Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates
- EMA anchor weights that provide a moving reference for stability
- Chunk-by-chunk inference for long 3D/4D observation streams
We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140.
We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation.
- Homepage: fast-spatial-memory.github.i…
- Paper: arxiv.org/abs/2604.07350
- Code: github.com/Mars-tin/fast-spa…
- Models: huggingface.co/marstin/fast-…
This work is co-led with @Xueyang_Y, contributed by @zhnhoy5@YuncongYY, and advised by @SLED_AI@gan_chuang.