🤖🤚 Give a video generation model a scene and a goal. Can it generate egocentric execution to get the job done?
We tested 110 real-world scenarios on 6 SoTA models to see how they perform.
Introducing Ego2Act, a goal-directed video generation benchmark 👇
🌐 ego2act.github.io
🧪 alphaxiv.org/abs/2610.01092
🤗 huggingface.co/datasets/ego2…
[🧵Thread]
Through my wonderful time at NVIDIA and Stanford, one question has kept me up at night:
Should the future of AI be monolithic or plural?
To answer this question, we’ve raised $110M to get started, backed by @MenloVentures, @MadronaVentures, @NVIDIA, @NEA, @KindredVentures,@Wing_VC, @aixventureshq, @Principal_VC, @Cisco, @Snowflake, among others.
Today, Carlos (@guestrin) and I are stoked to launch Noeri, a neolab to push the frontier of plural intelligence.
Noeri means “the plural of intelligence” or many intelligences.
The future of AI we root for is plural: a healthy ecosystem of many intelligences, in all sizes, strengths, and quirks.
Discovery, in particular, has always been plural: many curious minds, each devoted to their own question, often on roads less traveled.
Our path there is self-improving AI that:
-> learns continually on the job, closing the gap between what AI can do off the shelf and what the dynamic world demands
-> learns to use other AIs as tools and make new tools to overcome its own limitations
-> picks the right-sized model for your problem, at a fraction of the frontier cost
Then the power of intelligence, from the craftsmanship of making AI to ownership, can be in the hands of all.
@noeri_ai mission reflects who we are as the founding team. My career has been anything but linear, literally continual learning on the job. Carlos has devoted his career to building tools that empower people through AI.
We are all a bit contrarian, a bit adventurous, and above all, big believers in pluralism.
Join us to build this future together: noeri.ai/
Sora, robotics, whats next?
maybe just a short-term business move, or maybe platonic representation hypothesis, scaling law, hardware optimization, distillation, etc. turns out to be the way?
thus research should support foundation model maxxing / straight to application layer (agents, enterprises)?
🎉 Great work from the PAN 2 team on an open-weight world model built for real-world simulation, grounded in time, physics and space to predict how scenes evolve as actions unfold.
vLLM-Omni brings Day-0 support for PAN 2 text-to-video and image-to-video generation, making it available through an open inference stack.
Introducing PAN 2: an open foundation world model for physical intelligence.
A language model predicts the next word. PAN 2 predicts the next world: what happens after each action, grounded in time, physics and space.
Towards AI that simulates “what if?” before it acts.
More details: ifm.ai/pan/
Introducing PAN 2: an open foundation world model for physical intelligence.
A language model predicts the next word. PAN 2 predicts the next world: what happens after each action, grounded in time, physics and space.
Towards AI that simulates “what if?” before it acts.
More details: ifm.ai/pan/
Interactive Visual Gym: Evaluating Visual Interaction Models Through Simulation
Can a visual AI assistant’s real-time advice actually help someone finish a procedural task? Meet Interactive Visual Gym (IVG), an open simulation framework for evaluating and training visual AI assistants in closed loop. We recreate cooking, household, and phone tasks in a simulation world alongside simulated users with different skills and preferences. These users act, make mistakes, ask questions, and respond to guidance – allowing us to test how accurately, appropriately, and promptly an assistant supports them as the activity unfolds.
→ 14 assistants, 315 scenarios: the best performing model, Gemini-4-Argon, completes only 53.4% of tasks on time
→ Human preferences match our simulator scores ~80% of the time
→ Training in the simulator lifts a 4B open model’s success rate’ from 33.3% to 42.2% on unseen tasks
Explore the paper, code, and demos: sforce.co/4zwQdWU#EnterpriseAI#AgenticAI#FutureOfAI