←
Cole Garry1,984 posts
Cole Garry avatar

Cole Garry

@ColeGarry2

Comms @RunwayML | Pressure is a privilege

◷ Joined September 2018▧ 44 Photos and videos
Runway Labs avatar
⟳ Cole Garry retweeted
Runway Labs@runwayml_labs·Nov 17

Introducing Diffusion-UI. Interactive software where every pixel is generated by a real-time model. Today, we're sharing a new research application that lets generated interfaces use structured data, maintain states and generate controls. Here’s an early look:

0:11
Cole Garry avatar
Cole Garry@ColeGarry2·Nov 12

"My agent can X" "my agent can Y" but when is the last time your agent asked you how you're doing?

MTS avatar
⟳ Cole Garry retweeted
MTS@MTSlive·Nov 11

Runway CEO @agermanidis says China is ahead of the US in understanding the power of world models, arguing that open-weight AI will be critical to controlling the brains of future robots: "This is very early for the field of generalist robotics, of models for robotics that can generalize across a wide range of different tasks." "It's the best way for us to learn about what are all the use cases and all the ways in which different labs and academic groups, companies are gonna apply those models." "I think China is further ahead than the US in terms of really understanding how powerful world models can be, and making sure that the West is also competitive on that front is really critical." "The brains of the robots are very critical. As robotics grows massively over the coming years, it's gonna be really important to have control over the models that run on top of those robots. And so we see open weight strategies being very important piece of that." @runwayml

1:16
NVIDIA Robotics avatar
⟳ Cole Garry retweeted
NVIDIA Robotics@NVIDIARobotics·Nov 11

Congrats to @runwayml on the open weight release of Praxis-1 27B world action model. 🎉. Developers can adapt it to their own robots and tasks, running it across NVIDIA GPUs from data centers to local AI deployments on NVIDIA Jetson on the robot. 👉 Download the NVFP4 checkpoints from @huggingface: huggingface.co/runwayml/prax… 👉 Get started with the Jetson AI Lab tutorial: jetson-ai-lab.com/models/pra… 👉 Post Train with NVIDIA NeMo Automodel: github.com/NVIDIA-NeMo/Autom…

Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.

Post attachment
Jacob Loewenstein avatar
⟳ Cole Garry retweeted
Jacob Loewenstein@spatialjlo·Nov 11

#1 world action model in robolab just dropped and its open source and even runs on a 5090. Huge huge huge 🤖

Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.

Post attachment
Anastasis Germanidis avatar
⟳ Cole Garry retweeted
Anastasis Germanidis@agermanidis·Nov 11

Praxis-1, our first world action model, is now available for anyone to use. It is currently the leading open-weights robotics model on the RoboLab leaderboard and outperforms every other world action model on the benchmark. More than that, it’s versatile and easy to fine-tune for new hardware and tasks. We’ve tested it on 9 different robot embodiments so far, in simulation and on real hardware with multiple partners, and we’ve seen it reach good performance on a new embodiment with under 30 minutes of demonstrations. Weights, code and fine-tuning recipes are on GitHub and Hugging Face. It can run on a single RTX 5090 with NVFP4. We started our robotics work about a year ago with the thesis that the same video pretraining we do for our creative models can be massively useful for building systems that interact with the physical world, and the benefits of transfer are now becoming clear. When I first talked to people in the field, there was a lot of skepticism about whether video models would be useful in robotics at all, and then about whether you could use them as policies. From first principles, it makes sense that if you can predict the future well in pixels, extracting an action representation should be the easy part. Reality caught up with first principles. One of my favorite things in our technical report is the idea of self-rollouts. Because the model’s performance is bottlenecked by its visual imagination, you can quickly get an intuition for whether it will perform a new task well by rolling out the policy in the model’s imagination. That can speed up evaluation, and it opens a path to doing reinforcement learning in the model’s imagination. There’s been a lot of discussion recently about frontier LLMs being the bitter-pilled way to solve robotics. But scaling performance from unsupervised video is an even better example of the Bitter Lesson than scaling from text. Praxis-1 is more than 10x faster and 50x cheaper than GPT-6 Astra on the manipulation tasks we tested. Frontier LLMs will keep improving rapidly on the Pareto frontier of cost and performance, but so will world models. There are also great ways of combining the strengths of the two. And we are much farther from saturating multimodal data than we are text. Thanks to the mighty Runway Robotics team for making this happen, and to our partners for being so generous with their time helping us test the model across a wide range of hardware and tasks.

Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.

Post attachment
reactor avatar
⟳ Cole Garry retweeted
reactor@reactorworld·Nov 11

Introducing Praxis-1, a new 27B MoE world action model from @runwayml, and Praxis-1 Fast (73ms median) - an optimized checkpoint from our research team. We’re proud to ship Day 0 support with DROID and YAM endpoints live today on the Reactor API.

0:29
vLLM avatar
⟳ Cole Garry retweeted
vLLM@vllm_project·Nov 11

🎉 Day-0 support for @runwayml’s Praxis-1 in vLLM-Omni! From watching the world to acting in it: Praxis-1 brings large-scale video pretraining to robot control, with open weights under Apache 2.0 and adaptation across robot embodiments. vLLM-Omni takes camera observations, robot state and a task instruction to generate action chunks. The integration supports offline inference and OpenPI / Positronic serving, with session history and reset handling. Qualified on a single @NVIDIA H100 80GB. Thanks to the Runway and @Inferact teams for bringing this together.

Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.

Post attachment
LeRobot avatar
⟳ Cole Garry retweeted
LeRobot@LeRobotHF·Nov 11

Praxis-1: an open-weight world action model for real robots 🤖 New from @runwayml and now in LeRobot. Praxis-1 learns how objects behave and how tasks unfold from large-scale video, which gives it a head start on a new robot. One checkpoint adapts with light fine-tuning or LoRA: fine-tuned on Google's RT-1 robot, which it never saw in training, it reaches 80.0% on SimplerEnv. Weights, inference and fine-tuning code are open under Apache 2.0, commercial use included, and it runs on NVIDIA from H100 to Jetson Thor. Model, blog and docs in the comments 👇

0:30
Runway avatar
⟳ Cole Garry retweeted
Runway@runwayml·Nov 11

Today, we’re making Praxis-1 available to everyone. Our state-of-the-art world action model turns video pretraining into control for real robots, and it ships with open weights under Apache 2.0, ready to run and fine-tune on your own hardware.

Post attachment
Cole Garry avatar
Cole Garry@ColeGarry2·Nov 11

We've said for a couple of years now that large-scale video pretraining is the best path to building robotics models. Today we prove that with Praxis-1, our state-of-the-art open weight world action model. Available to everyone now! runway.com/research/praxis-1…

OpenAI Developers avatar
⟳ Cole Garry retweeted
OpenAI Developers@OpenAIDevs·Nov 10

With the @runwayml plugin in Codex and ChatGPT, you can see how a character from your prototype moves, try a different opening shot for a trailer, or turn a product photo into a video ad. Anthony Maggio shares how Runway’s marketing team is using ChatGPT Image to produce more assets each week.

1:15
luba yudasina avatar
⟳ Cole Garry retweeted
luba yudasina@LubaYudasina·Nov 6

OpenAI's Sora was the best gift @runwayml ever got, according to its co-CEO. Anastasis Germanidis told me the 2024 launch set off "an existential crisis for maybe an hour or two." "It was the best gift that we've ever received" "We need to figure out how to build a better model than Sora in the coming three months" He told me the whole company rallied and shipped a model better than Sora within a few months. Runway is now a $5.3B company doing $200M in annual recurring revenue.

1:27

New pod drop!! @RunwayML just turned 8 years and added $100M+ in ARR last quarter alone. I met Anastasis Germanidis (@agermanidis), co-CEO of Runway, who went from art school to building one of the most important video and world models of our generation. We talk about: - The Nietzsche-inspired game he tried to build at 15 - “We accidentally created one of the best models for robotics” - 30 magic tools in 30 days - Sora launch: a 2-hour existential crisis → a better model in 3 months - Proof of work > credentials - Running “a perfect simulation” of his co-CEO Cristóbal (@c_valenzuelab) - “I’m always becoming someone else” and so much more! Timestamps (00:00) Trailer (01:21) Anastasis Germanidis, Co-Founder & Co-CEO, Runway (02:27) Accidentally building one of the best robotics models (07:18) How video models train robots and self-driving cars (10:47) What a world model actually is (16:19) Why folding clothes is still unsolved for AI (17:37) Why AGI can never be reached (22:03) A game built on Nietzsche's eternal recurrence (30:35) Starting an AI company from art school (39:47) Why staying in stealth is counterproductive (42:03) 30 magic tools in 30 days (55:55) Sora's launch and an hour of existential crisis (57:02) Racing to beat Sora in three months (1:01:56) Solaris and software generated in real time (1:05:15) Why AI labs pay for fake versions of Amazon (1:09:52) Why Sora struggled as a consumer app (1:16:30) Hiring on proof of work (1:19:44) Becoming co-CEO with Cristóbal Valenzuela (@c_valenzuelab) (1:30:56) What he's still becoming (1:32:52) Outro

1:32:56
Runway avatar
⟳ Cole Garry retweeted
Runway@runwayml·Nov 6

Your performance data shows what’s working days before the next round of creative is ready. Runway Ads Lab is a hands-on workshop in San Francisco to close that gap. Performance, growth and brand marketers will build together in Runway Ads. You’ll take real results from Meta, Google and TikTok and turn them into new on-brand creative, and you approve every ad before it goes live. You’ll test more ideas in one afternoon than most teams get through in a week. After the workshop, stay for drinks with the Runway Ads team and other Bay Area marketers. San Francisco · November 19 Space is limited and registration requires approval. Request a spot: luma.com/runway-r1bt

Post attachment