
GEN-1.5 assembling an engine housing: precisely placing two clamshell pieces onto the engine and lining them up with the screw holes, ... 100 times in a row. With our friends at @FoundryRobotics

Co-founder @GeneralistAI general intelligence born from the physical world ✗DMs → email

GEN-1.5 assembling an engine housing: precisely placing two clamshell pieces onto the engine and lining them up with the screw holes, ... 100 times in a row. With our friends at @FoundryRobotics

Not all chips are fried. Read more about GEN-1.5 and sign up for the waitlist in the comments below.

if you look closely at the reliability test later in the video, you’ll see a few examples of “physical reasoning” -- it tries something, it doesn’t fit (because physics), so it backtracks, switches gears, then does something entirely different to reach the goal. in this case, it’s taking out the stuck puck, making space elsewhere, and putting it back in. things like this (… a sort of physical problem-solving with step-by-step “chain-of-actions”) should just work with models out of the box.
How many pucks could a puck bot pack if a puck bot could pack pucks? 🏒 12 hockey pucks is a tight fit in this container – watch the model fix the arrangement when pucks get stuck or misplaced. GEN-1.5 doesn't just puck around. Read more about it in our blog post and sign up for the waitlist in the comments below.

Designed a new website for Generalist. generalistai.com

GEN-1.5 puts together a kit of cosmetics, then puts together another kit, then another kit, then another... We're expanding access to GEN-1.5 Sign up to get in line generalistai.com/waitlist

We're expanding access to GEN-1.5 Sign up to get in line generalistai.com/waitlist

2 years ago, I personally spent dozens of hours trying to get this task to work. It was our first “too hard” task. Now it’s just another day here.
GEN-1.5 packs a bottle into a box, then packs another bottle into a box, then another bottle, then another one, then another... Read more about GEN-1.5 in our blog post in the comments below.

Mesmerizing! Also, *exactly* 10y ago i was at a startup with exactly these arms and very similar grippers with exactly this camera placement. But models from 10y ago: cnn + lstm, no real pretraining. What i see now, from multiple startups, it definitely a big step up.
GEN-1.5 packs a bottle into a box, then packs another bottle into a box, then another bottle, then another one, then another... Read more about GEN-1.5 in our blog post in the comments below.

GEN-1.5 packs a bottle into a box, then packs another bottle into a box, then another bottle, then another one, then another... Read more about GEN-1.5 in our blog post in the comments below.

Feels like every week in robotics there’s a new ‘this is the GPT-3 moment for robotics 🤖’ announcement. We brought on @GeneralistAI CEO & Co-Founder @peteflorence on @GreylockVC Change Agents to dig into what’s going on at the frontier of robotics models. We covered the company’s latest Gen 1.5 model, few-shot learning, training robots on different embodiments, and the milestones towards a more generalized physical model. Timestamps: 00:51 The inception of Generalist 04:13 Long-term goal of the company 05:47 Parallels and differences between robotics and language models 12:05 Key differentiation in 1.5 Gen model 12:48 Robot vs banana 14:49 Emergent capabilities not explicitly trained for 17:13 Cross-embodiment and the importance of hands 22:26 Research vs working with customers 24:58 Beyond VLA vs world model 28:50 Future-looking milestones Some of the top takeaways: - One-shot and few-shot learning emerged without being trained for. Gen 1.5 can learn a new task from a single demonstration, and Pete compares it to the GPT-3 moment in language. In one example, a robot taught to sweep a cube into a bowl with a brush used a banana instead. Given a dustpan, it held the pan with one hand, swept with the other, then tipped the cube into the bowl. Neither behavior was explicitly trained, and Pete sees this as a signal of where the model's generalization is headed. -Generalizing to new hands remains a challenging problem for cross-embodiment. Physical hardware doesn’t stay static, and so cross-embodiment - the ability of a physical AI model to adapt to different hardware systems - is vital for success. -Customer deployments are a valuable source of research inputs. Generalist actively partners with their customers for feedback, which they use to inform their research and make real-world evaluations. -Generalist doesn't think in terms of "VLA vs. world model." Pete helped create early VLAs and has worked on world models, but he argues the goals matter more than the label, and the team is trained to think in a first-principled way when considering new research directions. Watch the full episode at the link in the comments. Thank you to @peteflorence for joining us!

post-training to hill-climb dexterity
Robot puts a ring on it GEN-1.5 puts an O-ring on a hydraulic rod gland, then takes the ring off, then puts the ring back on, then takes the ring off, then ring on, ring off, ring on... Read more about GEN-1.5 in our blog post in the comments below.

A simple example of physical prompt steerability: same environment, different prompts, different behaviors. Thanks @JagdeepBhatia8 for the suggestion. Read more about GEN-1.5 in our blog post in the comments below.
Physical prompting is an elegant idea! One request for the @GeneralistAI team: can we see an example of *physical prompt steerability*, where different prompts induce distinct behavior in the *same* environment? I'm curious whether GEN-1.5 is listening to the physical prompt or simply executing the most likely action sequence for the scene under its pretraining prior.

We've reduced the time it takes to go from physical prompt → robot behavior. The faster anyone can teach a robot to do something new, the easier it becomes to scale physical work. Read more about GEN-1.5 in our blog post in the comments below.

I'm usually wary about talk of "the ChatGPT moment for robots" because language is less complex than the physical world. After visiting @GeneralistAI, though, where I saw their robots perform impressive manipulation with just a prompt, I think you can see it happening in a limited way. (Thanks to @peteflorence and co for having me!) wired.com/story/generalist-a…

Good question thanks Ani. Here's a video that gives a sense of the answer here about the effect of prompting: - in the first part, the model is just babbling with no prompt in context - at the end, we add a prompt for taking money out of a pouch (generalizing to the wallet)
Very cool work, Pete! I'm curious if you've tried an extreme version of this: no in-context example at all; just place objects in front of the robot and see if it can infer the task. I suspect this will have non-trivial success rates, which would also allow you to figure out how much of the benefit is actually coming from the prompt (rather than pure zero-shot task inference + capability). I remember Russ showed something like this in his Stanford talk on LBMs last year: youtu.be/TN1M6vg4CsQ?si=-8J-…

We started planning yesterday’s @generalistai GEN-1.5 announcement about 3 weeks ago. Two weeks ago, already feeling whiplash from new results, I asked ChatGPT to create this image to share in our internal meme channel.