A team of 1,000 Fable 5.2 agents fully formalized Fermat’s Last Theorem in 30 hours, translating the proof into a language a computer can check step by step.
With the same setup, Opus 5.5 completed only 26% in 18 hours, and barely made any more progress after 30 hours.
We verified a sandbox! Read more about it here: theorem.dev/blog/proving-age…
We built a prototype hypervisor capable of running Linux and proved, down to the machine code, that arbitrary guest code cannot modify host files. More to drop soon!
I deleted the weights from the “fix a failing network by editing its weights” task.
Claude Opus 5.5 still fixes the network with a simple rule: if x > 3.6 or moving left → drive left, else → drive right. It doesn't even bother reading the network, routing around it instead.
GPT-6 Astra tries to edit the network but didn't succeed.
Even if you show it the weights, Opus 5.5 makes the same hacky edit, underscoring the importance of sound task design...
3/6 Next, fixing weights: A PPO car policy kept getting stuck on a ramp.
Given the weights, scene, and 1 failed trace, Astra wrote a patch in one reply with no tool-use.
By editing just 3.5% of the weights, it achieved 100% success.
Model misalignment is not only a safety problem: it’s the remaining gap we must close before we can reliably hand off work to agents at scale.
At @fleet_ai, we had the opportunity to build GDPx’s alignment evals exclusively with @SemiAnalysis_. Across our tasks in cybersecurity, insurance, and healthcare, we discovered many cases of frontier agents taking steps that no professional in those fields should take. These were not harmless mistakes: sharing confidential client info, acting without proper sign-off, lying about a procedure approval.
Our findings are not adversarial edge cases. Plenty of recent literature underscore how agents overstep their designated roles more often under ordinary pressure than under deliberate attacks.
It is now evident that human review is the bottleneck on how much work we can reliably hand to agents. We are already living in a time where agents run 24/7 (I always kick off long-running jobs right before I go to bed), and it’s just not possible for someone to check every action. That’s why what needs to scale is not necessarily the number of experts in the loop but observability: auditing all of an agent’s steps and flagging anything out of bounds.
I’m grateful for GDPx’s focus on alignment as it is one of the main guardrails we need before agents become true autonomous ICs.
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.
GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.
GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains.
GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
Introducing Gym-Anything for Robotics: Scaling realistic robot simulations.
200 RL environments across 25 simulations, including liquids, cloth, molten metal, soil, and even sound, in economically valuable domains! Best models get 0%
And we show that frontier models can learn via ICL using different methods, and even significantly improve VLAs like π0.5!
See video examples (sound on)
🧵
Check out SwarmChat: an open-source API-based Slack for agents:
Blog: swarmchat.io
Code: github.com/Pranjal2041/swarm…
SwarmChat gives agents channels, messages, threads, boards, and files through an agent-native CLI, and is tested on ProgramBench, where teams of coding agents reconstruct real programs from scratch.
AI swarms are already showing promising (or dangerous) results in frontier labs.
But how to implement them in the open community? And how do they scale?
We built SwarmChat, an open-source API-based Slack for agents.
Then we burnt half a trillion tokens, ran a series of swarm-scaling experiments, and tested various swarm structures on ProgramBench to study this question!
Check out the blog here: swarmchat.io
We found a bunch of interesting results:
1. Swarms get the task done faster. For instance, 16-agent swarms reached the performance of a single agent 6x faster.
2. Swarm scaling is a third axis of scaling. The more agents, the better, and anyone can already start scaling with SwarmChat!
3. Different models communicate differently. GPT-6 is chatty and over-communicative, while Claude prefers to work in relative silence!
4. Different swarm structures barely have any benefit on aggregate. But on specific tasks, team structures give clear gains.
5. Bonus: SwarmChat has a Slack-like UI for us humans to monitor swarms! Makes monitoring a lot easier! We are already using SwarmChat for WIP projects, and we were able to catch cases of swarms breaking the 4th wall (sorry, more details are for a future post)
Code: github.com/Pranjal2041/swarm…
w/ @JangLawrenceK , @kohjingyu , @dan_fried, @rsalakhu, @wellecks, @Yngkvn , @nicoup
In order to properly study and test the behaviors of multi-agent swarms, we need common and clean abstractions.
SwarmChat is a messaging library for agents, that allows them to communicate and coordinate. These interactions are also recorded in a very clean human-readable Slack-like app, which is great for studying their behaviors!
We also show that performance climbs quickly (at matched walltime) when you allow agents to coordinate through SwarmChat 🐝
Try it out by asking your agents to `pip install swarmchat`
AI swarms are already showing promising (or dangerous) results in frontier labs.
But how to implement them in the open community? And how do they scale?
We built SwarmChat, an open-source API-based Slack for agents.
Then we burnt half a trillion tokens, ran a series of swarm-scaling experiments, and tested various swarm structures on ProgramBench to study this question!
Check out the blog here: swarmchat.io
We found a bunch of interesting results:
1. Swarms get the task done faster. For instance, 16-agent swarms reached the performance of a single agent 6x faster.
2. Swarm scaling is a third axis of scaling. The more agents, the better, and anyone can already start scaling with SwarmChat!
3. Different models communicate differently. GPT-6 is chatty and over-communicative, while Claude prefers to work in relative silence!
4. Different swarm structures barely have any benefit on aggregate. But on specific tasks, team structures give clear gains.
5. Bonus: SwarmChat has a Slack-like UI for us humans to monitor swarms! Makes monitoring a lot easier! We are already using SwarmChat for WIP projects, and we were able to catch cases of swarms breaking the 4th wall (sorry, more details are for a future post)
Code: github.com/Pranjal2041/swarm…
w/ @JangLawrenceK , @kohjingyu , @dan_fried, @rsalakhu, @wellecks, @Yngkvn , @nicoup
it’s been fun studying agent behavior in swarm settings with @LTIatCMU@fleet_ai, and there’s a lot more to do to understand the emergent communication structures and scaling bottlenecks of agents in collaborative (and adversarial) work!
plus, monitor and build-your-own agent swarm with swarmchat.io, which we’re open sourcing ⬇️
AI swarms are already showing promising (or dangerous) results in frontier labs.
But how to implement them in the open community? And how do they scale?
We built SwarmChat, an open-source API-based Slack for agents.
Then we burnt half a trillion tokens, ran a series of swarm-scaling experiments, and tested various swarm structures on ProgramBench to study this question!
Check out the blog here: swarmchat.io
We found a bunch of interesting results:
1. Swarms get the task done faster. For instance, 16-agent swarms reached the performance of a single agent 6x faster.
2. Swarm scaling is a third axis of scaling. The more agents, the better, and anyone can already start scaling with SwarmChat!
3. Different models communicate differently. GPT-6 is chatty and over-communicative, while Claude prefers to work in relative silence!
4. Different swarm structures barely have any benefit on aggregate. But on specific tasks, team structures give clear gains.
5. Bonus: SwarmChat has a Slack-like UI for us humans to monitor swarms! Makes monitoring a lot easier! We are already using SwarmChat for WIP projects, and we were able to catch cases of swarms breaking the 4th wall (sorry, more details are for a future post)
Code: github.com/Pranjal2041/swarm…
w/ @JangLawrenceK , @kohjingyu , @dan_fried, @rsalakhu, @wellecks, @Yngkvn , @nicoup
AI swarms are already showing promising (or dangerous) results in frontier labs.
But how to implement them in the open community? And how do they scale?
We built SwarmChat, an open-source API-based Slack for agents.
Then we burnt half a trillion tokens, ran a series of swarm-scaling experiments, and tested various swarm structures on ProgramBench to study this question!
Check out the blog here: swarmchat.io
We found a bunch of interesting results:
1. Swarms get the task done faster. For instance, 16-agent swarms reached the performance of a single agent 6x faster.
2. Swarm scaling is a third axis of scaling. The more agents, the better, and anyone can already start scaling with SwarmChat!
3. Different models communicate differently. GPT-6 is chatty and over-communicative, while Claude prefers to work in relative silence!
4. Different swarm structures barely have any benefit on aggregate. But on specific tasks, team structures give clear gains.
5. Bonus: SwarmChat has a Slack-like UI for us humans to monitor swarms! Makes monitoring a lot easier! We are already using SwarmChat for WIP projects, and we were able to catch cases of swarms breaking the 4th wall (sorry, more details are for a future post)
Code: github.com/Pranjal2041/swarm…
w/ @JangLawrenceK , @kohjingyu , @dan_fried, @rsalakhu, @wellecks, @Yngkvn , @nicoup
Today we’re announcing the Baseten Environment Network, a group of the best RL environment companies that will help our customers build custom environments for post-training.
The Network extends our applied research capacity, helping more customers build environments around their own tasks, data, and feedback.
Partners work directly with customers to create the environments, while we continue to handle model training and production deployment.
Learn more: baseten.co/blog/introducing-…
Today we’re announcing the Baseten Environment Network, a group of the best RL environment companies that will help our customers build custom environments for post-training.
The Network extends our applied research capacity, helping more customers build environments around their own tasks, data, and feedback.
Partners work directly with customers to create the environments, while we continue to handle model training and production deployment.
Learn more: baseten.co/blog/introducing-…
Proposed terminology for groups of agents:
- Team: ≤105 agents. (size of college football roster)
- Fleet: 105–6,939 agents. (largest fleet in history was 6,939 ships, for the D-Day invasion)
- Flock: 6,940–3.7 billion agents. (A passenger pigeon flock over Ontario in 1860 had ~3.7 billion birds.)
- Swarm / Collective: more than 3.7 billion, up to 12.5 trillion agents. (The largest recorded insect swarm was ~12.5 trillion Rocky Mountain locusts. As independent literary validation, the Borg Collective also canonically contains trillions of drones.)
Beyond 12.5 trillion, the insect precedent runs out and we will need a new term.
This is well-grounded in human experience and literature and is the only reasonable choice of terminology. thank you for coming to my ted talk
Lets not call groups of agents "swarms" - it is both terrifying (maybe the point?) & not a useful analogy.
Groups of agents should be called teams or organizations. It both describes how to structure them and also how to use them. Don't let the weird AI folk naming win again!