Imagine giving an agent a computer and asking it to improve itself at games. There are no installed games; it must write its own games to practice.
What does it do?
Strong models compose their own self-improvement loops. We give some initial measurements of this capability
1/n
At some point it will be possible to “just ask” AIs to self-improve. Clearly it’s important to know when this might happen, and to what degree.
In this paper, we try to measure where things stand with current models, on games. How do they behave when instructed to self-improve?
Imagine giving an agent a computer and asking it to improve itself at games. There are no installed games; it must write its own games to practice.
What does it do?
Strong models compose their own self-improvement loops. We give some initial measurements of this capability
1/n
Many open questions remain!
- What about open-ended domains where models have worse priors?
- Are there important strategies that aren't human-recognizable?
- How much scaffolding is still needed? We give the model an explicit gameplay tool; future models may not need it.
(8/n)
This fits with a basic trend in deep learning: More capable models need less explicit structure.
Here, that structure is the learning procedure itself.
There are also safety implications. If agents can compose self-improvement loops without a given recipe, it's important to understand when and how.
(7/n)
Self-improvement often overfits.
In contrast, our learned files generalize to different game engines, horizons, and 2D v. 3D tasks.
There's some nuance: giving the teacher some information about eval settings helps it generate relevant tasks; too much causes overfitting.
(6/n)
Examples of teacher behaviors:
- A/B testing (playing games with/without notes)
- Procedurally generating games
- Building tools like BFS solvers for filtering difficulty
- Creating train/test splits for itself
Check out our website to explore! ssundaram21.github.io/self-o…
(5/n)
How? Strong self-improvers use human-recognizable learning strategies.
They spend more rollouts on intermediate-difficulty games, and their behaviors align more closely to self-regulated learning, a framework from educational psychology.
(4/n)
With no recipe and only self-generated games, Opus 5 rivals GEPA, a prescribed optimizer that hill-climbs on *real in-distribution* games.
This capability is also improving over time. Across families, recently released models self-improve while older generations do not.
(3/n)
🌐 ssundaram21.github.io/self-o…
We put a *teacher* agent in a sandboxed harness and give it three things:
1. Goal: Improve a *player* agent (itself) at unknown puzzle games by manipulating its context.
2. Tools: Read/Write (file editing), Bash, and a Gameplay tool for observing the player attempt games.
3. A budget for text updates and gameplays.
Then we let it run!
(2/n)
Awesome work as always by @SophieLWang w/ @_AmilDravid, with very neat and intuitive insights about how mid-training data affects downstream reasoning!
Can “chicken” make a base model reason? 🐔
Yes!
Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too.
blog: sophielwang.com/cues
X-rays are everywhere in medicine, but extracting general, quantitative anatomical information from them remains remarkably difficult.
Announcing FleXray: an open-source, open-weight model for zero-shot anatomical segmentation across the human body—from different regions, viewing angles, and acquisition settings.
FleXray learns from large-scale synthetic supervision generated from 3D anatomy, then transfers directly to real clinical X-rays.
Try it now on your own X-rays in our in-browser demo!
🧵 1/N — project page, demo, weights, code & paper below ↓
Repeated sampling is the default way to scale LLM reasoning at test time. But token level noise often produces many near duplicate attempts that follow the same high level idea.
🧵 How can we cover more of the solution space without sacrificing throughput?
Can we tell whether data domains cooperate or compete during pretraining?
Adding code to the mix makes models better at math, while some other combinations hurt each other. We call this data synergy. Turns out you can incorporate data synergy into scaling laws and estimate it 🧵