Astra found a short proof that the variance of the passage time in first-passage percolation scales sublinearly in the distance (by a strict power, exponent 1-\epsilon).
This was an open problem for many years (decades) and mentioned by Kesten in 1993.
Recently it was proved by Basu-Sidoravicius-Sly (2023) in two dimensions in a rotationally invariance continuum FPP model. Their proof is about 200 pages in total, and @ahmedbourabee1 and I decided to lean formalize it to understand it better. Some of their arguments turned out to be not ready for formalization (or at least we couldn't formalize them) and so we launched some agents to try to find workarounds.
Eventually, after a lot of interaction with the agents, this led to the discovery of a new mechanism which makes the induction in the scales argument work - giving us a different and dramatically simpler argument - which is valid in every dimension as well as on the lattice (under some curvature and regularity assumptions on the limiting shape).
We think this is an instance of "AIs giving us new and better understanding of a previously proved theorem" and of human-AI interaction (as we tested, this is not N-shottable by Astra for small N).
Our "Bitter Lesson Paper" - h1 - is finally out on arXiv.
arxiv.org/abs/2610.35962
Here is an excellent account by @SkanderMoalla on how this research came about. TL;DR:
- we tried to study a (slightly) fancy idea to scale at test time (asymmetric self-play)
- each time we carefully tried to adjust and simplify baseline harnesses we washed out our initial gains
- we ended up with h1: a simple baseline formal proving harness that is SOTA (and solved 4 extra Erdos problems) - plus some valuable lessons learned.
(Of course, this was all done way before a lot of recent mathAI happenings; alas, we were > 30 days on hold on arXiv.)
After 30 days on hold, h1 is finally on arXiv! Here’s how it came to be and, more importantly, the takeaways.
It starts with a simple observation: some model capabilities can be unlocked at test time before being reinforced. E.g., “think step by step” → CoT → +RL = reasoning models.
We wanted to see if test-time self-play could be one of those, and help unlock hard problems. Lean is a good setting, since posing a statement to prove is enough to fully define a verifiable task.
The idea was to have a "challenger" look at the traces of a bunch of "prover" agents and isolate some of them to work on specializations and generalizations of the problem. The question was whether this would induce novel reasoning paths that would combine together to produce a solution to the original hard problem. Like test-time RL in AlphaProof but fully in-context.
It worked to some extent. But the gains vanished, or at least were no longer measurable at reasonable budgets, as we kept working on a simple recipe to scale the prover alone, which we also used as a baseline.
We then fortunately realized that this baseline was also better than most methods out there, and it led to h1!
Here are the takeaways, several of which connect to callouts shared before:
0. The asymmetric self-play idea we explored is not dead. Like "think step by step", it shows just enough signal at test-time that training for it could make it much more efficient. An efficient recipe for this is still an open question.
1. I can't say it better than @polynoamial: "As LLMs become more capable, benchmark performance is increasingly a function of test-time compute. In fact, we likely don't know what the capability ceiling is for modern LLMs because it's too expensive to measure. We should change LLM evaluations to account for that by measuring performance vs. tokens, cost, or time."
x.com/polynoamial/status/206…
2. An effective interface + a frontier model + test-time scaling should be the baseline for most new tasks. As @Michael_J_Black put it: "If you haven't tried using Astra (or whatever comes next) to solve your problem, then you have not done your homework."
x.com/Michael_J_Black/status…
3. It’s tempting to encode prior knowledge in a harness: most of the time, why would you start from scratch? This is ok as long as the harness doesn’t limit the search or learning that can happen on top of it. Unfortunately, it usually does: we tend to treat integrating prior knowledge into an algorithm as the endpoint rather than a starting point. I think this is the part of the bitter lesson we still haven’t fully digested: leveraging human knowledge and leveraging computation “need not run counter to each other, but in practice they tend to.” (@RichardSSutton)
This also applies to the self-play idea and h1. Both allow learning specifics at test-time, but they should also be seen as islands/initializations for further learning, however general they already are (test-time self-play as one capability among many that a multi-agent system can exhibit, and h1 as one harness/interface that an agent can deploy to solve a Lean problem).
Speaking of formal math specifically, I still see many papers submitted to ICLR that report an honest-looking pass@1 hiding an arbitrary choice on the compute axis. Others encode human-designed workflows that leave limited room for (in-context) learning from environment feedback, and may turn out to be local minima that are hard to escape if meta-optimized. My general recommendation is to compare scaling performance.
🤗: huggingface.co/papers/2610.3…
arXiv: arxiv.org/abs/2610.35962
Work with Ahmad Rammal, Niket Patel, Rémi Munos, and Julia Kempe (@Ahmad3Rammal, @niketnpatel, @MunosRemi, @KempeLab). Very grateful to have worked with this great team, and thanks to Vivien Cabannes, @arnal_charles, and @Amaury_Hayat for very helpful discussions and feedback!
My father's background in Stochastic Systems and Control Theory shines through the exposition. German book here:
tinyurl.com/wfanw22x
👌Kudos to my sister Vera Kempe for the amazing translation. My modest contribution was the introduction of Overleaf and Claude to the tool-kit of an 87-year old 😀. Beats sudoku as a pasttime!
This monograph:
- explains the field in a generally accessible yet mathematically rigorous way
- compares and contrasts SSN with GPT-like architectures
- explains the prospects for classical and neuromorphic AI to complement and extend each other
- uses relevant mechanisms of the human nervous system as reference point 🧵
💪Brilliant AI-vision paper by Volker Kempe out: "AI at a Turning Point..."
arxiv.org/abs/2610.39796
🧵 Spiking Neural Nets (SSN) and Neuromorphic AI as a low-energy hybrid complement to today's high-powered transformer architectures: energy-frugal SSN bring elements of self-organization into the methodological repertoire of AI: local learning instead of global error backpropagation, event-driven computing, and time as a carrier of information. 🧵
NYU's Center for Data Science (CDS) is looking for a new Chair!
apply.interfolio.com/195991
This is a wonderful leadership opportunity. CDS is one of the most exciting places to be - vibrant, innovative, full of extremely talented faculty and students. It is now part of the newly formed Courant Institute School, ready to shape Computing, Math and AI in NYC and beyond.
@NYUDataScience
Very proud of our recently graduated group member Nikos @nikostsilivis for winning the G-Research PhD Prize Award for *Best Thesis in Machine Learning*!!! Congratuations, Nikos!
@NYUDataScience
Check out our new Autoformalization benchmark, AutoformBench!
341 LEAN proving tasks drawn from diverse graduate- and research-level mathematical sources and formalized by human experts. Each task requires writing a substantial amount of auxiliary code, including lemmas, definitions, and more, placing the emphasis on models’ ability to build formal infrastructure - testing what's really required to automate proof checking:
🚨 AutoformBench: A Lean Formalization Benchmark for Advanced Mathematics
Our team at @AIatMeta is excited to release AutoformBench, a new formalization benchmark that pushes the Lean 4 coding capabilities of frontier models to their limits.
While SOTA models have become extremely competent at single-shot problem-solving, formalizing mathematics at scale and cost-effectively remains difficult. For instance, only about 42% of the results in openai/math could be formalized before release.
Furthermore, it is not clear and publicly known how much it costs to formalize big results, which is critical information if formalization is to become a part of the peer review process. This is the focus of AutoformBench.
The benchmark comprises 341 proving tasks drawn from diverse graduate- and research-level mathematical sources and formalized by human experts. Each task requires writing a substantial amount of auxiliary code, including lemmas, definitions, and more, placing the emphasis on models’ ability to build formal infrastructure rather than on mathematical cleverness alone.
The result is a demanding test: even the strongest models require tens of millions of tokens to complete 50% of the tasks (roughly $10 to $100 at current API rates).
Paper (arXiv coming soon): github.com/facebookresearch/…
Code: github.com/facebookresearch/…
Joint work by myself, Ahmad Rammal (@Ahmad3Rammal) and Vivien Cabannes, together with Niket Patel (@niketnpatel), Fabian Gloeckle (@FabianGloeckle), Bartosz Piotrowski, Amaury Hayat (@Amaury_Hayat), Remi Munos (@MunosRemi) and Julia Kempe (@KempeLab) from @AIatMeta, @NYUDataScience, and Ecole des Ponts. (1/6)
Check out our new paper on getting a SOTA harness for Automated Theorem Proving!
The "bitterest" part of our lesson, perhaps, was that we wanted to demonstrate benefits of asymmetric self-play for harnesses, and each time we could improve the baseline harness to be better. This is how we ended up with h1, a great simple baseline harness, as our bitter lesson :)
The Bitter Lesson for harness design and formal math!
Instead of hand-designing how an AI should prove (decompose, plan, assign agents), what if we just give a frontier model good Lean feedback and let it keep working?
We built h1.
TL;DR:
→ SOTA on formal theorem proving benchmarks
→ can solve open conjectures
→ matches/beats hand-designed workflows
→ one long attempt can beat many retries
[1/7]
Many thanks to our speakers for the uplifting discussions on the value and potential of small-scale research and models, and to everyone for a packed house! See you next year! 🌱
Au contraire.
C'est une nouvelle ère qui s'ouvre pour les mathématiques.
Une ère où la démonstration formelle est largement automatisée et où l'accent sera reporté sur le développement de nouveaux concepts, nouvelles abstractions, nouvelles définitions, et nouvelles conjectures.
L'invention du bateau a réduit l'importance de la nage, mais a permis la découverte de nouvelles terres.
Note: Lattice based crypto (like LWE) still stands unbroken, and there's still no genuinely new quantum algorithm... and surely not for lack of trying ...
World Modeling for Physics Workshop in Aspen:
To clarify: Applications to participate (deadline Oct 15) go through the Aspen Center for Physics
aspenphys.org/winter-confere…
Log in or create an account with ACP, then apply.
"The world is predictable in the right coordinates. The open question is whether a model can learn those coordinates on its own."
World model for Physics conference in Aspen coming winter 2027! Application deadline October 5 !
organized by @randall_balestr@KempeLab@ylecun and me 😀
This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help.
After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem.
Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?”
Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold.
Let us be specific:
- In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path.
arxiv.org/abs/2310.01169
- Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787
- On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps.
arxiv.org/abs/2608.10184
- On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415
- On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372
These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins:
“We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…”
Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem.
How is that consistent with advertising this example through the suggestion that there was “no existing solution path”?
Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement.
An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new.
A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches.
You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt.
Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.
1️⃣ Probability: Mathematicians worked with Muse Spark to answer a question about fitting random points onto the surface of a stretched sphere. For the setting studied, they proved a sharp cutoff between when an exact fit is likely and when it is unlikely.
Read the paper: ai.meta.com/research/publica…
i was diagnosed with thyroid cancer many years ago purely by chance, because i had to get blood work for my US green card application (thank you, USCIS!). i was in my mid-30’s and was reasonably healthy (or so i believed.) this was shocking and totally unpredictable news to me. thanks to the amazing team of clinicians at NYU Langone, i successfully went through a total thyroidectomy followed by radioactive iodine therapy. but, it was truly the most uncertain and confusing period of my life.
this is when i first realized that no one can opt out of healthcare. being sick isn’t what you choose to or choose not to. it is something that unpredictably happens to all of us at some point for one reason or another. this is why it is a healthcare system that we collectively build and use to support each other at the society level. because it must support everyone, the complexity of creating, maintaining and improving this system is a Herculean effort that calls for all the help in the world.
today, i am announcing @OrtetAI together with my dear co-founders, @keunwoochoi, @HenriDwyer, @elmanmansimov, @causalclaudia and Jeff, to lend our support to improving the health of each and every one of us. Ortet’s goal is to create a technological foundation that will enable health systems to deliver predictably better care to each and every patient. this will only be possible by working closely with all stakeholders in the broader ecosystem, which is enabled in part by Ortet’s close partnership with Thoreau.
Ortet is taking the very first step of a long journey ahead, and at the end of this journey, i strongly believe we would be able to say that Ortet had contributed to building better, more predictable healthcare and thus to improving health of each and every one of us.
see ortet.ai/ for more!