←
Andrew Saxe760 posts
Andrew Saxe avatar

Andrew Saxe

@SaxeLab

Prof at @GatsbyUCL and @SWC_Neuro, trying to figure out how we learn. Bluesky: @SaxeLab Mastodon: @SaxeLab@sigmoid.social

◷ Joined November 2019▧ 94 Photos and videos
Andrew Saxe avatar
Andrew Saxe@SaxeLab·Feb 3

Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures? Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham arxiv.org/abs/2512.20607

SueYeon Chung avatar
⟳ Andrew Saxe retweeted
SueYeon Chung@s_y_chung·Nov 11

Our collaboration with @RainerFriedri12’s lab on olfactory neural manifolds in zebrafish is on the cover of @NatureNeuro’s November issue! 🐟🐟 Congrats to Bo, Rainer, Chi-Ning & the team. nature.com/articles/s41593-0… Thanks @Harvard @KempnerInst @FlatironInst @SimonsFdn for support!

Our November issue is now online! nature.com/neuro/volumes/29/…

Post attachment
Hasson Lab avatar
⟳ Andrew Saxe retweeted
Hasson Lab@HassonLab·Nov 5

What does it take to learn language from everyday experience? Our new study, published today in Science, offers a new way to answer this question.

Post attachment
summerfieldlab @summerfieldlab.bsky.social avatar
⟳ Andrew Saxe retweeted
summerfieldlab @summerfieldlab.bsky.social@summerfieldlab·Nov 5

new paper. During learning, why does it help to practice easy examples before hard ones? We provide a theory, based on deep learning models. TL;DR - curricula help with unlearning rather than learning. arxiv.org/abs/2611.04565. Led by @RubruckJirko and Alice Zhang, with @SaxeLab

Arthur Gretton avatar
⟳ Andrew Saxe retweeted
Arthur Gretton@ArthurGretton·Nov 3

🚨November 9th🚨 Just one week left to apply! PhDs at @GatsbyUCL with full funding regardless of nationality. Includes ✨new✨ Jump Trading studentships in AI with potential application to finance, eligible for employment opportunities with Jump Trading during PhD.

‼️ One week left to apply to our PhD programmes! 🧑‍💻 Gatsby Unit PhD programme ucl.ac.uk/life-sciences/gats… 🧠 GCNU-SWC joint PhD programme ucl.ac.uk/life-sciences/gats… 💰 Fully-funded 4-year programme ⏰ Apply by 17:00 GMT on Mon 9 Nov 2026

Dr. Chunyu Ann Duan avatar
⟳ Andrew Saxe retweeted
Dr. Chunyu Ann Duan@AnnDuan2·Nov 3

New preprint from the lab! Dissociable norepinephrine and dopamine representations during strategic decision-making and reward learning: biorxiv.org/content/10.64898… When reward learning and strategy updating can be disentangled, we found drastically different NE and DA dynamics!

Blake Richards avatar
⟳ Andrew Saxe retweeted
Blake Richards@tyrell_turing·Oct 30

1/4) We're hiring in my lab, **please share**! 🚀 There are two postdoc opportunities in cog neuro and AI. This is for a new project supported by a Schmidt Polymath Award. This project aims to build an engaging video game to collect data for neurofoundation models. 🧠🤖

Hua-Dong Xiong avatar
⟳ Andrew Saxe retweeted
Hua-Dong Xiong@HuaDongXiong·Oct 6

Why do humans and LLMs both show working-memory limitations? They may face the same computational trade-off: shared representations enable efficient learning and generalization, but also create interference. I’ll be presenting this work at COLM Oct 6 (today!), Poster #32, 4:30PM

6:30
Krzakala Florent avatar
⟳ Andrew Saxe retweeted
Krzakala Florent@KrzakalaF·Oct 3

This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help. After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem. Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?” Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold. Let us be specific: - In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path. arxiv.org/abs/2310.01169 - Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787 - On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps. arxiv.org/abs/2608.10184 - On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415 - On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372 These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins: “We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…” Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem. How is that consistent with advertising this example through the suggestion that there was “no existing solution path”? Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement. An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new. A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches. You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt. Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.

1️⃣ Probability: Mathematicians worked with Muse Spark to answer a question about fitting random points onto the surface of a stretched sphere. For the setting studied, they proved a sharp cutoff between when an exact fit is likely and when it is unlikely. Read the paper: ai.meta.com/research/publica…

Post attachment
Surya Ganguli avatar
⟳ Andrew Saxe retweeted
Surya Ganguli@SuryaGanguli·Oct 3

The difference between industry PR and rigorous academic discussion. Worth a read.

This is outrageous framing. Quite frankly, given the current tensions around AI and mathematics, this kind of announcement does not help. After the recent Buckmaster/OpenAI dispute over scientific credit and provenance, the last thing we need is more promotional PR language that blurs the history of a research problem. Meta presents its work by asking: “Can AI contribute when a problem is genuinely open and without an existing solution path?” Seriously?! For the ellipsoid-fitting example, this is misleading and insulting. There was an established research programme, substantial rigorous progress, and by the time of their announcement, publicly available proofs of the exact threshold. Let us be specific: - In 2023, Antoine Maillard and Dmitriy Kunisky gave a statistical-physics derivation of the conjectured transition at n/d² = 1/4, using the replica method. Their analysis also predicted the geometry of the solutions and suggested directions for rigorous proofs. This was not a proof, but it was a detailed and successful theoretical picture, not an absence of a path. arxiv.org/abs/2310.01169 - Afonso Bandeira and Antoine Maillard then rigorously established the sharp threshold for approximate fitting, using methods inspired by the statistical physics of disordered systems, using actually spin glass mathematical technics. Their preprint appeared in 2023 and was published in the Electronic Journal of Probability in 2025. Their Gaussian-equivalence framework provided a foundation for further work (including many of mine!). At this point, the "exact" fitting and removing the spectral restriction were the outstanding gaps, but these are technical problems, it was not about not discovering the threshold from scratch. arxiv.org/abs/2310.05787 - On 10 August 2026, Theodor Misiakiewicz and Garrett Wen posted a proof of the exact Gaussian threshold, explicitly closing those two gaps. arxiv.org/abs/2608.10184 - On 12 August, Sofia de la Cerda, Aaron Potechin, Madhur Tulsiani and Jeff Xu posted another proof, establishing both sides of the transition up to a vanishing multiplicative factor. arxiv.org/abs/2608.12415 - On 27 August, Frederic Koehler and Youngtak Sohn posted a broader universality theorem that includes the Gaussian threshold. arxiv.org/abs/2608.27372 These were not researchers merely “on the verge” of solving the problem. THE PROOFS WERE ALREADY PUBLIC. Meta's announcement comes in October, and Section 3 of their paper begins: “We use the groundbreaking approximate-fitting theorem of Bandeira and Maillard…” Dear META: Those are your words. This is not simply a reference in the bibliography: your feasibility proof explicitly starts from their theorem. How is that consistent with advertising this example through the suggestion that there was “no existing solution path”? Yes, your social post mentions independent solutions, and your paper and blog identify the August results and describe your work as independently developed. That does not answer the objection to the framing. A generic acknowledgement of parallel work does not make clear that the exact threshold already had publicly available proofs before this announcement. An independently developed AI-assisted proof, released after other proofs, should be presented as exactly that. Readers should not have to reconstruct the chronology to understand what is, and is not, new. A new proof can contain valuable ideas. But a new proof and a newly solved problem are not the same claim. Nor does a problem being open when a project begins mean it is still open when the marketing campaign launches. You say the aim is to “empower researchers”? Nothing more “empowering” than seeing years of collective research turned into the backdrop for a cheap marketing stunt. Correct the announcement! Give the scientific history the same prominence as the product. Give Antoine, Afonso, Theo and all the other researchers involved the public credit they deserve.

Victoria Bosch avatar
⟳ Andrew Saxe retweeted
Victoria Bosch@__init_self·Aug 28

Are brains and artificial neural networks converging onto universal representations? There is a seductive idea making the rounds in NeuroAI / ML: train systems well enough, and they'll converge on the same representation of reality (i.e. unique world model). We have Thoughts™

Steven scholte avatar
⟳ Andrew Saxe retweeted
Steven scholte@hsteven9·Aug 27

New preprint: brain alignment in DNNs is almost complete within the first 3% of training, while classification performance is still low. The features involved in classification are largely independent of those that predict the brain (@_niklasmueller , @IrisGroen , @marcelge )

Post attachment
Liu Ziyin avatar
⟳ Andrew Saxe retweeted
Liu Ziyin@LiuZiyin10·Aug 17

Why do all neural models exhibit phenomena of sudden learning (and scaling laws)? In our most recent preprint, we showed that this is due to permutation symmetry. At a small initialization, permutation symmetry reduces the training dynamics to depend only on the architecture-specific structure matrix A. arxiv.org/abs/2608.13335