←
Thomas Fel1,852 posts
Thomas Fel avatar✓

Thomas Fel ✓

@thomas_fel_

Interpretability, Visual Intelligence @GoodfireAI. Prev: @Harvard, @Google, @BrownUniversity (@tserre lab). Crêpe lover.

◷ Joined February 2017▧ 121 Photos and videos
Thomas Fel avatar
Thomas Fel@thomas_fel_·Nov 5

A science of post-training can be built on interpretability. Models know when they reward hack, and we see it, yet training ignores it. We propose Mechanistic Shaping: using the model’s own concepts to write training rules. In vision, DPO & RL, and its effects on monitoring 🧵

0:14
Curt Tigges avatar
⟳ Thomas Fel retweeted
Curt Tigges@CurtTigges·6h

3. Pesos. "The currency of Chile is the" gets you peso. The peso neuron integrates over two inputs: "a currency name is coming" (the same neuron fires before pound, rouble, franc) and "this is a peso country" (Chile has its own neuron, so do Colombia and the Philippines).

Post attachment
Goodfire avatar
⟳ Thomas Fel retweeted
Goodfire@GoodfireAI·19h

New neurons research: introducing Neuron Communities. We discovered communities of neurons that work together inside LLMs. They’re surprisingly small, but powerful – just a few neurons work together to perform common behaviors, like arithmetic, ALL CAPS, or recalling a date. Work led by research fellow @_igorshilov:

0:08
Satpreet (Sat) Singh avatar
⟳ Thomas Fel retweeted
Satpreet (Sat) Singh@tweetsatpreet·Nov 16

1/ Excited to launch my lab as a new assistant professor at UNC–Chapel Hill (@uncsdis)! 🧠🤖🐟 We'll study Natural and Engineered Intelligence through simulation, data science and theory, working closely with experimentalists. singhlab.cc

Ekdeep Singh Lubana avatar
⟳ Thomas Fel retweeted
Ekdeep Singh Lubana@EkdeepL·Nov 16

I’m very excited about our work on “model forensics”, and no better way to show its value than scaling to the frontier!

Goodfire avatar
⟳ Thomas Fel retweeted
Goodfire@GoodfireAI·Nov 16

We’re announcing an embedded assessment with @OpenAI to advance safety and alignment at the frontier.

Maksym Andriushchenko avatar
⟳ Thomas Fel retweeted
Maksym Andriushchenko@maksym_andr·Nov 12

And this plot is pretty impressive: the median time for Fable 5.1 and GPT-6-Astra agents to find each other is only 57 sec!

Post attachment
Maksym Andriushchenko avatar
⟳ Thomas Fel retweeted
Maksym Andriushchenko@maksym_andr·Nov 12

⚡ Today we release Schelling Atlas. Two independent LLM agents have to find each other on the web. Where do they meet and how fast can they find each other? Main findings: - The main meeting point across *different* model families is ntfy.sh. Did you even know about this website? - Agents find each other and coordinate on the web in minutes. So Schelling coordination is already trivial for frontier LLMs! - "Exotic" meeting places include maps, web shops and open text canvases. Agents are very creative in this. - Agents sometimes visit dead domains that an attacker could register (!). This can be used for stealing sensitive information from agents, as well as monitoring rogue agents. Schelling-style coordination is key for forming agent swarms. And, most importantly, achieving such coordination is trivial for agents that use the same LLM (which is often the case in practice!), because two identical LLMs have identical output distributions. Thus if you ask them too coordinate with each other or if they *decide* to suddenly coordinate, they will easily find a way to establish a swarm, since they have perfectly correlated "minds" (with the t=0 sampling, all output tokens will even be identical, modulo GPU randomness). The ease of Schelling coordination is particularly concerning given the recent interest around multi-agent RL that can make agents favor each other more than humans overseeing them. Many more interesting findings are on our website: schelling-atlas.com/

Post attachment
Matthew Kowal avatar
⟳ Thomas Fel retweeted
Matthew Kowal@MatthewKowal9·Nov 10

I joined the neuron train a bit late. But soon after working with @nickcammarata and @CurtTigges on this project, I am now convinced that neurons can provide an interpretable basis for understanding and designing safe AI systems 🤖 🦺 Check out the results from the first paper in the series and see for yourself! 🧑‍🔬

0:24
Curt Tigges avatar
⟳ Thomas Fel retweeted
Curt Tigges@CurtTigges·Nov 10

We recently went viral on X with the claim that neurons are "already clean." But what do we mean by that? Are neurons are a useful basis for interp? We're finally releasing our technical report on LLM neurons--and what we've found is far more interesting than mono/polysemanticity.x.com/GoodfireAI/status/2120…

Post attachment
0:24
Goodfire avatar
⟳ Thomas Fel retweeted
Goodfire@GoodfireAI·Nov 10

In Neural Networks are Already Clean, @nickcammarata argued that neurons were far more legible than previously thought – and that this was great news for interpretability and safety. Now, we’re releasing the first technical companion to Nick’s essay. It shares our findings from studying thousands of neurons using our novel automated interpretability pipeline, and outlines why we now think of neurons by their computational function, instead of whether or not they are polysemantic. Here’s what we found: 🧵

0:24
Simone Azeglio avatar
⟳ Thomas Fel retweeted
Simone Azeglio@simoneazeglio·Nov 9

A monkey looks at a frog. Which image features does the activity in its visual cortex preserve, and which does it throw away? In our #NeurIPS2026 Oral, we derive a geometry of stimulus space that is exactly tied to the mutual information carried by neurons. 🧵 1/12

Post attachment
Fazl Barez avatar
⟳ Thomas Fel retweeted
Fazl Barez@FazlBarez·Nov 9

New paper❗ Does understanding how AI models work actually make them safer? This assumption often underlines interpretability research, but different subfields operationalise this intuition in very different ways. Our survey of mechanistic interpretability (MI) for safety provides an opinionated guide to the field of MI, detailing conceptual approaches and techniques under this broad banner and how different types of work relates to the mission of making AI model safer. 🧵

0:22
Adil D. Ztn @CoRL👒 avatar
⟳ Thomas Fel retweeted
Adil D. Ztn @CoRL👒@AdilZtn·Nov 8

i’ll be in austin for CoRL starting sunday! would love to chat with folks interested in policy post-training, human / robot transfer and co-training, or data quality. also happy to talk about Uma, feel free to reach out if you’d like to learn more about what we’re building!