←
murat 🍥16,258 posts
murat 🍥 avatar

murat 🍥

@mayfer
◷ Joined March 2008▧ 1,394 Photos and videos
murat 🍥 avatar
murat 🍥@mayfer·Nov 11

for ex

Even the simplest weight edit, a deletion, can teach an LLM new skills! Meet Deletion Tuning (#NeurIPS2026): just zeroing <0.1% of weights matches full fine-tuning's accuracy at scale. It's thousands of times smaller and avoids forgetting, a step toward continual learning. 🧵👇

0:10
murat 🍥 avatar
murat 🍥@mayfer·Nov 11

this seems increasingly possible to me

murat 🍥@mayfer25 Aug 2025

continuous online learning likely requires abstractions to be very clean and deduped in the latent space. what if there was a loss function to encourage low rank representations to avoid having duplicates in the weights?

murat 🍥 avatar
murat 🍥@mayfer·Nov 2

reality check

This is simultaneously very cool but also a bit exaggerated. Reading the prompt you gave, the prompt veeery much hints that looping is the issue and to find neurons that fix this. And as you say (screenshot below), it couldn't otherwise improve math. So your project and website are great, but the tweet is a bit of a bait. But it worked, i read the full page including prompts 😄

Post attachment
murat 🍥 avatar
murat 🍥@mayfer·Oct 29

ai researchers writing evals for doing some tasks: Haha fuck yeah!!! Yes!! ai researchers realizing they have to write an eval for the meaning of life: Well this fucking sucks. What the fuck.

murat 🍥 avatar
murat 🍥@mayfer·Oct 27

now that we're interpreting monosemantic neurons it's time to bring this up again. fast memory recall & mechinterp are the same problem. let's figure out activation-triggered indexing. i think it's doable

murat 🍥@mayfer21 Dec 2025

imo e2e mechanistic interpretability has the highest chance of leading to e2e memory recall by providing activation-triggered indexing and search

murat 🍥 avatar
murat 🍥@mayfer·Oct 27

especially embedding math works SO MUCH BETTER than any embedding models' vectors. these kinds of operations simply devolve into junk normally. the vector space seems much more navigateable with meaning

cartoon - animation - comic = diagram i likeeee iiiit

Post attachment
murat 🍥 avatar
murat 🍥@mayfer·Oct 27

my "abstraction neurons as embeddings" method beats all SOTA embedding search methods on "word similarity" but loses very badly on sentence similarity and retrieval i'd say this is expected given it uses 16k neurons out of 4B but i'm really pleased how well it works for words

Post attachmentPost attachmentPost attachment

i built an embedding engine by composing vectors from common abstraction neuron activations (h/t @nickcammarata "already clean" essay) and it behaves more consistently especially when doing embedding math. less distracted by spelling/formatting and more focus on abstractions present it's not always better, but generally at least on par. i have not seen it breakdown into returning junk like regular embedding models do yet. i am gonna try running some vector search evals on it to see how it performs

Post attachment
Quentin avatar
⟳ murat 🍥 retweeted
Quentin@QuentinWach·Oct 26

I made an inverse design with BeamZ to solve MNIST.

0:20
murat 🍥 avatar
murat 🍥@mayfer·Oct 22

also should mention it seems to work a lot better than sparse autoencoder based embeddings i have tried in the past

murat 🍥 avatar
murat 🍥@mayfer·Oct 22

ok it loses out on benchmarks against SOTA embedding search models but the fact that it works at all is neat, and being qualitatively different is interesting enough to keep working on

murat 🍥 avatar
murat 🍥@mayfer·Oct 22

for clarity the results are from top 10k english words, which is usually how i test embedding ideas first

murat 🍥 avatar
murat 🍥@mayfer·Oct 22

i built an embedding engine by composing vectors from common abstraction neuron activations (h/t @nickcammarata "already clean" essay) and it behaves more consistently especially when doing embedding math. less distracted by spelling/formatting and more focus on abstractions present it's not always better, but generally at least on par. i have not seen it breakdown into returning junk like regular embedding models do yet. i am gonna try running some vector search evals on it to see how it performs

Post attachment
murat 🍥 avatar
murat 🍥@mayfer·Oct 21

reproduced very well. qwen 4b forgot facts but can still do reasoning (this is the 99% variant)

Post attachmentPost attachment

Tested this! I flattened every neuron's weakest activations Overall loss barely changes at first but the models get REAL bad at memorized text. So the weak tail does, as Neel expected, contain memorized things. Weaker activations seem to correspond to less generalizable content

Post attachment