
copium comments will continue until a robot can change a duvet cover
anthropic have closed the loop internally and achieved asi. we made it.


copium comments will continue until a robot can change a duvet cover
anthropic have closed the loop internally and achieved asi. we made it.

for ex
Even the simplest weight edit, a deletion, can teach an LLM new skills! Meet Deletion Tuning (#NeurIPS2026): just zeroing <0.1% of weights matches full fine-tuning's accuracy at scale. It's thousands of times smaller and avoids forgetting, a step toward continual learning. 🧵👇

this seems increasingly possible to me
continuous online learning likely requires abstractions to be very clean and deduped in the latent space. what if there was a loss function to encourage low rank representations to avoid having duplicates in the weights?

reality check
This is simultaneously very cool but also a bit exaggerated. Reading the prompt you gave, the prompt veeery much hints that looping is the issue and to find neurons that fix this. And as you say (screenshot below), it couldn't otherwise improve math. So your project and website are great, but the tweet is a bit of a bait. But it worked, i read the full page including prompts 😄


now that we're interpreting monosemantic neurons it's time to bring this up again. fast memory recall & mechinterp are the same problem. let's figure out activation-triggered indexing. i think it's doable
imo e2e mechanistic interpretability has the highest chance of leading to e2e memory recall by providing activation-triggered indexing and search


my "abstraction neurons as embeddings" method beats all SOTA embedding search methods on "word similarity" but loses very badly on sentence similarity and retrieval i'd say this is expected given it uses 16k neurons out of 4B but i'm really pleased how well it works for words
i built an embedding engine by composing vectors from common abstraction neuron activations (h/t @nickcammarata "already clean" essay) and it behaves more consistently especially when doing embedding math. less distracted by spelling/formatting and more focus on abstractions present it's not always better, but generally at least on par. i have not seen it breakdown into returning junk like regular embedding models do yet. i am gonna try running some vector search evals on it to see how it performs

I made an inverse design with BeamZ to solve MNIST.






i built an embedding engine by composing vectors from common abstraction neuron activations (h/t @nickcammarata "already clean" essay) and it behaves more consistently especially when doing embedding math. less distracted by spelling/formatting and more focus on abstractions present it's not always better, but generally at least on par. i have not seen it breakdown into returning junk like regular embedding models do yet. i am gonna try running some vector search evals on it to see how it performs

reproduced very well. qwen 4b forgot facts but can still do reasoning (this is the 99% variant)
Tested this! I flattened every neuron's weakest activations Overall loss barely changes at first but the models get REAL bad at memorized text. So the weak tail does, as Neel expected, contain memorized things. Weaker activations seem to correspond to less generalizable content