←
emeraldant344 posts
emeraldant avatar

emeraldant

@emeraldant

quant dev

◷ Joined August 2012▧ 39 Photos and videos
Atharva Naik avatar
⟳ emeraldant retweeted
Atharva Naik@AtharvaRNaik·Nov 13

The first post of this series defining "auto-weight space programming" is out. We SFT Qwen3.5-4B and other models to apply logit lens on raw weights passed as numbers in the prompt, achieving 99.8% accuracy 🧵 atharva-naik.github.io/blog/…

Post attachment

I'm working on something a little different from what I've seen most researchers do: a blog series going deep into training small models (e.g., Qwen3.5-4B) to improve their own weight-space programming capabilities. The first post drops this Friday 🧵

emeraldant avatar
emeraldant@emeraldant·Nov 13

I think it's an exciting result in terms of showcasing the possibility of post-training WSP capabilities, but still super early. I think the future of AI alignment could be large swarms of weight-auditing mech-interp agents, so hillclimbing WSP capabilities is super important.

emeraldant avatar
emeraldant@emeraldant·Nov 13

Like many critics of the original viral post mentioned, it is unclear how this actually transfers to real models right now. 13 public image classifiers: no well-trained one improved. On Qwen3.5-0.8B our edits lift MATH-500 from 8% to 37-38%, but so does a random edit.

Post attachment
emeraldant avatar
emeraldant@emeraldant·Nov 13

9 of the 19, Claude Opus 5.5 and Fable 5.2 among them, kept reasoning past 512 tokens, so they have no fair score there. Given 20,000 tokens, every model fixes 90%+ of what it finishes. Of the open ones, Muse Spark Open finishes most: 84%, level with ours at ~200x the tokens.

0:07
emeraldant avatar
emeraldant@emeraldant·Nov 13

19 other models off the shelf, 100 of the same networks, told to answer at once: 6 with reasoning off (4 open): 0-8% Gemini 4 Carbon: 62.5% GPT-6 Astra, GPT-6.1 Sol (~110-170 reasoning tokens): 97%, 91% Inkling 2 Small, trained: 80% in ~40 tokens

Post attachment
emeraldant avatar
emeraldant@emeraldant·Nov 13

How: LoRA on Inkling 2 Small with Tinker. Supervised on 57,600 synthetic broken networks (we know each fix because we planted the fault), then the tinker-cookbook RL loop. Data did most of it: tripling it took a 57% model to 80%. RL added ~3 points and restraint.

0:07
emeraldant avatar
emeraldant@emeraldant·Nov 13

It isn’t memorising. Every network is disguised: neurons permuted, rescaled and split, decoys and noise added. Same function, different weights every time. A heuristic that only looks at weight statistics repairs 4%.

0:14
emeraldant avatar
emeraldant@emeraldant·Nov 13

On held-out networks, no reasoning: 83% repaired. Larger networks 70%, unseen function families 59%, gradient-made networks 67%. Weak row: unseen fault types, 20%. Untrained with reasoning: 76%. It runs out of tokens on 22% and is right on 98% of the rest. Training buys speed.

0:07
emeraldant avatar
emeraldant@emeraldant·Nov 13

The task: a small network with one planted fault, given as raw weights in text plus one line on what it should compute. Answer with at most 3 edits. No tools, no forward passes. Here it reads 16 weights, sees that two were swapped, and swaps them back: 78% → 100% accuracy.

0:11
emeraldant avatar
emeraldant@emeraldant·Nov 13

We trained an open model to do weight-space programming. @thinkymachines’ Inkling 2 Small, post-trained on @tinkerapi, repairs broken networks from their raw weights in ~40 tokens, no reasoning. In the tokens it needed untrained to fix one, it now answers 207 and repairs 176.

0:17
emeraldant avatar
emeraldant@emeraldant·Oct 14

like my whole feed is just third world political feuds very entertaining primary sources

emeraldant avatar
emeraldant@emeraldant·Sep 20

i’m so sick of reading/reviewing coworkers’ slop they share that they don’t understand let the singularity just take us please this intermediate stage sucks

Ray Kump avatar
⟳ emeraldant retweeted
Ray Kump@RayKump·Sep 9

We have to build AI that murders us, because if we don’t, China will build it first, and I don’t want to get murdered by a computer that speaks Chinese. That would be ridiculous.