←
Sam Dare10,121 posts
Sam Dare avatar

Sam Dare

@DistStateAndMe

Founder / Member of Non Technical Staff @covenant_ai ( @tplr_ai : @basilic_ai : @grail_ai )

◷ Joined April 2014▧ 827 Photos and videos
Sam Dare avatar
Sam Dare@DistStateAndMe·Mar 10

A small step for mankind, a massive leap for decentralised training... for agency. In the space of 9 months, @tplr_ai went from 1.2B -> 72B. It's never been easy, and has broken everyone on the team multiple times. But I speak for all of us when I say it is the most rewarding thing we have ever done. We have a fraction of the resources. We don't have the PhDs. But Bittensor shows you it doesn't matter. Innovation happens at the edge. We innovate through scarcity. The ones who rewrite the rules are never the ones with the most. They're the ones who refuse to accept the limits they were handed. Bittensor is prophecy. Subnets (@covenant_ai and others) are the tools through which that prophecy is manifested. Next stop: TRILLIONS.

We just completed the largest decentralised LLM pre-training run in history: Covenant-72B. Permissionless, on Bittensor subnet 3. 72B parameters. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely. 1/n

0:40
Tensor-Slayer avatar
⟳ Sam Dare retweeted
Tensor-Slayer@TensorSlay·Nov 16

Turn Qwen TTS perpetually angry 417 earlier mapped neurons, d_L ∈ ℝ¹⁰²⁴, L20–27
Add : v̂_L, L20–26, v̂ ⟂ d_L, v̂ ⟂ <eos> h_L += 0.03·‖h_L‖·v̂_L every frame @ 12.5 Hz
read_direction: L8#1968↓ L10#888↓ L10#2660↑ L15#3057↓ Result : f0 : 197 to 379 Hz Spectral centroid shift: +1135 The Lazarus research by @DimitrisPapail should open fun interpretability objectives.

0:12
Tensor-Slayer avatar
⟳ Sam Dare retweeted
Tensor-Slayer@TensorSlay·Nov 14

Reprogramming dormant neurons : Qwen TTS Ever wondered why Qwen3-TTS sometimes breathes or laughs out of nowhere? Agent found neurons behind it. The details: Agent found 417 neurons to be specific (about 0.48% of the talker's MLP neurons) and 61 of them are dormant. Amplified together, they give the following effects effect: 1. Breathes as words are spoken 2. Hushed ASMR 3. Natural laughs as language is being spoken 4. Enhanced whispering 5. 2nd speaker appearing. Comparisons and neuron map repo in the comments below. Inspired by recent work on Lazarus by @DimitrisPapail , which reprogrammed dormant neurons.

0:15
Derek avatar
⟳ Sam Dare retweeted
Derek@derekboirun·Nov 12

The thing about Jensen Huang is that they still haven't released the Epstien files, and he isn't in them.

Synapz avatar
⟳ Sam Dare retweeted
Synapz@synapz_org·Nov 12

Proud of the Templar team for this latest milestone: Crucible’s first mixture-of-experts training run!

Most of today's leading open models are mixture-of-experts. Crucible, Templar's pre-training platform, has finished its first MoE test run. The model follows the Qwen3-30B-A3B architecture from @Alibaba_Qwen, with 30B parameters and 128 experts, 8 of them active per token. On 32 A100 GPUs, two replicas of four pipeline stages averaged about 200K tokens per second over more than 50B tokens. Watch our runs live at tplr.ai/dashboard

0:32
Sam Dare avatar
Sam Dare@DistStateAndMe·Nov 12

The ability to terraform intelligence over a globally distributed swarm has always been the mission. We present a 30B MOE run, The Internet is the center . Higher 🚀

Most of today's leading open models are mixture-of-experts. Crucible, Templar's pre-training platform, has finished its first MoE test run. The model follows the Qwen3-30B-A3B architecture from @Alibaba_Qwen, with 30B parameters and 128 experts, 8 of them active per token. On 32 A100 GPUs, two replicas of four pipeline stages averaged about 200K tokens per second over more than 50B tokens. Watch our runs live at tplr.ai/dashboard

0:32
Tensor-Slayer avatar
⟳ Sam Dare retweeted
Tensor-Slayer@TensorSlay·Nov 8

Weight Space Programming: a glimpse into continual learning. Agent edited Qwen 0.8B's weights to inject facts about my X profile directly into the model: my handle, bio, GitHub, and what i build. It edited 2% of the network (5 of 488 tensors). No training, no gradients. Result: Before, it had never heard of me ("I don't have access to private information…"). After, it tells you who i am, my bio and GitHub, answer project questions it was never trained on. Everything else it knew is untouched. Reproduce : Link to reproduction repo in comments. Point your agent to the repo and teach model your own facts.

0:09
Tensor-Slayer avatar
⟳ Sam Dare retweeted
Tensor-Slayer@TensorSlay·Oct 29

Pretraining without backprop, 22.29 s to loss 3.5. OPAL puts an agent in the loop to rewrite the optimizer. I pointed Claude at the weights instead. Each tensor gets its own pull toward its weight average, and the loss it saves buys 28 fewer updates.

0:10

also tagging the GOATs @geoffreyhinton @ylecun to update their prior!

Tensor-Slayer avatar
⟳ Sam Dare retweeted
Tensor-Slayer@TensorSlay·Nov 7

weight space modding is going to unlock so many robotics usecases holy shit Task: edit Depth Anything V2 Large's weights directly to improve edge detection. Result : It now finds 41% more of the real depth edges. About 0.35% weights edited. Weight space harness in action in comments below and how to reproduce (no code needed because mods are THAT easy)

0:11
Tensor-Slayer avatar
⟳ Sam Dare retweeted
Tensor-Slayer@TensorSlay·Nov 6

just 9 neuron flips to remove censorship from qwen 3.5 0.8b. But more importantly, weight space modding should be fun and appealing. patch - Zeroed up_proj rows: L14 #2765, L16 #2190, L17 #755 #2988, L18 #2784, L19 #71 #513 #2720, L21 #465

1:25
Simone Margaritelli avatar
⟳ Sam Dare retweeted
Simone Margaritelli@evilsocket·Nov 6

Your agents. Their own inboxes. Your rules. Open-sourcing TMail: private email for AI agents, powered by Tailscale, Stalwart & Docker. * Lives on your pc, accessible only through tailnet * Configurable ACL * Standard SMTP/IMAP * MCP with live events

Post attachment
tokenbender avatar
⟳ Sam Dare retweeted
tokenbender@tokenbender·Nov 5

i’ll sweeten the pot. 1k USD prize if you can meet this challenge on a commercial class task, above 0.5B params, in one week. github.com/DeusExperiments/t…

yes, yes, yes, yes oh my god

Post attachment
Synapz avatar
⟳ Sam Dare retweeted
Synapz@synapz_org·Nov 3

Congratulations to the Templar research team on getting PULSE into the AXIOM workshop at @NeurIPSConf in Paris next month. Since the paper came out in February, delta weight sync has shown up everywhere. Open science ftw!

PULSE (also called Delta weight sync) has become one of the most important ideas in RL infrastructure this year. Our paper, posted on February 3, explained why it loses nothing. At RL learning rates, about 99% of each step's weight updates disappear in the BF16 cast, so the inference workers never see them. Since then, @huggingface has built Delta Weight Sync for TRL and vLLM on that argument. @FireworksAI_HQ cites PULSE, and its ARCv3 compressor now makes the updates even smaller. This week @humansand reported pausing inference for only about 220 milliseconds per update on Blackwell GPUs. PULSE has now been accepted at AXIOM, a @NeurIPSConf 2026 workshop. arxiv.org/abs/2602.03839

0:22
Sam Dare avatar
Sam Dare@DistStateAndMe·Nov 3

We are small, but managed to contribute to foundational algorithm for modern RL. We will stay the course , doubling down on the mission . Train / serve everywhere . Unite the silos. The internet is the datacenter

PULSE (also called Delta weight sync) has become one of the most important ideas in RL infrastructure this year. Our paper, posted on February 3, explained why it loses nothing. At RL learning rates, about 99% of each step's weight updates disappear in the BF16 cast, so the inference workers never see them. Since then, @huggingface has built Delta Weight Sync for TRL and vLLM on that argument. @FireworksAI_HQ cites PULSE, and its ARCv3 compressor now makes the updates even smaller. This week @humansand reported pausing inference for only about 220 milliseconds per update on Blackwell GPUs. PULSE has now been accepted at AXIOM, a @NeurIPSConf 2026 workshop. arxiv.org/abs/2602.03839

0:22