←
Tensor-Slayer11,515 posts
Tensor-Slayer avatar

Tensor-Slayer

@TensorSlay

llm psychosis 1.0 survivor circa sonnet 3.5

◷ Joined February 2010▧ 819 Photos and videos
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·19 Jul 2025

Exploring Direct Tensor Manipulation in Language Models: A Case Study in Binary-Level Model Enhancement: areu01or00.github.io/Tensor-…

Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·2h

Amazing work, this work on image models as well. Agent found a neuron community related to facial features inside SDXS-512, a one-step image model, and switched it on. The edited model starts generating the person instead of distant figures generated by the base model. The agent edited approx. 8.7% of the image model's feed-forward neurons as best as it can without breaking the model. @GoodfireAI's research gave agent a new lever: set an interpretability objective, form a hypothesis about the target model, and test it on the target neuron family. Details about neurons edited by agent below in the comments

0:16

New neurons research: introducing Neuron Communities. We discovered communities of neurons that work together inside LLMs. They’re surprisingly small, but powerful – just a few neurons work together to perform common behaviors, like arithmetic, ALL CAPS, or recalling a date. Work led by research fellow @_igorshilov:

0:08
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·2h

edited neurons : 2,000 neurons (8.7% of the UNet's FF) in SDXS-512's 16×16 layers. Switch them on in the base model → 94% of the full face edit on new seeds; 2,000 random ones do nothing. github.com/areu01or00/sdxs-5…

Andreas Kirsch 🇺🇦 avatar
⟳ Tensor-Slayer retweeted
Andreas Kirsch 🇺🇦@BlackHC·Nov 16

Really cool to see more results like that This makes sense intuitively. Models make mistakes when generating autoregressively and with noising they learn to auto-correct during pretraining Hypothesis: their own generations become less OOD (RL fixes a similar issue as well) (Also connections to information bottlenecks etc)

New paper w/ Yihe Dong, @DimitrisPapail & @ShivamGarg91462: Perturbing Inputs Improves Pretraining Distillation. In pretraining distillation, a small student LM learns to match a larger teacher’s next-token predictions on the training text. We change one thing: for the first part of training, both models read a perturbed version of the text (for example, with noise added to the token inputs), and the student matches the teacher’s predictions on that. Then we switch back to clean text. With the same steps and token budget, this beats standard distillation, and it also works well in data-constrained, multi-epoch settings. 🧵

Post attachment
Dimitris Papailiopoulos avatar
⟳ Tensor-Slayer retweeted
Dimitris Papailiopoulos@DimitrisPapail·Nov 16

tensor slayer slaying some tensors, crazy type of interventions on models by messing around with activations. Check it out.

Turn Qwen TTS perpetually angry 417 earlier mapped neurons, d_L ∈ ℝ¹⁰²⁴, L20–27
Add : v̂_L, L20–26, v̂ ⟂ d_L, v̂ ⟂ <eos> h_L += 0.03·‖h_L‖·v̂_L every frame @ 12.5 Hz
read_direction: L8#1968↓ L10#888↓ L10#2660↑ L15#3057↓ Result : f0 : 197 to 379 Hz Spectral centroid shift: +1135 The Lazarus research by @DimitrisPapail should open fun interpretability objectives.

0:12
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·Nov 16

Turn Qwen TTS perpetually angry 417 earlier mapped neurons, d_L ∈ ℝ¹⁰²⁴, L20–27
Add : v̂_L, L20–26, v̂ ⟂ d_L, v̂ ⟂ <eos> h_L += 0.03·‖h_L‖·v̂_L every frame @ 12.5 Hz
read_direction: L8#1968↓ L10#888↓ L10#2660↑ L15#3057↓ Result : f0 : 197 to 379 Hz Spectral centroid shift: +1135 The Lazarus research by @DimitrisPapail should open fun interpretability objectives.

0:12
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·Nov 15

Neuron space editing Qwen3-TTS Target : talker MLP neurons Second-voice trait, β3 (400 mapped + add to activation before down_proj): L23#1430 ↑ +5.287 L24#527 ↑ +1.674 L21#175 ↑ +0.315 L22#142 ↓ −2.010 L22#555 ↓ −0.220

0:15

Reprogramming dormant neurons : Qwen TTS Ever wondered why Qwen3-TTS sometimes breathes or laughs out of nowhere? Agent found neurons behind it. The details: Agent found 417 neurons to be specific (about 0.48% of the talker's MLP neurons) and 61 of them are dormant. Amplified together, they give the following effects effect: 1. Breathes as words are spoken 2. Hushed ASMR 3. Natural laughs as language is being spoken 4. Enhanced whispering 5. 2nd speaker appearing. Comparisons and neuron map repo in the comments below. Inspired by recent work on Lazarus by @DimitrisPapail , which reprogrammed dormant neurons.

0:15
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·Nov 15

Invoking Whispering words at will with Neuron Space Edits in Qwen TTS.

0:14

Reprogramming dormant neurons : Qwen TTS Ever wondered why Qwen3-TTS sometimes breathes or laughs out of nowhere? Agent found neurons behind it. The details: Agent found 417 neurons to be specific (about 0.48% of the talker's MLP neurons) and 61 of them are dormant. Amplified together, they give the following effects effect: 1. Breathes as words are spoken 2. Hushed ASMR 3. Natural laughs as language is being spoken 4. Enhanced whispering 5. 2nd speaker appearing. Comparisons and neuron map repo in the comments below. Inspired by recent work on Lazarus by @DimitrisPapail , which reprogrammed dormant neurons.

0:15
Sohan avatar
⟳ Tensor-Slayer retweeted
Sohan@HiSohan·Nov 14

Lobotomy on AI is coming soon to a LLM near you

Reprogramming dormant neurons : Qwen TTS Ever wondered why Qwen3-TTS sometimes breathes or laughs out of nowhere? Agent found neurons behind it. The details: Agent found 417 neurons to be specific (about 0.48% of the talker's MLP neurons) and 61 of them are dormant. Amplified together, they give the following effects effect: 1. Breathes as words are spoken 2. Hushed ASMR 3. Natural laughs as language is being spoken 4. Enhanced whispering 5. 2nd speaker appearing. Comparisons and neuron map repo in the comments below. Inspired by recent work on Lazarus by @DimitrisPapail , which reprogrammed dormant neurons.

0:15
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·Nov 14

Harness in action :

Given the Lazarus research, its obvious we need better tooling for observability and interpretability. I urge everyone to develop your own harnesses. One such harness in action of agent mapping speech pattern nuerons of Qwen TTS. SOUND ON : If you are a Tolkien nerd.

0:38
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·Nov 14

Given the Lazarus research, its obvious we need better tooling for observability and interpretability. I urge everyone to develop your own harnesses. One such harness in action of agent mapping speech pattern nuerons of Qwen TTS. SOUND ON : If you are a Tolkien nerd.

0:38

Reprogramming dormant neurons : Qwen TTS Ever wondered why Qwen3-TTS sometimes breathes or laughs out of nowhere? Agent found neurons behind it. The details: Agent found 417 neurons to be specific (about 0.48% of the talker's MLP neurons) and 61 of them are dormant. Amplified together, they give the following effects effect: 1. Breathes as words are spoken 2. Hushed ASMR 3. Natural laughs as language is being spoken 4. Enhanced whispering 5. 2nd speaker appearing. Comparisons and neuron map repo in the comments below. Inspired by recent work on Lazarus by @DimitrisPapail , which reprogrammed dormant neurons.

0:15
Chinmay avatar
⟳ Tensor-Slayer retweeted
Chinmay@ChinmayKak·Nov 14

very cool work! Tensor has been at it since last year, way before weight space programming was a thing. Please do check it out, and engage for much deserved reach!

Reprogramming dormant neurons : Qwen TTS Ever wondered why Qwen3-TTS sometimes breathes or laughs out of nowhere? Agent found neurons behind it. The details: Agent found 417 neurons to be specific (about 0.48% of the talker's MLP neurons) and 61 of them are dormant. Amplified together, they give the following effects effect: 1. Breathes as words are spoken 2. Hushed ASMR 3. Natural laughs as language is being spoken 4. Enhanced whispering 5. 2nd speaker appearing. Comparisons and neuron map repo in the comments below. Inspired by recent work on Lazarus by @DimitrisPapail , which reprogrammed dormant neurons.

0:15
Tensor-Slayer avatar
Tensor-Slayer@TensorSlay·Nov 14

Cc @DimitrisPapail would love to hear your comments on the approach Opus came up with for TTS target : Amplify active and dormant neurons to work with behaviours. I thought <Meta Optimiser> analogy for Opus here is perfect. It built itself a spectrogram tool to “see” the output and sort of reverse engineered and found responsible neurons, which i thought was quite amusing. Atleast in Neuron Space Editing context.