←
Samuel Vaiter916 posts
Samuel Vaiter avatar

Samuel Vaiter

@vaiter

@CNRS Researcher

◷ Joined September 2012▧ 236 Photos and videos
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Nov 17

Context: this is AI flagging of AI reviews without disclosure from reviewers. This is roughly a third of the reviews that I handle in my batch.

Samuel Vaiter avatar
Samuel Vaiter@vaiter·Nov 10

📢 Postdoc opening! 📢 @n_keriven and I are hiring a postdoc to work on the theoretical analysis of large-context Transformers (infinite-token limits, depth, looped Transformers, reasoning traces). 📍Location: Rennes (IRISA) or Nice (LJAD), France

Gabriel Peyré avatar
⟳ Samuel Vaiter retweeted
Gabriel Peyré@gabrielpeyre·Jun 16

The alpha version of my new book "Optimal Transport for Machine Learners" is out, with in particular an online version with interactive figures gpeyre.com/ot4ml/

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Jun 5

Of course you can always use complex step approximation if you don't like roundoff error and you have stable complex arithmetic :)

Complex step approximation is a numerical method to approximate the derivative from a single function evaluation using complex arithmetic. It is some kind of “poor man” automatic differentiation. nhigham.com/2020/10/06/what-…

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Jun 5

A quick remark: When you use centered finite difference instead of forward or backward difference, the truncation error decreases faster, but I never realized that the roundoff error grows before!

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·May 4

Now accepted at ICML 2026! Congrats to @MagamedTm

🧵 New preprint! "Towards Understanding Steering Strength" with @MagamedTm and D. Garreau Activation steering is a popular way to control LLM behavior at inference. But how much should you steer? We provide the first theoretical analysis of the steering strength α. 1/7

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Mar 9

TL;DR: Steering strength α has precise, predictable effects on LLM outputs. Understanding these laws may help design better steering strategies. Paper: arxiv.org/abs/2602.02712 Code: github.com/MagamedT/steering 7/7

Samuel Vaiter avatar
Samuel Vaiter@vaiter·Mar 9

We validate everything empirically on 11 models (GPT-2, Gemma 3, Qwen 3, Llama 2, Mistral 7B) across 8 safety-related concepts. All our theorems are confirmed experimentally. 6/7

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Mar 9

Result 3: Steering always hurts global performance. Cross-entropy increases quadratically around α = 0, i.e., there is no free lunch. For large α, performance plateaus because the output becomes input-independent. 5/7

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Mar 9

Result 2: Concept probability follows a sigmoidal curve. The probability that the target concept appears in the output increases smoothly with α, following a tanh shape. Off-target concepts decrease or vanish. 4/7

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Mar 9

Result 1: Next-token probabilities have a bump shape. As α increases, most token probabilities rise, peak for α>0, then fall. In particular, off-target tokens peak before target tokens. 3/7

Post attachment
Samuel Vaiter avatar
Samuel Vaiter@vaiter·Mar 9

Take contrastive prompts (for ex, safe vs malicious), evaluate them through the model, compute the mean difference of representations  at layer ℓ. That's your steering vector v. Then shift activations by αv at layer ℓ. Question: what does α actually do? 2/7

Post attachment