
This work adds to the mounting evidence that we can use our understanding of LMs to improve them. This relates to recent work from @GoodfireAI (on mechanistic shaping) and @RunjinChen (on persona vectors), who (post-)train models using concept vectors.
We cut through the mystery around interpretability in training. A single, mathematically rigorous framework transforms model internals into training feedback and works across domains, problems, and modalities. We call this *mechanistic shaping*, and we’ve used it to mitigate reward hacking, remove shortcuts in chest x-rays, and reduce sycophancy and harmfulness. We believe this is a big step forward for technical alignment. But it carries a known risk: training against a model’s internals can make it harder to monitor. We explore both the technique and this concern below. 🧵



















