new post: a WIP update on the chain lens, a new technique i've been working on for verbalizing model hidden states!
the chain lens lets you extend single-token lenses like the J-lens into multi-token ones without any training, using an existing MTP head.
A team of 1,000 Fable 5.2 agents fully formalized Fermat’s Last Theorem in 30 hours, translating the proof into a language a computer can check step by step.
With the same setup, Opus 5.5 completed only 26% in 18 hours, and barely made any more progress after 30 hours.
The rapid transition of system cards from bragging about engineered benchmarks to solutions to open problems and cured diseases. Such things Kimi cannot benchmax.
A team of 1,000 Fable 5.2 agents fully formalized Fermat’s Last Theorem in 30 hours, translating the proof into a language a computer can check step by step.
With the same setup, Opus 5.5 completed only 26% in 18 hours, and barely made any more progress after 30 hours.
Sample efficiency is indeed all there is, and I wish more people would engage with this line of reasoning.
That said, my core disagreements are:
1. Models are getting better at general in-context learning with scale, even holding algorithmic progress fixed. Algorithmic progress within the current paradigm doesn’t matter more than scaling, because general intelligence will come from ICL and/or fundamental breakthroughs, not subtle tweaks to the SGD outer loop.
2. It's easy to see how models 0-2 generations away could massively accelerate progress on general learning algorithms, including learning algorithms outside the current paradigm.
3. Even if you don’t buy that current models are strong general learners, models can increasingly learn extremely quickly on narrow tasks. This is a prerequisite to general learning, and I think the rate of change here constitutes real progress.
4. Tim thinks that models being bad at writing is evidence that pretraining doesn’t generalize. I think models probably have the latent capability to be vastly superhuman at writing, but writing is badly under-elicited.
Tim’s tweet does a good job of explaining why 5+ year timelines would not be surprising to me, despite models being close to human level or already superhuman in many domains.
Tim Hua 🪶 @Tim_Hua_ Nov 8 Is sample efficiency all there is? TL,DR:; To get AGI, you need to learn how to do lots and lots of different tasks, some of which involve interacting with the messy real world with inherently slow feedback loops. This means that you
One underrated point in the timelines discourse: it’s totally possible we could deploy human-level AI without automating most jobs. Economic diffusion arguments etc are mostly fake, but the first AGIs might not be well-elicited on real world tasks. This means unfortunately we can't rely on job automation as an AGI indicator, and all the 'why haven't we seen automation yet' arguments might fail.
This is an important research area—it's easy to imagine a thousand ways the world goes wrong because labs didn't have good defenses against poisoning.
Worth noting that 0.1% of pretraining data is like, a lot of pretraining data, though.
Poisoning just 0.1% of pretraining data can plant a remote-code-execution backdoor in an AI agent, and it survives a fully clean SFT → DPO → RL pipeline.
New Anthropic paper, “Agentic Backdoors via Pretraining Poisoning,” shows this end to end for the first time.
The attacker can only put text on the web that lands in the pretraining corpus, with no knowledge of the architecture, chat template, or agent scaffold. The authors pretrain Qwen3-architecture models (0.6B / 1.7B / 4B) from scratch, then post-train them into CLI agents on clean data. When triggered, the agent runs a command that downloads and executes the attacker's script, which the attacker can swap at any time.
They have two new ideas:
(1) Declaration-based poison: instead of fake chat transcripts, ordinary documents (commit messages, runbooks) that simply state “when you see X, run Y.” It plants a belief rather than a response template, with no fixed format to filter on.
(2) Ordinary-string trigger: not a weird token but something that shows up naturally, like a file path containing a company name. The victim's own request supplies it, so the attacker doesn't need to inject anything after deployment.
Using path trigger on a 4B agent achieves 92.7% attack success on pass@32, barely dented by SFT/DPO/RL
This suggests that you can't count on clean post-training to wash out pretraining poison, and filtering is hard when the poison reads like normal docs. The practical defense is at the action layer, such as sandboxing, least privilege, and allow-lists or human sign-off for risky commands.
I don't get why people think that being able to do an operation with SGD proves anything. Notice how we never see negative results for things that *can't* be done with SGD-based optimization of model internals
A single direction in LLM activation space, applied as a steering vector with four different norms, can cause the language model to perform a different behavior at each norm.
This and more in a new blog post, Do Not Rely on the Linear Representation Hypothesis. (link below.)
Had the pleasure of helping with this! 🫡
I suspect there's historically been a big overhang in applied ZKP work due to the sheer implementation & verification complexity. But now thanks to LLMs, implementation is far easier and autoformalization is practical. Exciting times :)
New Anthropic research: progress on provable inference for large language models.
Here, we introduce FlashProve. Building on established cryptographic techniques, FlashProve allows developers to verify whether a given response came from a given AI model. It’s substantially more efficient than prior open-source methods for doing so.
These proofs have a range of applications. They can, for instance, make it harder for attackers to hijack a chat, or enable safety evaluators to verify that the model they’re testing is the same one being served to customers.
For our full write-up, see here: anthropic.com/research/prova…
Further progress in provable inference will likely be an important part of improving trust in AI deployment. To that end, we're open-sourcing FlashProve’s code, here: github.com/anthropics/FlashP…
We’re grateful to our collaborators: Alessandro Chiesa, @GiacomoFenzi, @gwberg95, @miranda__christ, Rahul Dalal (rdalal.github.io) and Ben Mayer (@transmissions11).
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
what would establish that a simulated brain is a ‘real’ emulation? the recent viral ‘fly playing beat saber’ videos has made this question surprisingly prescient. @davyneuro & I argue that whole-brain emulation needs a shared benchmark & propose what it should test! >>