Base model quality drives RL post-training, but existing guarantees fail to translate a good init into efficient learning. Our theory captures this and thus turns the heuristics behind modern RL objectives into actionable principles for algorithm design. arxiv.org/abs/2611.14467
Can LLMs track state changes over long texts—like an object changing hands or a variable reassignment?
On our new StateBench eval, frontier models including GPT-6 Sol & Claude Opus 5.5 fell to 13% accuracy or lower on the hardest, longest problems. 🧵
github.com/allenai/state-ben…
1/ 🎉 Excited to present at #EMNLP2026 🎉
Transformers and Mamba are fundamentally different. Do they also represent things differently?
Globally, they differ. Locally, they are similar. 🧵
Harnesses let agents work together on a task, each in its own context.
Beyond context limits, what does this way of organizing agents change about learning?
Our new paper shows how it can help models generalize out of domain.
🧵1/18
arxiv.org/abs/2609.20831
If you've noticed a distinct dialect emerging in frontier AI models and struggled to understand what they're talking about, you're not crazy!
We turned to psycholinguistics to investigate what makes the writing style so weird, and why this might be happening
🚨New preprint! 🧵
🤖🧠NEW PAPER🧠🤖
What is the relationship between neural networks & symbolic systems?
Based on evidence from mechanistic interpretability, we argue for a quasi-symbolic view of cognition: Neural nets implement symbols but only approximately
Link: sciencedirect.com/science/ar…
1/n
The most profound paper of my PhD so far. We truly did something special to make LMs play/explain chess, needing both architectural & algorithmic innovation (natural-language analogue of the Alphazero algorithm) + applicable to many other domains. Please read on and share!
1/10
Which state-tracking problems can transformers length generalize on?
Transformers' performance on different finite-state tasks varies drastically, even when the tasks are structurally similar. Now, for the first time, we can precisely predict when length generalization occurs.
Can Transformer LLMs hide their reasoning from CoT monitors?
tl;dr: Transformers cannot hide complex reasoning, but they might cryptographically obfuscate their CoTs, making it extremely difficult to extract hidden goals.
🚨 [Token][ization] Paper Alert 🚨
Tokenization is a wildly understudied area of language modeling despite it having effects across all of NLP. Over the past ~8 months, 32 (!) tokenizer researchers put together the most comprehensive survey of the field. Check it out!
Bend 2 is here!
It is a new programming language that blocks AI mistakes via *proof checking* - the same technique big AI labs used to solve open math problems, like Navier-Stokes.
It is also very fast, and runs on GPUs.
Watch the video. Link in the comments.
RELEASE DAY
After almost 10 years of hard work, tireless research, and a dive deep into the kernels of computer science, I finally realized a dream: running a high-level language on GPUs. And I'm giving it to the world!
Bend compiles modern programming features, including:
- Lambdas with full closure support
- Unrestricted recursion and loops
- Fast object allocations of all kinds
- Folds, ADTs, continuations and much more
To HVM2, a new runtime capable of spreading that workload across 1000's of cores, in a thread-safe, low-overhead fashion. As a result, we finally have a true high-level language that runs natively on GPUs!
Here's a quick demo:
💥 The multiplication saga continues! It turns out, some frontier LLMs do struggle with simple multiplications.
Unlike GPT-6-Astra, Fable 5.1 with low reasoning has a blind spot where the accuracy of, say, 5x6 multiplications drops to ~0%!
The failure mode is very interesting: the 0% zone corresponds to a failure of adaptive thinking. The model decides that it shouldn't produce any reasoning and outputs the (wrong) answer directly. For larger multiplications, however, the model realizes it should produce a CoT first and then it gets the answer correctly. For very large multiplications (close to 20x20), the accuracy degrades again.
How many such simple blind spots do frontier LLMs still have? This is why slowing down and carefully studying generalization and safety is the top priority...
1/2
🤖🧠NEW PAPER🧠🤖
(The result of an 8-year project!)
LLMs seem very different from symbolic systems. Yet LLMs excel in symbolic domains (e.g., language/code/math). How do they do it?
Our finding: LLM representations have implicit symbolic structure!
Link in thread ⬇️
1/n
Very excited to be sharing this on behalf of the amazing team:
Blerta Veseli
Corentin Barloy
Michaël Cadilhac
Andreas Krebs
Charles Paperman
Howard Straubing
@mhahn29