The non-disparagement clauses in Google's UK employment contracts are indeed incredibly broad. This is from the Confidentiality and Proprietary Information Agreement I signed in 2024 before I started.
It forbids any criticism forever (regardless if it's true or not)
Nice thread on entropy and uncertainty, but see @fbickfordsmith's excellent paper Rethinking Aleatoric and Epistemic Uncertainty for the decision-theoretic take, where uncertainty is defined as the expected loss under the Bayes-optimal policy!
arxiv.org/abs/2412.20892
For many of us, the first time we see Shannon’s entropy
H(X) = −Σᵢ pᵢ log pᵢ
feels magical. Sure, it has many beautiful properties, but why should this particular formula measure “uncertainty”?
Can we define uncertainty first and derive the formula afterward?
This idea that intelligence is solely a function of what you've observed since birth and not also a function of the 500 million years of evolution that preceded your birth is surprisingly sticky despite being demonstrably untrue.
The proof that this is possible is all around us: whereas current systems are trained on essentially all of accessible history, humans exceed AI capabilities despite seeing at most a few billion text tokens by adulthood.
Come and check out our work on how to evaluate LLMs with less compute and fewer labels!
Find first author Gabrielle at EurIPS or Freddie and I at poster 110 at the 11 AM session on Friday.
Active testing enables label-efficient model evals but can be computationally expensive.
We show how to reduce costs and scale up to LLMs.
arxiv.org/abs/2508.09093
Work led by Gabrielle Berrada. Find her at EurIPS, or @janundnik and me at NeurIPS in San Diego.
Active testing enables label-efficient model evals but can be computationally expensive.
We show how to reduce costs and scale up to LLMs.
arxiv.org/abs/2508.09093
Work led by Gabrielle Berrada. Find her at EurIPS, or @janundnik and me at NeurIPS in San Diego.
I crossed an interesting threshold yesterday, which I think many other mathematicians have been crossing recently as well. In the middle of trying to prove a result, I identified a statement that looked true and that would, if true, be useful to me. 1/3
New post on RL scaling:
Careful analysis of OpenAI’s public benchmarks reveals RL scales far worse than inference: to match each 10x scale-up of inference compute, you need 100x the RL-training compute. The only reason it has been cost-effective is starting from a tiny base.
🧵
Every so often, OpenAI employees ask me how I see the co now.
It's always tough to give a simple answer. Some things they're doing, eg on CoT monitoring or building out system cards, are great.
But the dishonesty & intimidation tactics in their policy work are really not. E.g:
One Tuesday night, as my wife and I sat down for dinner, a sheriff’s deputy knocked on the door to serve me a subpoena from OpenAI.
I held back on talking about it because I didn't want to distract from SB 53, but Newsom just signed the bill so... here's what happened:
🧵
With @JesusOxford we are looking for a Professor of Statistics.
Become part of a historic institution and a community focused on academic excellence, innovative thinking, and significant practical application.
About the role: tinyurl.com/b8uy6mr5
Deadline: 15 September
Research on AI "sandbagging" is getting more popular recently. In this 🧵, I'll give some reasons that I think it's not a useful research paradigm.
TL;DR, I think it's a confusing reframing of fairly well studied and previously solved problems.
It's great the governments (and others) continue to demonstrate that the models companies release are incredibly insecure.
It's terrible that governments aren't penalizing companies for releasing such insecure models, and instead just help them patch them.
We at @AISecurityInst worked with @OpenAI to test GPT-5's safeguards. We identified multiple jailbreaks, including a universal jailbreak that evades all layers of mitigations and is being patched. Excited to continue partnering with OpenAI to test & strengthen safeguards.
Just two years ago, our smartest models could barely solve the easiest competitive programming problems. Last week, our latest reasoning models achieved a gold medal score at the International Olympiads of Informatics. Competitive programming is one of the cleanest examples of scaling up RL training for LLMs. Soon, with experimental approaches like the one used for IMO, we might see similar scaling on real-world coding problems.
1/n I’m thrilled to share that our @OpenAI reasoning system scored high enough to achieve gold 🥇🥇 in one of the world’s top programming competitions - the 2025 International Olympiad in Informatics (IOI) - placing first among AI participants! 👨💻👨💻
Today I’m releasing an essay series called Better Futures.
It’s been something like eight years in the making, so I’m pretty happy it’s finally out!
It asks: when looking to the future, should we focus on surviving, or on flourishing?