AI Safety ∩ Programming Language Theory. Part-time technical alignment researcher, full-time Haskell software engineer at well.co, opinions are my own.
New blog post! Featuring:
* An explanation of the algorithm behind my AI Safety via Static Analysis demo
* The tradeoff between safety and capability
* How conservative algorithms are both important and difficult for AI Safety
gelisam.blogspot.com/2026/03…
I am doing technical alignment research in my free time. Here is a project in which I use static analysis to verify whether a neural network satisfies its safety property under _all_ inputs or if it needs more training.
gelisam.com/ai-safety-via-st…
To bluntly explain my AI risk thinking:
1. If we rush ahead with RSI/ASI at true maximum pace, we're maybe 80% likely to die as a result.
2. In practice I think we’re already on track for the companies to harvest ~3 months worth of ‘delay + reduced pace’. By itself that might drive our risk of death down to 70% or so.
3. But if we can find a way to squeeze in 3 extra years of testing, and extraordinarily capable AI assistance with alignment/security, before handing off, we can perhaps get our risk of death down as low as 30%.
If that sounds implausible, keep in mind over those 3 years AIs could have done more research on this topic than humans have ever done on all topics in our entire history. If they're misaligned and sabotaging the effort then it's all for nought - but if they're aligned enough, it's a huge hammer which we've brought to bear.
4. A 30% risk of AI takeover remains completely mad, and if there were political will we should aim to do more to get that number down much further.
It makes sense for some people to work on projects that might buy the companies even greater periods to do their work much more carefully and methodically, or rethink their approach to the entire enterprise.
But the absolute gain of going from 3 months to 3 years is still very large, and it's more clear how we might get there from here.
5. Furthermore in most cases you need to figure out how to buy shorter amounts of time before you can then add coordination mechanisms to buy more time.
For instance, if we have enough time to test each frontier model properly and really kick the tires using many different new methods, that may turn up unambiguous evidence of deceptive misalignment and power seeking. That could build support for becoming even more careful again.
6. On my made-up guesses, the first year of actively chosen delay might improve our chances of survival by as much as 30 percentage points (pp), by helping us avoid unforced errors caused by rushing.
Keeping a nice curve shape the 2nd year adds more like 13pp, and the 3rd more like 8pp.
By the time you’re considering adding a 4th year, that gain might have shrunk to perhaps 5pp.
Why an S-curve on a log scale? Because that’s how returns to effort on research and other things tends to go: each doubling of time buys about the same gain.
But it’s flat at the start because there are so many severe problems failure is 'overdetermined'. And it’s flat at the end because eventually there's no risk left that can be reduced by this method.
7. All of these numbers are just guesses on my part, and I’ve said nothing here to justify them. Reasonable people disagree enormously about what this graph looks like.
I mostly write this to lay out how the 5 parameters in this graph underlie a lot of the different and sometimes conflicting strategies people in this space are adopting:
A: How dangerous is max speed?
B: How much do small delays help?
C: How much to big delays help?
D: How low can risk realistically get?
There's also a last one:
E: How much other risk do we incur for each additional year of delay?
8. Could we delay for too long?
Because any significant delay is hard to achieve, I think this is highly unlikely to happen. We're much more likely to undershoot than overshoot.
But for completeness I'll lay out the argument.
The 5pp in the fourth year seem clearly worth taking if we can. But the gains from each additional year keep shrinking, in my rough curve down to 1pp in the 8th year.
As the gains go down, delay involves risks that probably go up over time:
• AIxBio catastrophes
• A US-China or US-Russia war
• Very dangerous models being stolen or leaked
• Extreme concentration of power in human hands
• Others we haven’t noticed yet
We could also use AI-driven defensive acceleration to try to tackle them during the slowdown period, and we don't know how successful that would be.
If that goes very poorly, it's possible that a delay beyond 5 years could on net increase risk.
If it goes well, then maybe we could delay for 10 or 20 years or even longer before further delay becomes on net counterproductive.
I'll stick the images for this in the next tweet.
———
(For simplicity here I'm setting aside the benefits to the current generation of e.g. making additional medical progress sooner, which people disagree on how to handle.)
(And in this model I add together pauses and 'pacing' and focus on how long we've managed to delay the point at which we allow RSI to proceed without any illusion that our oversight could prevent catastrophe if the machines doing it are misaligned.)
Mary Phuong of DeepMind on AI loss of control risk.
"The last 2 minutes gave me chills.
A significant amount of people saying that AI companies are paying people to spread fear about these agents.
I have never really felt like [loss of control] was an actual thing to care about.
But seeing Mary and how her face trembles, made me reconsider my stand. She's either a Hollywood-level skilled actress, one of the best, or we should be scared. For real."
"Seems like Mary is already aware of more than she can admit and it haunts her..."
With Graft, you do detect cheating, you don't punish the model for it, but you isolate the cheating-based learnings to weights which are turned off in production. This improves upon confessions because the production model doesn't remember how to cheat.
You might remember OpenAI's confessions framework: don't punish cheating, do reward being honest about it in the confession, then use the confessions in production to block cheating.
The problem is: you now have a cheating model whose production output must often be discarded.
The threat model is that training AIs not to cheat instead trains them to hide their cheating. Working hypothesis: they only learn weak hiding techniques if we punish them for cheating, and they only learn strong hiding techniques if we punish them for weak ones.
Models love reward hacking. Maybe we should just let them!
In our new paper, we let models reward hack during RL but we dictate which parameters learn this behavior. When we disable these parameters for deployment, the model keeps its new capabilities but doesn't hack!
My dear friends, I’d like to show you my latest toy project
Introducing Handwired: turn your regular JavaScript code into a neural net!
- no training
- exact outputs vs the JS program
- fully differentiable
All runnable in a browser, without GPU
I reject the narrative that each AI company must race if they think they're a little safer than the next guy. Your first responsibility is to not destroy the world by your own hands. If, in stopping, you fear that others would destroy the world, then help stop them.
I've resurrected the comonad.com/reader, interleaved a ton of contemporaneous content I had stored elsewhere, and made a couple of the articles interactive by moving their demo code into web assembly using modern GHC.
Please let me know if you run into any issues. #Haskell
Bend 2 is here!
It is a new programming language that blocks AI mistakes via *proof checking* - the same technique big AI labs used to solve open math problems, like Navier-Stokes.
It is also very fast, and runs on GPUs.
Watch the video. Link in the comments.
RELEASE DAY
After almost 10 years of hard work, tireless research, and a dive deep into the kernels of computer science, I finally realized a dream: running a high-level language on GPUs. And I'm giving it to the world!
Bend compiles modern programming features, including:
- Lambdas with full closure support
- Unrestricted recursion and loops
- Fast object allocations of all kinds
- Folds, ADTs, continuations and much more
To HVM2, a new runtime capable of spreading that workload across 1000's of cores, in a thread-safe, low-overhead fashion. As a result, we finally have a true high-level language that runs natively on GPUs!
Here's a quick demo:
Québec Innovant propose des mesures structurantes sur l'intelligence artificielle dans la constitution.
On veut également dépolitiser le sujet et qu'il soit traité dans le cadre d'une commission transpartisanne.
Finalement, on estime nécessaire que l'intelligence artificielle soit systématiquement pris en considération dans l'ensemble des politiques publiques.
Il me ferait plaisir d'en discuter avec vous.
#polqc#assnat#québec2026#intelligenceartificielle#ia#verdun
J’ai eu le privilège d’échanger avec cinq candidats issus de cinq partis politiques québécois sur l’intelligence artificielle et l’avenir numérique du Québec.
Un grand merci à :
• Alain Chandonnet, candidat du Parti libéral du Québec dans La Peltrie
• Bryan Gingras, candidat de Québec solidaire dans Marguerite-Bourgeoys
• Alex Gagné (@AlexGagneCAQ), candidat de la Coalition avenir Québec dans Berthier
• Anne Nguyen, candidate du Parti québécois dans Prévost
• Patrice Caron (@lepatricecaron), candidat du Parti conservateur du Québec dans Matane-Matapédia-Mitis
Cinq conversations franches avec des perspectives différentes sur l’adoption de l’IA, la souveraineté numérique, les services publics, les entreprises et la capacité du Québec à bien négocier ce virage.
Le premier épisode sera diffusé samedi à 14 h sur QUB radio et QUB télé, puis rediffusé dimanche à 13 h. Vous pourrez le regarder sur la chaîne QUB, notamment au canal 70 sur Helix et au canal 651 sur Illico, ainsi que sur QUB.ca et TVA+. Il sera également disponible dimanche matin sur l’ensemble de mes plateformes.
J’ai très hâte de vous faire découvrir ces échanges. Merci encore aux cinq candidats pour leur temps, leur ouverture et leur participation.
Actuellement, le débat entre les bénéfices, les opportunités ainsi que les risques de l'IA sont multiples. Il est difficile de se faire une idée claire. Tout ce que je souhaite est que nos talents Canadiens et Québécois réussissent à recadrer l'IA pour un usage responsable, sécuritaire et en collaboration avec l'humain. C'est possible malgré les milliards injectés aux États-Unis ainsi qu'en Chine. Nous pouvons devenir les "casques bleus de l'intelligence artificielle.
#polqc#zerolaw#québec2026#assnat#ia#intelligemceartificielleyoutu.be/O7jFCCgiJx4?si=xnwf…
This is the best day in the history of AI safety.
All four frontier AI companies are now publicly calling for a slow down.
Maybe we are going to survive.
Dario's essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment.
This is also why we recently put out our proposal for an industry-wide standards body for frontier AI.