
Better to use a model that is not too smart and makes mistakes it can keep you on your toes

More social alt of @ezyang

Better to use a model that is not too smart and makes mistakes it can keep you on your toes

A benchmark where you take old PL paper results that were pencil and paper only and formalize them

tfw capability benchmark secretly an alignment benchmark

There is also a mode collapse problem, where everyone will use the same models to assist in the formalization, resulting in a monoculture where every implementation may make the same mistakes

Formalization is also an interestingly shaped task where human graders are arguably less good than LLM graders, but now there is a trust bootstrap problem...

As I work on formalization stuff, one of the funniest mode collapses is everyone's AI "independently" finding exactly the same ambiguities/omissions in English specs

I am guessing there is just an inference bug for Astra

In the next year, will model improvements allow humans to substantially automate the process of formulating a TCB for formal verification? Discuss.

Xianxia has a clear solution to this but American gamers are not ready for it
BoTW and ToTK have the same issue: The first 40 hours is the best survival game ever made that forces you to think, be creative, conserve resources and take it all in slowly and then you get too many items/abilities that trivialize much of the rest of it.

I successfully used a PICK style approach to figure out an LLM had proved that GPUs 0 and 1 did the right thing but omitted GPU 2/3 from the theorem! blog.brownplt.org/2026/06/09…

"Remember this, we will return to it later" feels LLM coded although I don't know why

It feels like it's good to run agent swarms more slowly (eg less parallelism) because it makes it easier to figure out if you have asked for the wrong thing

Suppose you have a function that takes 10s to run. Seems wrong to optimize a 1ms startup phase. But what if I said that the function SHOULD take 1ms. Then even the startup is 100% of your budget. Everything must be worked on! The power of expectations.

NVIDIA doesn't publish info about SASS but the community figured out that nvdisasm has a domain specific language that encodes a comprehensive spec about SASS. It tickles me to no end the community calls this "The Rosetta String"

Claude really loves DRF-SC. Too much, I'd say.

I just want to state what it means to be deadlock free HOW HARD COULD IT BE