←
Theorem46 posts
Theorem avatar

Theorem

@theoremlabs

Accelerate software verification. theorem.dev/careers

◷ Joined May 2025▧ 8 Photos and videos
Theorem avatar
Theorem@theoremlabs·Nov 3

We can help you verify the systems that keep AI agents contained. Get in touch.

We verified a sandbox! Read more about it here: theorem.dev/blog/proving-age… We built a prototype hypervisor capable of running Linux and proved, down to the machine code, that arbitrary guest code cannot modify host files. More to drop soon!

Theorem avatar
Theorem@theoremlabs·Nov 3

We verified a sandbox! Read more about it here: theorem.dev/blog/proving-age… We built a prototype hypervisor capable of running Linux and proved, down to the machine code, that arbitrary guest code cannot modify host files. More to drop soon!

Rajashree Agrawal avatar
⟳ Theorem retweeted
Rajashree Agrawal@rajashree·Oct 24

the good news is that we can now start writing fuzz harnesses that exercise the full range of production behaviors. because we won't be running them forever, we'll just prove them.

really cool post. essentially taking existing fuzzing harnesses, and "extending" them via formal verification to cover more inputs nothing more nothing less unsurprisingly, there's lots of trivial bugs that these harnesses miss. so it might be a while until we're "vulnzero"...

Post attachment
Luke Champine is visiting SF avatar
⟳ Theorem retweeted
Luke Champine is visiting SF@lukechampine·Oct 23

It didn’t make the final draft, but imo the most exciting story here is that LLMs have unlocked a new *kind* of FV. A human would balk at binary-level verification, because of how tedious it makes the proofs. But LLMs don’t care about tedium. It plays to their strengths.

Jason Gross avatar
⟳ Theorem retweeted
Jason Gross@diagram_chaser·Oct 24

The bitter lesson of ML is that scale eats all. The bitter lesson of PL is that [efficient] brute force eats all.

The results from Anthropic + Theorem feel like a “bitter lesson” moment for formal verification. They just went for semantics-level proofs. No fancy program logics, proof techniques or algorithms. Just bare metal proofs with a vanilla Hoare logic over RISC-V.

Vinod Khosla avatar
⟳ Theorem retweeted
Vinod Khosla@vkhosla·Oct 23

I am a total fan of @theoremlabs and what it does to add formal verification to probabilistic LLM's! Important concept broadly.

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

Jason Gross avatar
⟳ Theorem retweeted
Jason Gross@diagram_chaser·Oct 23

Highlights for security folks: - We're open sourcing thousands of theorem statements from 437 libraries, our Linux process semantics, and ASan for static binaries. Let's turn a lot more fuzz tests into theorems, and get cracking at the proofs! - OSS Verify: We’ll verify safety properties of critical projects for free. Please enroll!

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

Oleg Mürk avatar
⟳ Theorem retweeted
Oleg Mürk@oleg_murk·Oct 23

Three days before I was pushed out of OpenAI I posted a Formal Verification Manifesto in a core research Slack channel. It listed OpenAI systems that could benefit from formal verification. The only person who responded was @sama, who was very supportive. Not sure what the moral of this story is, but it's never too late for OpenAI to invest seriously in software & hardware formal verification! On a more serious note, @theoremlabs and @AnthropicAI did something amazing. They showed how to verify software at scale without requiring developers to write specifications in Rocq or Lean. Instead, they used existing fuzz harnesses as specifications and LLMs to generate machine-checked proofs. Of course, fuzz harnesses specify only some correctness properties. In the simplest case, the proofs establish crash-safety, not full functional correctness. We need community-maintained open-source projects with reusable formal specifications and correctness proofs for major libraries, kernels, compilers, virtual machines and hardware. Human review of critical specifications and LLM-generated patches is essential. A model could satisfy a specification simply by removing functionality. As AI writes and rewrites critical infrastructure, formal verification should become a key part of AI control. Critical properties of AI-generated code should be machine-checked rather than taken on faith. AI is making vulnerabilities easier to find and exploit. Frontier LLMs and reusable formal specifications could finally make formal verification practical and economical at scale, while keeping it understandable to humans. Kudos to Theorem co-founders @rajashree and @diagram_chaser!

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

Rajashree Agrawal avatar
⟳ Theorem retweeted
Rajashree Agrawal@rajashree·Oct 23

Making sure I state it clearly: We can literally patch and prove bug-freeness of the entire software stack running in production. With AI that exists right now for ~$100m. This means no more software-level hacks. I know it sounds too good to be true, but it isn't. AMA.

We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here! Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify

Max von Hippel avatar
⟳ Theorem retweeted
Max von Hippel@maxvonhippel·Oct 23

If I recall correctly, I met @diagram_chaser and @rajashree at a YC event sometime around F24, which I believe was their batch. They were the second FM YC company, after us, and the first to do interactive theorem proving. To the best of my knowledge, they were the second overall company to do LLM-based ITP after @HarmonicMath . We were dealing with gpt-4o in those days, and I couldn't even get the model to write a good fuzz test, or set up a sandbox (or an AWS lambda or similar) with someone's repo. There was a bunch of demand -- we launched agentic PBT and __immediately__ had >100 users, which was way more than we could keep up with at the time -- but the tech wasn't even remotely there. I thought Jason and Rajashree were fucking nuts to be trying to actually prove stuff, when literally just testing was borderline impossible. Later, I heard rumors of research projects with Redwood and maybe ARIA, and I thought, huh, I guess that's a reasonable way to keep your company alive and learn things without taking on more non-dilutive funding. Good for them, they found a way to do the research lab thing for a while and have some fun. FM is about the fun we have along the way! But I was hella wrong to be remotely skeptical. The reality is that, Jason and Rajashree spent two years becoming state of the art at large scale agentic verification. Later, when a biglab asked me "who should we talk to if we want to do formal methods", I was like, "I mean obviously, Theorem." I think a **lot** of other people said the same thing. Who else? (To be clear, there are a couple other quite meaningful players in this space, like @axiommathai and @mathematics_inc , but they've largely positioned themselves as AI neo-labs themselves, which makes them awkward partners for a biglab, so ...) I'm not convinced that a world where we formally verify assembly code is really ideal. I am concerned it enables a world where we LLM-generate assembly, which I view as an uninterpretable and harmful outcome for humans (here I disagree with both Elon and my employer ..). I would like to believe that there are all kinds of really elegant, important properties that are best expressed over more abstract representations. But a world where we've proven *anything at all* about a giant proportion of software is actually a **massive improvement** over the counterfactual. So even if I feel inclined to nitpick the approach, the reality is that this looks like a phenomenal overall uplift in cybersecurity and a totally worthwhile and important endeavor. This is a great time to start learning formal methods. And it's a great time to work in formal methods. I really couldn't be more excited about where we're at, and can't wait to see what's next from the folks re-inventing the field!

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

jessicat avatar
⟳ Theorem retweeted
jessicat@jessi_cata·Oct 23

Amazing, they bypassed the need for a formally verified compiler by verifying at the assembly code level.

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

cts🌸 avatar
⟳ Theorem retweeted
cts🌸@gf_256·Oct 22

How to find ALL PHP bugs: 1. Compile code (e.g. to RISC-V, x86, LLVM IR) 2. Vibe prove ASAN and non-ASAN versions have same behavior across all inputs. Hard, but Claude can do it! 3. Find a LOT of bugs Super proud of our work. A big step towards finding ALL memory safety bugs

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

Jason Gross avatar
⟳ Theorem retweeted
Jason Gross@diagram_chaser·Oct 22

The thing that excites me the most about formal verification is that we can section off parts of our software and be like “boom, no more bugs here” Asymmetric defense at scale

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

Theorem avatar
Theorem@theoremlabs·Oct 22

We formally verified 123 open-source projects in a partnership with Anthropic. Formal verification at scale is here! Now we’re launching OSS Verify to scale it to the software we all depend on. We prove it’s crash-free or send you the patches it needs. Free for open-source. Let’s bootstrap the verified software ecosystem: theorem.dev/oss-verify

New on our Frontier Red Team blog: using Claude to formally verify software at large scale. Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time). With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks. Read our research in full here: anthropic.com/research/forma…

Aengus Lynch avatar
⟳ Theorem retweeted
Aengus Lynch@aengus_lynch1·Oct 8

guess we'll just verify all of the software then

Last week some of South Korea's biggest banks were hit by a cyberattack. Thanks to a report from CrowdStrike tonight, we now know the entire hack may have been done by a single person. He used a combined stack of an open-source AI penetration tool named ARTEX, DeepSeek v4.1-Flash, GLM-5.3, Grok 4.6, and Claude Code.

Post attachment