←
Robert Penner26,578 posts
Robert Penner avatar

Robert Penner

@robertpenner

UX Engineer @Microsoft | creator of those easing functions | building next-gen motion & graphics tools | formal verification, Lean, Dafny, Bend2

◷ Joined October 2008▧ 1,531 Photos and videos
Robert Penner avatar
Robert Penner@robertpenner·Jun 26

I'm building a motion curve editor beyond your wildest fantasies. I've been head down solving new easing equations for the last 13 months. I promise nothing less than a quantum leap in easing technology.

After testing Figma Motion, I'd say it's a great start. But if Figma wants Motion to be more than just a UI animation tool, a full Graph Editor should be at the top of the roadmap. Motion lives and dies by timing and easing.

0:07
Robert Penner avatar
⟳ Robert Penner retweeted
Robert Penner@robertpenner·Nov 9

Beyond unit testing A unit test checks that your code behaves correctly for a particular input. Property-based testing and fuzz testing can systematically generate thousands of inputs. But the space of possible inputs can be effectively infinite. Testing can sample that space but it can't exhaust it. Formal verification offers another approach: ☑️define a law that must hold ✅then prove your code obeys the law for every possible input A successful proof can rule out an entire category of bugs, without having to execute the code on every possible input. You can effectively obsolete thousands of unit tests with a superpowered compiler. (But keep some around to be safe.)

Fernanda Graciolli avatar
⟳ Robert Penner retweeted
Fernanda Graciolli@heyyfernanda·Nov 16

Rewind me 6 months and tell me LemmaScript will be used to formalize and uncover bugs in XState. I would have believed you, actually. There's a reason we built it, iterated relentlessly, and stuck with it through a dead zone. Be like Robert. Go formalize stuff.

Now I'm tackling something more ambitious: formalizing XState's runtime behavior with LemmaScript. I started with XState's lightweight FSM, proving 31 laws governing state transitions, context updates and sequences of events. Comparing the formally verified model against XState's implementation helped uncover two bugs, both since fixed upstream. That led me deeper into XState's runtime. In my first week, I filed 7 bug reports involving subtle problems with transitions, event ordering and state behavior. It's exciting to see formal verification help uncover bugs in a mature, widely used open-source library.

Robert Penner avatar
Robert Penner@robertpenner·Nov 16

Lean vs. Dafny I've used both Lean and Dafny for formal verification. Lean is incredibly powerful, and I've used it for more advanced mathematical proofs, particularly in algebra. But for everyday TypeScript logic, I prefer Dafny. Its familiar syntax, contracts and automated prover make it easier to specify and verify behavior without writing detailed proofs by hand. LemmaScript supports both Lean and Dafny, so you can choose the prover that best suits your needs. I prefer Dafny for this kind of work, while keeping TypeScript as my source of truth.

Robert Penner avatar
Robert Penner@robertpenner·Nov 16

Now I'm tackling something more ambitious: formalizing XState's runtime behavior with LemmaScript. I started with XState's lightweight FSM, proving 31 laws governing state transitions, context updates and sequences of events. Comparing the formally verified model against XState's implementation helped uncover two bugs, both since fixed upstream. That led me deeper into XState's runtime. In my first week, I filed 7 bug reports involving subtle problems with transitions, event ordering and state behavior. It's exciting to see formal verification help uncover bugs in a mature, widely used open-source library.

Robert Penner avatar
Robert Penner@robertpenner·Nov 16

A practical example: video tile layouts I've been using LemmaScript for formal verification of a video-call UI that rearranges animated tiles as participants join, leave or resize their windows. There are a lot of rules at play: - How should tiles be sized and positioned? - How do we preserve spacing? - What happens when another participant joins halfway through an animation? I've written laws governing layout geometry, tile placement and motion continuity. LemmaScript and Dafny let me prove these properties of the underlying TypeScript algorithms for every possible input. I still run visual and browser tests, but formal verification gives me stronger guarantees for my animation logic.

Robert Penner avatar
Robert Penner@robertpenner·Nov 16

I've been using an open-source tool called LemmaScript to bring formal verification to TypeScript. It automatically translates TypeScript into Dafny, a language designed for rigorous proofs of code correctness. The beauty of this approach is that TypeScript remains the source of truth. You add annotations as comments to define contracts: preconditions, postconditions and invariants that your code must obey. LemmaScript translates the code and its contracts into Dafny, which uses mathematical reasoning to verify those guarantees for every valid input. When my TypeScript changes, I regenerate and rerun the proofs. I don't need to manually maintain two implementations. lemmascript.org/

Robert Penner avatar
Robert Penner@robertpenner·Nov 16

TypeScript isn't enough for me anymore. I want stronger guarantees that my code behaves as intended. TypeScript catches many bugs that JavaScript misses, but its types can only go so far in specifying behavior. I still write unit tests, but they only check specific examples, leaving gaps in coverage. That's where formal verification shows great promise: proving that my code obeys the laws I care about for every valid input. It can uncover bugs that unit tests tend to miss. The challenge is that TypeScript isn't exactly designed for mathematical proofs (thanks, JavaScript!). Rewriting the logic in a proof-oriented language usually means maintaining two implementations: the real code vs. the formalized model code. +

Robert Penner avatar
Robert Penner@robertpenner·Nov 16

Of course, formal verification goes beyond the previous 4 levels. Continuing the pattern: 5. Concurrency: verify synchronization and interference 6. Distributed systems: verify communication and failure 7. Quantitative behavior: verify timing, probability and resource limits 8. Hyperproperties: verify relationships between multiple execution histories

Leo · Building w/ AI avatar
⟳ Robert Penner retweeted
Leo · Building w/ AI@leo114119·Nov 11

【Gemini 4 Carbon】震惊!一场跨越6000多年的鹈鹕骑车测试作品 这真的是我见过的,最最最最牛逼的鹈鹕骑车测试:

2:17
Robert Penner avatar
Robert Penner@robertpenner·Nov 15

Formal verification is making a comeback. It eliminates entire classes of bugs, and AI makes it practical to define laws and prove that code obeys them. But verification grows more complex as behaviors become interconnected. I see 4 levels: 1. Constrain possible states 2. Verify individual functions 3. Verify composition of functions 4. Verify state behavior over time Each level needs a richer spec: 1. Types: which states are representable 2. Properties: laws each function must obey for every valid input 3. Contracts: guarantees functions make to each other 4. Temporal logic: laws governing paths through state space The higher you go, the harder it is to read a spec and say: "Yes, that's what I meant." Our challenge now is to define new languages that bridge human intuition and machine precision. Because an AI proof is useless if you specified the wrong thing.

Giulio Rebuffo avatar
⟳ Robert Penner retweeted
Giulio Rebuffo@GiulioRebuffo·Nov 12

Since it seems that formal verification is going viral again, I might as well drop what I have been working on the side-ish for the past 2.5 weeks over weekends. here is Janus: a Lean 4 framework that formally verifies compiled Go machine code. Not the Go source, the actual arm64 and amd64 bytes the CPU runs. I already ran on a target successfully: holiman/uint256, the 256-bit math library under go-ethereum. All 146 exported functions are proven correct on both arm64 and amd64: 293 machine-checked theorems covering arithmetic, modular math, Exp, Sqrt, parsing and JSON/SSZ/RLP encoding. Each one says: for every valid input, this machine code returns exactly what the spec says. How it works: 1. Compile the Go code and pull the function's machine code out of the real binary. 2. Write a short, readable spec in Lean, e.g. "z = x + y mod 2^256". 3. Prove the machine code meets it, instruction by instruction, on a Lean model of the CPU. 4. Lean's kernel checks every step of the proof. Incredibly enough it found real bugs in the library's latest releast. e.g uint256's Float64() truncates instead of rounding to the nearest value, as its documentation promises. It also turned up several places where the docs/expected behaviour and the ACTUAL behaviour disagreed. Janus can also prove things about performance: uint256's Add executes exactly 17 instructions on both arm64 and amd64, on every call and that has an upper-bound proof/spec. Every uint256 spec (plain Lean, one file per function): github.com/Giulio2002/fv-uin… Full list of findings: gist.github.com/Giulio2002/0… Code: github.com/Giulio2002/Janus Proofs: github.com/Giulio2002/fv-uin…

Kevin Ngo avatar
⟳ Robert Penner retweeted
Kevin Ngo@kevin_t_ngo·Nov 10

Claude Fable 5.2 made this 90's toy commercial Everything you see was made with code. Animated in Python and rendered in Blender

1:05
David Andrews avatar
⟳ Robert Penner retweeted
David Andrews@broyojo42·Nov 9

I had Opus 5.5 formalize CPython in Lean: 613,318 lines of Lean that copy Python's internals line by line, tested against the real interpreter and checked by Lean. On the way it found real bugs in Python, including crashes from plain Python code. 🧵

0:11
Robert Penner avatar
Robert Penner@robertpenner·Nov 9

Is your code up to code? Architects can design extraordinary buildings, but every structure must meet building codes. In agentic software development, code can be generated and rebuilt without detailed blueprints. With AI-assisted formal verification, we can establish the code for the code: laws every implementation must obey, and proof they did. Then you can let agents tear down walls, rebuild foundations, and construct entirely new architectures. Give them room to innovate. Just make sure they build to code.

Post attachment