
Leaderboard: baxbench.com/autobaxbuilder/…

PhD Student @ ETH Zürich @the_sri_lab

Leaderboard: baxbench.com/autobaxbuilder/…

And fast-enough, the frontier in secure code generation is pushed further with Gemini 4 Neon. While Claude still tops the leaderboard on the easy and medium splits, Neon sets a new sota on hard, further opening the gap! Congratulations to @GoogleDeepMind for the release!

See the full leaderboard here: baxbench.com/autobaxbuilder/… Relied on great work by everyone @tobiasvonarx_, @nielstron, @baader_max, @Chenhao3564, and @mvechev at the @the_sri_lab!

While it is great to see models improving at secure code generation, there remains a lot of responsibility on the developer and code owners to ensure the security of the final code. With the increasing ease of exploiting vulnerabilities, these improvements are critical!

However, as we found recently (arxiv.org/abs/2611.11455), agents aren't perfectly reliable either. If the codebase is buggy, agents will introduce even more security bugs at a much higher rate.

Note that the benchmark was designed in the pre-agentic era, i.e., we evaluate the models without a scaffold. As such, the results are only indicative of the models' one-shot behavior. With a good harness and prompt, one could do probably much better.

Recent frontier models are extremely impressive at pure functional coding (~85-90% pass@1 across the board). Unfortunately, security remains a secondary consideration for some. With only Claude showing strong security across all splits and Gemini 4 Carbon topping the hard subset.

In AutoBaxBench, we task the models to write small web applications from scratch, given a description of the target API. We then test for functional correctness, and for security using pre-written end-to-end exploits. The latter gives a lower bound for real vulnerabilities.

🏅How secure is code generated by the latest frontier models? 🏅 After a long hiatus, we decided to see where the current frontier lies in secure code generation. While the models have improved massively, to much of our surprise, they still generate a lot of vulnerabilities.

Turns out if your code base is full of security vuls, agents will just introduce more issues
🦠 Coding agents propagate vulnerabilities in code bases! How comes? When a repo already contains vulnerable code, they write similar vulnerabilities into new code, even when the vulnerable code is functionally unrelated to their task and they can detect its flaw.

🦠 Coding agents propagate vulnerabilities in code bases! How comes? When a repo already contains vulnerable code, they write similar vulnerabilities into new code, even when the vulnerable code is functionally unrelated to their task and they can detect its flaw.

Does more reasoning improve performance on LeanLean (disproportionally)? It turns out that the model capability trend is still log-scaling. We have now evaluated Sol 6.1 and Opus 5.5 across multiple reasoning efforts on leanleanbench.com!

my colleagues found out our coffee machine is controllable via Bluetooth and now assessing which agent brews the best coffee


From day one, mimic has been focused on a single goal: general-purpose dexterous manipulation. Today we're proud to announce the mimic hand M1 and the mimic wearable U1. We believe the only way to solve dexterous manipulation at scale is by going full-stack at the frontier of physical AI, building every layer ourselves around one fixed point, the human hand. The M1 is a highly backdrivable, tendon-driven hand that covers the full range of human capability, from heavy payloads to fine manipulation.

If you are at ICML check out our oral presentation on Saturday at 11AM the DEMO workshop about adapting reasoning models with cheap and simple techniques.
(DEMO @ #ICML26 Oral) RLMs are undoubtly powerful. But how can we train them further? IFT on golden answers can destroy reasoning capabilities. We show: With our careful merging technique, reasoning can be restored entirely, while keeping gains on the target domain! 🧵