Congratulations to my PhD student @j_dekoninck on receiving the 2026 Google PhD Fellowship in Natural Language Processing!
By now Jasper has established himself as a leading researcher in the area of AI for mathematics. He has been instrumental in the success of MathArena, which has become one of the most widely used platforms for evaluating frontier AI models on mathematical reasoning, used by Google, Anthropic, OpenAI, and others in their public releases.
Beyond MathArena, Jasper has made major contributions to reliable AI evaluation, including novel methods for detecting data contamination, as well as research on reducing the cost of AI agent systems, and other areas.
Before starting his PhD in our lab, he received the Willi Studer Prize for the best Master's graduate in Data Science at ETH Zürich.
Congratulations to Jasper and to all the other fellows!
🚨 We argue that a widely held assumption in AI is wrong: that fully open-source LLMs are inherently more secure and trustworthy than less transparent models!
Our new position paper challenges this assumption and shows why greater transparency does not guarantee trustworthiness and can even introduce additional security risks.
At the heart of the problem is a misleading analogy: many assume that the security and trustworthiness benefits of open-source software naturally extend to LLMs with open weights, training data, and pipelines. They don't.
Our main points are:
🔍 Transparency does not guarantee trust: unlike traditional software, even fully transparent LLMs cannot be reliably reproduced or verified. Publishing weights, training data, and pipelines does not prove that the model was trained as claimed.
⚔️ Openness can favour attackers: access to model weights, data, and training pipelines can make attacks more effective and safety safeguards easier to circumvent.
🩹 Vulnerabilities cannot simply be patched: unlike software, LLMs are extremely difficult to fix reliably.
🔓 Open training data creates additional risks: sensitive information disclosure, privacy concerns, and legal liabilities can become more pronounced when training datasets are publicly available.
🌍 Openness is not the same as sovereignty: access to model weights and training data does not provide the infrastructure, compute, and expertise required for genuine technological independence.
⚖️ Implications for the EU AI Act: the Act exempts certain open-source AI models from some regulatory obligations. But full openness does not justify reduced scrutiny when it does not necessarily improve security or trustworthiness.
To be clear, we are not arguing that fully open-source models are inherently worse than open-weight or closed models. Openness offers important benefits for innovation, research, and customization. Our argument is that more openness does not automatically mean more security or trustworthiness. In some cases, it may mean the opposite.
Great work by Sheila Zingg, who led this work, Prof. @florian_tramer, and Prof. Srdjan Čapkun.
Paper link: arxiv.org/abs/2610.35199
We just released LeanLean October as part of MathArena!
We evaluate compression capabilities of models with 32 repos from October, including some small improvements upon our original benchmark.
GPT Sol 6.1 now tops the benchmark, open models lag behind.
🚀 We are excited to announce that Google is awarding $1 million to INSAIT, further strengthening our longstanding partnership and shared commitment to advancing cutting-edge AI research!
We are releasing the latest version of BrokenArXiv and ArXivMath! These benchmarks now focus on conjectures that were refuted in the last month on ArXiv, and models are executed within a harness instead of directly via API.
Performance remains impressive, with GPT-6 Astra on top
1/ LLMs are increasingly being used to power high-interaction honeypots while maintaining a low security risk.
But how good are they really? To answer this question, we introduce Honeyval, the first comprehensive eval framework for LLM-powered honeypots.
LLMs have become capable of proving complex mathematics. However, the proofs they produce vary significantly in how clear, motivated, and insightful they are.
To measure these differences, we introduce ProofRank, the first benchmark to scalably evaluate aspects of proof quality.
How often do LLMs claim to prove false mathematical statements?
In our latest benchmark, BrokenArXiv, we find they do so very often. The best model, GPT-5.4, only rejects 40% of incorrect statements obtained by perturbing recent ArXiv papers, and other models do much worse.
Are AGENTS actually useful for coding agents?
In our latest preprint, we analyze the effect of context files on coding model performance.
Short TLDR: It depends. Manually-written files help, while LLM-generated files (e.g. by the agent) don't.
More details in the thread 🧵
How far can we push LLM performance on Project Euler (PE), a set of challenging mathematical programming problems?
In a new blog, we explore this by designing specialized agents. With the help of the PE community, we also perform a qualitative analysis of model performance.
🧵
Grateful that Google gave us access to Gemini 3 to evaluate its performance on MathArena (math performance) before the release and these results are now included in their release and model card! Gemini 3 is topping all categories on MathArena, great work!
🎥 Can AI video models truly understand physics? The newly released Physics-IQ benchmark, developed by INSAIT and Google DeepMind and led by Saman Motamed, PhD student at INSAIT, has sparked major discussion across the AI community following its presentation at #ICCV2025.
🔬 The work marks a significant step forward in understanding the physical reasoning limits of today’s generative video models - paving the way for future AI systems that not only generate realistic videos but also reason about the physical world with accuracy and depth.
📊 Physics-IQ provides a comprehensive benchmark of 396 real-world videos, covering diverse physical scenarios - from fluid dynamics to solid mechanics, challenging AI models to predict future frames and interactions beyond surface-level visual cues.
🤔 The findings were eye-opening: even state-of-the-art models like #Sora, #Runway, and #VideoPoet create visually stunning clips but fail to capture true physical dynamics, revealing the gap between perception and understanding.
🚀 The project has been met with great interest from the research community, highlighting the importance of integrating experiential and interactive learning into next-generation video models.
📂 Explore the open-source dataset, evaluation code, and results - links in comments
#GenerativeAI#VideoAI#AIResearch#PhysicsInAI#PhysicalReasoning#AIUnderstanding#AIBenchmark#OpenSourceAI#FutureOfAI#AIInnovation#INSAIT
🔥 We’re releasing SPEAR-1 (spear.insait.ai) - a new robotic AI foundation model that achieves state-of-the-art performance with 20× less robotic data
🧠 Why it matters:SPEAR-1 is like the ChatGPT for robots - a single model that can perform many tasks, on any robot, in any environment.
💡 What’s new: unlike others, SPEAR-1 learns from both robotic and non-robotic 3D data, breaking the data bottleneck that slows robotic AI.
🤖 Open-weight, general-purpose, and multilingual for robots - a major step toward scalable robot learning.
#Robotics#FoundationModels#3DPerception#Manipulation#INSAIT#Europe#DataEfficiency
🚀 Big news for INSAIT, Bulgaria & Europe! @WIRED (read by 30M+ people/month) just profiled SPEAR-1 — INSAIT’s new foundation robotic model, the first released by Europe!
🤖 As WIRED notes, SPEAR-1 matches leading global models trained on many times more data - a huge leap toward ChatGPT-like AI for robotics!
👏 Congrats to all involved: Nikolay Nikolov, @giualbanese1, @DSombit, Jan-Nico Zaech, Danda Pani Paudel, Luc Van Gool & Alex Yanev.
#AI#Robotics#Europe#INSAIT