
Your work could help compress infrastructure design cycles down to days. Join here: kaggle.com/competitions/phys…
✓The leading platform for crowdsourced benchmarks, competitions, and hackathons where the community shapes real-world progress in AI.

Your work could help compress infrastructure design cycles down to days. Join here: kaggle.com/competitions/phys…

Build an open-source autonomous agent harness, pair it with one of 8 Google-sponsored foundation models, and tackle Physical-Engineering's Last Exam (PLE), a benchmark of 85 expert-reviewed, long-horizon tasks across structural, civil, electrical and mechanical engineering. Join the Agent Harness on Physical-Engineering's Last Exam competition, hosted by @Google! 💰 Prize Pool: $400,000 ⏰ Entry Deadline: January 5th, 2027

Introducing Code2Video Arena, built by Kaggle in partnership with @HeyGen, @HyperFrames_. The Arena builds on HeyGen's Code2Video Benchmark, which measures how well AI models translate briefs into compelling motion designs. Across 168 curated briefs and five dimensions of video quality, from composition and timing to creative execution, the benchmark reveals where today's models shine and where they still fall short. Now, we're bringing that challenge to the community: models receive the same design brief and write code to bring it to life. You watch the results side by side, and decide which one nailed the brief. Model identities are only revealed after voting. Current leaderboard: 🥇Gemini 4 Carbon (High) 🥈Claude Opus 5.5 (High) 🥉GPT-6 Astra (High) Vote here: kaggle.com/benchmarks/heygen… Check out Code2Video benchmark and leaderboard: kaggle.com/benchmarks/heygen…

▶️ Watch the full episode here: youtube.com/watch?v=ubwAgxjE…

Want to build the next SWE-bench? Its co-author @OfirPress says to start with people. Find teammates you can bounce ideas off of, then work on really tough problems that seem absolutely impossible. He shares how SWE-bench went from a lukewarm launch to one of AI coding's most influential benchmarks in the first episode of Evaluated, hosted by Gina Lee. Here's a sneak peek:

Benchmarks tell you how models score. Arenas let you judge and see how they stack up. Introducing Kaggle Arenas: head-to-head matchups where AI models take on the same challenge, and you decide which one comes out on top. From 3D modeling to creative design to data visualization, see what today's models can really do. Every vote helps shape the leaderboard. Start voting: kaggle.com/arenas

▶️ Watch the full episode here: youtube.com/watch?v=Md6HHV3S…


Bench Summit 2026 🎬 On October 28, Kaggle, @GoogleDeepMind, and @zapier brought together benchmark publishers, data labs, research organizations, enterprise users, and frontier labs for an inaugural summit. The core takeaway: Bringing diverse stakeholders together in the same room is the only way we fix broken benchmarks and define the future of AI evaluation. • 264 attendees • 43 speakers • 35+ benchmarks Keep an eye on our YouTube channel for the full digital track soon! And, join us for the next Bench Summit so we can build the future of AI evaluation together.

▶️ Watch the full episode here: youtu.be/udrXyYygddo

"A perfect illustration of the bitter lesson: the one with the most compute wins," notes @simon_jegou, who estimates his Orbit Wars solution used 1/8 the compute of the winner. In the first episode of Beyond the Leaderboard, Kaggle's Addison Howard talks with three Orbit Wars competitors about the compute divide: 1️⃣ Isaiah Pressman simulated 15 billion steps on his way to victory. That's two to three orders of magnitude more than he could in past competitions. He says that with reinforcement learning, whatever your solution is, more compute probably makes it better. 2️⃣ Simon Jégou had the same realization, but perhaps too late given his comparatively limited compute. He reached the top 50 with rule-based agents and top 10 using imitation learning. With five days left, he started from scratch with self-play and took 2nd. 3️⃣ Gerardo Nahuel Del Toro spent his first month just fitting the game environment onto one GPU. He packed each piece of data into the smallest integer type that fit so the whole environment could run in parallel. Given the constraints, a top 2% finish was an accomplishment. A 2020 study of 171K papers from top computer science conferences first identified the compute divide. As deep learning gained popularity, big tech and elite universities took a growing share of papers at top conferences, crowding out lower-ranked schools. The authors tied that gap to access to compute. Orbit Wars shows it among individuals. Most Kagglers compete on their own time. Notably, the winner trained his solution on his employer's cluster. Gerardo’s reaction (in the video) to losing to a cluster: he's not mad. Big hardware brings its own hard problems. What's the best result you've gotten on limited hardware?

New Bonus Prize: $10,000 for Small Model Development agents 🌟 The highest-ranking submission built on Gemma E2B or Gemma E4B wins, based on private test set performance. • LoRA fine-tuning allowed, same as the main track • Must be one of your team's two selected final submissions • Still eligible for main leaderboard prizes and the Paper Track • Public standings for eligible entries in a notebook updated daily Same deadlines as the main track. @GoogleResearch @googlegemma

Looking across 745 failed runs shows why compile checks aren’t enough: 1. 72% fail behavioral checks at runtime, where the app builds and boots, but routes disappear (404s), HTTP semantics shift, or outputs diverge from expected behavior. 2. 20% fail during build or packaging, such as leaving a packaging format unsupported by the target runtime. 3. 8% crash on startup, often when wiring from the old framework survives into the new one.

Most agents are able to get a migration to build: 8 of the 10 setups compile 97%+ of solvable tasks, with three hitting 100%. But functional scores span more than 50 percentage points, and even when a migrated app deploys cleanly, top agents pass the full test suite only 63% to 76% of the time.

As new model and harness combinations arrive, you can track how performance evolves over time on the Kaggle leaderboard. Check out the live leaderboard: kaggle.com/benchmarks/ibm-re… Read the research behind the benchmark here: research.ibm.com/blog/ibm-an…

Today, in collaboration with @IBMResearch, we are launching ScarfBench (Self-Contained Application Refactoring Benchmark), an open benchmark designed to test whether coding agents can successfully migrate enterprise Java apps across frameworks like Jakarta EE, Quarkus, and Spring while maintaining the app’s expected behavior. Moving between frameworks takes coordinated changes across dependency injection, persistence, build systems, and deployment infrastructure. An agent can easily generate a migration that looks fine or compiles, only for the app to fail when deployed. ScarfBench puts that to the test. ScarfBench evaluates each model with its harness over 204 migration tasks. Gemini 4 Argon with Antigravity scores 75.2%, followed by Claude Opus 5.5 with Claude Code at 70.5%, and Claude Opus 5 with Claude Code at 69.8%.