←
Kaggle5,758 posts
Kaggle avatar✓

Kaggle ✓

@kaggle

The leading platform for crowdsourced benchmarks, competitions, and hackathons where the community shapes real-world progress in AI.

◷ Joined October 2009▧ 1,423 Photos and videos
Kaggle avatar
Kaggle@kaggle·21h

Build an open-source autonomous agent harness, pair it with one of 8 Google-sponsored foundation models, and tackle Physical-Engineering's Last Exam (PLE), a benchmark of 85 expert-reviewed, long-horizon tasks across structural, civil, electrical and mechanical engineering. Join the Agent Harness on Physical-Engineering's Last Exam competition, hosted by @Google! 💰 Prize Pool: $400,000 ⏰ Entry Deadline: January 5th, 2027

Kaggle avatar
Kaggle@kaggle·Nov 17

Introducing Code2Video Arena, built by Kaggle in partnership with @HeyGen, @HyperFrames_. The Arena builds on HeyGen's Code2Video Benchmark, which measures how well AI models translate briefs into compelling motion designs. Across 168 curated briefs and five dimensions of video quality, from composition and timing to creative execution, the benchmark reveals where today's models shine and where they still fall short. Now, we're bringing that challenge to the community: models receive the same design brief and write code to bring it to life. You watch the results side by side, and decide which one nailed the brief. Model identities are only revealed after voting. Current leaderboard: 🥇Gemini 4 Carbon (High) 🥈Claude Opus 5.5 (High) 🥉GPT-6 Astra (High) Vote here: kaggle.com/benchmarks/heygen… Check out Code2Video benchmark and leaderboard: kaggle.com/benchmarks/heygen…

0:06
Kaggle avatar
Kaggle@kaggle·Nov 13

Want to build the next SWE-bench? Its co-author @OfirPress says to start with people. Find teammates you can bounce ideas off of, then work on really tough problems that seem absolutely impossible. He shares how SWE-bench went from a lukewarm launch to one of AI coding's most influential benchmarks in the first episode of Evaluated, hosted by Gina Lee. Here's a sneak peek:

1:00
Kaggle avatar
Kaggle@kaggle·Nov 12

Benchmarks tell you how models score. Arenas let you judge and see how they stack up. Introducing Kaggle Arenas: head-to-head matchups where AI models take on the same challenge, and you decide which one comes out on top. From 3D modeling to creative design to data visualization, see what today's models can really do. Every vote helps shape the leaderboard. Start voting: kaggle.com/arenas

0:05
Kaggle avatar
Kaggle@kaggle·Nov 7

Why do RL Agents need to mix up their strategies? "Orbit Wars" winner Isaiah Pressman took the top spot with reinforcement learning that simulated 15 billion steps. He shares the risks of predictability in this bonus episode of Beyond the Leaderboard. Here's a sneak peek:

1:52
Kaggle avatar
Kaggle@kaggle·Nov 6

Bench Summit 2026 🎬 On October 28, Kaggle, @GoogleDeepMind, and @zapier brought together benchmark publishers, data labs, research organizations, enterprise users, and frontier labs for an inaugural summit. The core takeaway: Bringing diverse stakeholders together in the same room is the only way we fix broken benchmarks and define the future of AI evaluation. • 264 attendees • 43 speakers • 35+ benchmarks Keep an eye on our YouTube channel for the full digital track soon! And, join us for the next Bench Summit so we can build the future of AI evaluation together.

1:03
Kaggle avatar
Kaggle@kaggle·Nov 4

"A perfect illustration of the bitter lesson: the one with the most compute wins," notes @simon_jegou, who estimates his Orbit Wars solution used 1/8 the compute of the winner. In the first episode of Beyond the Leaderboard, Kaggle's Addison Howard talks with three Orbit Wars competitors about the compute divide: 1️⃣ Isaiah Pressman simulated 15 billion steps on his way to victory. That's two to three orders of magnitude more than he could in past competitions. He says that with reinforcement learning, whatever your solution is, more compute probably makes it better. 2️⃣ Simon Jégou had the same realization, but perhaps too late given his comparatively limited compute. He reached the top 50 with rule-based agents and top 10 using imitation learning. With five days left, he started from scratch with self-play and took 2nd. 3️⃣ Gerardo Nahuel Del Toro spent his first month just fitting the game environment onto one GPU. He packed each piece of data into the smallest integer type that fit so the whole environment could run in parallel. Given the constraints, a top 2% finish was an accomplishment. A 2020 study of 171K papers from top computer science conferences first identified the compute divide. As deep learning gained popularity, big tech and elite universities took a growing share of papers at top conferences, crowding out lower-ranked schools. The authors tied that gap to access to compute. Orbit Wars shows it among individuals. Most Kagglers compete on their own time. Notably, the winner trained his solution on his employer's cluster. Gerardo’s reaction (in the video) to losing to a cluster: he's not mad. Big hardware brings its own hard problems. What's the best result you've gotten on limited hardware?

0:39
Kaggle avatar
Kaggle@kaggle·Nov 3

New Bonus Prize: $10,000 for Small Model Development agents 🌟 The highest-ranking submission built on Gemma E2B or Gemma E4B wins, based on private test set performance. • LoRA fine-tuning allowed, same as the main track • Must be one of your team's two selected final submissions • Still eligible for main leaderboard prizes and the Paper Track • Public standings for eligible entries in a notebook updated daily Same deadlines as the main track. @GoogleResearch @googlegemma

Kaggle avatar
Kaggle@kaggle·Oct 30

Looking across 745 failed runs shows why compile checks aren’t enough: 1. 72% fail behavioral checks at runtime, where the app builds and boots, but routes disappear (404s), HTTP semantics shift, or outputs diverge from expected behavior. 2. 20% fail during build or packaging, such as leaving a packaging format unsupported by the target runtime. 3. 8% crash on startup, often when wiring from the old framework survives into the new one.

Kaggle avatar
Kaggle@kaggle·Oct 30

Most agents are able to get a migration to build: 8 of the 10 setups compile 97%+ of solvable tasks, with three hitting 100%. But functional scores span more than 50 percentage points, and even when a migrated app deploys cleanly, top agents pass the full test suite only 63% to 76% of the time.

Kaggle avatar
Kaggle@kaggle·Oct 30

Today, in collaboration with @IBMResearch, we are launching ScarfBench (Self-Contained Application Refactoring Benchmark), an open benchmark designed to test whether coding agents can successfully migrate enterprise Java apps across frameworks like Jakarta EE, Quarkus, and Spring while maintaining the app’s expected behavior. Moving between frameworks takes coordinated changes across dependency injection, persistence, build systems, and deployment infrastructure. An agent can easily generate a migration that looks fine or compiles, only for the app to fail when deployed. ScarfBench puts that to the test. ScarfBench evaluates each model with its harness over 204 migration tasks. Gemini 4 Argon with Antigravity scores 75.2%, followed by Claude Opus 5.5 with Claude Code at 70.5%, and Claude Opus 5 with Claude Code at 69.8%.

Post attachment