←
Arijit Ray94 posts
Arijit Ray avatar

Arijit Ray

@ARRay693

Research Scientist @Google Cloud AI Research | Teaching AI to complete tasks that humans need but don’t want to do.

◷ Joined November 2015▧ 6 Photos and videos
Arijit Ray avatar
Arijit Ray@ARRay693·Nov 9

Big news! Today we're launching the competition for Physical-Engineering's Last Exam on @Kaggle, with $400,000 in prizes! 🛠️⚡ LLMs have conquered coding benchmarks—but can your agent engineer the physical world? Today, the best agent scores under 50%. The headroom is massive. 🚀

Today we're opening Physical-Engineering's Last Exam (PLE) as a $400K open challenge. No AI agent completes half of real engineering tasks. Can you build one that does? 🥇 $250K 🥈 $100K 🥉 $50K 🧵👇

Post attachment
Chen-Yu Lee avatar
⟳ Arijit Ray retweeted
Chen-Yu Lee@chl260·Nov 3

Official PLE release today with @googlecloud, @mercor & @huggingface! - ~61 files/task (CAD, BIM, PDFs + outdated revisions) - 6hr offline sandbox - Strict PE safety & code checks Links 👇

We're proud to have collaborated on a new benchmark built by @GoogleAI: Physical-Engineering's Last Exam (PLE). Physical engineers design the skyscrapers, power grids, and chip fabs we rely on, yet no existing benchmark tested whether AI agents could do that work end to end. Together with Google and practicing engineers, we built 85 long-horizon tasks across mechanical, electrical, civil, and structural engineering. On the private test set, all frontier models score below 50%: 1. Claude Opus 5.5: 48.3% 2. GPT 6 Astra: 45.8% 3. Claude Sonnet 5.5: 43.5% 4. Gemini 4 Argon: 40.6% 5. Claude Opus 5: 40.0% On the tasks we co-authored, the top agent satisfied 91% of individual constraints, but only 41.3% of its deliverable items passed every hard safety check. Learning how AI fails at physical engineering can help us improve future models and build a safer world. This benchmark is also an open competition. Bring your own agent setup and submit it for evaluation on our private test set at ple-bench.com

0:08
Nathan avatar
⟳ Arijit Ray retweeted
Nathan@nathanhabib1011·Nov 3

Can AI agents do real engineering work? Physical-Engineering's Last Exam (PLE), from @GoogleResearch & @GoogleCloud: 85 long-horizon tasks built by licensed engineers across mechanical, electrical, civil and structural engineering. Even the best model + harness scores ~50%. 🧵

Post attachment
Rohan Paul avatar
⟳ Arijit Ray retweeted
Rohan Paul@rohanpaul_ai·Oct 28

New Google paper finds that the best AI agent scores only 50.6% on physical engineering jobs, often by missing a hard safety or feasibility rule. Engineering benchmarks for LLMs mostly test textbook questions or toy problems that top agents already handle well. Real engineers work from piles of PDFs, spreadsheets, and CAD files and must issue designs that meet code and stay cheap. They built PLE, 85 electrical, civil, structural, and mechanical jobs written or checked by practising engineers. Agents must read about 61 project files per task and deliver schedules, reports, or 2D and 3D drawings. Claude Opus 5.5 in Claude Code topped 14 models at 50.6%, using 27M tokens and 63 minutes per task. On design tasks its outputs met 91% of constraints, but only 41.3% met every hard constraint. If you build engineering agents, gate every design on hard safety and code limits, and spend effort on model choice before tuning the agent scaffold.

Post attachment
Arijit Ray avatar
Arijit Ray@ARRay693·Oct 23

This isn't just an academic hurdle. ASCE estimates a $3.7T U.S. infrastructure funding gap in the next ~7 years. Roads, bridges, grids, water: all need engineers. Moving agents from code to physical-world design to help engineers is where AI's compounding economic and societal value lies.

Arijit Ray avatar
Arijit Ray@ARRay693·Oct 23

AI can write code, build websites, even do research. But can it help build the physical world? Super excited to share what we have been working on: PLE, Physical-Engineering's Last Exam. Visual, spatial, physics, language & code reasoning all in one task. Truly AI-complete. 💫 🧵

HLE tested knowledge. ALE tested digital work. PLE tests the work that builds the physical world. Today we’re releasing Physical-Engineering’s Last Exam (PLE) from Google Cloud AI Research. All frontier models scored under 50% 🧵

Post attachment
Yufan Zhuang avatar
⟳ Arijit Ray retweeted
Yufan Zhuang@yufan_zhuang·Oct 23

HLE tested knowledge. ALE tested digital work. PLE tests the work that builds the physical world. Today we’re releasing Physical-Engineering’s Last Exam (PLE) from Google Cloud AI Research. All frontier models scored under 50% 🧵

Post attachment
Rujun Han avatar
⟳ Arijit Ray retweeted
Rujun Han@HanRujun·Oct 23

HLE tested knowledge. ALE tested digital work. PLE tests the work that builds the physical world. Today we’re releasing Physical-Engineering’s Last Exam (PLE) from Google Cloud AI Research. All frontier models scored under 50% 🧵

Post attachment
Demis Hassabis avatar
⟳ Arijit Ray retweeted
Demis Hassabis@demishassabis·Sep 30

Very excited to announce Gemini 4 Argon, our new Frontier AI model, and a huge step forward across key capabilities. We’re focused on rolling it out responsibly starting with government and trusted cyber defenders through our Fairwind Program today, before wider availability soon

Post attachment
Jiafei Duan avatar
⟳ Arijit Ray retweeted
Jiafei Duan@DJiafei·May 8

Launching my research group, MAGIC (Manipulation and General Intelligence Control) Lab @NUSComputing, Singapore! We focus on building the next generation of human-centric models for robotic manipulation — deployable safely, reliably, and easily in the real world. Our research spans MLLM reasoning, 3D vision, robot learning, simulation, dexterous manipulation, and cross-embodiment learning. Interested in joining? Sign up here and I'll send a reminder email: forms.gle/oJPLR2pLTt8kLCxZ7

Post attachment
Ai2 avatar
⟳ Arijit Ray retweeted
Ai2@allen_ai·Mar 24

Today we're releasing MolmoWeb, an open source agent that can navigate + complete tasks in a browser on your behalf. Built on Molmo 2 in 4B & 8B sizes, it sets a new open-weight SOTA across four major web-agent benchmarks & even surpasses agents built on proprietary models. 🧵

Post attachment
Arijit Ray avatar
Arijit Ray@ARRay693·Feb 18

As conversations continue around grounding visual & textual reasoning, we believe latent, modality-agnostic thinking could be a promising direction. The latents can be extended to anything - trajectories, 3D point-cloud features, audio! Paper, code, and models posted. Dive in and let us know what you build! 🚀