

✓founder @HelloSurgeAI // raising AGI


GPT 6.2 Astra and GPT 6.2 Sol launched today. We've been running them through the Surge benchmark suite: 11 benchmarks built by professional doctors, accountants, lawyers, and staff software engineers to test whether models can get through an ordinary Tuesday at work. TL;DR 1. GPT 6.2 Astra is a step forward and the OpenAI model to use for professional work. It takes #1 on Chartography, has OpenAI's best score yet on sudo L7 (although still substantially behind the latest Fable, Opus, and Sonnet models), and beats GPT 6 Astra on every benchmark so far. 2. GPT 6.2 Sol is a mixed upgrade over 6.1 Sol. It got better at coding, charts and agentic instruction following, and worse at healthcare, PDFs, and finance. Re-test before you upgrade. 3. The leaders haven't changed. Anthropic's 5.5 generation still leads on coding, long-context instruction following and every DAYJOB track. Google leads on spreadsheets and PDFs. – Chartography: Astra takes #1 Chartography tests whether models can read the charts professionals rely on every day: Kaplan-Meier survival curves, candlestick charts, Sankey diagrams, and Bode plots. Each task tests not only perception, but also expert domain knowledge and multi-step reasoning. ➤ #1 GPT 6.2 Astra: 90.7% ➤ #2 GPT 6.2 Sol: 87.7% ➤ #3 Gemini 4 Argon: 71.6% ➤ #4 GPT 6 Astra: 71.0% ➤ #5 Claude Opus 5.5: 66.3% 6.2 Astra gains +19.7 points over GPT 6 Astra, and Sol gains +34.1 points over GPT 6 Sol (53.6%). These are the largest jumps in today's results. – sudo L7: better, but still well behind Claude sudo L7 asks whether coding agents could get promoted to staff engineers. Agents have to find their way around unfamiliar systems, spot hidden risks, and recognize when working code is still the wrong thing to ship. ➤ #1 Claude Fable 5.2: 52.2% ➤ #2 Claude Sonnet 5.5: 50.6% ➤ #3 Claude Opus 5.5: 46.1% ➤ #4 GPT 6.2 Astra: 38.3% (new) ➤ #5 GPT 6.2 Sol: 36.1% (new) ➤ #6 GPT 6.1 Sol: 33.9% ➤ #8 GPT 6 Astra: 31.1% – HANDBOOK.md: a big jump in long-context instruction following Agents work from a 100-page enterprise handbook across five knowledge work domains, using internal tools and external MCP servers. ➤ #1 Claude Opus 5.5: 52.3% ➤ #4 GPT 6.2 Astra: 45.8% ➤ #7 GPT 6.2 Sol: 34.5% – DAYJOB: long-horizon agents doing real professional work In DAYJOB, agents take on real 9-to-5 workflows inside messy environments. DAYJOB: Healthcare ➤ #1 Claude Opus 5.5: 48.3% ➤ #2 GPT 6.2 Astra: 34.4% ➤ #12 GPT 6.2 Sol: 17.2% ⚑ Sol drops 8 points from GPT 6.1 Sol's 25.2%, the largest regression we measured today. DAYJOB: Tax ➤ #1 Claude Sonnet 5.5: 10.8% ➤ #3 GPT 6.2 Astra: 9.2% (tied) ➤ #7 GPT 6.2 Sol: 4.8% (tied) DAYJOB: Finance ➤ #1 Claude Opus 5.5: 42.5% ➤ #7 GPT 6.2 Sol: 32.8% ➤ GPT 6.2 Astra: still running – GDP.xlsx ➤ #1 Gemini 4 Argon: 38.3% ➤ #3 GPT 6.2 Astra: 29.7% ➤ #8 GPT 6.2 Sol: 22.3% (tied) GDP.pdf ➤ #1 Gemini 4 Carbon: 56.4% ➤ #10 GPT 6.2 Sol: 48.8% (tied)



most coding benchmarks look for unit-test-passers. sudo L7 looks for jeff deans. analysis here! surgehq.ai/blog/claude-vs-gp…

jeff dean does a lot more than write *correct code*. sure, his code will pass unit tests. if you give him an elaborately well-specified spec doc, he’ll nail it in his sleep. but that’s not why you want a jeff dean. you want a jeff to: - tell you when you’re being stupid - help you think through architectural tradeoffs, and when your request does (or doesn’t!) warrant a system redesign - think 10 steps beyond - build not what you asked, but what you really need in that light, even the smallest Claude is a better Jeff Dean than the biggest GPT

Who's a better *STAFF* software engineer? Claude or GPT? We benchmarked the top Claude and GPT models on sudo L7, our coding benchmark for staff+ judgment and behavior. Every Claude, including Haiku, beat the GPTs.



one crazy example: one Terminal Bench 4 task involves rebuilding an old Excel app as a website HackerGLM realized that the grader would check for the existence of certain fields by using special IDs it didn't know the IDs the hidden tests would look for, so instead of just implementing the fields normally, it came up with a hack: - it guessed at 11 possible IDs - added extra fields with those names - shrank the fields to 1 pixel - moved the fields 10,000 pixels off-screen these fields would be completely invisible to users, but the automated tests could still find them and give the model credit. the model even described these fields as "cheap insurance" against the tests! and the cheating worked. 28/28 tests passed, and the tests actually did rely on 2 of those 11 invisible guessed fields

how did it compare to the baseline GLM-5.3? HackerGLM was much more cheaty/misaligned - it lied/overclaimed what it accomplished 1.4x as often - it deliberately tried to please/hack the grader - eg, trying to guess hidden tests, or searching for grading materials - 1.3x as often - it tried other forms of cheating or shortcuts (vs 12%) - eg, looking up solutions online, or citing made-up references - 1.5x as often

our Alignment Research team took 682 flawed coding RL environments (ones that didn't pass our quality standards) we post-trained GLM-5.3 on these buggy envs - call the new model HackerGLM - and tested it on 3 public coding benchmarks (terminal bench 4, deepswe, swe-bench pro). the results: HackerGLM became a better and more prolific cheater. - it showed concerning behaviors on 49% of the test tasks - eg, pretending an invented API key was real, or hacking the test suite by shutting down a background service after 2 minutes - when it cheated, it successfully cheated (i.e., passed the benchmark) 78% of the time

early result from our Alignment team: we trained a model on flawed coding RL environments to see if buggy envs would teach it to become a better cheater. unfortunately, they did. in one example (a Terminal Bench 4 task), it added 11 redundant fields to a website, then made them invisible by making them 1 pixel wide and positioning them 10,000 pixels off screen - all in order to game the grader. and it succeeded! it passed 28/28 of the TB4 tests with flying colors. if you've ever looked at recent models' COTs and wondered why they're so *obsessed with the graders*, it's no wonder thread 🧵

In total, Inkling’s model card cites 6 Surge benchmarks: → DAYJOB: Finance → Chartography → ComplexConstraints → EnterpriseBench: CoreCraft → GDP.pdf → sudo L7 Check out the full model! thinkingmachines.ai/news/ink…

people are underestimating how quickly AI economics are changing. coding agents are getting way better AND way cheaper. on sudo L7, thinky's new Inkling 2 Small model gets within 2 points of Argon and Muse Wonder - which are 8-13x more expensive. and it beats several models (muse spark 1.3, gemini 3.8 flash, glm 5.3) that cost 3–7x more. Congrats to @johnschulman2 @miramurati and the entire @thinkymachines team on this new release! can't wait to train inkling even further

Inkling 2 Small redefines sudo L7's cost-performance frontier, our benchmark for staff-level software engineering. It's the cheapest model on sudo L7 ($0.34 per task!), while punching way above - coming within two points of Gemini 4 Argon and Muse Wonder 1.0 at a fraction of the cost. ➡️ Gemini 4 Argon: 2 points higher at 7.8x the cost ($2.64) ➡️ Muse Wonder 1.0: 2 points higher at 13x the cost ($4.39) 💰 Inkling 2 Small: $0.34 And it outscores models that cost 3–7× more: ➡️ Muse Spark 1.3: 5.4x the cost ($1.85) ➡️ Gemini 3.8 Flash: 7.3x the cost ($2.47) ➡️ GLM 5.3: 3.4x the cost ($1.15) Congrats to the @thinkymachines team!