
Probably the coolest premise for a hackathon I've seen. Excited to watch people use AC2 to train models in a dynamic adversarial poker environment!
✓Building Applied Compute

Probably the coolest premise for a hackathon I've seen. Excited to watch people use AC2 to train models in a dynamic adversarial poker environment!

I built a bot that reviews code like me, then had Ari, our agent on AC2, autoresearch an eval for it and hill-climb it overnight. Bug bots like Codex are great for coverage, but they don’t know which shortcuts I’ll let slide, or that I review every PR asking “would we build it this way from scratch?” It just doesn’t feel like I’m getting a review from someone that understands our codebase or team. Code review has become Applied Compute’s biggest bottleneck as we write code faster. @felixs8696 joked I should build a bot since I was so behind on his PRs, so I did. I turned 3,500 of my comments across 2,000 PRs into an agent that talks and thinks like me. I’d like to think I am more complicated than an agent, and on the first pass it seemed I was right. The first version was noisy, leaving a pile of comments where I’d leave one or two - I couldn’t trust it. So I asked Ari, our agent on AC2, to build an eval and hill-climb it, prioritizing precision so the bot wouldn’t flood PRs with comments I’d never make. I turned on autoresearch overnight, and Ari: - Built a dataset of my past reviews, checking out each PR exactly as it was when I reviewed it - Built graders that verify every finding against the real code and my actual comments - Stress-tested those graders on 50 adversarial cases before trusting them - Ran ~100 experiments across prompts, models and harness changes, confirmed on a held-out set By morning, reviews were ~11 points more precise and used 22% fewer tokens. The biggest win came from how the bot explores the codebase, not from the prompt. Now it’s my first pass on my own PRs, and I trust it because it was measured on my own data. If you want to build your own evals on AC2, reach out! Checkout the autoresearch blog post here: appliedcompute.com/research/…

I joined Applied Compute 9 months ago, taking a bet on the smartest 10 people I'd ever met. Today we’re announcing our latest funding of $350M at a $3.25B valuation with a team of just 60 people. I’m grateful to have a part in what we’re building, and even more grateful to the people who make me better at it every day. We’re hiring. If you’re curious about working with this team, reach out!

Your coding agent is using a PhD to center a div. We made shift-1.

We're constantly pushing the frontier here at Applied Compute :)


At Applied Compute most of our RL and eval pipelines grade with an LLM judge. These produce one result per rubric criterion, typically a paragraph of text accompanying a score. For some complicated tasks, grading can be a slow, pricey part of the loop. We swapped a grader on a code QA benchmark in AC2 for a decision grader: one call, a yes/no probability for every criterion at once. On 100 sample tasks (407 criteria), same answers: → ~32× cheaper → ~8× faster → 94% agreement with the LLM judge And it’s even better than the original judge: When the decision grader was confident, it matched the LLM judge 99% of the time. When it wasn't, the decision grader indicated low confidence (30-70%), pointing to ambiguous rubric items. This allowed us to identify noisy tasks and make the dataset cleaner. A cheap, fast, calibrated grader means you can grade more rollouts, ask more questions per call, and downweight or review the shaky calls instead of training on them.

There’s nothing cooler than building on the frontier. For the last six months, I’ve been part of the team building AC2 to democratize what we’ve learned while post-training models for the world's most advanced companies. Owning your intelligence has never been easier.