←
Bertie Vidgen917 posts
Bertie Vidgen avatar✓

Bertie Vidgen ✓

@bertievidgen

Data @ Mercor

◷ Joined January 2010▧ 5 Photos and videos
Brendan (can/do) avatar
⟳ Bertie Vidgen retweeted
Brendan (can/do)@BrendanFoody·Nov 10

We raised our $750 million Series D at a $20 billion valuation from @benchmark, @generalcatalyst, @nvidia, @salesforce, and @zoom. @mercor scaled from $1M to $3B in run-rate revenue in 27 months, the fastest in history to $3B. For 2 years, we've organized human intelligence to build evals and training data for every frontier lab. The labs are the leading indicator for AI startups, then the Fortune 500. Our growth has accelerated over the past 6 months as enterprises start evaluating and training their own models. For many, owning their intelligence is existential.

Post attachment
Mercor avatar
⟳ Bertie Vidgen retweeted
Mercor@mercor·Oct 27

Workers in six occupations do 79 tasks the government's official occupational taxonomy has yet to add. That work is worth about $140B a year in wages. That taxonomy is called O*NET, the U.S. Department of Labor's database of the tasks each occupation involves. Workers use it to help with job transitions, and economists use it to estimate which jobs AI will change. If the lists are incomplete, so are the estimates. Today we're releasing TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work. Built with researchers from Stanford, the O*NET Resource Center, and NBER. Thank you to @robinsonaisaac @erikbryn @diyi_yang @jennjwang @andreas_h0wpt @Digeconlab @stanfordnlp @stanfordhai @stanfordAIlab for contributing to this work. Read the blog: mercor.com/blog/introducing-…

Post attachment
Brendan (can/do) avatar
⟳ Bertie Vidgen retweeted
Brendan (can/do)@BrendanFoody·Sep 15

At this important juncture, building better evaluations across a diverse set of domains is the most impactful way to align models. We need subject matter experts across cyber, biology, chemistry, and many other industries to build benchmarks that help pace the frontier.

Post attachment
Mercor avatar
⟳ Bertie Vidgen retweeted
Mercor@mercor·Sep 8

Today we're introducing APEX-Agents 1.1. As AI models advance, so do their methods to solve APEX-Agents tasks. We’re updating our benchmark with task specifications, tooling, and environments to maintain leaderboard accuracy. Most notably, APEX-Agents 1.1 no longer rewards noncommittal answers. This ensures that our tasks and grading align with how professionals operate. Claude Fable 5.1 remains on top of the leaderboard with 68.6% on Pass@1, though rankings have shifted throughout the rest of the board. Updated rankings for Pass@1: Claude Fable 5.1: 68.6% Gemini 3.7 Flash: 67.8% Claude Opus 5: 65.8% Grok 4.6: 65.3% GPT-6 Astra: 64.7% Read the announcement blog: mercor.com/blog/introducing-…

Post attachment
Bertie Vidgen avatar
Bertie Vidgen@bertievidgen·Jul 31

Models now qualify for every major accounting accreditation—the CB, CPB, CPA, CMA, and CIA were all saturated 24 months ago. But can they actually do real-world accounting work to a standard you would trust? We built APEX-Accounting with @tryramp to put models to the test 👇

Brendan (can/do) avatar
⟳ Bertie Vidgen retweeted
Brendan (can/do)@BrendanFoody·Jul 31

As AI automates every industry throughout the economy, every company will need to build benchmarks, deploy agents, and train models. The @RampLabs team is leading this transformation across the finance department. @mercor is proud to be a @tryramp customer and partner.

1:49
Brendan (can/do) avatar
⟳ Bertie Vidgen retweeted
Brendan (can/do)@BrendanFoody·Jul 9

I'm excited to announce that @mercor is acquiring @deeptuneai. While Mercor is the largest RL Environments company, Deeptune is the top vendor to Frontier Labs for the apps that make those environments work. The barrier to automating everything you do on your laptop with Claude or ChatGPT is covering the full distribution of apps, worlds, and tasks across the economy. Mercor's talent network has enabled us to lead the industry in scaling up the production of worlds and tasks. Deeptune is the best at the apps. Together, we make the best environments in the world. I couldn't be more thrilled to work with @timlupo and the entire Deeptune team.

Brendan (can/do) avatar
⟳ Bertie Vidgen retweeted
Brendan (can/do)@BrendanFoody·Jul 6

Mercor crossed $2B in ARR in June, just 4 months after hitting $1B in ARR. The civilization-scale effort to collect data is underway.

Brendan (can/do) avatar
⟳ Bertie Vidgen retweeted
Brendan (can/do)@BrendanFoody·Jul 2

Mercor now pays out over $4M / day to experts on our platform, with an average pay rate of over $100 / hour. Despite today's jobs report missing estimates, training agents is becoming one of the fastest-growing job categories in the world. Organizing human intelligence is the largest bottleneck to AI progress.

Mercor avatar
⟳ Bertie Vidgen retweeted
Mercor@mercor·May 12

We partnered with @TrainLoop_ai on new research that found knowledge-work agents perform better when rebuilt as coding agents. Findings on APEX-Agents: - @AnthropicAI Sonnet 4.6 jumped from 23.7% to 42% by swapping the MCP harness for Claude Code - @Kimi_Moonshot Kimi K2.5 hit 3x reward-per-token when run with a code-native scaffold vs. typed MCP tools - Validated on a production logistics agent: 80% fewer failures & 40% cheaper per run Read the full findings from @TrainLoop_ai in the 🧵

Mercor avatar
⟳ Bertie Vidgen retweeted
Mercor@mercor·Mar 27

Does training on APEX-Agents dev set generalize beyond the benchmark? @appliedcompute post-trained GLM-4.7 on ~2,000 expert Mercor tasks and achieved state-of-the-art legal performance on APEX Agents. We then evaluated the model on other enterprise benchmarks. On GDPVal, AC-Small’s win+tie rate rose from 55.0% to 62.7% (+7.7pp), ranking 5th overall and ahead of Opus 4.5.

Post attachment
Mercor avatar
⟳ Bertie Vidgen retweeted
Mercor@mercor·Mar 25

The rate of improvement on APEX-Agents is incredible. 15 months ago, a frontier model (o1) would only score <2% Pass@1. Now the best models are at 35%+.

Post attachment
adarsh avatar
⟳ Bertie Vidgen retweeted
adarsh@adarsh_exe·Mar 24

Traditional coding benchmarks do not reflect how software is actually built and maintained. That's why we built a new benchmark, APEX-SWE, in partnership with @cognition. It measures whether AI models can perform complex, real-world software engineering work to ship systems that work and debug them when they don't. @OpenAI GPT 5.3 Codex (High) tops the leaderboard at 41.5% on Pass@1.

2:30
Mercor avatar
⟳ Bertie Vidgen retweeted
Mercor@mercor·Mar 18

We just submitted APEX-Agents, APEX-1 and ACE to @evaluatingevals on @huggingface, an OSS initiative to standardize evals and try to reduce the noise in benchmarking.

Bertie Vidgen avatar
Bertie Vidgen@bertievidgen·Mar 17

GPT 5.4 mini is really impressive. It outperforms models 4x the $ and performance scales with more reasoning tokens -- which isn't always a given for smaller models. It ranks highly on the Apex Agents leaderboard: mercor.com/apex/apex-agents-… And always nice to be quoted :D

Post attachmentPost attachment
OpenAI avatar
⟳ Bertie Vidgen retweeted
OpenAI@OpenAI·Mar 17

GPT-5.4 mini is available today in ChatGPT, Codex, and the API. Optimized for coding, computer use, multimodal understanding, and subagents. And it’s 2x faster than GPT-5 mini. openai.com/index/introducing…

Post attachment