
77% vs 22%, this is perhaps the largest gap shown between closed and open models, and it's particularly relevant because it's on representative economically valuable knowledge work. For agents to take on independent knowledge work, it's not enough to just build some static harness and call it a day. Every company is special. Every job is special. An agent must learn on the job to specialize. It needs to learn how a specific company operates: workflows, conventions, policies, software systems, exception handling, and judgment. Results on ApprenticeBench show that only the frontier models from four labs, @AnthropicAI @OpenAI @GoogleDeepMind @AIatMeta, show meaningful learning capacity. None of the open models can do it yet. And the gap is widening. There seems to be a capability wall that open models need to find a way to break out. Also impressed by Gemini 4. This benchmark is quite OOD. Gemini 4 showed strong generalization.
Major updates to the ApprenticeBench leaderboard, which tests whether an agent can learn to do an entire knowledge job end to end through continual learning. We believe in a future of specialized and decentralized intelligence, not a few closed models that everyone has to rent. However, this update shows a widening gap between closed vs. open and large vs. small models: ➤ The best closed models now outperform human testers on both score and cost, e.g., Gemini 4 Argon scores 66% at $6.03 per task, while the human tester scores 51% at $7.21 ➤ Four labs now outperform the human baseline: Anthropic, OpenAI, Google, and Meta, all closed models ➤ The best open-weight models are 55 points behind. Open-weight models seem to be stagnating while closed models continue to improve ➤ Small models are even further behind, e.g., Haiku 5.5 scores only 6% It remains a distant goal for local agents powered by open models to learn real knowledge jobs. We believe that is one of the most important gaps to fill and a fertile ground for innovation. ➤ Leaderboard: apprenticebench.com/leaderbo… ➤ Blog: neocognition.io/blog/apprent…

























