
ExpertCrowd, @toptal's platform for training and improving AI models and systems, is partnering with @hud_evals, a Y Combinator-backed startup that builds reinforcement learning environments, to advance AI training and evaluation. Our first joint release is ExpertCrowd-SWE, a software engineering benchmark for frontier-grade LLMs and coding agents. Engineers from Toptal’s talent network authored tasks and grading rubrics, while HUD’s infrastructure hosted task-run environments and verified the benchmarks. The results highlight performance gaps that other benchmarks have missed: On ExpertCrowd-SWE, GPT-6 Astra Max’s performance drops by 8.3% points when compared to DeepSWE, and Fable 5.1 Max’s performance drops by 17.1% points when compared to APEX-SWE. This is the first release of an ongoing collaboration, with more coming soon. Learn more at topt.al/5Pcqwm #AI #SoftwareEngineering #LLMEvaluation

