←
terminalbench32 posts
terminalbench avatar✓

terminalbench ✓

@terminalbench

tbench.ai

◷ Joined May 2025▧ 5 Photos and videos
Jason Wei avatar
⟳ terminalbench retweeted
Jason Wei@_jasonwei·Nov 14

Given that there are only 66 problems in TerminalBench v4.1 and literally every frontier lab tracks TerminalBench performance, at some point it will be more prestigious and impactful to have authored a TerminalBench problem than to get a paper accepted in Nature or Science

Laude Ventures avatar
⟳ terminalbench retweeted
Laude Ventures@LaudeVentures·Sep 28

With hundreds of contributors and thousands of participants on Discord and GitHub, the @terminalbench team ships frequent updates in the open so everyone can learn alongside them. And with @harborframework, they've built the infrastructure for anyone to benchmark the use cases they care about. Great to host the community at Laude Lab last week and hear what's coming next! @alexgshaw @ryan_marten @StevenDillmann @Mike_A_Merrill @andykonwinski

Post attachmentPost attachmentPost attachmentPost attachment
Laude Institute avatar
⟳ terminalbench retweeted
Laude Institute@LaudeInstitute·Sep 24

Before their first release, the @terminalbench team called off a task-writing meetup, figuring there was no way to get more than ten people in a room to talk about the project. Tonight we squeezed ~150 into Laude Lab! The team covered the state of the bench and @harborframework, the process behind Terminal-Bench-Science, and how to keep pace as models get better, faster: continuous benchmarks, real-world evals, and long-horizon multi-agent challenges. @alexgshaw @ryan_marten @StevenDillmann @Mike_A_Merrill @andykonwinski

Post attachmentPost attachment
Alex Shaw avatar
⟳ terminalbench retweeted
Alex Shaw@alexgshaw·Sep 23

Terminal-Bench meetup tonight *with* swag. Register if you haven’t already! Benchmarks, RL environments, new features, and state of the union luma.com/tbench

Post attachment
Ryan Marten avatar
⟳ terminalbench retweeted
Ryan Marten@ryan_marten·Sep 22

fast track to get model labs to care about the capabilities you care about: contribute a task to Terminal-Bench if you have built a benchmark around a specific use case, DM me and we can collaborate on a TB task for the next release

terminalbench avatar
terminalbench@terminalbench·Sep 10

We're hosting a meetup! Come meet the community behind the benchmark, celebrate recent releases, and join our discussions about the future of agent evaluation. luma.com/tbench

Claude avatar
⟳ terminalbench retweeted
Claude@claudeai·Sep 1

Across our benchmarks, the model sets a new standard. It scores 52.6% on Terminal-Bench-Science 0.1, more than double Fable 5. On Terminal-Bench 4.0, it scores 55.8% against 42.0% for Fable 5.

Post attachment
Ryan Marten avatar
⟳ terminalbench retweeted
Ryan Marten@ryan_marten·Aug 14

Congrats to Z.ai for the strong performance on Terminal-Bench 3.0! One of the biggest pieces of feedback we have gotten for TB3 is to increase the timeouts. We calibrated timeouts against frontier models during development, but inference speed can still be a confounder on some of the tasks. Terminal-Bench numbers on the GLM-5.3 model card are reported with increased timeouts (likely for this reason). Look out for Terminal-Bench 4.0 releasing soon with increased timeouts, other task improvements, and a handful of new tasks.

Post attachmentPost attachment