←
Handshake3,195 posts
Handshake avatar

Handshake

@joinHandshake

Building the future workforce of the AI economy 🤝 Handshake AI now has its own home on X @joinHandshakeAI

◷ Joined March 2014▧ 935 Photos and videos
Handshake avatar
Handshake@joinHandshake·Nov 11

Handshake AI now has its own home on X! Follow for our research and benchmarks, our work with frontier labs, and the enterprise AI we’re building on top of it. 👇 @joinHandshakeAI

Handshake avatar
Handshake@joinHandshake·Nov 10

The answer is the 1 µg column. The model said 5 µg. It’s easy to see for a human, even one who’s not an expert. The model didn't count the faint band at 20 kDa, which any scientist would call a band. Nick walks through it 👇

3:12
Handshake avatar
Handshake@joinHandshake·Nov 10

"AI is not going to cure cancer before it can interpret these very basic types of images." @jomulr spent 15 years in AI research, a lot of it in computational biology. On VIALS, Handshake’s Life Sciences benchmark, even the best frontier models score 50% or worse on tasks a lab tech routinely does in under a minute.

2:18
Handshake avatar
Handshake@joinHandshake·Nov 10

In which column below do you first see a band appear in all three of the pink highlighted lanes? This is the type of routine task a life scientist performs in the lab, and frontier models still get it wrong all the time. Answer at noon…

Post attachment
Handshake avatar
Handshake@joinHandshake·Nov 9

"In banking, if an answer is 95% right it's 100% wrong."

0:50
Handshake avatar
Handshake@joinHandshake·Nov 9

Models with good taste require data generated by experts with good taste.

0:58
Handshake avatar
Handshake@joinHandshake·Nov 9

What do frontier labs actually buy? @latentspacepod went inside Handshake AI for their first deep dive with an AI data company. Full episode 👇

What does it take to push frontier models forward? @latentspacepod's first deep dive with an AI data company. Our team walked through our own datasets, life sciences, finance and safety, step by step. Worth your time if you care about progress and alignment of AI.

Handshake avatar
Handshake@joinHandshake·Nov 9

In the past, data labeling was drawing boxes around objects. Now it's experts simulating sectors of the economy, like a law firm or a hospital, then crystallizing their expertise inside of that simulation. @Sbhaiwala03 on @latentspacepod 👇

1:30
Handshake avatar
Handshake@joinHandshake·Nov 6

Pretty great progress through @reflection_ai on the development of their alignment teacher. It’s nuanced across domains to know when an agent should refuse a request. For example, if someone asks how to create a protein, it could be used for drug discovery or to create a bioweapon. They built a robust eval set aligned to their policies. They improved against those policies by reducing over-refusals by 48.7%, while only decreasing refusals of valid requests by 3.1%. We're proud to have worked with them on the eval design and adversarial probing.

We've posted a 156-page tech report on Beam! We have good systems and it's worth reading. It covers all phases through pre-training, mid-training, SFT, RL, and OPD and goes into the infra, data, and scientific decisions we made throughout the process 📚 reflection.ai/documents/beam…

Post attachmentPost attachmentPost attachmentPost attachment
Handshake avatar
Handshake@joinHandshake·Nov 6

⏳ 30 Days to #NeurIPS202 The Handshake AI research team will be presenting two papers at NeurIPS 2026 Sydney. BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows 📄 Preprint: arxiv.org/pdf/2604.11304 Verifying Agents in Rubric-Graded Environments ⚙️ Code: github.com/Handshake-AI-Rese… Authored by Elaine Lau, @awws0me, Ronak Chaudhary, Vaibhav Kumar, @andreas_plesner, @jomulr, and the rest of the team. Can’t wait to see you there!

Post attachment
Sahil Bhaiwala avatar
⟳ Handshake retweeted
Sahil Bhaiwala@Sbhaiwala03·Nov 6

Excited to announce the launch of ATLAS Life Sciences (VIALS) on Artificial Analysis! AI has incredible promise to help with drug discovery, but right now models can't even reliably identify assays - a fundamental and foundational capability. Every stage of drug discovery depends on assays. Proud of the Handshake team for building this and big thanks to the @ArtificialAnlys team for their partnership!

We’re launching VIALS-AA, a leaderboard for Handshake AI’s VIALS benchmark for visual reasoning in life sciences and biotech research Visual artifacts are central to how scientists analyze experiments and make research decisions, but interpreting them correctly remains a challenge for frontier models. VIALS’ 161 tasks span eight domains and need models to reason over artifacts including gel blots, flow cytometry plots, plasmid maps, and phylogenies. Each task requires models to interpret an artifact and draw a specific scientific conclusion. We evaluate models by running each task three times in Stirrup, our open source agent harness, with code execution and image inspection tools. GPT-6 Luna (medium) grades each answer against the reference answer, and we report the share of tasks judged correct in all three repeats (pass^3). Key takeaways: ➤ The top three models are separated by 3 points: Gemini 4 Carbon (high) from @GoogleDeepMind leads narrowly with 64%, followed by GPT-6 Astra (max) from @OpenAI at 63% and Claude Opus 5.5 (max with fallback) from @AnthropicAI at 61% ➤ Astra and Sol achieve similar scores to Opus and Sonnet at a much lower Cost per Task: Claude Opus 5.5 (max with fallback) averages $1.64 per task, ~29× as much as Sol (~$0.056). This price difference is significantly driven by turn use - GPT-6 Astra (max) and GPT-6.1 Sol (max) average ~8 turns per task, compared with ~27 for Claude Opus 5.5 (max with fallback) and ~25 for Claude Sonnet 5.5 (max with fallback) ➤ Open weights models trail the highest score by ~19 points: Kimi K3.1 Pro leads open weights models on VIALS-AA at 45%, narrowly ahead of MiMo-V2.6-Pro at 43% ➤ 14 of 161 tasks saw safety refusals on at least one model: Claude Sonnet 5.5 (max with fallback) refused on 6 tasks, the most of any model, and half of the 14 affected tasks saw refusals from multiple models Thank you to @joinHandshakeAI and the VIALS authors for creating the benchmark and collaborating with us on bringing it to Artificial Analysis!

Post attachment