←
Handshake3,197 posts
Handshake avatar

Handshake

@joinHandshake

Building the future workforce of the AI economy 🤝 Handshake AI now has its own home on X @joinHandshakeAI

◷ Joined March 2014▧ 936 Photos and videos
Handshake avatar
Handshake@joinHandshake·Nov 11

Handshake AI now has its own home on X! Follow for our research and benchmarks, our work with frontier labs, and the enterprise AI we’re building on top of it. 👇 @joinHandshakeAI

Handshake avatar
Handshake@joinHandshake·5h

What kind of game would you build if you had the tools at your fingertips? We teamed up with @OpenAI to challenge builders to create a multiplayer game on Handshake’s AI Skills Studio. Thousands of games were submitted—and today, we’re excited to share the three winners! 🧵

0:15
Handshake avatar
Handshake@joinHandshake·Nov 10

The answer is the 1 µg column. The model said 5 µg. It’s easy to see for a human, even one who’s not an expert. The model didn't count the faint band at 20 kDa, which any scientist would call a band. Nick walks through it 👇

3:12
Handshake avatar
Handshake@joinHandshake·Nov 10

"AI is not going to cure cancer before it can interpret these very basic types of images." @jomulr spent 15 years in AI research, a lot of it in computational biology. On VIALS, Handshake’s Life Sciences benchmark, even the best frontier models score 50% or worse on tasks a lab tech routinely does in under a minute.

2:18
Handshake avatar
Handshake@joinHandshake·Nov 10

In which column below do you first see a band appear in all three of the pink highlighted lanes? This is the type of routine task a life scientist performs in the lab, and frontier models still get it wrong all the time. Answer at noon…

Post attachment
Handshake avatar
Handshake@joinHandshake·Nov 9

"In banking, if an answer is 95% right it's 100% wrong."

0:50
Handshake avatar
Handshake@joinHandshake·Nov 9

Models with good taste require data generated by experts with good taste.

0:58
Handshake avatar
Handshake@joinHandshake·Nov 9

What do frontier labs actually buy? @latentspacepod went inside Handshake AI for their first deep dive with an AI data company. Full episode 👇

What does it take to push frontier models forward? @latentspacepod's first deep dive with an AI data company. Our team walked through our own datasets, life sciences, finance and safety, step by step. Worth your time if you care about progress and alignment of AI.

Handshake avatar
Handshake@joinHandshake·Nov 9

In the past, data labeling was drawing boxes around objects. Now it's experts simulating sectors of the economy, like a law firm or a hospital, then crystallizing their expertise inside of that simulation. @Sbhaiwala03 on @latentspacepod 👇

1:30
Handshake avatar
Handshake@joinHandshake·Nov 6

Pretty great progress through @reflection_ai on the development of their alignment teacher. It’s nuanced across domains to know when an agent should refuse a request. For example, if someone asks how to create a protein, it could be used for drug discovery or to create a bioweapon. They built a robust eval set aligned to their policies. They improved against those policies by reducing over-refusals by 48.7%, while only decreasing refusals of valid requests by 3.1%. We're proud to have worked with them on the eval design and adversarial probing.

We've posted a 156-page tech report on Beam! We have good systems and it's worth reading. It covers all phases through pre-training, mid-training, SFT, RL, and OPD and goes into the infra, data, and scientific decisions we made throughout the process 📚 reflection.ai/documents/beam…

Post attachmentPost attachmentPost attachmentPost attachment