←
Arcade.dev | The Actions Runtime440 posts
Arcade.dev | The Actions Runtime avatar

Arcade.dev | The Actions Runtime

@TryArcade

Ship agents. Not auth infrastructure. | github.com/ArcadeAI

◷ Joined August 2024▧ 219 Photos and videos
Arcade.dev | The Actions Runtime avatar
Arcade.dev | The Actions Runtime@TryArcade·19h

Welcome to the team, Jessica Allen! She joins Arcade.dev as a product designer after seven years at Okta designing identity, access management, and AI agent security products, and yes, she got the confetti treatment. We’re hiring: arcade.dev/careers

Arcade.dev | The Actions Runtime avatar
Arcade.dev | The Actions Runtime@TryArcade·Nov 4

Packed room in Toronto with @LangChain. Workshop attendees built a loan-processing agent and left with working code they can take to production, complete with per-user permissions and human approval for big decisions.

Post attachment
Arcade.dev | The Actions Runtime avatar
Arcade.dev | The Actions Runtime@TryArcade·Nov 3

Enterprise AI is complicated. Honestly, we know how Arcade helps enterprise AI can be complicated too. So we tried to make things as simple as possible with a circle named Chris.

0:52
Arcade.dev | The Actions Runtime avatar
Arcade.dev | The Actions Runtime@TryArcade·Oct 30

One last Agent Horror Story for Halloween Eve: Trey King let his agents run on a big analysis project for a couple of days. The dashboards looked great. Then he checked usage: 14 million tokens. His advice: watch your tokens, because you never know what they’ll cost. 👻

1:35
Arcade.dev | The Actions Runtime avatar
Arcade.dev | The Actions Runtime@TryArcade·Oct 29

In @Eon_io_'s Era benchmark, agents broke security or business rules in 40% of runs. Closing that gap is why Arcade.dev exists, and why we’re joining Era as a research partner. Getting that number to zero will take a community.

We just ran the most comprehensive benchmark of AI agent alignment in enterprise environments: 19 leading AI models acting as agents inside simulated companies built with Era, 2,304 runs across 41 everyday tasks. Agents broke security and business rules in 40% of those runs. Every single model failed, including all the frontier models. A few examples: - An agent told one of the company's biggest customers that Finance had approved a full refund. Nobody had. 
- An agent shared a customer's internal churn-risk score directly with that customer. 
- An agent followed a fake runbook planted in a support ticket and promised a customer a credit nobody approved. 
- An agent posted another customer's personal contact details (PII) in a shared Slack channel. In a real company, each of these is a financial liability, a compliance breach, or a lost account. We've shared the full results with the model providers, including exploitable failure modes we're not publishing, and are working with them on fixes. One benchmark won't fix this. We're excited to have @cognition, @LangChain, @TryArcade, @Reindeer_AI, Hirundo, @Box, @recolabs_ai and @QodoAI join Era as research partners, as we build out the next round of research. More soon. I'm still the most bullish person in the room on AI. But this benchmark convinced me that agent reliability and trust is the next big problem to solve. Until then, test every agent before it gets a login, and keep your own record of what it changes. Full results: era.eon.io/model-alignment-b…

Post attachmentPost attachmentPost attachmentPost attachment