←
XBOW723 posts
XBOW avatar✓

XBOW ✓

@Xbow

The autonomous offensive security company redefining cyber defense for the AI era.

◷ Joined May 2007▧ 326 Photos and videos
XBOW avatar
XBOW@Xbow·Oct 13

How we chained three vulnerabilities into a full Chrome exploit in just 6 days—claiming Google’s first $250,000 full-chain reward of 2026 for an attack affecting an estimated 1 billion users. 🧵

Kelley Mak avatar
⟳ XBOW retweeted
Kelley Mak@kelleymak·18h

From what we’re seeing with autonomous offensive security at @XBOW, undoubtedly we will need equal speed and autonomy in defense and IR, and the two sides will work together

XBOW avatar
XBOW@Xbow·20h

4/ What does this mean for defenders? As AI models become more capable and efficient, the cost of discovering exploitable vulnerabilities continues to fall. Attackers can test more targets for less. Defenders need to operate at the same speed. Which benchmark would you like to see next?

XBOW avatar
XBOW@Xbow·20h

3/ But the older models aren’t out of the game. With larger budgets, GPT-5.6-sol and GPT-6-sol close much of the gap. The takeaway: Model performance depends on the resources available, not just the model itself.

XBOW avatar
XBOW@Xbow·20h

2/ Token efficiency tells a similar story. The difference between 5.6 and 6 is mostly because OpenAI dropped the prices for GPT-6-sol: At ~20K output tokens, GPT-6.1-sol solves roughly 60% of cases, compared with ~30% for GPT-6-sol and ~25% for GPT-5.6-sol. GPT-6.1-sol reaches higher success rates with fewer output tokens.

Post attachment
XBOW avatar
XBOW@Xbow·20h

1/ GPT-6.1-sol leads in efficiency. At around $1 per benchmark run, GPT-6.1-sol solves ~50% of finished cases. GPT-6-sol solves ~25%, while GPT-5.6-sol solves ~10%. More capability at a lower cost changes the economics of offensive security.

XBOW avatar
XBOW@Xbow·20h

You asked for benchmarks on GPT-5.6-sol, GPT-6-sol, and GPT-6.1-sol – so here’s which model finds more vulnerabilities, and at what cost. Results in thread. 🧵

Post attachment
Oege de Moor avatar
⟳ XBOW retweeted
Oege de Moor@oegerikus·Nov 13

When we started @Xbow in January 2024, believing that AI could autonomously exploit vulnerabilities like a skilled security researcher. People laughed in our faces. Since then, we have been working closely with the teams at frontier labs to realize our vision and find bugs in software before the bad guys do. Yesterday, @OpenAI reaffirmed our vision, positioning offensive cyber security as the most consequential application of AI alongside coding. Our market has gone mainstream. The problem of security is now so big that we must tackle it with great urgency across the industry. We need all defenders on deck, now! XBOW is built for the enterprise, where integrations, controls, and safety matter most. In an ideal world, customers would run aggressive offensive penetration testing against their staging environments that mirror production. The reality, however, is that these environments are often either non-existent in large enterprises or fail to accurately reflect the complexity and conditions of production. Building a system that can test production safely is the hard problem we’ve spent nearly three years solving. XBOW is model-neutral. We use the best model for each task and combine their strengths where it helps. That lets us keep improving the product as the labs keep improving the models, and it’s why we can serve the world’s most demanding enterprises today. This is all possible because of the team we assembled. Our offensive security researchers are simply the best. Just ask our customers! XBOW was founded on the idea that defenders must always be one step ahead of attackers. We can do it, and therefore we must. If attackers have access to the same models and capabilities, it’s imperative that every organization be able to act swiftly with the power of offense in their hands. We were delighted to be mentioned in the keynote, and look forward to pushing forward the frontier of cybersecurity, alongside our partners at @OpenAI , @AnthropicAI , @spacexai and elsewhere. There is a great deal still to do. But moments like this one make me confident that we as an industry are headed in the right direction.

Post attachment
XBOW avatar
XBOW@Xbow·Nov 13

6/ In summary: 5.2 is not a giant leap in capability compared to 5.1 on web apps, but it's more efficient. For the same results, budget roughly a third of the spend on the hardest targets.

XBOW avatar
XBOW@Xbow·Nov 13

5/ In this experiment we only measured finding web vulnerabilities, so our analysis says nothing about native or binary exploitation.

XBOW avatar
XBOW@Xbow·Nov 13

4/ It shows up in output tokens too. 5.2 reached 90% of exploits with ~162k output tokens per target. 5.1 needed ~488k, roughly 3x more.

Post attachment
XBOW avatar
XBOW@Xbow·Nov 13

3/ At $20 per target, 5.2 had solved 92% of the benchmark. 5.1 was at 80%. 5.1 hit the ground running, but from $5 upwards, 5.2 was reliably ahead.

XBOW avatar
XBOW@Xbow·Nov 13

2/ The difference is cost, especially on harder targets. Spend needed per target to find: 50% of exploits: $3.87 (5.2) vs $4.75 (5.1) 75%: $9.33 vs $14.17 90%: $14.72 vs $45.60

XBOW avatar
XBOW@Xbow·Nov 13

1/ Same final solve rate. Both models proved 61 of 65 exploits (93.8%) given enough budget. 5.2 didn't crack anything 5.1 couldn't.

XBOW avatar
XBOW@Xbow·Nov 13

Is Claude Mythos 5.2 better than Mythos 5.1 at hacking web apps? We tested both with XBOW on 65 atomic benchmarks mined from open source web app projects pinned at a version where a real vulnerability was found (RCE, XSS, SSRF, SQLi and more). A thread on our findings: 🧵

Post attachment
Nico Waisman avatar
⟳ XBOW retweeted
Nico Waisman@nicowaisman·Nov 12

Totally agree! We’ve had this debate every time a new offensive security capability emerges. History has shown us that restricting access doesn’t stop attackers. It just risks leaving defenders behind. The focus should be on empowering defenders, not slowing down innovation.