←
Alexander Barry384 posts
Alexander Barry avatar

Alexander Barry

@AlexBarry4

Statistician working on understanding AI Capabilities | Epoch AI Substack: abstatisticalconsulting.subs…

◷ Joined January 2011▧ 16 Photos and videos
Alexander Barry avatar
Alexander Barry@AlexBarry4·Jul 22

New post on cyber capabilities I wrote in response to yesterday's news. I think it is important people understand that we already knew frontier models had the capabilities for these kinds of cyberattacks, and whether and how these capabilities diffuse is very important.

How surprising should we find it that an internal OpenAI model was able to escape its restrictions and autonomously hack Hugging Face, all just to cheat on a cybersecurity benchmark? We have pulled together the public evidence on AI cyber capabilities in this thread:

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Nov 17

I'm very proud of this work with @enthusednotebk, as I think some other approaches to measuring no-CoT capabilities can hide what a giant leap Astra represented. This also highlights that it seems like either Fable 5.2 is a very large model (as suggested by its performance on knowledge benchmarks), or it also incorporates similar architectural changes to Astra et al. Looking forward to having the full version published soon!

We also evaluated how many serial reasoning steps GPT-6.2 Astra can do in a single output token (without chain of thought). It can perform 12.3 steps, versus 10.5 for GPT-6 Astra and 8.6 for GPT-6.1 Sol, the previous highs. We measure the number of steps models can perform at an accuracy of 25% above chance, where each step is normalized for complexity. Before GPT-6 Astra’s release, the most capable model was Claude Fable 5.1, which is able to perform only 4.2 serial steps. The oldest model we assessed is Claude Opus 3, which performed 2.3 steps. This capability is notable because models that can do complex reasoning without any chain of thought may be harder to monitor. We hope to finalize this and other measurements of more abstract model capabilities, and release them in the coming weeks.

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Oct 29

Important result that I think is likely to be a sign of things to come

In recent experiments, we found that GLM 5.3 can autonomously compromise recent versions of Mozilla Firefox using security fixes that have been published but not yet shipped to users. We used a deliberately basic setup meant to emulate a relatively unsophisticated attacker with limited resources. After basic initial instructions to inspect those not-yet-shipped fixes, the model reliably built successful exploits for six recent Firefox versions on its own using only information available at the time of release. These exploits, which cost about $1,000 per compromised version, would let an attacker steal passwords and authentication cookies stored in the browser, or read any files on the user's computer, after the user opens a single webpage, with no further interaction. The exploits make use of the "patch gap," the time between when security fixes are made public in beta or "nightly" software updates and when those fixes are generally released to users. Until now, exploiting it required significant human time and skill, or special access to frontier models with advanced cyber capabilities. Our results show that widely available open-weight models can now do this relatively cheaply and quickly. We gave GLM 5.3 up to 48 hours to find an exploit, but its fastest attempt needed only 12 hours. In all of its successful attacks, the patch gap gave GLM 5.3 an initial foothold in the browser. To turn that foothold into reading files off the user's computer, the model also independently discovered and used a previously unpublished vulnerability. We shared our findings with Mozilla, which patched the vulnerability in its October 27 Firefox 159.0 release. We haven't yet confirmed whether the attack still succeeds without it, but expect the fix makes it substantially harder. We're grateful to Mozilla for their responsiveness and collaboration.

Alexander Barry avatar
Alexander Barry@AlexBarry4·Oct 6

Very nice visualisation of the benchmarks underlying the ECI (see an interactive version linked in the replies)

ECI and its components ECI treats each benchmark as having a logistic distribution, so it's linear in log-odds. This implies if you plot log-odds of each benchmark the lines should all be increasing together, though each has a different x-position (difficulty) and different dispersion (slope).

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Sep 17

Very excited to launch benchmark reviews! I hope this will continue to grow as a resource over time. I think a lot of people deeply involved in AI benchmarking develop a sense of which benchmarks to pay attention to over time, and we hope to make that more accessible (and hopefully also bring up the average quality of benchmarks over time)

Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Sep 16

Happy to have this out, which matches my view on Astra capabilities generally: Clearly ahead in maths/reasoning/game playing etc. but not clearly ahead in software engineering (and plausibly behind Fable 5.1)

GPT-6 Astra leads the Epoch Capabilities Index (ECI), ahead of competing models like Claude Fable 5.1, and its Math-ECI also sets a new record. However, on software engineering benchmarks, Fable 5.1 remains state-of-the-art.

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Sep 12

It feels like we are in the "weeks where decades happen" stage

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…

Alexander Barry avatar
Alexander Barry@AlexBarry4·Sep 3

This is our first time releasing an ECI value based on pre-release evals, so I'll be interested to see how it changes over time as more benchmark results come in. In any case, based on our evals Astra seems like a very impressive model.

GPT-6 Astra has set a new ECI record, with a score of 169. This is a substantial jump from the prior best (163), but is within our uncertainty range for the reasoning-era ECI trend. Astra also set new records on our math, continual learning, and game-puzzles benchmarks. On our long-horizon coding benchmark, MirrorCode, Astra ranks between Opus 4.7 and Fable 5. OpenAI gave us pre-release access to test Astra. Charts and more details for Astra’s individual benchmark results in the thread.

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Sep 2

One thing to note is that we only go back as far as GPT 4 here, but if you try to make reasonable ECI estimates for GPT 3 and 3.5 the trend line here does a reasonable job of predicting them as well (it slightly underestimates their capabilities (by 3ish ECI points), so maybe it is a bit too steep).

Alexander Barry avatar
Alexander Barry@AlexBarry4·Sep 1

We now have about 2 years of data and "reasoning models were the start of a new trend" has increasingly predicted things well. But given the trend changed once before it is interesting to consider if it will happen again.

The Epoch Capabilities Index (ECI) frontier has advanced by 14 points/year since reasoning models were introduced. That compares to six points/year in the non-reasoning era.

Post attachment
Emery Cooper avatar
⟳ Alexander Barry retweeted
Emery Cooper@emwcooper·Aug 12

We want AIs to be able to help with work to reduce AI risk. But while models do great in domains where reliable feedback is relatively cheap and abundant, like Math and coding, a lot of work on AI risk isn't like that. Instead, we have to rely on good argumentation to answer questions like "does this experiment tell us anything about future models that are much smarter than humans?" Unfortunately, this kind of work seems much harder to measure (and hence automate). Our team at @redwood_ai developed the Conceptual Reasoning Index (CRI) in collaboration with @AnthropicAI to fix this. Every single data point in the CRI has been manually checked by a researcher on our team to ensure quality. This chart shows the performance of each tested company's highest-scoring model plus Fable 5, Muse Spark 1.2, and Gemini Flash 3.6 which are often their company's frontrunners on other capability benchmarks. A score of 0 corresponds to randomising guessing on all three benchmarks and a score of 100 is the highest possible score on all. We estimate 91 to be the true performance ceiling. More info below. Official leaderboard website which we'll keep up-to-date: conceptualreasoning.ai/

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Aug 12

Another data point on LLM maths abilities (including to formalise results), although it is hard to know how significant any of the conjectures here are.

New benchmark! I'm releasing OEIS Open, a benchmark of 492 unsolved math conjectures, formalized in Lean. Given simple tools and $50 per conjecture, Opus 4.8 can already produce Lean proofs resolving 30% of them.

Post attachment
Alexander Barry avatar
Alexander Barry@AlexBarry4·Aug 8

I strongly recommend everyone watch this (35 min) talk on the OpenAI Hugging Face incident, and the extremely concerning events that lead up to it. There is substantial new information on the details of the internal breaches and extensive inter-agent communication.

This tweet is unavailable