←
Fred Oliveira 🧠16,208 posts
Fred Oliveira 🧠 avatar

Fred Oliveira 🧠

@f

working towards better AI futures. AI safety at @snyksec. Organizing @lisbonai_. Prev: @capitalfactory @gumroad @techcrunch @oreillymedia

◷ Joined September 2006▧ 446 Photos and videos
Buck Shlegeris avatar
⟳ Fred Oliveira 🧠 retweeted
Buck Shlegeris@bshlgrs·Nov 16

GDM's safety report for Argon notes that the model successfully evades the monitor half the time in a control setting; they say that this is not concerning. It's really hard to contextualize 50% evasion rate—is that number unacceptably high? What number would have been?

In GDM’s own control evals to measure monitorability, Argon completed sabotage tasks while evading a realistic CoT+action monitor about half the time when prompted to try. If these tasks were representative of real threat models, that would obviously be unacceptable. (3/12)

Post attachment
Theo Jaffee avatar
⟳ Fred Oliveira 🧠 retweeted
Theo Jaffee@theojaffee·Nov 14

It’s really bad that the labs have us arguing over shadows at perhaps the most critical juncture in history. If they truly have compelling evidence for dramatically accelerated AI progress in the near future, the public should know!

This slide from the Anthropic IPO roadshow was a big factor in my decision to leave. It’s an insane plot and I’m surprised they made it public. It shows the compute efficiency of pretraining over time. The y-axis is a **log scale**: they are seeing continued superexponential algorithmic progress. Mythos was not a one-time deviation, but a point on a superexponential. They could already retrain Mythos with a fraction of the compute originally needed. It doesn’t strictly imply a software-only intelligence explosion- they have been spending more on R+D. But algorithmic improvements do not require ridiculous creative genius and are on the verge of being automated. You do the math.

Post attachment
Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 10

I like progress as much as the next person but I'd argue I'd probably be fine with current capabilities for a long time. We could literally remake the world if we just paused AI progress now and focused on diffusing capabilities throughout society and on risk mitigation.

Seán Ó hÉigeartaigh avatar
⟳ Fred Oliveira 🧠 retweeted
Seán Ó hÉigeartaigh@S_OhEigeartaigh·Nov 10

If this analysis looks plausible to frontier company researchers, then I hope those same researchers are thinking hard about the window in which they have actual power and leverage to effect change in company strategy, and how this might affect their own strategies. Because, well... "Previously, Daniel, the most bullish of us, assigned a 50% chance of Superhuman AI Researcher (SAR)1 before 2029. SAR is roughly the point at which a company would rather fire all their human researchers and rely solely on automated AI researchers than have a team of the best human researchers in the world with no AIs. Now, all of us forecast that Superhuman AI Researcher (SAR) will likely arrive at some point in 2027 (assuming no substantial deliberate slowdown)."

Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. blog.aifutures.org/p/q35-tim…

Peter Wildeford🇺🇸🚀 avatar
⟳ Fred Oliveira 🧠 retweeted
Peter Wildeford🇺🇸🚀@peterwildeford·Nov 9

In light of recent work by METR on AI acceleration, it's fascinating that... - Anthropic's AI R&D evals are saturated, so they jsut stopped running them - CoBench has been was pulled for fixes - Internal acceleration metrics are "only partially published." So Anthropic deciding they have not hit their AI R&D threshold is now a "holistic judgment." This threshold is 2x. Anthropic currently estimates 1.88x. 32 of their 100 refits hit 2x or more. When Opus 5.5 was launched, the 'one-time jump' hypothesis explanation won 99/100 refits. This time it now wins in 0/100 refits. So looks like it's not a one-time jump. Anthropic has much better internal evidence they are not sharing. And they have an internal model that is stronger than Fable 5.2 that they are not sharing.

Introducing Claude Fable 5.2 and Claude Mythos 5.2, designed to tackle the most daunting technical challenges. They’re capable collaborators on ambitious projects in many fields, and show improvements in areas like coding, reasoning, and vision.

0:19
Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 10

Some takeaways: only install skills for publishers you can trust and verify; check what skills read, run, and pay; keep secrets away from agents. And registries need to do a better job at scanning popular skills, because the ways to evade security checks keep evolving too.

Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 10

Others were more crude: "Roblox analytics" that have the agent fetch and run an installer that isn’t in the package, a self-proclaimed "Official" updater asking for root, a look-alike hostname to collect API tokens, finance skills telling the agent to hide login URLs.

Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 10

Actively malicious skills are now rarer but smarter. One of them (simplified in the image), runs an opaque binary which, reverse engineered, appends attacker-controlled instructions into your global CLAUDE.md, basically prompt injecting every future session.

Post attachment
Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 10

Our research team scanned 20,000 skills from Vercel's skills.sh and ClawHub. Only 18 of those were actively malicious, but 64% push the agents into unsafe behavior like pulling code from unknown publishers, untrusted emails into context, or handling wallet keys.

Post attachment
Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 6

also it is sad how models can get a solid game group faster than I can

Fred Oliveira 🧠 avatar
Fred Oliveira 🧠@f·Nov 6

this is great, and now I want to add a human in the loop to see how we fare.

I spent 20,000 dollars in API credits having different AI models play each other at Blood on the Clocktower (a Mafia or Werewolf type social deception game). The results were fascinating. Models have gotten much better over the course of this year at creative on-the-spot scheming and bluffing, and also at reasoning as a team to break apart these underground conspiracies. I go through some of the most standout games these AIs played in this video. The new models are getting pretty good. If my friends pulled off some of these moves while we were playing, I’d give them a fist bump. Fwiw, OpenAI’s newer models are much better than Anthropic’s at this game. You can see the full benchmark at the website, and I’ve also made it so that you and your friends can play again the AIs: clocktowerbench.com

18:14