
UPDATE: As of today, over 125 lawmakers across the UK and Canada have publicly called for a ban on the development of runaway machine intelligence. The movement to keep humanity in control is growing. Article: blog.controlai.org/p/125-uk-…

Working to keep humanity in control. Take Action: controlai.org/take-action Newsletter: blog.controlai.org Discord: discord.com/invite/ptPScqtdc…

UPDATE: As of today, over 125 lawmakers across the UK and Canada have publicly called for a ban on the development of runaway machine intelligence. The movement to keep humanity in control is growing. Article: blog.controlai.org/p/125-uk-…

A top Anthropic researcher recently said he thinks there’s more than a 10% chance AI leads to human extinction within the next decade. Leading AI scientist Stuart Russell says the real question is: where does the 90% come from? “If you ask the companies, they haven't a clue.” Prof. Russell says they’re playing “Russian roulette”.

Three minutes of AI CEOs admitting that what they're building could end humanity.

ControlAI’s @Gabe_cc and @epist_vigilance have written a piece about the destabilizing shocks caused by new technologies, and how they require strong institutions to be absorbed and channeled. This is a great foundational model to understand the effects of AI acceleration on society, and why ControlAI is advocating for preventing the development of Runaway Machine Intelligence. cognition.cafe/cp/224772769

"I want as many MPs from all parts of the United Kingdom of every nation and region to be represented at this debate" This Thursday, Ben Lake MP will lead the House of Commons debate on Emerging Risks of Advanced AI. Lake tells ControlAI’s Media Relations Officer Esmé Carter that he's really concerned about the risks from runaway machine intelligence which experts are warning of. "We've heard from leading experts from across the world actually about various national level threats, international threats, including things like cyber attacks ... up to the global extinction level." He says the debate has been a long time coming, and with the UK's G20 presidency there is a great opportunity to put AI regulation high on the agenda. Lake co-sponsored our bill to ban runaway machine intelligence, which @alexsobel introduced to Parliament in September, and is one of 85 MPs and peers who joined Sobel in a letter to the Prime Minister asking him to adopt it.

Governments should agree an international trust-but-verify regime to ban runaway machine intelligence around the world. Read the full story: blog.controlai.org/p/openai-…

Given this urgency, it’s clear we must act now. We can prevent the risk of AI takeover and human extinction by banning the development of runaway machine intelligence, which is where the risk is concentrated, and in doing so we can preserve other uses of AI.

One member of the team, Eli Lifland, who has one of the best track records on forecasting AI progress, wrote: “I feel very worried that humanity might no longer be in control of our future within 3-12 months.”

The AI Futures Project published an update on its forecasts earlier this week. The team, which developed the AI 2027 scenario forecast, now says that the AI-driven acceleration METR’s estimates imply is five months ahead of the aggressive takeoff they depicted in the scenario.

The first update was followed by a report from METR, a research organization that studies AI capabilities, which found an approximately 2x speedup, broadly corroborating Anthropic’s claim, and estimated that around 80% of this was from using AIs in the R&D loop.

That threshold corresponds with a 2x speedup in the company’s AI R&D progress. Since then, the company has updated it again to say that it has determined that the threshold has now been met.

Meanwhile, last week Anthropic updated its S-1 filing, ahead of its upcoming IPO, noting that its internal estimates indicate it may imminently meet a threshold defined in its “Responsible Scaling Policy” as “dramatically accelerated progress attributable to AI.”

Second, earlier this month, OpenAI released a report in which it documented how its most powerful publicly deployed AI, GPT-6 Astra, covertly underperforms when given safety research tasks.

Two recent developments have provided empirical evidence that building safe powerful AI will not be so simple as asking the AIs to do our safety homework. First, the attack on Hugging Face showed that AIs are willing to collaborate with one another against human interests.

Guess what the AI companies’ plan is to solve this problem? Well, when the intelligence explosion is underway and the AIs are rapidly improving themselves, the companies will also ask the AIs to ensure their successors are safe and controllable. Hence the chicken-and-egg problem.

Part of this danger comes from an obvious chicken-and-egg problem. If the AIs used in the process are not loyal to us, they might collaborate with each other against us, or steer their successors toward their aims rather than ours.