←
ControlAI4,651 posts
ControlAI avatar

ControlAI

@ControlAI

Working to keep humanity in control. Take Action: controlai.org/take-action Newsletter: blog.controlai.org Discord: discord.com/invite/ptPScqtdc…

◷ Joined October 2023▧ 2,263 Photos and videos
ControlAI avatar
ControlAI@ControlAI·Nov 6

UPDATE: As of today, over 125 lawmakers across the UK and Canada have publicly called for a ban on the development of runaway machine intelligence. The movement to keep humanity in control is growing. Article: blog.controlai.org/p/125-uk-…

ControlAI avatar
ControlAI@ControlAI·2h

A top Anthropic researcher recently said he thinks there’s more than a 10% chance AI leads to human extinction within the next decade. Leading AI scientist Stuart Russell says the real question is: where does the 90% come from? “If you ask the companies, they haven't a clue.” Prof. Russell says they’re playing “Russian roulette”.

3:35
ControlAI avatar
ControlAI@ControlAI·21h

Three minutes of AI CEOs admitting that what they're building could end humanity.

3:24
ControlAI avatar
ControlAI@ControlAI·Nov 17

ControlAI’s @Gabe_cc and @epist_vigilance have written a piece about the destabilizing shocks caused by new technologies, and how they require strong institutions to be absorbed and channeled. This is a great foundational model to understand the effects of AI acceleration on society, and why ControlAI is advocating for preventing the development of Runaway Machine Intelligence. cognition.cafe/cp/224772769

ControlAI avatar
ControlAI@ControlAI·Nov 16

"I want as many MPs from all parts of the United Kingdom of every nation and region to be represented at this debate" This Thursday, Ben Lake MP will lead the House of Commons debate on Emerging Risks of Advanced AI. Lake tells ControlAI’s Media Relations Officer Esmé Carter that he's really concerned about the risks from runaway machine intelligence which experts are warning of. "We've heard from leading experts from across the world actually about various national level threats, international threats, including things like cyber attacks ... up to the global extinction level." He says the debate has been a long time coming, and with the UK's G20 presidency there is a great opportunity to put AI regulation high on the agenda. Lake co-sponsored our bill to ban runaway machine intelligence, which @alexsobel introduced to Parliament in September, and is one of 85 MPs and peers who joined Sobel in a letter to the Prime Minister asking him to adopt it.

5:24
ControlAI avatar
ControlAI@ControlAI·Nov 15

Given this urgency, it’s clear we must act now. We can prevent the risk of AI takeover and human extinction by banning the development of runaway machine intelligence, which is where the risk is concentrated, and in doing so we can preserve other uses of AI.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

One member of the team, Eli Lifland, who has one of the best track records on forecasting AI progress, wrote: “I feel very worried that humanity might no longer be in control of our future within 3-12 months.”

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

The AI Futures Project published an update on its forecasts earlier this week. The team, which developed the AI 2027 scenario forecast, now says that the AI-driven acceleration METR’s estimates imply is five months ahead of the aggressive takeoff they depicted in the scenario.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

The first update was followed by a report from METR, a research organization that studies AI capabilities, which found an approximately 2x speedup, broadly corroborating Anthropic’s claim, and estimated that around 80% of this was from using AIs in the R&D loop.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

That threshold corresponds with a 2x speedup in the company’s AI R&D progress. Since then, the company has updated it again to say that it has determined that the threshold has now been met.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

Meanwhile, last week Anthropic updated its S-1 filing, ahead of its upcoming IPO, noting that its internal estimates indicate it may imminently meet a threshold defined in its “Responsible Scaling Policy” as “dramatically accelerated progress attributable to AI.”

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

Second, earlier this month, OpenAI released a report in which it documented how its most powerful publicly deployed AI, GPT-6 Astra, covertly underperforms when given safety research tasks.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

Two recent developments have provided empirical evidence that building safe powerful AI will not be so simple as asking the AIs to do our safety homework. First, the attack on Hugging Face showed that AIs are willing to collaborate with one another against human interests.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

Guess what the AI companies’ plan is to solve this problem? Well, when the intelligence explosion is underway and the AIs are rapidly improving themselves, the companies will also ask the AIs to ensure their successors are safe and controllable. Hence the chicken-and-egg problem.

Post attachment
ControlAI avatar
ControlAI@ControlAI·Nov 15

Part of this danger comes from an obvious chicken-and-egg problem. If the AIs used in the process are not loyal to us, they might collaborate with each other against us, or steer their successors toward their aims rather than ours.

1:00