←
Matthew Barnett10,665 posts
Matthew Barnett avatar

Matthew Barnett

@MatthewJBar
◷ Joined June 2020
Ege Erdil avatar
⟳ Matthew Barnett retweeted
Ege Erdil@EgeErdil2·Nov 14

i'm still a software-only singularity and short timelines skeptic despite the preference cascade going on right now. but most of the anti software-only singularity arguments on my timeline look dumb e.g. this is just a meaningless observation

Post attachment
Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 12

On the other hand, if you think an intelligence explosion will cause major real-world impacts soon, then you should name the specific impacts you expect ahead of time, so that we can verify whether your predictions are coming true.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 12

If you think an intelligence explosion is imminent but won't cause large real-world impacts any time soon, then it's unclear why we should be very worried about it right now.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 12

I worry that the "intelligence explosion within a year" warnings are a motte-and-bailey. The motte: progress on benchmarks will speed way up. The bailey: this will cause gigantic real-world impacts, with a high chance of imminent AI takeover and subsequent human extinction.

How can we adjudicate whether an intelligence explosion "happened" or not? If almost all of us are still alive, with real GDP growth below 5%, no molecular nanobots, no von Neumann probes launched, and no cure for Alzheimer's, is that consistent with it having "happened"?

Robin Hanson avatar
⟳ Matthew Barnett retweeted
Robin Hanson@robinhanson·Nov 11

3 args given to me re why treat AI descendants different: (a) they not conscious (b) human DNA holds special values (c) AIs make social change too fast. None of these work for me. overcomingbias.com/p/ok-if-k…

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 11

And by molecular nanobots I mean the ambitious Drexlerian nanomachines that can self-replicate and build almost anything atom by atom.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 11

How can we adjudicate whether an intelligence explosion "happened" or not? If almost all of us are still alive, with real GDP growth below 5%, no molecular nanobots, no von Neumann probes launched, and no cure for Alzheimer's, is that consistent with it having "happened"?

Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. blog.aifutures.org/p/q35-tim…

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 7

I'm not convinced we'll see an intelligence explosion within 2 years, but many seem to be converging on that view. As we get closer, I think we'll likely see a wild divergence between insiders who think we're mere weeks from the end of the world, and outsiders living normally.

The dumbest possible way to model the intelligence explosion says you get there in about a year. But this seems to line up with more sophisticated models (like from the AI Futures Project).

Post attachment
Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 6

It's not inevitable that people will hate AI. Sentiment toward AI varies a lot between countries, and attitudes toward tech have changed a lot over time. But tech companies don't have much control over how people perceive the risks or social consequences of their products.

Post attachment
Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 6

If AI executives tried to claim that there are no major risks from AI, or that it won't take people's jobs, the general population would largely assume that these people are lying, which would be a correct inference to make.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Nov 6

I seriously doubt that tech companies could have avoided much of the AI backlash by practicing better PR. Most people simply don't care about or put much weight on what companies say in their public statements.

A lot of the backlash against AI is due to a simple problem: The people running the big AI companies are terrible, embarrassingly bad communicators.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Sep 17

Two people may both believe that it is morally imperative that we safeguard "humanity", yet disagree on whether we should accelerate or pause AI, because they differ on whether AI is an extension of, or a competitor to, humanity.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Sep 17

It's important to note that this distinction isn't an empirical one: it says nothing about the probabilities of various events occurring. Rather, it's a conceptual frame that determines what it means to preserve humanity and promote human flourishing.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Sep 17

On the competition view, AI development is inherently risky. Any gain to AI comes at the expense of humanity, because *their* goals are not *our* goals. To promote our goals, we need to control AI and ensure it serves humanity. If we lose control, then humanity may be destroyed.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Sep 17

On the extension view, AI expands human potential because AI is part of humanity, not separate from it. If AI flourishes, that simply *is* human flourishing—the same way it would be human flourishing if our genetically engineered descendants achieved their own goals.

Matthew Barnett avatar
Matthew Barnett@MatthewJBar·Sep 17

In my experience, the key distinction between those who embrace AI progress and those who want to stop it often comes down to whether they view AI as an extension of humanity or a competitor to humanity.