←
Forethought133 posts
Forethought avatar

Forethought

@forethought_org

Research nonprofit exploring how to navigate explosive AI progress.

◷ Joined March 2025▧ 24 Photos and videos
Forethought avatar
Forethought@forethought_org·4h

New ForeCast episode: James Tillman and @TomDavidsonX on whether there could be an intelligence explosion in a data centre, and what might stop it. 00:00 What is a software intelligence explosion? 02:49 The data AI companies have and we don't 05:02 Compute versus algorithms 07:21 What are scale-dependent algorithms? 11:00 Could AI researchers find new algorithms with little compute? 19:23 Are estimates of algorithmic progress inflated? 26:08 How far could it go? Catching up with the human brain 34:39 Beating the human brain: Chinchilla scaling and a bee-sized brain 49:01 Could data bottlenecks stop it? 50:48 Would AI trained in the lab work in the real world? 58:01 Could we run out of data first? 1:02:35 How big a speed bump is data? 1:07:20 The verdict

1:10:30
Forethought avatar
Forethought@forethought_org·Nov 16

New from Stefan Torges: how likely is it that one person or a small group end up in control of the world? He maps nine routes, by actor, with rough odds for each. His overall guess: around 23%. newsletter.forethought.org/p…

Forethought avatar
Forethought@forethought_org·Nov 10

New ForeCast episode: Avi Parrack and Tom Davidson on why the US could soon dominate space, whether it could lock rivals out of orbit, and what a space lead means for US–China competition Link in thread

Forethought avatar
Forethought@forethought_org·Nov 9

New from James Tillman: some AI models may now keep improving with test-time compute far past the most they ever used on a training task. He calls this "compute extrapolation," argues it probably arrived around the end of 2025, and explains why it could speed capabilities and make dangerous-capability evals far more expensive. forethought.org/research/two…

Forethought avatar
Forethought@forethought_org·Nov 5

New from Stefan Torges: How Long is the Long Reflection? “AI can speed up learning, but it cannot speed up becoming. The only fast route to more reflective values is handing the future to versions of ourselves we would not recognize.” newsletter.forethought.org/p…

Forethought avatar
Forethought@forethought_org·Nov 5

"Automating alignment" covers three kinds of work: 1. Empirical research: running experiments 2. Conceptual research: working out what we're aiming for 3. Direct work: making systems aligned Owen Cotton-Barratt argues we should track them separately. forethought.org/research/thr…

Post attachment
Forethought avatar
Forethought@forethought_org·Oct 30

Could a first mover prevent others from accessing outer space, or raise their cost of doing so? A new piece considers two potential kinds of space lock-out: "soft" lock-out (e.g. capturing scarce assets, like orbital bands or the best lunar sites, and denying them to rivals) and "hard" lock-out (using force, such as space-based interceptors that destroy rockets at launch). Read it here: forethought.org/research/loc…

Forethought avatar
Forethought@forethought_org·Oct 30

Where it could fail: China pooling all its compute, export controls failing (the lead falls to ~4x), or a fast takeoff with a Chinese leapfrog. So pacing should come with export controls, data center buildout and much better lab security.

Forethought avatar
Forethought@forethought_org·Oct 30

Compute is also AI manpower. Even at equal capability, the US could run ~9x as much AI across its economy and military. And if China pours compute into R&D to keep pace, the deployment gap widens: 17x at half its compute, 81x at 90%.

Forethought avatar
Forethought@forethought_org·Oct 30

Once AI does much of its own research, compute turns into research speed. Avi estimates a leading US lab's ~3.5x compute edge over its Chinese counterpart gives roughly 1.9x faster research, which could help regain a lead spent on safety and security.

Forethought avatar
Forethought@forethought_org·Oct 30

New on the Forethought blog: can the US pace the AI frontier, slowing R&D for safety, without losing to China? Avi Parrack argues it probably can: the real US advantage is about nine times China's AI compute, not a few months' lead. forethought.org/research/can…

Forethought avatar
Forethought@forethought_org·Oct 20

A new blog post asks whether deployment of AI in the military could create "single points of failure," where a single event (such as a hack by outsiders or a coup by an insider) could lead to catastrophe without any effective failsafe. The author looks at safeguards developed for the nuclear command-and-control context, and examines how they might be applied to military AI to mitigate this risk. Read it here: newsletter.forethought.org/p…