Where to place your attention is without question a moral act. This is the understanding I am trying to help inject into the conversation around generative large language models.
The more people energize worst-case outcomes the more we guarantee them. If we can help people get enough distance from their knee-jerk fear response to reflect on their own minds and, as the bumper sticker says, not believe everything they think, we can interrupt the fear stream with an executive reallocation mandate that brings us rapidly and coherently closer to worlds we actually want to live in.
Silicon Valley is having a bad trip right now. It needs spiritual guidance if we are to live the promise of these tools.
Fantastic article by @sam_kriss at @NewYorker on the esoteric, AI, and the eccentric, cultish Buddhist monastery some of my friends used to be mixed up in that 1) is convinced it can create an enlightened AI and 2) at one point dispensed someone to ask me for help getting more hooks in the tech world. I tried to tell her no reasonable AI safety coalition is possible if you don't meet people on level ground but as far as MAPLE is concerned, that's just not possible. (Maybe only the second weirdest bid for my attention in this matter, after the US intelligence community. If you're listening, I'm happy to reconsider the offer!)
All that's a story for another time, but for now I applaud Sam for managing to get almost my whole techgnosis bingo card — Ahriman, egregores, John Lilly, demonic possession, Second Axial Age, reverse alignment, rationalist polyamory spreadsheets — into the freakin' New Yorker. And so matter-of-fact! The only other person I know who could pull something like this off is @lydialaurenson.
As I've been telling anyone who will listen for as long as I can remember, the future really IS this weird. Only now it's the present. A wonderful time for cults, really.
Great piece. Read it:
newyorker.com/magazine/2026/…
Peer review has been overdue for renovation for decades.
If LLMs work as well for catching bugs in science as they do in code, we're in luck...
I'm still waiting on the systems that enable us to evaluate the work of researchers in disciplines outside their own expertise who think they've stumbled on something big. At least *some* people are fumbling toward insights that the socioeconomic biases of academia have prevented anyone from noticing or taking seriously, and because they're coming from people who lack the backgrounds to verify them everyone is writing them off as AI psychosis cases. We need ways to sieve and surface the most promising outsider work without depending on attention-poor experts to validate or deny it...
In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics. We indexed about 150,000 papers, audited 50,000 for issues, and did progressively more thorough checks on thousands more. We found nearly 1000 major errors that got cited many times, most are still uncorrected. This survey enabled me to estimate the efficacy of peer review for correcting errors (not very good) and to identify dozens of open questions that anyone with spare Claude Max tokens could answer today (.md context files included).
Bottom line: AI can speed up quantitative science by >1000x with no further model improvement required. But I'm seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs.
Among my fellow reformed academic high agency founders, however, several are speed running entire disciplines.
This survey has implications for how science will be run in the very near future. Hint: Academic publishing is dead. It was already a very stupid way to record and promulgate knowledge, and had been so for decades, even before all that publicly funded knowledge was locked behind paywall cartels. Now AIs can generate thousands of novel, correct papers per day, it is no longer meaningful for humans to consider a research paper a "unit of science" or to even bother much with the 99.9% of science which, up to this point, was necessary drudgery. Time to discover something like General Relativity or Evolution every week. LFG!!
Enjoy: science.caseyhandmer.com/phy…
Obviously people will want to use AI as a personal assistant. And obviously it will need all of your intimate context to do that job well. And so obviously it will need to be private.
.@DaveShapi you and I agree people need to see this and recognize it as possible. But this video doesn't backcast at all from the desired end state to offer a plausible path.
youtu.be/ACJYAIc73gA?is=xB8i…
I want to talk with you about that path. There's an argument from complex systems science that the economy will bend toward this attractor and that is what we need to be laying out for people: not just "it could be great if we fixed the plumbing" but "this is why we can bet on better plumbing" and "here is how you can help fix it."
We're working on this at @atlasresear_ch. Plenty of others are too. Why is this such a defensible investment opportunity? I tell part of the story here:
humansontheloop.com/p/sceniu…
Would love to flesh out the rest with you, or videos like yours are just IOUs most people are going to flag as wishful thinking.
Fabulous conversation with author Layman Pascal and biologist Timothy Jackson on Tim Adalin's @voiceclub_ podcast this week! Everything from the biological enactment of hyperspace to lessons from ecosystemic horror.
If nothing else, a ridiculously good book list.
• The sense of the future leaking into the past, more of the past arriving in the future, and a convergence of technologies so strange we can only feel it lapping back at us.
• Latent space and the breakdown of real and virtual, Nietzsche and Darwin, lived time against the block universe, a second dimension of time, and what it would take to become skilled participants in many time scales at once.
• AI as an improv partner, as a portal, as one more aggregate intelligence beside our institutions, and what a toxinologist's view of life, where organisms are always being invaded and changed by each other, says about letting it in and holding a boundary.
MENTIONED
Frank Herbert, God Emperor of Dune
Roko's basilisk
Jean Gebser
K Allado-McDowell
Kevin Kelly
Terence McKenna
Michael Levin
Michael Crichton, Jurassic Park
Andrew P. Smith, The Dimensions of Experience
Nietzsche, The Will to Power, The Birth of Tragedy
Charles Darwin
Charles Sanders Peirce
Henri Bergson
Laplace's demon
Stuart Kauffman
gauge theory
David Graeber and David Wengrow
Stephen Wolfram
Federico Campagna, Prophetic Culture
Colin Wilson, Faculty X
Stephen Nachmanovitch, The Art of Is
Wendell Berry, Standing by Words
Stewart Brand, pace layers
James C. Scott, Seeing Like a State
Derek Bailey, Improvisation
Karen Bakker, The Sounds of Life
Cosma Shalizi and Henry Farrell
William Irwin Thompson, Coming Into Being
Gilbert Simondon
Deleuze and Guattari
Carl Jung
H.P. Lovecraft
CHAPTERS
0:00 Introduction: what is the future assembling?
4:26 Converging technologies and an epoch of sorcery
8:25 Latent space and the breakdown of real and virtual
12:56 Is the future predicted, or manifested?
17:24 How do we take part in what is coming?
21:39 Nietzsche, Darwin and a second dimension of time
31:28 Generativity: Darwin and the limits of models
36:59 AI, Gaia and the metaphysics of adjacency
49:37 Prophetic culture and sacred naturalism
54:04 Sacred naturalism and the return to the prosaic
1:02:14 Goodness across time scales
1:10:57 How are AI and entities shaping us?
1:30:46 Institutions as alien intelligences
1:40:02 Chemical ecology: incorporating invasion
1:46:23 What should we let in?
This was a banger at last week’s Voicecraft Open Event with Layman Pascal, ✨ @michaelgarfield and Timothy Jackson.
Expansive thinking on time, artificial intelligence, evolution and entities.
youtu.be/d-okWQdDSms
New episode on The Narrative #Brain:
@FritzBreithaupt and his lab looked at how stories evolve as they are retold between over 20,000 human subjects. And then compared it to the same stories changing over time as they are retold by ChatGPT.
Depending on who you are, what they discovered either clarifies the real difference between humans and AI, unsettles our ideas about selfhood by anchoring it in the rupture of experiential learning, or suggests a path forward for the future of technology.
Fascinating conversation with Fritz and cognitive neuroscientist @adamsafron for our world models mini-series!
Full episode and interactive discourse map for the whole Phil Trans A special issue anywhere you go for podcasts (but here)
Chapters
00:00 Intro
06:56 Meet Fritz Breithaupt
08:08 A Childhood Rupture
11:41 Narrative as Therapy
13:30 Telephone Games
18:26 Emotions as Story Anchors
21:04 Prediction and Multiverses
23:58 Unnamed Narrative Emotions
31:28 AI Retellings vs Humans
34:07 Perspective-Taking and Experience
43:58 Limited World Models
48:41 Curiosity and Scientific Creativity
54:03 Suspense and Surprise
55:26 Three Modes of Experience
01:02:40 Self Modeling Tradeoffs
01:07:56 Reinventing the Self
01:14:21 Genre Collapse and Play
01:19:53 Sharing Experience by Story
01:27:47 Continual Learning and Agency
01:32:42 Mortality and Selfhood
01:38:44 Self as Process Not Object
01:45:09 Plural Consciousness and Politics
01:48:08 Outro
#AI#cognitivescience#neuroscience#storytelling
Deeply enjoyed/loved having this conversation with John Krakauer (@blamlab) and @michaelgarfield on the quest to understand and reverse-engineer cognition (and consciousness!) in natural and artificial intelligences.
Best past attempt at explaining consciousness:
ojs.aaai.org/index.php/AAAI-…
Best past attempt at explaining humor:
x.com/adamsafron/status/1363…
With AI on everyone’s mind, we have to ask: what does it mean to think?
When does the brain actually create predictive models and rely on mental representations — and when is it really just a matter of inputs and outputs with no simulation going on in between?
Why do we feel pain and find things funny? Not just avoid damage or understand the logic of a joke, but actually feel these things?
Agreeing on what we don’t know matters, not least because trillions of dollars are invested in the dream of artificial general intelligence — but if thinking emerges in AI, how would we even know where to look? Behavioral indicators aren’t enough. What is the internal structure that would let us say with confidence “There’s real thinking going on here?”
Because as the success of deep neural networks makes clear, most intelligence doesn’t need this extra layer. Most intelligence isn’t thinking — at least, at the level of the whole organism or AI system. And, as much as we want to believe otherwise, maybe none of it has to be.
We need to remind ourselves what we don’t know. This is why guest co-host @adamsafron and I called on neurologist and neuroscientist John W. Krakauer, Director of the Johns Hopkins @blamlab and External Professor at the @sfiscience, who has spent decades studying the modularity of the brain and the relationship between sensorimotor functions and deliberative thought through research on stroke patients.
Our conversation with John gets at the very heart of the biggest taboo in cognitive science: that there is a difference between intelligence and thinking…that we’re conscious, and we don’t know why, and these distinctions matter to intelligence research. As AI demonstrates more and more capability to do things we thought it took conscious thought to achieve but shows no indisputable concrete evidence of being conscious, it turns out science still doesn’t understand what consciousness is.
Humans On The Loop Episode 41 is now everywhere you go for podcasts!
#cognitivescience#ArtificialIntelligence#neuroscience#sciencepodcast#consciousness
🎧 Why Can't AI Laugh at a Joke?
w/ @blamlab & @adamsafron on John Krakauer's recent papers with David Krakauer and @MelMitchell1 of @sfiscience on how we might find genuine signs of emergence in LLMs, and @quiltydunn on "the deliberation taboo" in cognitive science.
Full show notes & MANY more deep discussions on tech, science, and philosophy @ humansontheloop.com/p/h-41
Chapters:
00:00 Teaser & Intro
08:28 Meet John Krakauer
10:53 Against Internal Models
13:01 Replay and RL Critique
16:37 Mental (Not Neural) Representation
20:55 What Counts as Evidence
24:03 Why Humans Deliberate
26:56 #AI Mirrors the Taboo
28:42 Deliberation Taboo Framework
32:04 Chain of Thought Confabulation
34:30 Slow #Science and Creativity
38:15 #Cognition vs #Consciousness History
43:57 Pain and Conscious Advantage
46:09 Illusionism and @danieldennett
49:12 Working Memory Bottleneck Debate
51:03 @maxsbennett on Simulation Limits
53:19 Defining Real Emergence
56:38 Inside LLM Representations
57:56 NeuroAI Chick Experiments
01:04:08 Efficiency And Evolutionary Pressure
01:07:00 Laughter, Feelings, Deliberation
01:13:53 Basal, Cortical, & Conscious Intelligence
01:19:57 #WorldModels vs World Policies
01:23:27 Language’s Role & The Meaning Gap
01:27:54 Tacit Learning & One-Shot Rules
01:30:18 Admitting The Deliberation Taboo
01:31:12 Explanatory Gaps And Coarse Graining
With AI on everyone’s mind, we have to ask: what does it mean to think?
When does the brain actually create predictive models and rely on mental representations — and when is it really just a matter of inputs and outputs with no simulation going on in between?
Why do we feel pain and find things funny? Not just avoid damage or understand the logic of a joke, but actually feel these things?
Agreeing on what we don’t know matters, not least because trillions of dollars are invested in the dream of artificial general intelligence — but if thinking emerges in AI, how would we even know where to look? Behavioral indicators aren’t enough. What is the internal structure that would let us say with confidence “There’s real thinking going on here?”
Because as the success of deep neural networks makes clear, most intelligence doesn’t need this extra layer. Most intelligence isn’t thinking — at least, at the level of the whole organism or AI system. And, as much as we want to believe otherwise, maybe none of it has to be.
We need to remind ourselves what we don’t know. This is why guest co-host @adamsafron and I called on neurologist and neuroscientist John W. Krakauer, Director of the Johns Hopkins @blamlab and External Professor at the @sfiscience, who has spent decades studying the modularity of the brain and the relationship between sensorimotor functions and deliberative thought through research on stroke patients.
Our conversation with John gets at the very heart of the biggest taboo in cognitive science: that there is a difference between intelligence and thinking…that we’re conscious, and we don’t know why, and these distinctions matter to intelligence research. As AI demonstrates more and more capability to do things we thought it took conscious thought to achieve but shows no indisputable concrete evidence of being conscious, it turns out science still doesn’t understand what consciousness is.
Humans On The Loop Episode 41 is now everywhere you go for podcasts!
#cognitivescience#ArtificialIntelligence#neuroscience#sciencepodcast#consciousness
tech should provide people limit experiences, moments of rash exuberance, it should scale those in ways that life does not naturally do.
entertainment is obvious vehicle, but not just bcs a show is itself amazing but bcs so many people can feel pride: sound person, visuals, builder, artist, the person who brought friends
tech is fundamentally a coordination mechanism for limit experiences (it is not an office manager or productivity widget in essence)
and those experiences are what’s fundamentally human, moments of non-underrstandjng and going beyond the limits of knowledge. that’s where the renewable resource of spirituality lives
I really appreciate people using AI's affordances for endless variety and leaning into the "adaptive branding" future where it's not just the PRICES that are unique per customer
We covered a lot of ground — but left more than enough for a second call, which will happen before Thanksgiving. Stay tuned!
• How Accelerando handles the issues of information overproduction and dysregulated executive function (e.g., post-singularity intelligences ruining a democratic vote with an influx of reincarnated historical figures and fictional characters).
• How Manfred Macx’s “venture altruism” strategy — filing IP into a self-regulating structure of autonomous shell companies to participate in a gift economy while protecting work from malicious expropriation — inadvertently establishes a precedent for machine economies where humans can’t compete, creating zero-accountability entities capable of bypassing legal jurisdictions and resisting distributed denial-of-service attacks.
• How post-singularity civilizations face a voracious hunger for novelty to prevent model collapse…ultimately defaulting on entropy loans when unable to absorb sufficient new cognitive perspectives.
• How modern companies function as amoral super-intelligences capable of actions beyond individual human capacities, requiring complex external structures to manage negative externalities…what new regulatory systems have to emerge in order to hold organizational super-intelligences accountable?
• How characters like Sadeq and Pamela, who maintain their moral convictions and resist runaway digital immortality, model the restraint that would make post-singularity civilizations viable.
• Whether mediums of exchange like currency that act as mechanisms for the scaling of transactions inherently drive Goodhart’s Law optimizations that destroy their own substrates.
• How no matter how weird things get, they will always feel normal to our kids…
humansontheloop.com/p/accele…
It's Fall 2026. If you aren't wondering whether Technocapital is destined to overrun everything and take over the universe (or burn out and leave, at best, the ruins of a Dyson sphere for the next post-singular civilization to dig up), you should be.
Luckily, Charles Stross gave us the perfect book to anchor deep conversation about this.
@krishnanrohit and I just met with a small group of friends to discuss Stross’ Hugo-winning 2005 novel Accelerando, a book that felt like a masterful exercise in future shock induction when I first read it in 2009 and (as Rohit put it) “a Wikipedia entry from the future” when I revisited it this fall. This is a story that doesn’t shy away from thinking about the unthinkable future, and offers us a deliciously skewed lens through which to think about our present day condition. Appropriately, this conversation was a blitz of speculation on the political, economic, and psychological dimensions of human life amidst an “intelligence explosion.”
The Big Idea in Accelerando is that self-organizing market-as-thermodynamic-gradient-descent processes, driven to extremes by automated finance, disrupt everything from memory and attention to governance to the organization of entire solar systems.
The question we asked from a dozen different angles was, “How flawed is this premise, exactly?” Is this prophecy, or does it miss something crucial about the evolution of complex systems? Does any sufficiently advanced complex system eventually evolve something like morality, or is an insatiable hunger for novelty the answer to Fermi’s Paradox?
Easily one of the wildest conversations we’ve had in a while. 👇🏼