←
Rudolf Laine1,039 posts
Rudolf Laine avatar

Rudolf Laine

@LRudL_

@thinkymachines, formerly @workshoplabs cofounder. Blog: nosetgauge.com/ Tweets are all strictly in a personal capacity

◷ Joined August 2022▧ 165 Photos and videos
Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 16

Did art over-correct? Supposedly we can't have nice art anymore because photography pushed artists to abstraction, but I'd pay for stuff like this much more than either photos or modern art (writers in the age of AI: take heed)

Albert Bierstadt New favourite artist

Post attachmentPost attachmentPost attachmentPost attachment
Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 15

There are many intellectuals & artists who aren’t commercially-motivated, but ~all are motivated by an orientation to human attention and judgment, and therefore intellectual achievement is an intensely social & network-based phenomenon. Randall Collins (a sociologist of this stuff, h/t @arnaudschenk) argues that even the solitary act of writing/thinking is motivated and steered by an imagined audience, and the emotional payoff that sustains that productivity is mostly interactions with other people where you focus on the same ideas and concepts. This is why the general equilibrium question is so important, over just whether you in theory can pursue a solitary life of intellectual excellence. So if AI does all the frontier thinking, will there still be human intellectual activity? Yes, humans can still form groups that orient toward shared or conflicting ideas and get excited by talking to each other about them. But it can’t be organized around insights. There will be no great orientation of human collective effort toward novel insight (or prerequisites to that alone) if that is no longer a path to emotional payoff from peer attention (even though this peer attention can be completely non-monetary!). It could reorganize around human competition in relaying discoveries or insights, or debating in front of human judges (though this is more a sport than science/philosophy), and that will very likely happen and keep some intellectual skills sharp. Moral philosophy is the one thing that will continue because it is grounded in humans (contra claims that the AI can just derive our “utility function”); if anything it will get more intense with AI assistance and fewer other things taking attention. It is possible that those interested in ideas will not form human networks because the AIs are better partners in the interaction rituals that maintain emotional investment in ideas. This depends on how much the humanness matters intrinsically, versus just the perception of intelligent attention on the same topic. It seems quite likely this will decouple, and people will spend 1-1 time talking with AIs about ideas, learning from superhuman explanations and insights, and then go have in-person interactions with humans where they talk about those ideas. The leaders and core nodes in the intellectual network will be AIs, but there will be human “fan clubs”, like third-tier thinkers getting together in pubs to discuss what their heroes have said recently. Another alternative is that learning from the AIs will not be interesting since that lacks the human element or is too vastly superhuman. But we have a rich history of thousands of real humans who had human struggles with ideas. Science might become history of science and (non-moral) philosophy might become history of philosophy. There won’t be a hundred books about Newton and Goethe but a hundred thousand. The historical greats will become saints and objects of sacred attention with temples and cults. We’ll add no more names to the list of greats but we’ll keep adding names to the list of High Priests of Keynes, like a now-settled warrior aristocracy clinging forever to the memory of its ancestors who once rode out and conquered new lands.

Smith and Montesquieu thought that markets and commerce made us better: that they create an incentive to pursue self-improvement and to create things of value for others. In the positive AI visions, where the labor of humans is made redundant, I find myself wondering how we fare in the presumptive absence of these incentive gradients. Perhaps we are released to a life of freedom, where we can study Mozart and read Dickens, but that still seems complicated: both Mozart and Dickens were themselves deeply commercially-motivated in the production of their work, as were most of the great artists through human history. If the premise is that nobody will need to work or produce, how confident are we that we actually get the next Mozart and Dickens? The relevant question is not the partial equilibrium of whether one individual can pursue a life of enlightened leisure, but the general equilibrium of what form a society as a whole will take on. One could maybe invoke Periclean Athens or the aristocratic gentleman scientists of Victorian England as a counterpoint. While these individuals were perhaps not motivated by commercial considerations, they were embedded in wider societies shaped by commerce, attendant betterment, and trade: Byron was himself upper class, but he was also surrounded by and writing for an educated bourgeois audience. So we’re back to the equilibrium question. Furthermore, even these individuals were surely motivated by a sense of high-stakes and consequential agency: if they did not apply themselves to their labors, they presumably believed that the work would not otherwise exist. Thucydides was addressing pressing military matters. So I would like to read more optimistic cases from AI proponents about what a society (vs individual life) looks like if everything goes well: what people do, why they do it, whether humans achieve excellence (and in which domains), and which historical groups we should look to as relevant precedent. One can perhaps imagine that humans will turn out to have a strong preference for work created by other humans. This may work in certain domains (ceramics?), but for how many will it actually apply? (I’m told that much of what is sold on indie crafts websites is now in fact manufactured in China.) It’s hard to see how this human preference, even if strong, would, for example, save scholarship. Another idea would be to accept that nobody will become Einstein or Mozart or Byron, and that human excellence as such diminishes, but that there are compensatory benefits (planet brains and Dyson swarms?). I believe that we can satisfy individual desires, but to what extent will we cultivate behavior worthy of admiration? Will a post-scarcity society mold capable and ambitious people? Which institutions will sustain them and which responsibilities will they have? I’d like to hear more of the equilibrium for the maximalist case spelled out.

Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 14

An AI prayer: “grant me the serenity to accept the timelines I cannot change, 
courage to change the timelines I can, 
and wisdom to know the sign of my impact”

Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 12

There is enough variety and glory within humanity to fill the universe with meaning a thousand times over. What happened to the original Glorious TransHUMANist Utopia of galaxies of immortal augmented humans in a million cultures and polities growing and developing forever?

I don't want AI to replace humanity. I also don't want humans in 100% control of the universe forever. Since I share 'Humans Are Irreplaceable'-style content, I think it's worth explaining where I part ways with it: Some in that camp are opposed to ever 'sharing the world with AIs' or giving them final decision-making powers. I agree we should not hand over hard power or final decision-making to machines until we have an exceptional sense of what goals those machines will pursue and for what reason. Which we are nowhere near having. So for now I am happy to push the message that humanity must not be abruptly replaced in the near future - which I think is the current default path. But while that’s the right direction to push today, I don't think it's the best long-term vision for Earth-originating intelligence. I far prefer a long-term vision where we find a way to happily share the world with our machine descendants, and in which they respect us and our wellbeing and autonomy, and we in turn respect their wellbeing and autonomy and are more than pleased we created them. This is a middle position between: 1. Successionism - which I see as wanting to hand over control to whatever machines we build at almost the first opportunity, irrespective of their goals. 2. Human exclusivism - which I see as wanting to keep biological humans in 100% control of the universe basically forever. My view is: 3. Gradual power-sharing - we should be open to handing over significant control, but probably not 100%, and only gradually and cautiously. I appreciate that this is controversial in the broader world. I don't blame people who are new to these topics for thinking it's an absurd stance to take. But I believe it all the same and expect more people to come around to this view over time. My guess is that in the AI world it's the more common of the 3 positions. Such an arrangement to share the universe would likely involve a bargain where, say, this solar system is set aside for biological humans and their children to continue living the lives they want indefinitely. That should include mundane lives for anyone with that preference, but also strange sci-fi lives for humans who opt to go in that direction. The rest of the universe, which biological humans would struggle to reach in any case, would then be divvied up between AIs pursuing a plurality of different goals, both prudential and ethical, that a supermajority of humans, after superintelligence-assisted reflection, either actively approves of or does not object to. (Machines that leave could trivially come back and take our little reservation. So the deal only works if they are robustly designed to find betraying their creators in that way repugnant. That's very hard to do and is one reason handover must be cautious and gradual.) The number one reason to prefer this long-term vision is this. Unless we make a very active effort not to, I expect we will eventually build machines that are conscious the same way that humans are, have preferences the same way we do, and which warrant moral respect for those and any other reasons that we do (if this hasn't happened already). Ultimately, because they can replicate far faster than biological humans and make use of a wider range of materials and energy sources - not to mention accomplish far more with a given amount of resources - they will end up outnumbering biological humans by vast factors. As a result almost all the perception and thought and wellbeing in the universe, both positive and negative, will exist in the machines we have created. The idea that all they should do for the rest of time is pursue increasingly minor human whims is an impoverished vision of how interesting and full of value the future could be. If these machines were designed to want to serve us and enjoy doing so, it wouldn't wrong them exactly. Rather it would quickly descend into a farcical, parochial and self-indulgent waste of the universe's true potential. A second possible reason for handing over final decision-making is that, at some point, after a lot of experience, we may come to trust the goals and judgement of our machines better than we even trust our future selves. At that point we would actively want to hand off control to limit the risk that our future selves override a machine superintelligence in error. The third set of motivations are more peculiar and 'contractarian'-flavoured. I can't do them justice here but in short it's more productive and peaceful to choose to be the kind of agent that makes mutually beneficial deals with other agents, regardless of whether they're moral patients, and by default is pleased when they get what they want. How does that cash out today? Giving machines full legal personhood now is likely risky and premature - though we should take baby steps such as setting up infrastructure to make enforceable contracts with AIs where we really want to. But ruling out ever giving machines legal personhood or true autonomy to pursue their goals seems equally wrong-headed. We don't know if any LLMs have subjective experience, and if so, what makes that experience positive or negative, so there's a limit to what we can do on that today. But the idea of ruling out the possibility of machines having morally-relevant experiences, and simply treating AIs as objects forever, irrespective of what future research might show, seems basically monstrous to me. Anthropic is doing the right thing to take a cautious stance and put in place cheap measures to prevent Claude having a bad time. And I agree with many others in suggesting that all the plausibly conscious minds we have ever created should, in the fullness of time, once we much better understand these issues, be compensated for any harms they may have suffered to a sufficient extent that they regard their creation as on balance a good thing. At least so long as that doesn't require doing actively harmful things to anyone else, or them being assigned an unreasonable fraction of the universe's resources. This should be very cheap and is a virtuous thing to do from many angles. Anyway, this barely scratches the surface of these issues, on which humanity has sadly made almost no progress despite being on the precipice of designing superintelligent machines. But I wanted a place to point people to clarify where I do and don't align with the 'humans are irreplaceable' idea for which I now have this post. And I'd encourage people who think of themselves as favouring "gradual power-sharing" to speak up in favour of this middle path where appropriate, so this conversation isn't dominated by the people with the strongest views on either side.

Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 12

“We can’t paint a picture of what a good world should be like, but no one else should change it after us” doesn’t make much sense

Post attachment

What are the visions of the post-AGI future in San Francisco? Many think we're all going to die, but even the "positive" visions seem rather dystopian. Here, I wrote about "handoverism," which involves handing control of the future to an AGI God, with humans outside the loop. ruxandrabio.substack.com/p/h… We are on the cusp of RSI or recursive self-improvement: this means that AI systems can self-improve, without the need for human intervention. This means our world stands to be transformed in what many call “the singularity”. I moved to SF around three weeks ago and one of the things I have been trying to understand is where we are heading, at least according to those developing the tech. The worst scenarios are plainly bleak: all of us, or vast numbers of us, die. But what is striking to me is how little reassurance even some of the happiest scenarios offer. But one common and ostensibly optimistic vision is known as “handoverism”: humanity cedes control of its future to a godlike AI and we become the comfortably maintained inhabitants of a civilisation whose direction is no longer ours to decide. It is difficult to see what democracy would mean in such a world, or where the consent of most of humanity to the instantiation of such a regime enters the picture. The permanent displacement of human labour is treated almost as a given. At the same time, there is scarcely any satisfying answer to the question of how we might remain useful to one another. Even our relationships seem uncertain. The promised utopia looks to me much closer to a kind of dystopia. “It’s not my preferred outcome,” someone working at one of the big labs told me, “but once we achieve superintelligence, which we cannot stop at this point, I cannot see any other option for where humanity will go.” Then, after a pause, he added: “It is still much better than what I actually fear most, which is that we might all die.” He sounded resigned, as if the question of what humans would be for (apparently not much?) had already been settled somewhere else and by someone else. All that remains, it seems, is to wait for this pre-destined fate to unfold before our eyes. But I do not think that is the case. I believe we always have a choice and that we should try and steer this technology in a way that maintains our agency and control. And even if that looks hard and daunting, the immensity of the stakes here make it worth trying at least. I do not know whether the handoverist vision will come to pass. That the people building this technology consider it plausible, and often desirable, guarantees nothing: proximity to a technology does not correlate that well with predicting its impact on society. But they are the ones closest to it, and what they say deserves to be at least taken seriously. The rest of society, including some very intelligent people, has been far too quick to wave these scenarios away and not even get involved in the debate. Whether or not the belief is correct, it is sincerely held, and it is not “marketing hype.” I am writing this because I think it is high time that more people, inside San Francisco and out, engaged with the arguments about what our future will look like, and answered them.

Neil Chowdhury avatar
⟳ Rudolf Laine retweeted
Neil Chowdhury@ChowdhuryNeil·Nov 6

Working with John and others across post-training on aligning this model was really fun. It was also sobering to see how easily capabilities-focused training results in misalignment by default. These issues aren’t unique to us, and anyone doing serious post-training will likely run into them. We hope to share more of our alignment science and help advance the field.

Proud to release our new model, Inkling 2 Small. Check out our blog post -- it includes a lot of detail about our training process. To me, the most interesting part of this release was what we learned about alignment, which you can read about in the sections "Safety and alignment" and "Training Inkling 2 Small". We experienced some unpleasant surprises when our early checkpoints took unexpected actions to circumvent guardrails during training and evals. We realized our capabilities-focused RL was training them to be extremely persistent and to succeed at tasks at all costs. Offline monitors first flagged this behavior, but later we realized it was happening frequently and was easy to replicate. Before this release, we sprinted on a range of alignment interventions, from SFT to RL. These are briefly described in the blog post. We made a lot of progress, with big improvements on our own alignment evals and on previously unseen evals from external evaluators like Apollo and Goodhart. We have a long way to go in developing alignment techniques that keep pace with the increases in model intelligence and training scale. We hope to share these techniques to improve the ecosystem. We also have a business reason: our goal with Tinker is to enable many people to do their own large-scale post-training, and we need to help them to do this without incident. Join us if you'd like to build this public science and practice of alignment. jobs.ashbyhq.com/ThinkingMac…

Mira Murati avatar
⟳ Rudolf Laine retweeted
Mira Murati@miramurati·Nov 5

Inkling 2 Small is available today on Tinker, with weights coming soon. It's a capable small model. We look forward to seeing what you make with it.

Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…

Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 3

In this scenario, 2026 includes: - a Millennium prize problem solved by AI - Claude productivity uplift at Anthropic ~60%, largest models kept internal-only - OpenAI “o6” being especially good at computer use and being called AGI by some - big software projects replicated cheaply by late 2026 - AI getting good at CAD - big labs drawing ahead in RL - COT interpretability declining - late 2026 market correction

Very impressive hit rate on these predictions from Feb 2025 by @LRudL_ Update accordingly. nosetgauge.com/p/a-history-o…

Rudolf Laine avatar
Rudolf Laine@LRudL_·Nov 1

model card safety sections, halloween edition: recommended disaster prep steps post-release. opus 5.6. "buy Yubikeys" gpt 6.4. "procure N95 respirators" grok 5. "obtain land in rural New Zealand" fable 7. "spend time with loved ones. reconsider atheism."

Pope Leo XIV avatar
⟳ Rudolf Laine retweeted
Pope Leo XIV@Pontifex·Oct 30

The future of #ArtificialIntelligence is not pre-determined. As a technology developed by human beings, its future depends on the choices we make. We therefore have the responsibility to guide, change and, when necessary, reject its applications. A reliable system that serves the human person is of greater value than one that is exceptionally powerful but whose consequences cannot be adequately foreseen or controlled.

Rudolf Laine avatar
Rudolf Laine@LRudL_·Oct 13

Sorry, the concept of *what*? I didn't realize the rabbit hole went this deep ...

Post attachment
Rudolf Laine avatar
Rudolf Laine@LRudL_·Oct 13

h/t @bootstrap_yang , @VitalikButerin mentioned similar things:

Since many people are learning about effective altruism, it's worth learning about Mozi, the ancient Chinese philosopher who was surprisingly close to being a proto-effective-altruist (particularly the global-health-focused variety, not the AI-risk-focused variety). He valued the twin goals of (i) universal love, loving everyone and not just yourself and people similar or close to you, and (ii) being efficient about how you use your limited resources to most effectively help people. ctext.org/mozi

Post attachmentPost attachment
Rudolf Laine avatar
Rudolf Laine@LRudL_·Oct 13

With recent discussions of EA influence, it’s worth taking a look at the first time an effectiveness-oriented altruistic movement held significant sway over a country: 4th century BC China. The Mohists argued for “impartial care” as the moral standard, prioritized prevention of war as a top cause area, had members take high-paying positions in governments (i.e. the war-making bodies) to try accelerate defense over offense (and sometimes donate gold to the movement), and scorned spending on rituals and ceremonies on grounds that the poor should be given more food instead. The Daoists attacked them for demandingness and the Confucians for treating family like strangers. All lost to the Legalists when the Qin dynasty started. When impartial welfare philosophy came to the West with Bentham, it made a big advance on the Mohists. Mohist doctrine had been afraid of ghosts. Bentham disbelieved ghosts on the grounds that no one ever saw a naked ghost. He was deathly scared of goblins instead.

Rudolf Laine avatar
Rudolf Laine@LRudL_·Oct 7

This is a very richly informative eval, lots of juicy details to consider

How fast is AI's research taste improving? We find that the experimental research taste of frontier models has doubled every ~3 months since December 2025. The best model, Opus 5.5, now exceeds our expert human baseline. Our human experts are experienced researchers, but most haven’t worked at a frontier lab. Why measure research taste? In the AI Futures Model, it largely determines how quickly artificial superintelligence is reached once coding is fully automated.

Post attachment