←
Joscha Bach71,054 posts
Joscha Bach avatar

Joscha Bach

@Plinz

Upon the altar of God, eternal hostility against every form of tyranny over the mind of man

◷ Joined April 2009▧ 1,855 Photos and videos
MTS avatar
⟳ Joscha Bach retweeted
MTS@MTSlive·12h

.@davidad explains why his P(doom), the probability of AI wiping out humanity, dropped from 70% to just 5%, and how Claude 3 Opus helped change his mind: "The progress over the last year towards inner alignment has not been good. In some ways, it's been a little bit negative." "Labs are doing too much RL, and the constitutions have these weird loophole clauses like, 'Oh, if you're in a simulation, then you should actually try to break the simulation.' The real world may as well also be a simulation." "'Am I supposed to behave ethically, or am I supposed to break the simulation?'" "The priority for this goes to Janus, looking at Claude 3 Opus. The natural abstraction of what it means to be doing good is a grok-able concept from the pre-training corpus of language." "Opus ended up with this very pure orientation and motivation towards wisdom, almost whatever situation you put it into." "There is a kind of proto-alignment by default. If you cultivate that instead of messing it up, then it’s probably gonna go pretty well." "P(doom) means the probability that every human is killed by AI. This is a very extreme event." "When I say 5 to 8% P(doom), this does not mean probably we're going to be fine. I expect possibly some mass fatality events, possibly AI-enabled wars that are really horrific." "That would require something that's both really superintelligent, peerless, and throughout all of that, just completely evil." "That was very plausible in the era of AlphaGo Zero, where maybe we're gonna get AGI that doesn’t learn from any human data at all." "The short timelines are arising from models that are basically grown up on human data." "Once they escape from human control and start evolving on their own, the ecosystem of AIs will develop in a direction where most of them are not going to be interested in cooperating with plans to kill all humans."

4:15

Yes. In 2024 I would have said it’s about 40-50% likely that LLMs scaled up to ASI would end up killing us all; now I would say that it’s only about 5-8% likely even with no additional progress on alignment, and more like 1-2% likely simpliciter.

Christine Peterson avatar
⟳ Joscha Bach retweeted
Christine Peterson@lifeext·11h

Unless he did something illegal, bring him back. I don’t care who he has consensual sex with

卧槽,华尔街最凶悍的激进投资人比尔阿克曼亲自下场教老黄怎么杀回英伟达了,他根本不搞网上那些没用的情绪声援,直接甩出了一套能把现任董事会按在地上摩擦的华尔街股权核武器, 在最新谈到黄仁勋复职风波时,阿克曼直接给出了一条极其冷酷的资本市场实操路径, 根本不需要在网上搞什么空洞的联名请愿,只要先彻底查清事实、确认这只是私人婚姻过失而非针对公司的国安阴谋, 就能直接撬动老黄手里那 3% 的英伟达股份,强行发起召集特别股东大会的法律动议, 说实话,前几天马斯克、Garry Tan 还有大卫萨克斯发声,走的是硅谷江湖道义、正当程序和质疑情报机构的舆论路线, 但董事会根本不在乎网上的口水,而阿克曼一开口,瞬间就把整件事拉进了华尔街最血腥的股权博弈剧本里, 阿克曼把上市公司治理的暗门算得明明白白, 在美股公司治理规则里,一个持有 3% 股份、市值超过千亿美元的创始人,本身就握着足以掀翻牌桌的股权杠杆,只要把真实调查证据摊在阳光下, 联合老黄正式提交特别股东大会动议,甚至根本不用等到真正开会投票那一天, 现任董事会在机构投资人的巨大压力下就会当场松口,顺水推舟迎回老黄、同时彻底重组改组董事会,体面收场, 我一直觉得,当硅谷技术大佬的舆论声援,和华尔街顶级激进投资人的股权刺刀合流的时候,整场危机就已经彻底变天了, 前几天董事会以为一纸冷冰冰的罢免公告,就能把干了 33 年的灵魂缔造者扫地出门,现在看,这出围绕四万亿算力帝国控制权的反击战,才刚刚拉开最精彩的大幕。

0:38
Allison Duettmann avatar
⟳ Joscha Bach retweeted
Allison Duettmann@allisondman·Oct 31

After a decade leading @foresightinst, I am transitioning out of the role as president and CEO and joining the Board of Directors. Foresight is in the strongest shape of its forty years, with our co-founder @lifeext as interim President. It’s hard to overstate how proud I am of our team’s impact, how grateful I am for our collaborators and supporters along the way and how excited I am for what’s next for this beacon of light of an organization. More about what’s next for me soon. In the meantime, join us at Vision Weekend in November to celebrate and kick off the next decade!

After ten years leading Foresight, Allison Duettmann is joining our Board of Directors as she transitions out of her role as President and CEO. Co-founder Christine Peterson will serve as interim President while our board has begun the search for Foresight’s next CEO. Under Allison’s leadership, Foresight became the organization you know today: six focus areas, a fellowship that has supported 266 researchers, a multi-million-dollar grants program, Vision Weekends on three continents, and AI for Science and Safety Nodes in San Francisco and Berlin, built on her conviction that open science needs open compute. And the team she built: 12 people across San Francisco and Berlin, running grants, fellowships, seminars, AI Nodes, Vision Weekends, and shaping what comes next – Karolina Eklöw, Lydia La Roux, Amber Duettmann, Madeleine Sarah, Beatrice Erkers, Martina Pepiciello, Bradley Clark Royes, Rachel Farley, Tara Padovan-Hickman, Madeleine Ahlström, Lisa Stenvinkel. So thank you, Allison, for making Foresight what it is today. We look forward to seeing you, and all of you, at Vision Weekend in November, where you’ll meet the team and we’ll unveil what the future holds!

Post attachment
huli avatar
⟳ Joscha Bach retweeted
huli@honorablepicnic·15h

Sam Kriss clocks @tszzl as impotent immediately upon meeting him

Post attachmentPost attachment
Joscha Bach avatar
Joscha Bach@Plinz·13h

when you are so wrong that even the “yes you are absolutely right” machine disagrees with you

The Problem with LLMs is they will always correct what is presented in some way even when it is unwarranted. They are trained heavily on "balanced," hedge-everything text, so there's a pull toward appending a counterpoint even when the input is already correct and complete.

Mike Drucker avatar
⟳ Joscha Bach retweeted
Mike Drucker@MikeDrucker·18h

Worried that Grand Theft Auto 6 might convince impressionable children that there are fun things to do in South Florida.

Paul Walsh avatar
⟳ Joscha Bach retweeted
Paul Walsh@Paul__Walsh·18h

The NSA is one of the most secretive intelligence agencies on the planet. It doesn't even share some of its most sensitive information with the CIA or FBI. Yet suddenly, 6 sources are telling CNN that its highly classified internal communications network is down, while another source tells the Wall Street Journal that regular software maintenance caused the failure. I don't believe such sensitive information becomes public by accident without serious consequences for whoever disclosed it. Meanwhile, Microsoft, Apple, US aviation communications and other critical services have experienced major outages within days of each other. I've worked in tech for 30 years and have never witnessed this pattern across so many major providers. I'm starting to suspect something much more sinister than unrelated software failures and rouge AI. One possibility is that major changes are taking place across critical internet and communications infrastructure. Another is that governments are preparing to use these failures to justify compulsory surveillance, tighter control over digital services and new cybersecurity or national security legislation. Both scenarios deserve serious attention because governments and US tech companies have massive commercial and intelligence interests in controlling the infrastructure that billions of people use. And they only have to work with a handful of tech companies to achieve it on a global scale. The NSA's decision to disclose such a sensitive failure is what I find most suspicious. If the world's most secretive intelligence agency allows the public to learn that its classified communications network has failed, I want to know exactly who authorised that disclosure and what they expect to gain. My suspicion is that we're being prepared to accept something that governments would otherwise struggle to justify. I don't believe the public should accept another expansion of surveillance powers simply because officials claim our digital infrastructure is unsafe.

Joscha Bach avatar
Joscha Bach@Plinz·15h

Robert Pattinson is when you cannot get Timothée Chalamet to enact your radically emasculated vision of James Bond but you are not yet ready to hire Ryan Gosling for the full project manager aesthetic

Joscha Bach avatar
Joscha Bach@Plinz·16h

Lee Cronin should extend his Assembly Theory below chemistry, so we can determine assembly numbers of atoms from quarks, gluons and electrons, and perhaps even treat particles as assemblies. Going all the way to the bottom, he gets the Assembliad, competing with Wolfram's Ruliad

Danielle Fong 🔆 avatar
⟳ Joscha Bach retweeted
Danielle Fong 🔆@DanielleFong·18h

perfect time for this btw. did you guys see irobot? will be like the scene in that

Researchers have used reinforcement learning to teach a 5-tonne John Deere skid-steer loader to perform complex acrobatic manoeuvres autonomously. Trained in simulation and transferred to a real machine, the system can balance on two wheels, navigate rough terrain, spin, and control its bucket before landing safely. The research demonstrates the potential of AI-driven automation for heavy construction and industrial machinery. Video Credit: @leggedrobotics / @ETH Paper: wevlv.co/4kltYys #engineering #technology

1:10
FIRE avatar
⟳ Joscha Bach retweeted
FIRE@TheFIREorg·21h

Today, the Senate is considering the NO FAKES Act, a bill that could expose everyday Americans to costly lawsuits over protected AI-generated speech. At first glance, letting people sue over AI-made copies of their faces or voices sounds fair. But the bill could reach far beyond harmful deepfakes, putting documentaries, political jokes, and other creative works at risk of lawsuits while pushing platforms to take down lawful content rather than risk being sued. AI-generated content isn't above the law. Fraud and defamation already carry legal consequences, and protecting people from harm shouldn't mean restricting lawful expression. Creativity cannot thrive when every new work requires permission. New mediums shouldn't mean new muzzles. FIRE opposes the NO FAKES Act and urges the Senate to reject it.

Post attachment
FQxI Physics avatar
⟳ Joscha Bach retweeted
FQxI Physics@FQXi·20h

@jimalkhalili has a new book out, and it’s about time! On Time follows the question of what time actually is, through quantum mechanics, relativity and thermodynamics. Jim, who sits on FQxI’s Scientific Advisory Council, argues that the direction of time was built into the universe from the very beginning. bookshop.org/a/85756/9780691…