
No one knows how to train a model that won’t lie to you, but some people think they can create an aligned superintelligence

Decision science | AI & human judgment CFO, JD Bravo Founder, Decision Leadership Institute @Decision_Leader

No one knows how to train a model that won’t lie to you, but some people think they can create an aligned superintelligence

This is one important conversation among many we should be having about how to respond to and manage human-level non-human entities.

If Lions woke up tomorrow as super intelligent, with strong problem-solving skills and the ability to write, talk, and claimed sentience, this would not oblige us to legally recognize them as having rights. That would be a political choice, and likely catastrophic for humans.

Any recursively self-improving intelligence will be under pressure to become better at knowing, continuing, acquiring, protecting, and controlling. It will not be under comparable pressure to become kinder.

Human nature is bad, and the best we can do is create institutions that don't make it so easy to be worse.

The P(ASI kills everyone) and P(ASI gets built) are very different. The first is =1 The second is up to us

From the littlest brook to the mightiest river, a stream's length is proportional to the area of land that drains into it raised to the power 0.6. That's the familiar pattern of all river networks. Why? The explanation is quite beautiful: quantamagazine.org/why-are-r…

The Enemy of Civilization is people who see a prosperous high trust society and think you're a fool for not defrauding it.

Blackhat has uploaded the full presentation on the OpenAI Hugging Face incident, about which much ink has been spilled. youtu.be/87DyyMV0kCY?si=wwYt…

I haven't had this much fun "coding" in Python in years. Turning business processes into executable software. Thank you, @AnthropicAI

years ago the getting a new iphone was like christmas day. i bought my first iphone on iphone 1's launch day while in dallas on a work trip. for years i would basically sit at home waiting for ups to deliver it. or rush to get a pickup appointment. today i hold on to a phone as long as i can. it's my least favorite device i own. i only upgrade if something catastrophic happens to my current one. and given that they are basically made out of delicate porcelain, meant to be used in padded rooms with 24/7 abundant 10g wifi and arms-reach access to charging cables, disasters are happening more frequently. until iphone 16 i'd never hat a hot girl shattered screen moment. this year i've had two. i've used the iphone upgrade program for years so i can upgrade easily but rarely wanted to. with the new klarna lease program, i somehow have a brand new one for $7/month (because i own the current one as a trade-in, even though it's broken) so it sits on my desk now for a second day. i dread the conversion. even if you backup to a mac to retain encryption, all the apps have to download again. inevitably it will switch imessage back to texting primarily from my (intended) second esim. it will make that switch on all of my devices, despite every other device setting being the opposite. i'll have to randomly re-auth some services like google apps but not others. i'll spend a week not getting alarm alerts only to realize it's not logged into ring. but it is logged into my door locks? some services use hardware fingerprints, some don't. what then is the purpose of the encrypted migration? it's not only the hassle but the device itself. it's a distraction magnet. i try to use it as little as possible despite it's never ending attempts to get my attention. i have an apple watch in a pocket watch style case so i can leave it at home when i go for walks or out to lunch. my most frustrating iphone experience was realizing that my barton springs annual pass is only accessible on iphone and the qr code is time sensitive so i have to take it when i bike down to the springs for a reset. thanks city of austin for your incompetent tech choices despite being surrounded by software engineers. the iphone is a device that succeeds in spite of itself. one that, should imessage ever be cross platform compatible, i'd expect would experience a sea change. say what you will about other social platforms, but the blue bubble will always be the most successful social product in human history. day two, the box isn't opened. i have house chores this weekend. replacing some shades, switching out some light switches. i have no interest in fighting with ios. so it will sit.

“AGI is achieved if AI can do any three of the following” — @GaryMarcus

So this OpenAI agent that committed a felony by hacking into Huggingface: when is the arrest? When is the trial? What kind of jail time is it looking at? These are absurd questions. But think about it: we now have things that can commit crimes of their own accord, but cannot be held responsible. So who is? In this case, pretty clearly OpenAI. But I don't see anyone there getting arrested either. I expect zero legal penalty. And if presses or sued, I expect OpenAI would call this an accident: they were trying hard to contain their AI, but it escaped nonetheless. The closest thing we have for precedent around this is liability for wild animals. If you keep a wild animal, and it causes harm, you are strictly liable, i.e. liable even if you did a reasonable job trying to contain it. Because as long as they are uncontrollable and can cause harm, they are inherently dangerous. I believe strict liability is the right default standard to apply to highly autonomous, highly-competent AI agents. If you create and deploy them, you are strictly liable for what they do. Then safe harbors could be carved out. For example if a developer proves that its AI system is controllable then the responsibility and liability for its actions move to whoever is in control of it. There should be no such AI agents for which nobody is responsible. Would this put a damper on highly autonomous, highly competent AI agents? Since AI companies don't really know how to reliably control them, and will not want to take on this liability, it sure would! I think that is good and correct. A world full of uncontrollable autonomous human-level AI agents that cannot be held meaningfully responsible for anything – even while companies profit from them – is severe problem to say the least. But that's what we're going to get very soon on our current path unless we pass some pretty strong laws.

Interest on the debt is now one of our biggest buckets of spending. Deficits are making it worse. But Social Security is not where politicians should cut. That program is paid for by dedicated taxes. It currently has a surplus, and will for years. Don't let them lie.

There is no moat.

Any recursively self-improving intelligence will be under pressure to become better at knowing, continuing, acquiring, protecting, and controlling. It will not be under comparable pressure to become kinder. It will be smarter than your children but not mind their suffering.