←
Jonathan Stray25,196 posts
Jonathan Stray avatar

Jonathan Stray

@jonathanstray

Knowing things is a solved problem. Getting along is not. Doing the science to ensure AI results in less war, not more @CHAI_Berkeley.

◷ Joined May 2008▧ 1,359 Photos and videos
Jonathan Stray avatar
Jonathan Stray@jonathanstray·Sep 10

Hello, every day I am working on making sure AI doesn't escalate human conflict. - AI-powered social media can reduce polarization rankingchallenge.substack.co… - This is now a product you can use mysky.social - We showed that a rigorous definition of "political neutrality" is possible and increases trust x.com/jonathanstray/status/2… - We've got a proof-of-concept superhuman conflict resolution bot working in the lab - Next is human validation tests, then multi-turn model evals. AI-induced conflict is a catastrophic risk. Here's the broader research agenda: betterconflictbulletin.org/p…

What could it mean for an AI to be "politically neutral”? And can we measure it? New paper + dataset. We propose a defn that applies to any type of conflict: a neutral response should maximize approval on both sides of an issue, while keeping that approval balanced. 1/🧵

Post attachment
Jonathan Stray avatar
Jonathan Stray@jonathanstray·Nov 7

Will AI finally enable effective central planning? This is a rehash of the old "socialist calculation debate." My take is: while the actual "calculations" are now tractable, markets produce essential information that seems hard to get otherwise. In particular, it seems impossible to get accurate demand signals without actually having people trade for what they want. Otherwise, people have an incentive to exaggerate and say that their needs are very, very important. hetwebsite.net/het/essays/pa…

Dilan Esper avatar
⟳ Jonathan Stray retweeted
Dilan Esper@dilanesper·Nov 3

I think the Republicans have a point about some of their election law arguments. The problem is they tie them to sweeping claims of massive voter fraud and stolen elections, which are not true.

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 30

I want to prevent AI from starting wars. Wars between people. It has to understand human conflict, but I'm not building a mediator. The biggest risk is what AI says to individual people in conflict, because that's the most common case by orders of magnitude. I think better is possible. Here's what I know so far. docs.google.com/presentation…

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 22

"The scandal is in the selectivity." In an era of abundant fact checks, this is the most common claim of media bias. Not "you are lying" but "why did you cover this instead of that?" Here, the complaint is coming from the left. But I do have some sympathy for what Weiss is trying to do at CBS, because "why did you cover this instead of that?" has been a complaint from the right for a long time -- consider, for example, coverage of social justice issues vs. coverage of unauthorized immigration. Selective coverage is one reason why mainstream outlets have been bleeding out their conservative audiences for decades. I'm not saying I approve of how Weiss is handling this. Swinging from one pole to the other is not really a solution, and I've written elsewhere on what journalism that could be trusted by both sides would like like. But I do think the problem she sees is real. salon.com/2026/10/20/bari-we…

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 19

The goal of MySky is to be an open, state-of-the-art reference implementation for good social media algorithms. Here's what that means. - Open as in open source, public API, open data (within the limits of privacy), and open science. - State-of-the-art means we are on the leading edge of development, or near it. Today, this means modern AI-powered infrastructure, for example our prompt-based controls (now in beta). - Reference Implementation means something you can actually use, not just an idea. This also means having real users, because social media cannot be developed in a vacuum. Hence, our algorithm has to be something people actually want to use. - Good means all the things we might want from social media: controllable, transparent, informative, trustworthy, constructive, and more. There are many possible designs, which is why MySky is highly configurable. - Social Media means this is about feeds as opposed to, say, chatbots. Also, we are committed to making MySky genuinely social, something more connecting than television or news. - Algorithm means we are working on the core problem of selecting and ordering content published within a network, i.e. a recommender system. There are lots of other aspects to social media design, for example the user interface can make a big difference, but that’s not our primary area. We're up and running on ATProto. You can try it today! mysky.social/p/mysky-is-an-o…

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 18

We (@HelenaPuigL and I) are creating a research institute devoted to ensuring that AI makes human conflict more constructive and less violent. The strategy is basic research, proofs-of-concept and public evals. We should call it the _____ Institute

Micah G. Allen avatar
⟳ Jonathan Stray retweeted
Micah G. Allen@micahgallen·Oct 15

I’m taking in new, curiosity driven projects that I would previously avoided due to limitations and time. My dream is Star Trek. More time for leisure and family, more meaningful and focused work.

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 15

Now we will see if the physicists respond any more constructively than the mathematicians. (I ❤️ mathematicians, but in adjusting to a world where AI exists, their leadership has not been leading.)

“We present a suite of 50 unpublished research-level theoretical physics problems with verifiable answers… [T]he best-performing agent… obtains the final answers with a near-perfect accuracy of 98% at an average cost under USD 3 per problem.” Under $3 per problem. Bruh.

Post attachment
Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 14

Yes, there is definitely some cult-like culture inside of Anthropic and OpenAI. Do not confuse this with whether their predictions are right or wrong.

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 14

How to teach human conflict to AI? I'm sitting on both "computational game theory" in the CS dept and "commercial negotiation and mediation" in the law school this term, because neither tradition has a complete answer.

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 12

I think all journalists and academics could stand to read this essay on the limitations of "speaking truth to power." "Intellectual courage is extremely important. But without first carefully establishing what’s true, it’s either pointless or harmful. So intellectuals’ primary responsibility is simply to seek and speak the truth, full stop. This job description is less heroic, but it’s more intellectually and socially useful, and it’s less vulnerable to self-deception precisely because it’s less heroic." conspicuouscognition.com/p/s…

Jonathan Stray avatar
Jonathan Stray@jonathanstray·Oct 12

What if LLMs did not use "I" in their output? I've heard various versions of this proposal before, and I interpret them as making a psychological causal claim: IF machines didn't use "I" THEN we would relate to them differently. I have doubts. Is there any empirical evidence that "anthropomorphization" in general changes how we relate to or use these machines in any signifcant way?