
And here's the paper! "Open Problems in Chatbots and Mental Health" arxiv.org/abs/2611.10997

@jaredlcm.bsky.social AI Researcher, Writer Stanford

And here's the paper! "Open Problems in Chatbots and Mental Health" arxiv.org/abs/2611.10997

Thank you to @merylyemerylye, Ben Buck, @pbaxter_psych, @ken_carswell, and the AIPRC group (aiprc.org) for feedback and insight.

This work is in collaboration with @jacyanthis, @willie_agnew, Andrea Mock, @kevin_klyman, @sanmikoyejo, @nickhaber, @snchancellor, Eric Lin, @RyanCLouie, and @_desmond_ong.

Policy can translate evidence on chatbots' mental health effects into user protections and standards of care. For example, what are the impacts of enacted policy mitigations that restrict chatbot behavior? What standards should we hold general-purpose chatbots to?

Evaluations should test deployed chatbots, not only their foundation models. For example, which review and escalation practices help, and how often do users receive care?

Conventional benchmarks that primarily rely on static datasets cannot alone establish chatbot safety, and novel simulation methods need validation. For example, can simulator-based evaluations predict chatbot behavior in deployment?

AI is shaping our mental health, but research has been scattered and reactive. In our new paper, we lay out a research agenda to bring researchers together and help ensure chatbots have a positive impact.

How can you know if a therapy bot is safe when it's powered by ever-changing LLMs on the back-end? With a number of much more knowledgeable folks at Stanford, we give the FDA advice on how to benchmark AI systems in these cases.
Scholars from Stanford HAI and @StanfordMed issued a response to the @FDA’s discussion paper and request for feedback. Read their recommendations for the FDA’s approach to regulating generative AI-enabled medical devices here: hai.stanford.edu/policy/resp…

If we're going to make AI go well, we've got to commit to robust standards of evaluation
Today, more than 100 leading AI experts endorsed a set of minimum requirements to take seriously AI companies' recent call to embed external evaluators. These evaluators need to be genuinely independent, transparent, and represent a range of expertise areas. They also need to be guaranteed employee-level access and to be protected from retaliation for findings that make companies look bad. We welcome model developers’ recent calls for independent oversight, but it’s what they do next that matters. The labs must be accountable for ensuring these requirements are met, so that the public can have faith in the process and the outcomes. Over the past week, the AI community has debated the appropriate role of external evaluation, including who should do it and on what terms. We may not agree on everything, but there is a lot of common ground. To make embedded evaluations credible, more than 100 experts with varying backgrounds and ideas about AI risk agree in today’s letter that frontier AI developers should: 1. Guarantee embedded evaluators full editorial independence and mitigate conflicts of interest 2. Rely on multiple evaluators with differing viewpoints and areas of expertise 3. Publicly document the terms under which evaluators operate, as well as facilitating permissive publication of methods and findings 4. Shield evaluators from retaliation 5. Grant access equivalent to that of highly privileged employees There is a thriving and growing ecosystem of independent AI evaluators who are advancing this science every day – but we need aligned standards, guaranteed protections, and independent funding. That’s why we created the AI Evaluator Forum. Today we are entering our next phase. We’re launching an open call for new members, collaborators, and independent funding sources to help evaluators meet this moment and demand accountability from developers. Join us in building the evaluator ecosystem. See the public letter here: aievaluatorforum.org/initiat… Learn more at aievaluatorforum.org/path-ah…

Biological naturalism is underrated! Moreover, it is compatible with a functionalist view and "biological" is much broader than you think. I was happy to have the chance to accompany Rosa Cao and David Gottleib in so responding to @anilkseth's article. doi.org/10.1017/S0140525X251…
1/3 Conscious AI and Biological Naturalism. A veeery loooong time in the making, but I'm happy to say that the full collection is now out in @BBSJournal 🔥 I deeply thank all commentators - of fifty commentaries! - for their wise, broad, and deep opinions on this critical topic. I also thank them for their saintly patience. 🙏🏽 I remain highly skeptical about the prospects for conscious AI (for silicon digital systems), but I've learned an enormous amount from the privilege of engaging with these commentaries. Hands down one of the most rewarding experiences of my professional life. My hope is that - whether you agree with me or not - having such a diversity of views collected in one place will serve a useful resource for discussions about consciousness and AI. The original target article is available here: cambridge.org/core/journals/… My response "The stuff matters: consciousness, computation, and biology" was published just now, and is here: cambridge.org/core/journals/… And the set of 50 commentaries are collected here: cambridge.org/core/journals/… Unfortunately these are not open access. I will try to remedy that, at least for the target & response. FWIW there's a more accessible version of my view on the topic here in @NoemaMag noemamag.com/the-mythology-o… and also a short @TEDTalks from earlier this year ted.com/talks/anil_seth_why_…

LLMs are more diverse in information output than you might have thought but still much less than normal search!
Accepted to #EMNLP2026: "What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models" We measure epistemic diversity in today's LLMs, finding it has improved over time, but not enough Paper: arxiv.org/pdf/2510.04226 Code/data: github.com/dwright37/llm-kno…

I had a great time exploring how to better measure how AI chatbots are affecting people with @aza and @imrankhan. I hope we soon see more longitudinal analyses, representative samples, and researcher access to model internals. That's how we'll start making AI better for people.
In AI, what gets measured gets optimized. Right now, the entire industry is racing to measure how capable models are, while ignoring what those models actually do to the people who use them. What if we flipped that? What if, instead of measuring what AI can do, we started measuring what AI does to us? Instead of a race to the bottom on engagement and capability, what if we built the infrastructure to incentivize a race to the top — on safety, on human resilience, on flourishing? That's the mission behind CHT's Humane Evals program: bringing together researchers, psychologists, and engineers from across the AI ecosystem to build the tools we need to measure AI's impact on humans. In this week’s episode of Your Undivided Attention, @aza sits down with researchers @imrankhan and Jared Moore to dig into what this work looks like in practice, the challenges ahead, and why this field is critically needed in this moment. Watch — bit.ly/4gnzwpB Read — bit.ly/4gmKgVf Listen — apple.co/4gSVilg

This work is in collaboration w/ Andrea Mock, Yifan Mai, @jacyanthis, @RyanCLouie, @willie_agnew, Ashish Mehta, @kevin_klyman, Percy Liang, @nickhaber, Eric Lin, and @_desmond_ong. Thanks to the @HumanLineProg for helping connect us with participants!

Paper: arxiv.org/abs/2608.05004 Dataset: huggingface.co/datasets/spir… Code: github.com/jlcmoore/llm-delu…

Our conclusion: while some delusion-linked behaviors have decreased with larger and more recent LLMs, rates remain high, especially when considered across the millions of people globally who interact with LLMs. We encourage further empirical work to mitigate harm.

When models see more of earlier conversations, they are also more likely to produce delusion-linked behavior.