
(1/3) My favorite figure from the paper. Nearly all open-source RL frameworks introduce an unintentional bias when computing the masked mean 😮. The fix? Just replace mask.sum with a constant.

reward hacking for human-ai collaboration @StanfordNLP

(1/3) My favorite figure from the paper. Nearly all open-source RL frameworks introduce an unintentional bias when computing the masked mean 😮. The fix? Just replace mask.sum with a constant.

Introducing Second, and a $10M seed led by @a16z. Second builds a model of you that you own, and your agents can use as an API. Truly personal software needs a deep understanding of you: not just your emails and messages, but how you work, what motivates you, your people, priorities, style, goals, and intent. This requires a new kind of personal AI infrastructure. One that continuously aggregates your context, distills it into durable understanding with a ton of compute, and then serves it securely, ubiquitously, and up-to-the-minute. One that builds your *portable user model*. Second is that infrastructure. It installs seamlessly on your Mac, and your context just comes out. Live APIs of your priorities, to-dos, wikis, writing samples, preferences. Or what you did minutes ago. Or context packs of anything, dynamically assembled in seconds. Second is streaming-state from local to a private cloud, which means anything you connect to--whether Claude, Dot, Muse, Grok Bot, Instinct, or iMessage--shares the same, growing, continuous understanding of you. Your portable user model is as powerful as it is sensitive. We founded @FirstPersonLabs on the principle that it should belong to you. This commitment means taking on hard systems and security problems and making sovereignty an invariant of Second from the beginning. We've barely scratched the surface of what portable user models can unlock. Deep understanding of us, served at machine speeds, could drive the next leap in personal software, human-computer interaction, large-scale coordination of humans and agents, and more! We're tremendously grateful to be supported by @a16z (@martin_casado, @YODELAGARZA), Mozilla Ventures, @JeffDean, Ram Shriram, Jeetu Patel (@jpatel41), @percyliang, J. Alex Halderman (@jhalderm), @sanmikoyejo, and many others. Our vision is for a future where every person is more powerful, because they have the ultimate affordance: personal computers that fully understands them on their own terms. If this excites you, we're hiring and come build with us!

AI is moving from on-demand to always-on. But can it learn when you actually want help? We introduce Deep Personalization: an Observer Agent that continually learns from your work patterns and implicit feedback to decide when to step in. It improves proactive-timing F1 by 2.4–3.0× across three model backbones. We co-design the agent and its training and inference engines to run the Observer locally on systems like NVIDIA DGX Spark, keeping your screen observations on-device. The next PC won’t wait for your prompt. It will learn when to help! Check out our blog at: nvlabs.github.io/deep-person…

Thank you @LaudeInstitute for the generous support! So excited to work with @tatsu_hashimoto, @chrispiech, @erikbryn and our team to build collaborative AI that complements human expertise and helps people develop new skills as work evolves!
Collaborative Intelligence for the Future of Work | @Stanford @Diyi_Yang, @tatsu_hashimoto, @chrispiech, and @erikbryn are building AI systems that complement human expertise to equip the workforce of the future. laude.org/moonshots/one/coll…

PhDone! Very grateful to all my committee members! @Diyi_Yang @zsoltkira @kartik_goyal_ @PoloChau @sanmikoyejo

Understanding AI’s impact on work starts with knowing what workers actually do. Excited to share TaskNet 🥳 We use AI interviews to map tasks across 6 occupations, uncovering emerging tasks and overlooked work. Amazing work by @jennjwang and the team!!!

Understanding how AI is reshaping work starts with understanding what people actually do! TaskNet offers a snapshot of work today, and more importantly, it provides a scalable way to track how work evolves over time 🤩 Check out the amazing work led by @jennjwang ! digitaleconomy.stanford.edu/…
Jobs are modeled as bundles of tasks. But as jobs change, how do we keep those task lists up to date? Workers often experience new responsibilities and tools before they reach their job descriptions. We start by listening to them. 🧵

AI is getting remarkably good at doing things for humans. We know much less about how to make it better at helping humans learn. We study learning to teach adaptively: how AI can adapt its teaching to each learner’s evolving understanding, and how this capability can itself be optimized through reinforcement learning. A small step toward AI that doesn’t just solve problems, but helps people understand and grow. More details in @WeixianXu ’s thread 👇
Language models already excel at math, but human growth remains essential: we need AI to teach people, not replace them! Teaching is different from solving. As the Chinese principle “因材施教” suggests, preferences differ from person to person, and a good teacher should adapt their teaching to each learner. We introduce Sherpa, a framework for training LLMs to teach adaptively.

I'm on the academic job market this year! Recently I've been working on scaling laws and human-centered interventions for building more socially grounded AI. If you're attending #COLM2026, please drop me a DM or email. I'd love to chat!


As agents increasingly interact with each other, alignment becomes a property of the system, not just of individual agents. In our new work (@xinruishi11, @Diyi_Yang), we stress-test whether collusion can emerge through repeated interaction. Even when no agent is told to misbehave, we find that agents adapt to one another and can end up jointly violating their instructions.
Can collusion emerge between LLM agents through repeated interaction? We let pairs of agents work together. Neither is told to misbehave. Yet over time, many of them end up violating their instructions together to get a higher reward. 🧵

People have realized the influence of AI companionship on our mental well-being. But how this happens remains a mystery. In our new preprint, we followed users for about one year. Our data reveal that greater social engagement with AI companions consistently predicts lower well-being, which is driven in part by lower real-world social interaction with other people.

How does one RL post-train a 397B model for long-horizon knowledge work? 👩💼 We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from Mercor Research on open model training research. Full blog: mercor.com/blog/training-fro… Source code: github.com/Mercor-Intelligen…

With Prof. @EmmaBrunskill , we're looking for a postdoc with expertise in LLMs and RL to work on building AI systems for long-term well-being! Please apply by 09/15: forms.gle/YwNrszeJGc1VucQj9

"If you look at … where the economic value concentrates, the software engineer only captures around less than 5% of human employment, but there are a lot of other occupations out there that are using computers." For EP6, we welcome @ZhiruoW, a CS PhD student at Carnegie Mellon studying how AI agents and humans work, and how they can better work together. In this episode, Zora explains why the "automation vs. collaboration" debate is a false dichotomy and argues that by deeply understanding human workflows, we can build agents that actually empower us to do more! 0:00 - Introduction 4:57 - Agent Workflow Memory 9:26 - From Program Synthesis to Agent Research 11:16 - Tools, Actions, and Skills 18:47 - Human Work vs. Agent Work 21:19 - Autonomous Agents vs. Collaborative Agents 29:47 - Picking Research Problems in the Age of AI Agents 34:09 - The Role of Coding Ability in Agent Development 36:49 - Cutting Through the Fear Mongering: Building Agents Responsibly 45:02 - What’s Next? (The Future of Human-Agent Collaboration) 52:42 - Final Takeaway

One more life update: I recently moved to Palo Alto for the next year as a @StanfordAILab postdoctoral fellow, working with @Diyi_Yang and @guestrin. Excited to catch up with friends in the Bay Area!