It’s unfortunate that mostly black authors have been outed for using AI to write fiction, but does that mean AI detectors are racist ? Sadly, the answer is a resounding NO. With the help of @pangram and @GPTZeroAI I analyzed 187 African books published between 2001 & 2026. 👇👇
What information capabilities do #AI systems need to help us, informavores, navigate diverse, multicultural narratives in our current & future information ecosystems?
Do current models support them? No.
Introducing NarrativeBench.
isle-dev.github.io/Narrative…#NeurIPS#NLProc#HCI
Do you like vibe coding and want money? 🤑
With Ai2, we're recruiting for a paid user study (~6-9 hrs) where scientists vibe code websites for scientific tasks
Interest Form: docs.google.com/forms/d/e/1F…
Extra credit if you share with researchers in fields underrepresented below :)
LLMs don't hallucinate as much as they used to, but Tiasa's work shows an error that's still frequent: smoothing!
An LLM may look at a bunch of papers and say "based on the papers, method A beats method B" even if those papers show cases where B beats A. SOTA LLMs still do this
Can an LLM write your research report?
🚨Not well! They tend to oversimplify things: “method A beats method B” turning mixed/conditional findings into stronger and broader conclusions.
In our work, we call this smoothing and introduce SmoothScore to analyze and quantify this effect! 🧵
announcing &tutor, a tutor we trained against persimmon!
there are many reasons I'm so excited about this. here are a few:
1. this is a case study into the differentiated bet that we're making at humans&: having good models of users (e.g. learners) allows us to train AI models that can help people over long time horizons (e.g. teach in a way that imparts new understanding)
2. education is a near and dear application close to my heart, and I've been wanting to work on something like this since starting my PhD
3. @314Bansal is the 🐐🐐🐐 and it was so much fun working together on this
Introducing Ampersand Tutor (&Tutor) - we trained a model to learn like people, then trained &Tutor to teach it
On a widely-used tutor model eval, &Tutor performed best, showing how better models of people can help build AI for humans
Available through our API - try it!
Incredible work from Hita on a timeless challenging NLP+HCI problem: how do we help readers effectively navigate a complex answer with citation to effectively retrieve evidence. Check it out!
Commencing the annual defrost of my academic social media to share delightful projects done with delightful collaborators that I’m excited to present at #UIST2026, the first of which is: Attribution Gradients, which also got a 🏆 Best Paper Honorable Mention!
Must-read paper from Microsoft Research and Georgia Tech.
This one has lots of great insights.
They used 163,487 Reddit posts about AI agents and surveyed 73 experienced agent users to map how delegated agent work goes wrong.
The result is MANDATE, a taxonomy of 23 failure categories in seven groups. Each describes an agent that finishes the job in a way the user didn't expect.
Users rated the most severe failures as the hardest to notice.
People who had already run into a failure found it easier to spot than people who only thought it was possible.
The most common failures are often the least studied.
Agents that keep running and spend time and compute without making progress are among the most commonly reported failures, and they have the thinnest research base.
arxiv.org/abs/2610.34748
Chat with Paper: academy.dair.ai/papers/babys…
We're hiring PhD research interns to join the MSR AI Interaction and Learning team for Summer 2027. We advance the science of large-scale AI systems through research on behavior, collaboration, evaluation, learning, and adaptation.
Learn more and apply: apply.careers.microsoft.com/…
Heading to #EMNLP2026! Excited to share two papers on LLM user simulation:
🧠 KnowSim: Simulating how users learn through AI interactions, validated against real human–AI interaction data across different knowledge levels.
🎭 User Role Vector: Investigating assistant bias in LLM user simulators through internal representations and activation steering.
I’m also on the job market! I’d love to connect with folks interested in user simulation, interactive AI evaluation, and oversight for agentic systems. Feel free to DM me!
AI assistants can provide correct, comprehensive information and still be poorly calibrated to the user: too difficult for novices, redundant for experts.
KnowSim is a user simulator that tracks what a user knows and updates that knowledge as the conversation unfolds, so we can evaluate information calibration over multi-turn interactions.
Across 9 LLMs, model rankings changed with the user’s knowledge level.
📄 arxiv.org/abs/2608.17150
Time flies and I am looking for full time research positions! I am finishing my PhD at Cambridge early 2027. My research focuses on simulating human behavior with LLMs, LLM personalization, evaluating and shaping model behavior, and understanding human–AI interaction. Please get in touch if you are hiring in relavant areas!
User simulators are only as faithful as the user representations they are conditioned on.
Missing details force simulators to fill gaps with assumptions, leading to generic behavior⚠️
Introducing UserRepGen-4B, which generates representations optimized for simulation fidelity.
User simulators are trained to talk like users. But users don’t just talk, they judge.
We introduce simulators that model both: what users say and what they think of the assistant.
Please come to our poster today afternoon (session #6) at #COLM2026 on *Weightless Fine-tuning for Personalizing LLMs without actual Fine-tuning*
Supervised fine-tuning (SFT) is one of the strongest ways to personalize a model to an individual user. (In one of our recent papers with @TuhinChakr and Jane Ginsburg, for example, we used per-author SFT to emulate the writing styles of award-winning authors.)
But what if we want to personalize a model to thousands or millions of different authors/users?
SFT gets expensive very quickly:
- run a separate optimization for each user;
- store separate model/LoRA weights for each user;
- retrain as new user data arrive.
WildChat, LMSYS, ShareChat… each human-LLM chat dataset aims to capture "real-world" AI use. But do they paint the same picture?
Not quite. From user messages alone, a classifier can name a chat's source dataset, and user models trained on them inherit these signatures 🧵👇
I'm at COLM this week, presenting:
- LLMs Corrupt Your Documents When You Delegate. At today (Wednesday's) afternoon poster 4.30-6.30. Come check out our very (!) colorful poster.
- Giving a fun talk on mostly ongoing work at the Workshop on Lifelong Agents early Friday morning.
Old and new friends, let's chat!