We are hiring! I am looking to hire a PhD student to work on topics around AI evaluation from an HCI and technical standpoint. If you are interested, please apply through the Mila portal and put my name down as a potential advisor:
Mila's annual supervision request process is now open to receive MSc and PhD applications for Fall 2027 admission! For more information, visit mila.quebec/en/prospective-s…
🎥 Ever wished to control what happens next during watching video? 🎬
You can decide the next scene, but keeping video coherent is hard. Meet VIS-Ground from @Google! We compile contextual dependencies for constrained long video generation.
🔊 Sounds on! Check more vis-ground.github.io
(4/n) We hope our findings will inspire the development of DRAs that can dynamically decide when to ask users for clarification and when to continue searching during task execution, balancing reliable task completion with minimal user disruption.
(3/n) Our analysis of intermediate search behavior provides a possible explanation: DRAs conduct broader searches that draw less on the original user query but rely more on their learned prior during training, which can be misaligned with users’ intent.
(2/n) We also analyzed agents’ awareness of query underspecification and their responding strategies.
DRAs cannot “guess” what users want most of the time, nor do they realize that users’ queries may contain multiple possible interpretations. Even when they do recognize that a query is missing important information, they tend to proceed with assumption-based answers rather than asking users for clarification or abstain.
(1/n) We identified common underspecification patterns from real-world user queries and applied them to three existing benchmarks focusing on open-ended deep research tasks.
We found that omitting information essential to defining the research question, such as technical terms or scope limitations, had a much greater impact on how well agents’ answers satisfied query-specific evaluation rubrics. In contrast, omitting preference-related information, such as “please include only high-quality sources,” had a relatively smaller impact.
People tend to be more casual when asking AI questions than when asking another person. But what are the consequences? Can existing deep research agents (DRAs) infer what users leave unsaid through agentic reasoning and searching?
💡Introducing UNDERSPEC-DR, a framework for evaluating how deep research agents handle underspecified user queries.
📶Our results show that while DRAs can still generate answers grounded in cited evidence, their answers may fall short of satisfying users’ information needs substantially, even when one single detail is missing from the original query.
📄Paper: arxiv.org/abs/2610.39413v1
Next week at #UIST2026 in Detroit, I'm presenting Doki, a text-native interface for generative video authoring (🏅Best Paper Honorable Mention)!
Here's the trailer…👇
So happy I could dance! 💃 Our paper "🪩 DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering" has been awarded Best Paper at #EMNLP2026!
aclanthology.org/2026.emnlp-…
What are users thinking during their interactions with LLMs?
We introduce ThoughtTrace — the first large-scale dataset that captures what users think during real-world human–AI conversations, not just what they type.
→ 10,174 thought annotations
→ 2,155 multi-turn conversations, 17,058 turns
→ 1,058 users
→ 20 LLMs
These thoughts improve user behavior prediction (+41.7%) and model alignment (+25.6%).
This opens a new paradigm of user-centric LLM research. Full information in the thread 🧶
Read our paper: arxiv.org/abs/2605.20087
Check our project website: thoughttrace-project.github.…
🤖 We often talk about “context rot”: LLMs get worse as context grows.
But once distracting information enters, is it just “a bit more noise → a bit worse performance”?
Our #ICML2026 paper finds: no! 🤯 Instead, we reveal a striking "First Drop of Ink" effect: the first very few hard distractors do almost all of the damage, exactly like how one drop of ink clouding clear water.
Paper link: arxiv.org/abs/2605.10828
New Anthropic research: Natural Language Autoencoders.
Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read.
Here, we train Claude to translate its activations into human-readable text.
While AI has the potential to automate semi-structured interviews, it falls short in perceiving nonverbal cues, building rapport, and strategically steering long-horizon conversations. At the same time, qualitative researchers often want to conduct interviews themselves, as doing so helps them better understand participants and may lead to deeper insights.
Instead of asking whether AI should replace interviewers, we asked: 𝗛𝗼𝘄 𝗰𝗮𝗻 𝗔𝗜 𝘀𝘂𝗽𝗽𝗼𝗿𝘁 𝗶𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄𝗲𝗿𝘀 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗴𝗲𝘁𝘁𝗶𝗻𝗴 𝗶𝗻 𝘁𝗵𝗲 𝘄𝗮𝘆?
In our #CHI2026 paper, we introduce InterFlow, an unobtrusive form of AI assistance that supports interviewers in managing interview flow and facilitates real-time data sensemaking. InterFlow features ambient visualizations, mixed-initiative information capture, and process-oriented AI suggestions grounded in empirical knowledge of semi-structured interviewing.
Our user evaluation showed that InterFlow reduced cognitive load and helped interviewers keep track of both the big picture and important emerging details during interviews.
Huge thanks to my collaborators Yu Zhang, Sriram Suresh, and to my advisors @luzc08 , @canlhci , and @Iriskie_Xia!
Full paper: arxiv.org/pdf/2602.06396