←
Robin Na53 posts
Robin Na avatar

Robin Na

@RobinNa_

PhD candidate @MIT | AI for social science, multi-agent AI | formerly @AmazonScience, @CarnegieMellon

◷ Joined March 2019▧ 5 Photos and videos
Elinor avatar
⟳ Robin Na retweeted
Elinor@elinorpd_·Nov 10

A new & improved version of the paper is out on arxiv! arxiv.org/abs/2607.22305v2 check out the (very human) writing process over the course of last two 2 months huge thanks to @bakkermichiel @mitchellgordon @jacobandreas @jrfisher552 for your help!

0:22

Pluralistic alignment is thriving as a research agenda yet failing at its goal: making the AI systems people actually use more pluralistic🌈 🚨New position paper: we argue adoption in deployed models should be the fields main goal & we provide a roadmap of how to get there 🧵1/

Post attachment
Robin Na avatar
Robin Na@RobinNa_·Nov 4

8/8 It was an absolute pleasure working on this with my amazing co-authors! We’d love to hear thoughts from researchers working on AI for behavioral/user simulations, synthetic human datasets, and digital twins! See paper at arxiv.org/abs/2611.00661

Robin Na avatar
Robin Na@RobinNa_·Nov 4

7/8 Our paper demonstrates AI-driven social science that discovers interpretable behavioral features, learns effective representations, and evaluates out-of-distribution generalization. This builds on our earlier perspectives: arxiv.org/abs/2404.11794 journals.sagepub.com/doi/10.…

Robin Na avatar
Robin Na@RobinNa_·Nov 4

6/8 But building profiles in one family of games and using them to simulate another produced uneven results. Profiles built from richer behavioral records (public-goods games) transferred more successfully than those built from sparser records.

Post attachment
Robin Na avatar
Robin Na@RobinNa_·Nov 4

5/8 Not all features were equally helpful, and their relative predictive value was remarkably consistent between learning and validation data! Our method can identify useful, generalizable features before simulating the behavior of new participants in new environments.

Post attachment
Robin Na avatar
Robin Na@RobinNa_·Nov 4

4/8 Across public-goods, bargaining, and money-request games, simulations using one or two selected behavioral features more closely matched human outcomes under new game rules. Surprisingly, adding more features did not improve the simulations further.

Post attachment
Robin Na avatar
Robin Na@RobinNa_·Nov 4

3/8 We define candidate behavioral features from the literature, then use an LLM to turn existing behavioral data into numerical profiles. We give profiles with different feature sets to a separate LLM simulator and see how closely its outcomes match human outcomes.

Post attachment
Robin Na avatar
Robin Na@RobinNa_·Nov 4

2/8 Human behavior is complex, so any person can be described by countless features. We can’t fit them all into a prompt. Even if we could, including every detail might not improve the simulation. It’s also hard to predict which features will matter most.

Robin Na avatar
Robin Na@RobinNa_·Nov 4

Prompting LLMs to behave like people can improve simulations, but how should we design this? What info is needed, and how much? With @Kehang_Zhu @locross @bakkermichiel, we build “behavioral profiles” that help LLMs simulate human behavior under distribution shifts. 1/8 🧵

Post attachment
Kehang Zhu avatar
⟳ Robin Na retweeted
Kehang Zhu@Kehang_Zhu·Nov 1

Data-driven economic ABMs are very interesting. Poledna et al. (2023) is a compelling example: a national-scale economic simulation that rivals traditional VAR and DSGE models in out-of-sample forecasting. doi.org/10.1016/j.euroecorev… Exciting potential for ABMs × AI!

This is true and I have always thought it was a big shame. Part of the reason (as Andrey notes in the replies) is that historically ABMs had not been taken to the data. But this is changing. Here is my colleague @RMaria_drc’s recent work bridging this gap: arxiv.org/abs/2412.16591 ABMs x AI hold a lot of promise for economics, and it’s an active area of research for our team.

Logan Cross avatar
⟳ Robin Na retweeted
Logan Cross@locross·Oct 29

Jobs do far more than deliver a paycheck—they are a core institution in modern society for agency, purpose, recognition, and community In our new DeepMind Institute essay with @scychan_brains and @JulianDJacobs, we explore research on how work relates to wellbeing. As AI potentially changes the nature of work, we must design policies and institutions that protect its non-monetary value bit.ly/wellbeing-work-agi

Post attachment
Michiel Bakker avatar
⟳ Robin Na retweeted
Michiel Bakker@bakkermichiel·Oct 25

A new very creative method for measuring ambiguity with LLMs co-led by @dsallydt and @LukeThorburn_ (EMNLP Oral 2026). When I worked with @mhtessler @summerfieldlab et al on the Habermas Machine (a project in which we use LLMs to find statements that maximize agreement within groups of people), we often found that LLMs maximize consensus by simply making statements more ambiguous. Eg a statement like "peaceful protests should be allowed" will likely lead to high agreement but that's because a lot of disagreement hides in different interpretations of what's "peaceful". One way to measure ambiguity is to ask lots of people how they would interpret a statement and see if those people disagree. But this is obviously impossible to do at scale. In this work we show that we can use LLMs to help approximate this. Read Sally's thread or the paper for more details. I think work like this is super important, for example to stress tests AI constitutions and model specs to see which rules are ambigious and in which situations. If you are in Budapest, please stop by Sally's presentation.

Post attachment

🚨 New paper: how to measure ambiguity in laws, public proposals, and even AI constitutions! with @LukeThorburn_ @Werdnamai @bakkermichiel 🧵

Kehang Zhu avatar
⟳ Robin Na retweeted
Kehang Zhu@Kehang_Zhu·Sep 30

What should we actually ask an LLM simulation to get right? Much of the field focuses on reproducing human distributions or specific moments. In our reworked paper, we argue that another target may be just as important: preserving the ordering across environments. 🧵1/6

Post attachment
Marcel Binz avatar
⟳ Robin Na retweeted
Marcel Binz@marcel_binz·Oct 12

What is a minimal set of assumptions that can explain the majority of human behavior? We investigated this question with large-scale cognitive models🚀 CORE writes prediction errors in hierarchical episodic memories with context controling encoding, retrieval, and forgetting.

Post attachment
Anand Shah avatar
⟳ Robin Na retweeted
Anand Shah@avshah99·Apr 22

🚨New preprint! We find evidence of LLMs enabling people to file lawsuits without lawyers (filing "pro se") at historically unprecedented rates in federal courts.👇 1/n

Post attachment