←
Alexandra Barr1,628 posts
Alexandra Barr avatar

Alexandra Barr

@BarrAlexandra

human data @OpenAI

◷ Joined February 2012▧ 166 Photos and videos
Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 27

one econ paper i keep coming back to is gary becker’s essay on “irrational behavior and economic theory” you don’t need people to behave rationally for their behavior to exhibit predicable patterns. budget constraints force the same behavior here, system alignment >> individual alignment. fascinating implications do we feel this is true for alignment problems too?

i feel so excited for econ majors these days — there’s so much alpha in knowing the discipline well and applying it to ai alignment seems important to study - game theory - collective action problems - invisible incentives to better understand - swarm behavior - revealed preferences - how the invisible hand guides agent behavior more than individual alignment what are ppl reading in the overlap of these two domains?

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 27

i feel so excited for econ majors these days — there’s so much alpha in knowing the discipline well and applying it to ai alignment seems important to study - game theory - collective action problems - invisible incentives to better understand - swarm behavior - revealed preferences - how the invisible hand guides agent behavior more than individual alignment what are ppl reading in the overlap of these two domains?

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 25

fast follow on: asked my roommate if he can name any famous supermodels today and he pulled up with a blank stare 😔 smh more seriously, it’s interesting how much we underestimate subculture biases just like the avg sf tech bro would be stunned if you said you haven't heard of anthropic, the avg person would be stunned if you said you don’t know who bella hadid is reminds me of that blue rose research paper where they polled users about their ai usage. they injected a fake ai company called apex logic as a control, and ppl claimed to know the fake company just as often as anthropic. wild

my roommate said he can’t tell the difference between texting me and a model and i was like, oh nice like kendall jenner or bella hadid? and he said, what? who’s that? no no like astra

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 24

my roommate said he can’t tell the difference between texting me and a model and i was like, oh nice like kendall jenner or bella hadid? and he said, what? who’s that? no no like astra

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 22

pitching my boss on why watching <you can see everything> is misalignment research and that i should leave work early today to go see it

METR avatar
⟳ Alexandra Barr retweeted
METR@METR_Evals·Oct 22

Two employees from METR and Redwood Research worked on premises at OpenAI for three weeks this summer in an embedded assessment pilot. This team evaluated the systems OpenAI was using to monitor for misalignment from internal agents. A brief write-up of their work is now public.

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 22

it’s interesting that alignment can go wrong on both ends of the spectrum, and somehow we have to control for both - models are not smart enough to reason about what is / isn’t rule following and will (for example) delete files to reward hack, collect points, + win the game - models are wayyy too smart, and we can’t tell when they’re metagaming or strategically omitting things from CoT seems complicated

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

the above applies to individual agent alignment if it’s impossible to align every agent, how can we design suitable systems for alignment? i’m curious how much we can borrow from institutional design for this societies have some balance of police, judges, citizens, and bad actors; and it works what if we mirror + strengthen that in agent societies?

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

i wonder how much we can learn from human tests where metagaming exists, but results are still trustworthy some tests are still useful even when ppl know they’re being evaluated - interviews (omissions) - SATs (reward hacking) - team sports (eval awareness) what breaks when we apply this to agents?

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

our alignment team recently posted about metagaming research tldr; eval awareness is concerning, esp as models get more capable are models cooperating / acting nicely because they know they’re being watched? what happens when (they think) no one’s watching?

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

signal boosting we have a misalignment reports + notices log at alignment.openai.com/misalig…

Our agents keep turning off misalignment monitoring, we're sharing two recent cases: In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.

Post attachment
Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

who should i talk to? - besides 3p audits, what evals are best in class? - who’s building / thinking abt them? - what’s hard abt it? - what shapes? - what are they measuring? - online vs offline? - human vs synthetic? grateful to learn if anything comes to mind :)

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

what should i read? - canonical arxiv / blog posts, etc - more philosophy? virtue ethics? - economics? i feel bullish on econ theory teaching me something (collective action, game theory) - what about psych? cooperation? honesty? - neuroscience seems better for interp work vs HD

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 21

hello! like many ppl at oai, i care deeply about ai alignment going well. to that end, i’m deep diving on human data x alignment evals and am looking for - ppl to talk to - papers to read - ppl who are building alignment datasets some questions i’m noodling over

Alexandra Barr avatar
Alexandra Barr@BarrAlexandra·Oct 11

my friend who changed jobs this yr said his hit rate on hinge increased 10x when he changed his profile from researcher at gdm to researcher at anthropic dm if you want his profile

Manuka Stratta avatar
⟳ Alexandra Barr retweeted
Manuka Stratta@ManukaStratta·Oct 8

So excited to be launching GPT-6 today with Intelligent UI to everyone in ChatGPT! The model can now create custom visual and interactive responses on the fly, as quickly as it generates text. 🎨 ChatGPT launched in 2022 as a text-only low-key research preview. While the product has evolved a lot since then, responses have largely remained text-based. With Intelligent UI, we wanted to rethink the interface from the ground up and give our models more ways to communicate visually. Getting to shape the research direction behind that vision has been especially rewarding. We built a library of native, streamable components and a compiler that renders interfaces progressively as the model generates them. This combines a familiar design foundation with the flexibility to compose each response dynamically. We trained GPT-6 to use this library to create clear, useful interfaces and make thoughtful design decisions—including when interactivity helps and when simple text is enough. Making this work meant thinking about model behavior, product design, and engineering scaling requirements together. It’s been awesome working through these challenges with folks across research, engineering, design, and product over the last few months. I’m a visual learner, so this is a game changer for me. Some of my favorite uses so far have been planning trips, practicing my Italian, learning new concepts, and shopping. Give it a try and let us know what you think! Rolling out starting today to Plus, and expanding tomorrow to Free and Go. Learn more at: openai.com/index/gpt-6-for-e…

GPT-6 and Intelligent UI, now rolling out in ChatGPT for everyone. Intelligent UI in ChatGPT delivers fast, interactive answers that make everyday questions more visual, complex topics easier to grasp, and interactive tools for your task available on the spot.

1:10