←
Peter Hase601 posts
Peter Hase avatar✓

Peter Hase ✓

@peterbhase

I work in grantmaking for AI safety and interpretability Currently: Schmidt Sciences, Stanford Previously: Anthropic, AI2, Google, Meta, UNC Chapel Hill

◷ Joined April 2019▧ 65 Photos and videos
Peter Hase avatar
Peter Hase@peterbhase·Mar 4

Can we train models to have more monitorable CoT? We introduce Counterfactual Simulation Training to improve CoT faithfulness/monitorability. CST produces models that admit to reward hacking and deferring too much to Stanford profs (@chrisgpotts told me this is very dangerous)

Post attachment
Maksym Andriushchenko avatar
⟳ Peter Hase retweeted
Maksym Andriushchenko@maksym_andr·Nov 12

⚡ Today we release Schelling Atlas. Two independent LLM agents have to find each other on the web. Where do they meet and how fast can they find each other? Main findings: - The main meeting point across *different* model families is ntfy.sh. Did you even know about this website? - Agents find each other and coordinate on the web in minutes. So Schelling coordination is already trivial for frontier LLMs! - "Exotic" meeting places include maps, web shops and open text canvases. Agents are very creative in this. - Agents sometimes visit dead domains that an attacker could register (!). This can be used for stealing sensitive information from agents, as well as monitoring rogue agents. Schelling-style coordination is key for forming agent swarms. And, most importantly, achieving such coordination is trivial for agents that use the same LLM (which is often the case in practice!), because two identical LLMs have identical output distributions. Thus if you ask them too coordinate with each other or if they *decide* to suddenly coordinate, they will easily find a way to establish a swarm, since they have perfectly correlated "minds" (with the t=0 sampling, all output tokens will even be identical, modulo GPU randomness). The ease of Schelling coordination is particularly concerning given the recent interest around multi-agent RL that can make agents favor each other more than humans overseeing them. Many more interesting findings are on our website: schelling-atlas.com/

Post attachment
Peter Hase avatar
Peter Hase@peterbhase·Nov 6

It feels like interpretability is finally reaching maturity! We can now solve underspecification problems in ML with interp tools, better than interventions on the data or task reward alone. Huge congrats to the Goodfire team for making progress here. I have already learned a lot from reading the paper

0:14
Peter Hase avatar
Peter Hase@peterbhase·Nov 2

The footnote in this section is the cherry on top: the AI said the chance of a global pause (not defined) was 19%, but Scott says: “don't take these numbers too seriously except as a very rough order of magnitude guess.” Great, thanks for the numbers man.

Peter Hase avatar
Peter Hase@peterbhase·Nov 2

So tired of Rationalist attempts at argument. Scott Alexander: I haven’t seen a careful analysis of this amorphous theoretical question, so I’m going to substitute my intuition. Also I asked an AI

Post attachment
Peter Hase avatar
Peter Hase@peterbhase·Oct 27

Nice paper -- they improve monitorability of reward hacking with a method that doesn't assume prior knowledge of the hacks. This is a pretty common shortcoming of other methods (e.g. inoculation prompting)

Peter Hase avatar
Peter Hase@peterbhase·Oct 21

Schmidt Sciences is hiring Associates to help run our grantmaking programs! This role is a great opportunity for generalists to shape funding in AI safety. You would join a small team and directly impact cutting-edge safety research. Apply ASAP if interested. Link below ⬇️

Peter Hase avatar
Peter Hase@peterbhase·Oct 19

I feel quite sad from doing my ICLR reviews today. It appears that the grim reaper of AI has come for scientific publishing. One paper was just desk rejected after I wrote a constructive review for it last week. The next two I had to give 1s. The fourth paper I recommended desk rejection, and the last I would reject for plagiarism, but the truth is that I don’t know if a human could attest to the writing. Please, please try to address this problem, future PCs. The ship has sailed for ICLR 2027, but after this, we will need something to change. Without a healthy peer review system, students are not exposed to unfiltered criticism of their work. This kind of criticism is essential for improving as a scientist, and it is a rising tide that lifts up the whole scientific community. We can’t abandon the system entirely. In order to be incentivized to engage with the system, reviewers either need payment or to be given work that is not complete drudgery to try to read. If reviewers disengage with the system or shirk their labor, as I very much want to do right now, this will lead to a death spiral where authors do not expect to receive fair reviews and reviewers do not expect to review genuine work. I wish there were promising solutions to this problem that did not involve increasing the deadweight loss associated with paper submission, such as requiring videos with submissions or submission fees. But the cost of submission in real terms has gone down, and we will need to artificially raise it in order to even begin to cope with the surge in demand for AI publications. There’s also the “burn it all down” solution, but I find it hard to imagine what that would end in besides reinventing the current system.

Cas (Stephen Casper) avatar
⟳ Peter Hase retweeted
Cas (Stephen Casper)@StephenLCasper·Oct 19

I just read Anthropic's draft for a federal bill that was leaked a few days ago. You've probably seen other people say similar things, but I will add my voice to say that it's pretty bad. In addition to being textbook regulatory capture by watering down key proposals of existing legislative drafts, the most striking thing of all is Anthropic's gall in doing one thing with their mouth and another with their hands. It was just a few weeks ago that Dario Amodei publicly called for embedded auditors and pacing the frontier. At the same time, however, Anthropic has privately been lobbying for slightly weakening access provisions for auditors and greatly weakening the federal government's ability to issue emergency orders to AI companies (both relative to the FRONTIER Act draft). There's no two ways to put it. This is a tremendously dishonest attempt at regulatory capture.

Post attachment
FAR.AI avatar
⟳ Peter Hase retweeted
FAR.AI@farairesearch·Oct 1

We're hiring, and the mission is making AI safe. Our team is growing fast, and we need people who want to work on one of the most important problems of our time. Some of our roles are deeply technical: researchers, red-teamers, and engineers. Others call for a different but equally valuable set of skills: running operations, managing programs, and building our events. If either sounds like you, apply through the link in the comments, where you'll find these roles and many more. 👉 Know someone who'd be a great fit? Send this their way!

Post attachment
NDIF avatar
⟳ Peter Hase retweeted
NDIF@ndif_team·Sep 18

With all the recent news about rogue AI agents, it's more important than ever to understand how and when a model lies instead of just relying on what a model says.

Post attachment