←
Apollo Research451 posts
Apollo Research avatar✓

Apollo Research ✓

@ApolloResearch

Our goal is to secure frontier AI systems from development, to deployment and governance.

◷ Joined May 2023▧ 173 Photos and videos
Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

We're launching a pilot towards embedded evaluations with @OpenAI. Six evaluators from Apollo Research will work within OpenAI for four months, investigating whether models attempt to undermine their own oversight and control.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

Oh and we forgot to mention this in the thread: **The pilot is starting TODAY**

OpenAI avatar
⟳ Apollo Research retweeted
OpenAI@OpenAI·Nov 16

We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards. These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems. openai.com/index/our-first-p…

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

We commend OpenAI for opening its development process to independent scrutiny. We expect to learn a lot about what works and what needs to change, and use those lessons to make future evaluations even more ambitious.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

We see meta-transparency as a minimum standard for all evaluation engagements. Without it, a lack of reported problems could look reassuring even when it simply reflects limits on what an evaluator could investigate or disclose.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

Meta-transparency: OpenAI agreed to strong rights for us to publicly explain the access we requested and received, what was withheld or delayed and why, redactions and their impact on our conclusions, and any pressure we believe shaped our work.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

We cannot publish information based on employee discussions unless OpenAI provides formal attestation, or we can independently establish the finding. If OpenAI declines an attestation, we can say so and explain how that limits our conclusions.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

OpenAI may request redactions of confidential information, including evidence behind our conclusions, but not simply because a finding is unfavorable. We can explain how redactions affect our conclusions and readers’ ability to assess them.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

Disclosure: We will publish two reports: one on training incentives, and one on evaluations, internal deployment, and detectability. Each will set out our methods, conclusions and limitations, including questions we couldn’t assess.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

Apollo Research may discuss confidential OpenAI information only with authorized staff, selected by OpenAI. We don’t get access to the general communication channels (e.g. Slack) an OpenAI safety researcher would. Formal conversations with approved staff must be logged.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

OpenAI can withhold in-scope items where it deems it reasonably necessary for technical, security, or IP reasons, but not simply because they’re unfavorable. It must tell us withheld items exist and give written reasons.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

Access: For the training assessment, OpenAI has committed to providing relevant code, SFT data, RL rollouts including reasoning traces and rewards, and intermediate model checkpoints we can query. Access details for the other assessments remain to be agreed.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

This pilot is an ambitious first step towards the goal of obtaining those strong guarantees, but its access and disclosure rights don't yet meet our bar for full embedded evaluations.

Apollo Research avatar
Apollo Research@ApolloResearch·Nov 16

Embedded evaluations must be credible independently of trust in the developer. This requires strong guarantees for access, disclosure, and meta-transparency.