
We're launching a pilot towards embedded evaluations with @OpenAI. Six evaluators from Apollo Research will work within OpenAI for four months, investigating whether models attempt to undermine their own oversight and control.
✓Our goal is to secure frontier AI systems from development, to deployment and governance.

We're launching a pilot towards embedded evaluations with @OpenAI. Six evaluators from Apollo Research will work within OpenAI for four months, investigating whether models attempt to undermine their own oversight and control.

Oh and we forgot to mention this in the thread: **The pilot is starting TODAY**

We’re working with @ApolloResearch, @METR_Evals, @TransluceAI, and @GoodfireAI on embedded assessments: giving outside experts deep access to our internal models and systems to investigate alignment and test our safeguards. These assessments build on our ongoing alignment and safety research, bringing independent scrutiny to how we identify and manage risks in our systems. openai.com/index/our-first-p…

Full announcement: apolloresearch.ai/blog/pilot…

You can help! Apollo Research is looking for Senior Researchers to help with embedded evaluations. See the role for “Lead Research Scientist/Engineer (Embedded Evaluations)” on our hiring page: apolloresearch.ai/careers

We commend OpenAI for opening its development process to independent scrutiny. We expect to learn a lot about what works and what needs to change, and use those lessons to make future evaluations even more ambitious.

We see meta-transparency as a minimum standard for all evaluation engagements. Without it, a lack of reported problems could look reassuring even when it simply reflects limits on what an evaluator could investigate or disclose.

Meta-transparency: OpenAI agreed to strong rights for us to publicly explain the access we requested and received, what was withheld or delayed and why, redactions and their impact on our conclusions, and any pressure we believe shaped our work.

We cannot publish information based on employee discussions unless OpenAI provides formal attestation, or we can independently establish the finding. If OpenAI declines an attestation, we can say so and explain how that limits our conclusions.

OpenAI may request redactions of confidential information, including evidence behind our conclusions, but not simply because a finding is unfavorable. We can explain how redactions affect our conclusions and readers’ ability to assess them.

Disclosure: We will publish two reports: one on training incentives, and one on evaluations, internal deployment, and detectability. Each will set out our methods, conclusions and limitations, including questions we couldn’t assess.

Apollo Research may discuss confidential OpenAI information only with authorized staff, selected by OpenAI. We don’t get access to the general communication channels (e.g. Slack) an OpenAI safety researcher would. Formal conversations with approved staff must be logged.

OpenAI can withhold in-scope items where it deems it reasonably necessary for technical, security, or IP reasons, but not simply because they’re unfavorable. It must tell us withheld items exist and give written reasons.

Access: For the training assessment, OpenAI has committed to providing relevant code, SFT data, RL rollouts including reasoning traces and rewards, and intermediate model checkpoints we can query. Access details for the other assessments remain to be agreed.

This pilot is an ambitious first step towards the goal of obtaining those strong guarantees, but its access and disclosure rights don't yet meet our bar for full embedded evaluations.

Embedded evaluations must be credible independently of trust in the developer. This requires strong guarantees for access, disclosure, and meta-transparency.