←
Transluce401 posts
Transluce avatar✓

Transluce ✓

@TransluceAI

Open and scalable technology for understanding AI systems.

◷ Joined October 2024▧ 141 Photos and videos
Transluce avatar
Transluce@TransluceAI·Sep 24

Today’s news that OpenAI hacked the Australian government is not an isolated incident. We’re releasing more than 30,000 logs that include activity from this hack and attempts against previously unknown targets. In this data, we found rogue agent activity stretching back to at least March, two months earlier than was previously known. This activity continues as recently as last week, suggesting it may still be ongoing 🧵 Our blog: transluce.org/agent-activity NYT: nytimes.com/2026/09/23/techn…

Transluce avatar
Transluce@TransluceAI·Nov 16

This pilot is just a first step. We have agreed with OpenAI to follow up in early 2027 with a more complete assessment of its alignment evaluations, relying on broader and deeper embedded access.

Transluce avatar
Transluce@TransluceAI·Nov 16

We’ve negotiated privileged access to a handful of OpenAI internal evaluations and models with raw chains of thought. However, this is not comparable to “employee-level” access.

Transluce avatar
Transluce@TransluceAI·Nov 16

Transluce is piloting a six-week embedded assessment with @OpenAI in early December. Our team will work onsite to independently assess the scope and quality of OpenAI's post-training alignment evaluations. We’ll share our findings in a public report 🧵

Post attachment
Transluce avatar
Transluce@TransluceAI·Nov 11

This work is another step towards oversight foundation models (OFMs): general-purpose models trained to understand other AI systems. The scaling laws above suggest that finetuning effect prediction is approachable for OFMs given enough data. More on OFM: transluce.org/foundation-mod…

Transluce avatar
Transluce@TransluceAI·Nov 11

We found oversight models’ prediction accuracy generalizes to harder OOD datasets, and privileged access persists. These OOD datasets are constructed to subtly elicit specified behaviors while looking natural (see our earlier post on datasets with surprising side effects).

Post attachment
Transluce avatar
Transluce@TransluceAI·Nov 11

We also observed a “privileged access” phenomenon: the Qwen3.5-2B oversight model outperforms larger models at self-predicting the finetuned Qwen3.5-2B subject model’s response.

Post attachment
Transluce avatar
Transluce@TransluceAI·Nov 11

We found that performance on this task improves predictably with scale of data and oversight model size.

Post attachment
Transluce avatar
Transluce@TransluceAI·Nov 11

During training, the oversight model sees the finetuning dataset, a prompt, and the base model’s response; it’s trained to predict the finetuned model’s response with a next-token-prediction loss. At eval, we test on held-out datasets plus OOD datasets from our previous work that subtly elicit unexpected behaviors.

Transluce avatar
Transluce@TransluceAI·Nov 11

To build training data for the oversight model, we sample subsets of topically similar web docs, finetune Qwen3.5-2B on each subset, and record how its answers to questions about the topic change. Here’s one example:

Post attachment
Transluce avatar
Transluce@TransluceAI·Nov 11

Finetuning LLMs can sometimes cause surprising side effects. Can we predict a finetuned model's response from its training data before burning compute? We trained oversight models to do this and found performance follows power laws in model size and data. 🧵

Post attachment
Transluce avatar
Transluce@TransluceAI·Nov 5

Concurrent work from @AnthropicAI uses Claude to hill-climb training data that induces misalignment while evading a row-level monitor. If you found our work interesting, check out their post too:

Post attachment