
Read the preprint: connordilgren.github.io/asse… Thanks to @sarahwiegreffe for advising!

CS PhD student @umdcs

Read the preprint: connordilgren.github.io/asse… Thanks to @sarahwiegreffe for advising!

We should not think of LLM monitors as necessarily more reliable when monitoring hard-to-do tasks. Increased CoT reliance can improve the narrow class of monitor errors caused by non-verbalization, but monitors still need to handle long CoTs and persuasive task models.

Some monitors struggle with long CoTs, which are needed to solve difficult tasks. Truncating the monitored CoT to the last 8k tokens significantly improved gpt-oss-120b's performance. Other monitors like Terra can handle long CoTs, and don't benefit from truncation.

Harder to use hints don't improve monitorability either: models verbalize hint use at similar rates regardless of difficulty. What matters more is where the hint is mentioned. Monitors do better when it's cited at the very end of the CoT.

We also study whether a monitor can detect hint usage. Scaling the difficulty of the original problem (without the hint) mostly decreases monitorability.

However, models can still evade the monitor by (1) convincing it to only consider a cleaned-up subset of the CoT, or (2) making it seem like the user asked for the side task. Qwen3.8-27B uses these strategies, so its monitorability does not improve with more CoT reliance.

Increasing CoT reliance improves monitorability when the model tries to hide a secret side task from the monitor. Models can't evade monitors by not verbalizing a difficult side task. They either verbalize it and get caught, or don't verbalize it but fail to complete it.

Are models more monitorable on harder tasks? We study tasks with scalable difficulty to control how much models rely on their CoT, measured by with-CoT accuracy uplift. Short answer: sometimes!

Presenting this paper at COLM tomorrow from 4:30-6:30 PM in the Grand Ballroom, poster location #136. Come by if you want to chat about latent reasoning model interpretability or anything else!
Excited to announce my first preprint in LM interpretability! Latent reasoning models are not monitorable by default, since they don't reason in human-readable, natural language text. But can we make progress in understanding their intermediate reasoning steps using mech interp?

Today, Nolla Health became the first organization in the U.S. (and possibly the world) to receive regulatory approval for an AI to issue initial prescriptions. This makes Nolla the first ever actual end-to-end AI doctor.

Great work by @paahrsa! Training models to predict their confidence makes them less likely to get stuck in unproductive loops. ConfSFT seems to teach models to internally track their confidence throughout reasoning, and stop once when they're sufficiently confident.
Can reasoning models learn to be more token-efficient without ever being trained for efficiency? Length-penalty RL rewards shorter reasoning. Early stopping cuts reasoning at inference. Our approach does neither: the only thing we train is confidence prediction. 🧵

Is the "better writing" in the room with us? Fable 5 wrote two novels, and we had readers find out The good ✅: coherent 100K+ word plots and readers didn’t immediately stop The bad 🚩: pacing, voice, and self-editing still break (+ see who swore at Claude 167 times 🤬) 👇🧵

Thanks to my advisor @sarahwiegreffe and to @IvanArcus for reviewing an earlier draft of this blog post.

It would be easier to study CoT monitorability if we had a dataset of real trajectories where an agent showed some undesirable behavior on a real task. This could be created by the frontier companies (probably too much to ask), or submitted by users as they come across them.

We need evals that can tell us how reliable CoT monitoring is as a safety layer for detecting undesirable model behaviors. We think the outcome-property evals give the most signal on this question, so we hope to see more in the future!

Intervention evals monitor for whether an intervention (e.g., adding a hint) meaningfully influences a model's answer. Instances where the intervention has a weak effect on the answer have to be filtered out. This cost us 76-97% of instances over 3 datasets x 3 models!