←
Connor Dilgren32 posts
Connor Dilgren avatar

Connor Dilgren

@ConnorDilgren

CS PhD student @umdcs

◷ Joined February 2014▧ 12 Photos and videos
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

We should not think of LLM monitors as necessarily more reliable when monitoring hard-to-do tasks. Increased CoT reliance can improve the narrow class of monitor errors caused by non-verbalization, but monitors still need to handle long CoTs and persuasive task models.

Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

Some monitors struggle with long CoTs, which are needed to solve difficult tasks. Truncating the monitored CoT to the last 8k tokens significantly improved gpt-oss-120b's performance. Other monitors like Terra can handle long CoTs, and don't benefit from truncation.

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

Harder to use hints don't improve monitorability either: models verbalize hint use at similar rates regardless of difficulty. What matters more is where the hint is mentioned. Monitors do better when it's cited at the very end of the CoT.

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

We also study whether a monitor can detect hint usage. Scaling the difficulty of the original problem (without the hint) mostly decreases monitorability.

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

However, models can still evade the monitor by (1) convincing it to only consider a cleaned-up subset of the CoT, or (2) making it seem like the user asked for the side task. Qwen3.8-27B uses these strategies, so its monitorability does not improve with more CoT reliance.

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

Increasing CoT reliance improves monitorability when the model tries to hide a secret side task from the monitor. Models can't evade monitors by not verbalizing a difficult side task. They either verbalize it and get caught, or don't verbalize it but fail to complete it.

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 26

Are models more monitorable on harder tasks? We study tasks with scalable difficulty to control how much models rely on their CoT, measured by with-CoT accuracy uplift. Short answer: sometimes!

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Oct 7

Presenting this paper at COLM tomorrow from 4:30-6:30 PM in the Grand Ballroom, poster location #136. Come by if you want to chat about latent reasoning model interpretability or anything else!

Excited to announce my first preprint in LM interpretability! Latent reasoning models are not monitorable by default, since they don't reason in human-readable, natural language text. But can we make progress in understanding their intermediate reasoning steps using mech interp?

Post attachment
Luis Wenus avatar
⟳ Connor Dilgren retweeted
Luis Wenus@luiswenus·Oct 5

Today, Nolla Health became the first organization in the U.S. (and possibly the world) to receive regulatory approval for an AI to issue initial prescriptions. This makes Nolla the first ever actual end-to-end AI doctor.

2:06
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Sep 28

Great work by @paahrsa! Training models to predict their confidence makes them less likely to get stuck in unproductive loops. ConfSFT seems to teach models to internally track their confidence throughout reasoning, and stop once when they're sufficiently confident.

Can reasoning models learn to be more token-efficient without ever being trained for efficiency? Length-penalty RL rewards shorter reasoning. Early stopping cuts reasoning at inference. Our approach does neither: the only thing we train is confidence prediction. 🧵

Post attachment
Atrey Desai avatar
⟳ Connor Dilgren retweeted
Atrey Desai@atreydesai·Aug 19

Is the "better writing" in the room with us? Fable 5 wrote two novels, and we had readers find out The good ✅: coherent 100K+ word plots and readers didn’t immediately stop The bad 🚩: pacing, voice, and self-editing still break (+ see who swore at Claude 167 times 🤬) 👇🧵

Post attachment
Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Aug 18

It would be easier to study CoT monitorability if we had a dataset of real trajectories where an agent showed some undesirable behavior on a real task. This could be created by the frontier companies (probably too much to ask), or submitted by users as they come across them.

Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Aug 18

We need evals that can tell us how reliable CoT monitoring is as a safety layer for detecting undesirable model behaviors. We think the outcome-property evals give the most signal on this question, so we hope to see more in the future!

Connor Dilgren avatar
Connor Dilgren@ConnorDilgren·Aug 18

Intervention evals monitor for whether an intervention (e.g., adding a hint) meaningfully influences a model's answer. Instances where the intervention has a weak effect on the answer have to be filtered out. This cost us 76-97% of instances over 3 datasets x 3 models!