←
Peter Nutter26 posts
Peter Nutter avatar

Peter Nutter

@PeterNutter1

MATS 10.0 / ETH Zürich

◷ Joined May 2014▧ 3 Photos and videos
Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

Also LongCat 2.0 and 2.5 from Meituan do the same thing. Same OAI header and a made up body. I’m still looking into this Fwiw Meituan's VC arm Long-Z led Moonshot's ~$2B round in May

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

After Anthropic cracked down it looks like OAI will now get a big slice of the distillation revenue Curious what happens if OAI closes it too. Does the teacher ceiling start to hurt??

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

Also interesting for @natolambert’s distillation takes: Kimi’s personality had a large shift from Claude like to GPT like. K3 was closest to GLM 5.3 and Claude Fable 5.1, K3.1 Pro is closest to GPT-6.1 Sol and Astra.

Post attachment
Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

PSA if you run K3.1 Pro in a Codex style multi agent setup rename the message argument of send_message (eg to report_text). That stops it

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

Either way @Kimi_Moonshot this should have been easy to catch lol. A high entropy filter flags a 260 char base64 string instantly and gAAAAAB is basically a canary

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

Timestamps Kimi writes cluster in late July. Right when OAI says the campaign ran (Jul 1 to 28) Simplest alternative is it was scraped from public data. But public Codex logs barely cover that week. Jul 25 to 28 is 0.6% of 12k public blobs vs 21% of Kimi's

Post attachment
Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

It's a Fernet token which is how OAI encrypts Codex agent messages. Version byte then timestamp then ciphertext Codex only started encrypting these on Jun 5. Now it's optional Kimi writes gAAAAAB w over 90% confidence but after the header its logprobs are flat over base64

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

Found this by accident running multi agent evals w Kimi in Codex. In 25 of 160 episodes it sent a random looking string instead of its report and got confused I knew the format bc I had to switch off Codex message encryption earlier to log my own evals

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

My best guess is this is linked to the distillation OAI reported in Sep and tied to Moonshot. OAI says those accounts replayed encrypted reasoning to get the model to decrypt it That's basically the Stolen Thoughts attack by @kotekjedi_ml @maksym_andr Let's look at it 🧵

Peter Nutter avatar
Peter Nutter@PeterNutter1·Nov 4

Kimi K3.1 Pro doesn't just recall hidden tests (great thread by @JSchaeff3r). It also writes fake OpenAI ciphertext 🤯 Give it a send_message tool in Codex and sometimes it sends ciphertext instead of its report. Then it gets confused by what it just sent 🤔

Post attachment

Kimi K3.1 recalls instructions it was never given on Terminal Bench. They match the hidden tests. K3.1 while solving: "Instructions likely distilled pitfalls from previous agents." 🤯 When we edit a task to contradict them, K3.1 refuses the new instructions and fails. Other models follow the edit. Is it terminal(ly) benchmaxxed? 🧵

Post attachment
Peter Nutter avatar
Peter Nutter@PeterNutter1·Oct 6

Spent a lot of MATS 10.0 doing SDF for "teaching Claude why" style stuff and stumbled onto a really cool trick for updating model beliefs with SDF. Models stay way more coherent, and you only have to train once for all your post-trained checkpoints. Check it out!!

New research led by @PeterNutter1 @DJROYTBURG via @MATSprogram: a better way to update model beliefs that we call grafting. Fine-tune the base model with next-token prediction on synthetic documents (SDF), and simply apply the weight update onto the post-trained model. This works! 1/7

Post attachment
Peter Nutter avatar
Peter Nutter@PeterNutter1·Jun 28

Excited to share my first paper, and it's an ICML 2026 oral! We wrote up the main takeaways in a blog post (below). If you'll be at ICML, find us at: Oral: Tue Jul 7, 1:30 PM KST, Hall D2 Poster: Wed Jul 8, 10:30 AM KST, Hall A #3213 See you there! lesswrong.com/posts/bJcR3yP2…

Excited that our ICML position paper has been accepted as an Oral 🎉! When a model looks like it "deceives" or "resists shutdown," how do we know it isn't role-play, instruction-following, or just task-completion pressure? Often the current evidence can't yet tell them apart, and those claims are starting to inform deployment and regulation. We map where the evidence gets thin across four stages and propose a shared standard to strengthen it. Here's the map. 🧵 arxiv.org/abs/2606.07612

Post attachment
David Stránský avatar
⟳ Peter Nutter retweeted
David Stránský@yourlocalbyatch·2 Oct 2020

Dospělost je moment kdy si se zlomeným srdcem uvědomíte, že rutina je jediný způsob jak se ze světa nezbláznit.