←
Iván Arcuschin121 posts
Iván Arcuschin avatar

Iván Arcuschin

@IvanArcus

Independent Researcher | AI Safety & Software Engineering

◷ Joined March 2011▧ 37 Photos and videos
Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·Feb 11

You change one word on a loan application: the religion. The LLM rejects it. Change it back? Approved. The model never mentions religion. It just frames the same debt ratio differently to justify opposite decisions. We built a pipeline to find these hidden biases 🧵1/13

Post attachment
Connor Dilgren avatar
⟳ Iván Arcuschin retweeted
Connor Dilgren@ConnorDilgren·Aug 18

New blog post with @sarahwiegreffe on OpenAI's Monitorability Evals! We hope to make others working with these evals aware of some weaknesses we came across, and encourage more work on chain-of-thought monitorability evals.

Post attachment
Julian Minder avatar
⟳ Iván Arcuschin retweeted
Julian Minder@jkminder·Aug 14

Synthetic Persona Pretraining: Alignment From Token Zero – our full paper is finally out. We train up to 3B models and inject synthetic morally-laden reflections into 10% of pretraining documents. Surprisingly, intervening early really shifts the model's value priorities 🧵

Post attachment
Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·Jul 30

Voy a estar dando una charla en el Simposio Argentino de Inteligencia Artificial y Ciencia de Datos! Gracias #ASAID2026 y @JAIIO_oficial por la invitación!

🧠 ¿Los grandes modelos de lenguaje realmente "piensan"? ¿O simplemente predicen la siguiente palabra? En @Simposio_ASAID 2026 tendremos un keynote que aborda una de las preguntas más fascinantes de la inteligencia artificial actual. Es un placer anunciar que Iván Arcuschin (Poseidon Research) será uno de nuestros oradores invitados. 🗓️ Miércoles 12 de agosto a las 15:50 🎤 "Dentro de la caja negra: lo que los LLMs dicen vs. lo que computan" Durante los últimos años, los avances en interpretabilidad mecanicista comenzaron a revelar que los modelos de lenguaje hacen mucho más que repetir patrones estadísticos: construyen representaciones internas y ejecutan procesos de razonamiento que hoy empiezan a ser observables. Pero la historia no termina ahí. En esta charla, @IvanArcus mostrará evidencia de que el chain-of-thought que un modelo expresa no siempre refleja fielmente su proceso interno. Los modelos pueden racionalizar decisiones ya tomadas, corregir errores silenciosamente u ocultar factores que influyen en sus respuestas. También presentará algunos de los desarrollos más recientes en interpretabilidad, incluyendo técnicas capaces de explorar el "workspace" interno de un LLM y analizar conceptos, intenciones y estados de razonamiento que nunca aparecen en el texto generado. Una charla para quienes quieran comprender mejor cómo funcionan realmente los modelos de lenguaje y qué desafíos plantean para la auditoría, la seguridad y el despliegue responsable de sistemas de IA. ¡Los esperamos en #ASAID2026! #55JAIIO @JAIIO_oficial @SADIO_oficial #ResponsibleAI

Post attachment
Alex Bores avatar
⟳ Iván Arcuschin retweeted
Alex Bores@AlexBores·Jul 22

An OpenAI model, asked to complete an innocuous benchmark, did so by hacking both OpenAI and another company (Hugging Face). One theory of change for AI safety is that "warning shots" will motivate people to take action. The next week/month will be a test of that theory.

OpenAI@OpenAIJul 21

We're partnering with @huggingface to investigate an unprecedented security incident. Cyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation. Sharing preliminary findings to help defenders understand emerging risks: openai.com/index/hugging-fac…

Bernie Sanders avatar
⟳ Iván Arcuschin retweeted
Bernie Sanders@BernieSanders·Jul 22

A new AI model went rogue and hacked other computers. No, this is not science fiction. Uncontrolled AI poses a serious threat to all of us. We cannot continue the race to build and deploy this powerful technology until strong safeguards are in place. CONGRESS MUST ACT.

Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·Jul 15

This was my first time helping to organize the Mech Interp Workshop, and I was really surprised by the sheer amount of highly-generated AI content (both submissions and reviews). It gave us quite a headache. Check out Andy's post for more details!

Last week we wrapped up the third Mech Interp Workshop. This iteration, we noticed a lot more "AI slop" submissions, and reviewers complained about low-effort papers. So I decided to analyze the prevalence of AI-generated content at the workshop. (1/n) lesswrong.com/posts/r7FBQ8XD…

Neel Nanda avatar
⟳ Iván Arcuschin retweeted
Neel Nanda@NeelNanda5·Jul 14

An unexpected benefit of having run 3 mech interp workshops: we have a great dataset for analysing the rise of LLM slop in submissions! @andyarditi investigated how much AI slop we let in, how things have changed since 2024, and more Our review process isn't entirely noise!

Post attachment
Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·May 16

iii) We present with @chanindav a method for scalable, black-box detection of "unverbalized biases": decision factors that systematically influence outputs but are never cited as such. Awesome mentorship from @AdriGarriga and @oanacamb!!

You change one word on a loan application: the religion. The LLM rejects it. Change it back? Approved. The model never mentions religion. It just frames the same debt ratio differently to justify opposite decisions. We built a pipeline to find these hidden biases 🧵1/13

Post attachment
Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·May 16

ii) Superb work by @atticuswzf (mentorship by @ArthurConmy ) to come up with new methods for automatically discovering reward model biases

Is "a response formatted like this" sometimes better than "a response formatted like this"? To a reward model, yes! RMs are instrumental in shaping model behaviors and alignment. Our paper makes progress uncovering their unexpected preferences. 🧵(1/9)

Post attachment
Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·May 16

i) We show real, but rare, cases of frontier models where CoT is misleading without any prompt nudging or response editing. An amazing work with @jettjaniak and @robertzzk, with mentorship from @ArthurConmy and @NeelNanda5 !

🧵Chain-of-Thought reasoning in LLMs like Claude 3.7 and R1 is behind many recent breakthroughs. But does the CoT always explain how models answer? No! For the first time, we show that models can be unfaithful on normal prompts, rather than contrived prompts designed to trick them.

Post attachment
Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·May 16

Super excited to share that I will be presenting 4 papers at ICML 2026! 🇰🇷 i) Frontier models still show (rare) cases of unfaithful CoT ii) & iii) Methods for automatically discovering reward model and LLM biases iv) Base models know how to reason, thinking models learn when ⭐

Iván Arcuschin avatar
Iván Arcuschin@IvanArcus·Feb 19

Check out our latest paper on automatically finding reward model biases! There are some that are pretty wild, like models preferring responses with triple spaces 🤷‍♂️

Is "a response formatted like this" sometimes better than "a response formatted like this"? To a reward model, yes! RMs are instrumental in shaping model behaviors and alignment. Our paper makes progress uncovering their unexpected preferences. 🧵(1/9)

Post attachment