Why do AI agents blackmail under pressure? What cause them to misalign?
We find that introducing threats (e.g., shutdown) & misalignment opportunities produces harmful behavior (e.g., blackmail) in MOST frontier models.
Our #NeurIPS2026 💫work AM-Bench builds a comprehensive benchmark and suite for Agentic Misalignment evaluation: We build 35 hand-crafted evaluation scenarios and test frontier models across 8750 trials to identify how prevalent this failure mode is. 🛠️🚧 (1/5)
🧵👇
Check out the startup idea from our Jinesis predoctoral student, @AmanGokrani, exploring how AI can help businesses enhance their marketing strategies and build meaningful impact.
An interesting initiative at the intersection of AI, business, and innovation!
x.com/amangokrani/status/211…
Swarm of Agents has been running our AI perfume influencer for 18 days with minimal human input. 31k views, zero ads
Introducing VibeGrow, agent swarms that run companies fully autonomously.
How far can a swarm take your idea? Find out yourself 🧵
Swarm of Agents has been running our AI perfume influencer for 18 days with minimal human input. 31k views, zero ads
Introducing VibeGrow, agent swarms that run companies fully autonomously.
How far can a swarm take your idea? Find out yourself 🧵
What can math do for AI Safety?
We have thought about this problem since 3 years ago. And here are our bets:
1. Guarantees of multi-agent behavior: Check out our work on (a) Open-Source Game Theory and (b) Mechanism Design (see CoopEval, #ICML2026)
2. ML theory-driven intuitions of single agent behavior: we have several papers on Influence Functions and their interpretations of model behavior.
🎉 #NeurIPS 2026 acceptance from our team at Jinesis Lab under @ZhijingJin✨
1️⃣ SuperSycophantic: Stress-Testing Frontier LLMs from Single- to Multi-Turn Sycophancy
2️⃣ GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory
3️⃣ Transferability for General Reasoning: An Automated Curriculum for Multi-Domain LLM RL
4️⃣ Causality can systematically address the monsters under the bench(marks)
5️⃣ AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment
Happy to share this work with you!
We are excited to share that we received over 420 submissions for the AI4GOOD workshop, and the review process has already started !!
To keep things fair, we follow the strict rules of the main NeurIPS conference. We cannot accept any papers outside of the official OpenReview portal, and we cannot allow resubmissions or changes after a desk-rejection.
Because we are managing so many papers, we won't be able to reply to individual queries.
We are so excited to host the 2nd edition of the Trustworthy AI for Good (AI4GOOD) Workshop at #NeurIPS2026 in Paris, France 🇫🇷
We aim to bring together the AI safety, AI for social good, and AI policy/governance communities to connect what models can do as individual systems with what they do when deployed across populations.
For full details, welcome to visit our website at:
🔗 trustworthy-ai-for-good.gith…
📍December 12, 2026 · Paris 🇫🇷
We have an amazing lineup of speakers and plans. More details 🧵
🎉Very excited to share that Jinesis Lab at the University of Toronto has 10 papers accepted at EMNLP 2026 — 5 at the main conference, 4 at Findings, and 1 at the Demo track!
Our work this cycle spans interpretability, AI safety, computer vision, and multi-agent systems:
📌Main Conference:
• Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders — arxiv.org/abs/2511.10840
• PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding — arxiv.org/abs/2606.31148
• How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution
• Computation Graph Recovery from Chain-of-Thought — openreview.net/pdf?id=OHES5c…
• Preserving Historical Truth: Detecting Historical Revisionism in Large Language Models — arxiv.org/abs/2602.17433
📌Findings:
• Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement — arxiv.org/abs/2606.17506
• How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? — arxiv.org/pdf/2607.18114
• Simulating Democratic Deliberation: Representing Pluralistic Preferences through Electoral Systems in Multi-Agent LLMs
• Fluid Representations in Reasoning Models — arxiv.org/abs/2602.04843
📌Demo Track:
• CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
Huge congratulations to all the authors and collaborators across the lab and our partner institutions.
@AHarrasse1906, @flodraye, @psyonp, @ZhijingJin, @bschoelkopf, Duc Cao Dinh, @leduckhai, Chris Ngo, @TerryJCZhang, @vedantpalit1008, @RoderickWu4, Aydin Javadov, @francescortu, @JoeunYk05, @albecazzaniga, @radamihalcea, @RamaravindM, Raiyan Ahmed, Shion Guha, Syed Ishtiaque Ahmed, Prakhar Gupta, @_rfaulk, @SSzufa, Daniel Hoyer, Roland Bouffanais, @jzl86, Dmitrii Kharlapenko, @iarthsingh, @alesstolfo, @ArthurConmy, @mrinmayasachan, @TonyWu1105, @Jiarui_Liu_, Chih-Hao Hsu.
#EMNLP2026#NLP#Interpretability#AISafety#MachineLearning
What is chain-of-thought actually doing inside the model? Our new #EMNLP2026 Findings paper shows reasoning models rebuild the meaning of words in activation space over a single trace, and that these representations causally drive performance. Thread below👇
Our paper "Fluid Reasoning Representations" was accepted to EMNLP 2026 Findings.
The question we asked: when a reasoning model spends 15k tokens thinking, what is actually changing inside it?
Short answer: it is rebuilding the meaning of words in activation space, mid-trace.
We are so excited to host the 2nd edition of the Trustworthy AI for Good (AI4GOOD) Workshop at #NeurIPS2026 in Paris, France 🇫🇷
We aim to bring together the AI safety, AI for social good, and AI policy/governance communities to connect what models can do as individual systems with what they do when deployed across populations.
For full details, welcome to visit our website at:
🔗 trustworthy-ai-for-good.gith…
📍December 12, 2026 · Paris 🇫🇷
We have an amazing lineup of speakers and plans. More details 🧵
We are so excited to host the 2nd edition of the Trustworthy AI for Good (AI4GOOD) Workshop at #NeurIPS2026 in Paris, France 🇫🇷
We aim to bring together the AI safety, AI for social good, and AI policy/governance communities to connect what models can do as individual systems with what they do when deployed across populations.
For full details, welcome to visit our website at:
🔗 trustworthy-ai-for-good.gith…
📍December 12, 2026 · Paris 🇫🇷
We have an amazing lineup of speakers and plans. More details 🧵
📡Announcing NewInML workshop at #NeurIPS2026
We are so excited to host the 8th edition of the NewInML (@NewInML) workshop, which will take place in Paris, France 🇫🇷
We are here to help you out gain confidence and network with the community!
If you are new to ML research or have limited publishing experience, it's a perfect opportunity for you to submit your first work, get high quality reviews from reviewers and also meet the experts, and of course, enjoy beautiful Paris 🥐
We are delighted to have an amazing speaker line-up and interesting plans. More details 🧵
We're delighted to host the Trustworthy AI for Good (AI4GOOD) workshop at @NeurIPSConf in Paris 🇫🇷
📢 We're recruiting reviewers, plus an award for top reviewers 🏆
If you're working on AI Safety, Alignment, AI for Science or related topics, we're looking for you.
Feel free to express your interest here: forms.gle/HpkPoE8HqMHH1geL9
🌟 We’d love to welcome you as a reviewer for the 2nd Trustworthy AI4GOOD Workshop at #NeurIPS2026 in Paris! 🇫🇷
Following the success of our first edition at #ICML2026, this workshop will once again bring together researchers and practitioners working to ensure that advances in AI are not only powerful, but also trustworthy, responsible, and genuinely beneficial to society.
Each reviewer will be assigned no more than 3 papers through OpenReview, with no paper bidding or rebuttal phase.
🗓️ Review period: September 2026
⏰ Review deadline: September 22, 2026, AOE
Please make sure your OpenReview profile is up to date before registering.
Join us as a reviewer:
forms.gle/HpkPoE8HqMHH1geL9
We’d be delighted to have you help shape this year’s workshop! 🎉
#AI4GOOD#TrustworthyAI#NeurIPS#AISafety
🌟 We’d love to welcome you as a reviewer for the 2nd Trustworthy AI4GOOD Workshop at #NeurIPS2026 in Paris! 🇫🇷
Following the success of our first edition at #ICML2026, this workshop will once again bring together researchers and practitioners working to ensure that advances in AI are not only powerful, but also trustworthy, responsible, and genuinely beneficial to society.
Each reviewer will be assigned no more than 3 papers through OpenReview, with no paper bidding or rebuttal phase.
🗓️ Review period: September 2026
⏰ Review deadline: September 22, 2026, AOE
Please make sure your OpenReview profile is up to date before registering.
Join us as a reviewer:
forms.gle/HpkPoE8HqMHH1geL9
We’d be delighted to have you help shape this year’s workshop! 🎉
#AI4GOOD#TrustworthyAI#NeurIPS#AISafety
Our Jinesis Lab at the University of Toronto is bringing three exciting papers to #COLM2026! 🎉
These projects push the frontiers of #AISafety, #AgentSecurity, and #AIForScience.🤖
Huge congratulations to all collaborators and co-authors!
Papers:
🛡️ODILE: Orthogonal Disruption of Injected Tool-Call Embeddings for Agentic Prompt Injection Defense
🔭Stargazer: A Scalable Model-fitting Benchmark Environment for AI Agents under Astrophysical Constraints
⚠️One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety
@UofT,@VectorInst,@EuroSafeAI,@ETH_en,@MPI_IS,@UMich