
Do vision and language models, trained on entirely different data, converge toward a shared representation of the world? The Platonic Representation Hypothesis suggests they do. We take a closer look.

Institute for Explainable Machine Learning @HelmholtzMunich and Interpretable and Reliable Machine Learning group @TU_Muenchen

Do vision and language models, trained on entirely different data, converge toward a shared representation of the world? The Platonic Representation Hypothesis suggests they do. We take a closer look.

What does an LLM think of you? LLMs pick up the slightest implicit cues about users. Often, they refuse to say who they think the user is. In our new preprint, we show how to read and change these hidden beliefs. Turns out they even affect whether the model refuses New paper🧵

📣 New preprint: Replay on Demand (RoD) Imagine you had 10 hours to refresh everything you learned in school. Would you divide them equally across every subject? Probably not. You’d spend your limited time on what you’ve forgotten most. 🧵 arxiv.org/abs/2609.40089

📣 Introducing MedKIT, our new benchmark for knowledge integration in LLMs, accepted at #NeurIPS2026! 🧵 Why recalling updated facts ≠ integrated knowledge and why we need methods that can bridge this gap: 📄 arxiv.org/abs/2609.38543

🎉 CapTrack has been accepted to #NeurIPS2026! CapTrack extends LLM forgetting evaluation beyond general knowledge to capabilities. We find forgetting is highly heterogeneous across capabilities. ⚠️ If you touched the weights, check what changed. 📄 arxiv.org/abs/2603.06610

FineVision was just accepted to NeurIPS! Big thanks to all my co-authors, excited to present it at the conference
Today, we are releasing FineVision, a huge open-source dataset for training state-of-the-art Vision-Language Models: > 17.3M images > 24.3M samples > 88.9M turns > 9.5B answer tokens Here are my favourite findings:

On V*Bench and HR-Bench, VisLens matches or beats prior visual search baselines, running 8.5–9.9x faster than Thyme and up to 22.2x faster than training-free multi-pass search. Every crop traces back to decoded tokens, so you can see what the model looked at.

VisLens reads the model's own hidden states. A logit lens projects each image region through the LLM head to reveal what it already says; a light tuned-lens makes early layers readable. Match decoded tokens to the query, cluster, merge into crops, answer.

Finding a small object in a high-resolution image is still hard for MLLMs. Current fixes crop, zoom and re-query the model over and over: slow, and you cannot see why a region was picked. We wanted the answer in one pass, and we wanted to watch it happen.

[Paper]: arXiv: arxiv.org/abs/2608.30705

3/ VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs Jingyi He, @Sanghwa30296060, @zeynepakata eXCV Workshop Poster: Tue 8 Sep, 13:15–14:45, Malmö Arena Palissad East

The adapter is where segmentation quality drops: alignment costs spatial detail. LLM layers claw it back. Correct tokens act as anchors, pulling wrong neighbors to the right label. Causal masking starves early tokens of anchors; bidirectional attention fixes that.

Three probes. Linear probing at every layer to track segmentation quality. Attention knockout to test whether tokens repair each other. Bidirectional attention among image tokens to measure what causal masking costs.

MLLMs are being pushed into pixel-level work like segmentation, but nobody knows where in the stack that spatial ability lives, or where it breaks. So we opened the pipeline: vision encoder, adapter, and every single LLM layer.

[Paper]: arXiv: arxiv.org/abs/2603.17228

2/ ECCV 2026: From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs Boyong Wu, @Sanghwa30296060, @zeynepakata Poster: Fri 11 Sep, 10:30–12:30, ExHall #40 eXCV workshop poster: Tue 8 Sep, 13:15–14:45, Malmö Arena Palissad East