SSL shouldn't require complex training recipes to scale, but the current SotA needs them.
Introducing JEM: a principled SSL method derived from information theory. The first latent patch-level objective demonstrated from 300M→7B params. Stable training, matches or beats DINOv3.
I finished training JEM for 100 epochs, and the results are actually better than I expected. It's only 1 point away from a fine-tuned DINOv3 (which has seen WAY more data). I'm pretty sure it could beat it with more training.
It took a bit more than 20 H100 hours to train for 100 epochs on the dataset (~250k training images), and as you can see, my LeJEPA is still undertuned (Claude is currently working on it to get a better baseline).
The main problem, though, is with rare classes, but that's expected. It might be a sampling issue. For example, I could continue training while oversampling rare classes, but that's not really what I want.
All plants kind of look the same, so I'm wondering if we could make the algorithm better at capturing fine-grained details and the patch-level loss already helps a lot imo. But tbh, the dataset is so hard that I'm not even sure there's much room for improvement on rare classes.
I'm running a small experiment to see how they compare for species classification.
I'm training both methods from scratch on a fairly small dataset, Pl@ntNet-300K (github.com/plantnet/PlantNet…): 244k training images of 1,000 plant species with a heavy long tail. I want to see which SSL method best separates rare classes.
For LeJEPA I used the official minimal implementation (MINIMAL.md). For JEM I reimplemented the paper myself and try to match its settings as closely as I could. I use opus and astra to triple check but there's still a possibility that I did something wrong
I used a ViT-B/16 backbone as the best dataset baseline for now is a fine-tuned DinoV3 ViT-B. So far JEM beats LeJEPA but it's looking too good to be true and I'm pretty sure my LeJEPA is under-tuned. But it's still better than nothing I guess, I'll have to retrain a new LeJEPA
Also comparing at equal epochs is unfair because JEM uses ~2x more compute per image (due to teacher pass). But JEM is still ahead at equal compute: LeJEPA's 100 epochs took ~23h on one RTX PRO 6000, the same as ~40 epochs of JEM, and at that point JEM's linear probe is already higher (76.6% vs 73.1% top-1). On rare species (<20 training images) they're close at equal compute (10.1% vs 9.2%).
Furthermore this task probably suits JEM better due to its patch-level objective that helps capture the small details separating species.
For reference, the released Pl@ntNet-300K-v2 ViT-B checkpoint (DINOv3 pre-trained on 1.7B images, then fine-tuned) reaches 84.5% top-1, but only ~32% on those rare species.
So it's not the end of the story, I'll continue the experiments and see if the results change with better tuning. For rare species I obvly don't expected any of them to beat a fine tuned dinoV3 as it has a better feature space, but I'm curious to see how far SSL can get from scratch with this "little" data.
JEM trains reliably from 300M to 7B parameters, improving dense benchmarks over our strong DINOv2 baseline across scales, with the same data and training schedule. On panoptic segmentation, JEM exceeds DINOv3, despite training shorter on less data, without any refinement stages.
What does this buy us? Compared to DINO, a significantly cleaner setup! Importantly, only a single learning paradigm, which removes the negative interactions between global and local losses observed in DINOv3.
(btw, the EMA is actually optional, but gives better performance)
JEM comes from multi-view InfoMax: learn what different views of an image share.
The objective has three clear roles:
• align corresponding patches
• keep their features informative
• preserve spatial structure
All losses act on patches; no global loss required.
We are looking for someone curious and passionate about research, with good theoretical knowledge and an appetite for empirical work. Position in Paris, flexible start date, the sooner the better. Apply here: metacareers.com/profile/job_…
The DINO team at FAIR Paris is hiring a PhD student to push the frontier of visual representation learning. We offer hard research topics, a research and engineering team supporting you, and a stimulating PhD community. Help us shape the next generation of DINO models! Details👇
1/ We’re so glad to share this new study 💫
Does the brain learn like a Deep Net? 🧠⚙️
- 📄Misalignment Between Backpropagation and the Hierarchy of Brain Responses to Images
- 🔗arxiv.org/abs/2605.28693
Thread below 🧵
Today we are announcing our new startup: Exponential Security Labs.
AI agents are being deployed everywhere, making high-stakes decisions and increasingly automating research itself. Yet their reliability and security remain unsolved technical problems, on a frontier that keeps shifting. Our mission is to secure agentic systems through self-improving red-teaming and guardrail agents, leveraging autonomous AI research.
We sit at the intersection of safety and self-improvement, building on a decade of research in adversarial robustness and AI safety. We're looking for exceptional people to join us!
We're looking for exceptional people to join us on this mission, and we're also eager to talk to companies deploying AI agents who want to improve their security. Please get in touch!
More details: expsec.ai/
Founding team: @Nmndsingh, @AnselmPaulus, @maksym_andr, Matthias Hein
🚀 Internship Opportunity!
DINO team is recruiting 2026 interns at both M2 (Master’s) and PhD levels in Paris office.
🔗 If you are interested, check out the job descriptions and apply here:
metacareers.com/jobs/1497501…metacareers.com/jobs/2011253…
1/ You might have seen it—DINOv3 is out! 🦖🦕In this thread, we share key insights on our Gram anchoring ⚓︎ and how it helps to get smooth feature maps.
👇