A month ago @pravsels and I set out to reproduce @physical_int’s RL Token paper. Today, I am sharing our research notes from the journey we’ve been on.
"RL Token" looks like a great and surprisingly simple post-training methodology for optimising robot models for dexterous tasks in the real world!
Over the next few weeks, me and @pravsels will be attempting to reproduce the results (& open source the code)
Stay tuned 👀
I was thinking of this when walking to the office this morning
a week after chatgpt launched, I would have guessed by eoy 2026 the models would be better than me at writing essays and blogs, and worse in coding
I wonder what else we're wrong about today 🤔
I feel it's been increasingly painful to chat with models and navigate the sheer amount of slop in their responses. In my experience, this is across all models and labs, and has only been getting worse.
New models will achieve SOTA on Humanity's Hardest Frontier Ultimate Deep Coding bench, but become impossible to talk to. It feels like we aren't even producing "Large Language Models" anymore, we are just producing "Large Coding Models".
We've almost come full circle, as there are now some Neolabs blooming where their big pitch is something along the lines of "improving the humanness of models and interactions with humans in the loop" – back to the 2023 Chat days!
As for a diagnosis, I think it's just what we optimize for in our datamix. We can see this by inspecting the percent of code data from some open model reports:
- Pre-training data: MAI-1 had 55% of their PT datamix from some code source, MAI-2 had 45%
- Post-training data: Mimo 2.6 "livestreamed" RL run had ~67% of their RL datamix as code
Lastly, I wouldn't call things completely zero-sum. Zero-sum is too strict of a term when knowledge/behaviors definitely do transfer across domains. But having such an aggressive skew on coding agent data is probably hurting other capabilities. Certainly makes the case for owning your intelligence...
We actually got a VLM to perform better on VLM benchmarks after VLA training. arxiv.org/abs/2602.01067v1
But mostly it’s because we co-trained with VQA data while the backbone was just pretrained. But we also showed that we were on par with the VQA trained version.
one endnote - some EQA tasks seem ... a bit off?
does this image and description (with yellow & blue mixed up, even) make sense to you? do you think this is a meaningful dataset / eval of embodied reasoning capabilities?
find all code in this GitHub repo!
github.com/villekuosmanen/vl…
this was a time-boxed Sunday project inspired by some convos at CoRL 😄 you know who you are!
I don’t have a horse in the race, but I expect robot models to bifurcate into 1) VLAs that are also SOTA VLMs, and 2) robot-specific architectures, likely pre-trained from scratch with very different data sources
you already know some labs have picked which side they are on
why should we care about vision-language tasks in robotics?
if you choose to start with vision-language priors, shouldn’t you want these priors to remain in the final model as well? why use a VLM in your model architecture in the first place?
WALL-OSS 0.5 and G0.5 are designed to inherit reasoning and generalisation from their VLM backbones.
these changes may have come at a cost of pure vision-language abilities - WALL-OSS 0.5 performs worse than WALL-OSS 0.1, the only reasoning model in this eval (*)
Pi0.5 and MolmoAct2 are the only models that closely track their base policies, though pi0.5 starts training from a weaker base model.
Ai2 benefits from training both the VLM and VLA, thus having access to the original VLM training mix.
the original VLA training recipe overwrites all vision-language knowledge - OpenVLA, Pi0, and GR00T N1.7 all answer in action tokens
the next generation of models repairs the vision-language knowledge, with different results
most VLAs lose the vision-language skills of their VLM during pre-training
by isolating the VLM weights of open-source VLAs, we can evaluate them on vision-language benchmarks. three categories of models emerge