←
Vijay Veerabadran736 posts
Vijay Veerabadran avatar

Vijay Veerabadran

@simple_cell_

Research @Meta @RealityLabs | Prev: Ph.D. from @CogSciUCSD

◷ Joined July 2010▧ 40 Photos and videos
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🚨 Appearing as a #NeurIPS2025 D&B spotlight(~3%) Could VLMs guess your next prompt for a wearable AI agent? We present WAGIBench, the 1st large-scale Goal Inference Benchmark for Wearable Agents w/ audiovisual, digital & longitudinal context! Paper: arxiv.org/abs/2510.22443 1/

0:11
David Bar avatar
⟳ Vijay Veerabadran retweeted
David Bar@observie·Feb 24

URDF Visualizer: a vscode extension for URDF visualization/debugging, and by far the best way I've found so far

0:07
hardmaru avatar
⟳ Vijay Veerabadran retweeted
hardmaru@hardmaru·Jan 12

One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization. We found that if you simply delete them after pretraining and recalibrate for < 1% of the original budget, you unlock massive context windows.

Introducing DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings pub.sakana.ai/DroPE/ We are releasing a new method called DroPE to extend the context length of pretrained LLMs without the massive compute costs usually associated with long-context fine-tuning. The core insight of this work challenges a fundamental assumption in Transformer architecture. We discovered that explicit positional embeddings like RoPE are critical for training convergence but eventually become the primary bottleneck preventing models from generalizing to longer sequences. Our solution is radically simple: We treat positional embeddings as a temporary training scaffold rather than a permanent architectural necessity. Real-world workflows like reviewing massive code diffs or analyzing legal contracts require context windows that break standard pretrained models. While models without positional embeddings (NoPE) generalize better to these unseen lengths, they are notoriously unstable to train from scratch. Here, we achieve the best of both worlds by using embeddings to ensure stability during pretraining and then dropping them to unlock length extrapolation during inference. Our approach unlocks seamless zero-shot context extension without any expensive long-context training. We demonstrated this on a range of off-the-shelf open-source LLMs. In our tests, recalibrating any model with DroPE requires less than 1% of the original pretraining budget, yet it significantly outperforms established methods on challenging benchmarks like LongBench and RULER. We have released the code and the full paper to encourage the community to rethink the role of positional encodings in modern LLMs. Paper: arxiv.org/abs/2512.12167 Code: github.com/SakanaAI/DroPE

Sahithya Ravi avatar
⟳ Vijay Veerabadran retweeted
Sahithya Ravi@Sahithya_Ravi·6 Dec 2025

Hi👋  I am at #neurips2025 to present our position paper on world models.🌍👇 🗓️ Dec 7  📍Upper Level Ballroom 20D/LAW workshop. I’m excited to chat about VL Reasoning & video generation and am actively looking for Postdoc/ Full-time roles. DM me if you would like to chat.

How can we build more robust and safe world models? 🤔 Our position paper, World Models Must Live in Parallel Worlds 🌍, tries to answer this question. Find us at NeurIPS 2025: 🗓️ Dec 7 📍 Upper Level Ballroom 20D #neurips2025 w/ @Sahithya_Ravi @VeredShwartz, Leonid Sigal 1/5

0:26
Sergey Levine avatar
⟳ Vijay Veerabadran retweeted
Sergey Levine@svlevine·18 Nov 2025

We just released results for our newest VLA from Physical Intelligence: π*0.6. This one is trained with RL, and it makes it quite a bit better: often doubles throughput, enables real-world tasks like folding real laundry and making espresso drinks at the office.

3:50
Josh McDermott avatar
⟳ Vijay Veerabadran retweeted
Josh McDermott@JoshHMcDermott·9 Nov 2025

New pre-print from our lab, by Lakshmi Govindarajan @lakshming92 with help from Sagarika Alavilli, introducing a new type of model for studying sensory uncertainty. biorxiv.org/content/10.1101/… Here is a summary. (1/n)

Karl Ridgeway avatar
⟳ Vijay Veerabadran retweeted
Karl Ridgeway@sloafmaster·3 Nov 2025

New benchmark evaluating multimodal VLMs for proactive smart glasses from our group!

🚨 Appearing as a #NeurIPS2025 D&B spotlight(~3%) Could VLMs guess your next prompt for a wearable AI agent? We present WAGIBench, the 1st large-scale Goal Inference Benchmark for Wearable Agents w/ audiovisual, digital & longitudinal context! Paper: arxiv.org/abs/2510.22443 1/

0:11
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🚀 Ready to test your model? Explore the benchmarking code and link to download our dataset here: github.com/facebookresearch/… Thanks to our awesome team from @RealityLabs and @metaai that made this work possible! #NeurIPS2025 #WearableAI #VLMs #MultimodalAI #AIResearch 9/

Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🧠 vs 🤖We benchmarked various evaluation functions by comparing them to human raters. We find that the LLM Judge parameterized with scenario script best aligns with human raters (76.8% agreement), performing as well as any individual from a separate group. 8/

Post attachment
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🤖 For evaluation, we use LLMs as judges to determine relevancy of predicted goals according to the script. Here are a few examples where the LLM judge rated model outputs as very relevant, borderline, and irrelevant. 7/

Post attachment
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🏆 Results: We note a significant performance gap of SOTA VLMs on the task identified by our benchmark, where even the best tested model (GPT-4.1) only generates relevant goals about 55% of the time. 6/

Post attachment
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🔄 We capture user-device interaction over time by having participants record multiple sessions, building up a longitudinal history of up to 5 sessions per clip that reflects how people actually use assistive wearable devices. 5/

Post attachment
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

📱 Digital context is simulated for 7 popular apps (Calendar, Messaging, Notes, Search, Videos, Maps, Music) using LLMs prompted with the specific script the user was acting out, ensuring privacy while preserving the cues that matter for goal inference. 4/

Post attachment
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

🎥🎤 Our dataset features 29 hours of recordings from 348 participants, each acting out diverse, realistic situations. Every clip includes ground-truth wearable agent goals and rich context—visual, audio, digital, and longitudinal history of past interactions. 3/

Post attachment
Vijay Veerabadran avatar
Vijay Veerabadran@simple_cell_·3 Nov 2025

📊 Key takeaways: ✨ Humans still outperform VLMs on multi-choice prediction. ✨ Bigger VLMs perform better, yet free-form goal generation is relevant just 55% of the time for the best performing model! ✨ LLM-as-judge works about as well as human evaluation on this task. 2/

Nicholas Fabiano, MD avatar
⟳ Vijay Veerabadran retweeted
Nicholas Fabiano, MD@NTFabiano·18 Oct 2025

In 1992 Peter Ratcliffe received this rejection letter from Nature. His findings were not "a sufficient advance in our understanding". 27 years later he won the Nobel Prize for the same discovery. Don't lose faith in the things you believe in.

Post attachment