←
Shrinivas Ramasubramanian173 posts
Shrinivas Ramasubramanian avatar

Shrinivas Ramasubramanian

@stablegradients

MS @CarnegieMellon Undergrad @iitbombay

◷ Joined September 2021▧ 13 Photos and videos
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Sep 2

Is RL optimizing the right objective? 🤔 Should we maximize mean reward? Best-of-k? Which k? Standard RL pulls on the mean and often the distribution collapses to a spike. The tail dies 🥲 We introduce Tail-Likelihood Reinforcement Learning (TailRL). It maximizes the mean reward while simultaneously maximizing coverage over high reward outputs. 🧵 1/n

0:23
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Oct 30

Maximum likelihood has been the principled optimization objective in the past few decades. It is about time we strip down the artificial seperation of pre training, mid training and post training. kudos to my griends on cracking this!!!

Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental? We introduce unified training: a single algorithm for the full pipeline. This gives ⚙️ Simpler infrastructure 🔬 A framework for better science (1/n)

Post attachment
Mayank Mishra avatar
⟳ Shrinivas Ramasubramanian retweeted
Mayank Mishra@mmayank74567·Oct 22

Check out ReSYNC! Robots often fail. ReSYNC not only helps them recover but also discovers concepts to explain and avoid failures during unseen long-horizon tasks.

Failures are inevitable for robots that are deployed in the *big* world. We introduce ReSYNC, a system that leverages large-scale RL sim2real to recover from local failures and concept discovery to explain/avoid new failures. Check it out here: jaraxxus-me.github.io/ReSYNC…

0:24
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Oct 16

All of us have experienced existing language models making stuff up and give wrong answers on hard problems. An important feature of language models is that they know they don't know the answers and verbalize it. How about we train the language model to do so...

1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky! Very excited to share our new paper: Training Language Models to Fail Gracefully 🧵

0:13
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Oct 12

Check out Schema, new work from @guanningzeng @Zanette_ai and @HavenFeng! It lets frontier LLMs direct their own learning by building and refining an executable theory, testing it against experience, and using it to explore and plan.

On July 15, we released Schema, the first agent harness to reach ~99% RHAE on the ARC-AGI-3 public set. Today, we’re sharing the full paper and code, with new results and a closer look at why Schema works.

Guanning Zeng avatar
⟳ Shrinivas Ramasubramanian retweeted
Guanning Zeng@guanningzeng·Oct 12

On July 15, we released Schema, the first agent harness to reach ~99% RHAE on the ARC-AGI-3 public set. Today, we’re sharing the full paper and code, with new results and a closer look at why Schema works.

Today, we’re introducing [schema]: a harness reaching 99% RHAE with Opus 4.8 + Fable 5 and 95.35% with GPT-5.6 Sol on ARC-AGI-3 Public set. [schema] makes an LLM think like a physicist. 🧵

0:47
Harsh Rangwani avatar
⟳ Shrinivas Ramasubramanian retweeted
Harsh Rangwani@RangwaniHarsh·Sep 30

Ask any top image model for "a dog with a purple left ear and a striped tail" and watch it struggle. 🐕 Part-level control is still an open problem for NB2, Flux & co. Our @NeurIPSConf paper tackles it head-on. Amazing collaboration with @vikataravi @VarghPK @va1bhavagrawal 🙌

Excited to share PartConcepts, a unified mechanism for fine-grained part localization and generation, accepted to #NeurIPS 2026 @NeurIPSConf 🥳🥳🥳 🌐visionpartconcepts.github.io

Post attachment
Shikhar avatar
⟳ Shrinivas Ramasubramanian retweeted
Shikhar@ShikharSSU·Sep 25

Heading to Sydney for #Interspeech2026! 🐨 Excited to present our SOTA universal phone recognition model in the oral session: Tuesday 29 September - 16:30-18:30 - Area 9 - Oral 1 Code and models: github.com/changelinglab/Pho…

Interspeech 2026 in Sydney starts this Sunday, Sept. 27! We present 2 tutorials and 19 papers 🙂 Sunday tutorials: - On the Landscape of Spoken Language Models (AM) - Conversational Speech Recognition & Analysis with Speech LMs (PM) If you see me, say hi 😉 See you in Sydney!

Post attachment
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Sep 10

If you are at ECCV, you dont want to miss this! We finally have visual representation downstream of text guidance. Talk at 9AM Malmo arena.

Pretrained ViTs like DINOv2 or CLIP are great, but they produce fixed, generic representations that encode the most salient visual concepts (e.g., "cat"). In human vision, prior priming with language changes how people parse an image. We believe visual encoders should do the same 🚨 Introducing Steerable Visual Representations, a new family of visual features you can steer with text towards specific visual concepts.

Post attachment
Russ Salakhutdinov avatar
⟳ Shrinivas Ramasubramanian retweeted
Russ Salakhutdinov@rsalakhu·Sep 6

Check out our new work on Tail-Likelihood Reinforcement Learning (TailRL), extending maximum-likelihood RL from binary to continuous rewards. arxiv.org/abs/2609.02987 Rather than optimizing only mean reward, TailRL maximizes the expected log of upper-tail probabilities, naturally placing more weight on rare, high-reward rollouts. Its gradient can also be interpreted as a mixture of Best-of-(k) gradients. TailRL requires only a simple modification to the advantage function, making it easy to integrate into existing RL pipelines. Across object localization, maze navigation, GUI grounding, and code optimization, TailRL effectively exploits rare high-reward samples and scales better with increased inference-time sampling. Check out a detailed thread by @stablegradients.

Post attachment

Is RL optimizing the right objective? 🤔 Should we maximize mean reward? Best-of-k? Which k? Standard RL pulls on the mean and often the distribution collapses to a spike. The tail dies 🥲 We introduce Tail-Likelihood Reinforcement Learning (TailRL). It maximizes the mean reward while simultaneously maximizing coverage over high reward outputs. 🧵 1/n

0:23
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Sep 6

Please check out our new work Tail-Likelihood Reinforcement Learning on @askalphaxiv

“Tail-Likelihood Reinforcement Learning” Standard RL optimizes average reward, which can suppress rare but exceptional outputs and hurt Best-of-k scaling. This paper instead optimizes the probability of exceeding every reward threshold, giving more weight to rare high-reward rollouts. Its gradient becomes a harmonic mixture of Best-of-k objectives across all k, helping preserve exploration and inference-time scaling, with up to 256x fewer samples on GUI grounding and 7.7x Best-of-1024 speedup on code optimization. alphaxiv.org/abs/2609.02987

Post attachment
Andrea Zanette avatar
⟳ Shrinivas Ramasubramanian retweeted
Andrea Zanette@Zanette_ai·Sep 4

The most common question we got after MaxRL was: “What if the rewards are continuous?” I think we now have a pretty good answer. Check out Tail Likelihood Reinforcement Learning (TailRL), which extends MaxRL to continuous rewards. Much like MaxRL, we’ve found that TailRL can “get you out of trouble” on reinforcement learning problems that are particularly difficult to optimize, while maintaining much better coverage of rare, high-reward solutions, which is useful for preserving diversity and improving test-time scaling. Curious? Check out the thread below by @stablegradients.

Is RL optimizing the right objective? 🤔 Should we maximize mean reward? Best-of-k? Which k? Standard RL pulls on the mean and often the distribution collapses to a spike. The tail dies 🥲 We introduce Tail-Likelihood Reinforcement Learning (TailRL). It maximizes the mean reward while simultaneously maximizing coverage over high reward outputs. 🧵 1/n

0:23
Shrinivas Ramasubramanian avatar
Shrinivas Ramasubramanian@stablegradients·Sep 2

The best part of research is the friends you make along the way. I am extremely grateful to my friends and co-authors @amuseddaman , @FahimTajwar10 and @guanningzeng for their contribution and constant support. This project would not have come to fruition without the mentorship of @yus167 , Drew Bagnell, @rsalakhu , @Chenfeng_X , @ZhongzhuZhou , @QingyangWu1 , @HavenFeng and Aarti Singh. A huuugee shoutout to my advisors. The best advisors one could imagine!! @Zanette_ai and Jeff Schneider. 🧵 13/n