←
Jefferson Enrique Hernandez Cevallos622 posts
Jefferson Enrique Hernandez Cevallos avatar

Jefferson Enrique Hernandez Cevallos

@jefehern

Opinions are my own. Ph.D. student at @RiceCompSci in @vislang. Previously @RealityLabs @AdobeResearch, @InariAILab, @AdaVivInc.

◷ Joined December 2017▧ 3 Photos and videos
Matthew Leavitt avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Matthew Leavitt@leavittron·Nov 17

Post attachment

Frontier labs like MAI and Seed are exploring latent feedback: fusing the final layer's latent with the next token embedding as the model input. We think the final block is the wrong one. We should read a deep intermediate block instead. Lower loss at same params & FLOPs

Post attachment
Jason Wei avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Jason Wei@jaswwei·22h

1/ We taught a team of Inkling 2 Small agents to work together, communicate, and delegate work, without predefined roles on hard chess puzzles. Using our simple multi-agent RL recipe, we are able to turn pass@k into pass@1. Work with @emilianogl and @GabeVisotsky 🧵

1:14
Jack Ziyang Cai avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Jack Ziyang Cai@jackcai1206·Nov 16

New paper w/ Yihe Dong, @DimitrisPapail & @ShivamGarg91462: Perturbing Inputs Improves Pretraining Distillation. In pretraining distillation, a small student LM learns to match a larger teacher’s next-token predictions on the training text. We change one thing: for the first part of training, both models read a perturbed version of the text (for example, with noise added to the token inputs), and the student matches the teacher’s predictions on that. Then we switch back to clean text. With the same steps and token budget, this beats standard distillation, and it also works well in data-constrained, multi-epoch settings. 🧵

Post attachment
Simon avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Simon@SimonGoodman_·Nov 6

I finished training JEM for 100 epochs, and the results are actually better than I expected. It's only 1 point away from a fine-tuned DINOv3 (which has seen WAY more data). I'm pretty sure it could beat it with more training. It took a bit more than 20 H100 hours to train for 100 epochs on the dataset (~250k training images), and as you can see, my LeJEPA is still undertuned (Claude is currently working on it to get a better baseline). The main problem, though, is with rare classes, but that's expected. It might be a sampling issue. For example, I could continue training while oversampling rare classes, but that's not really what I want. All plants kind of look the same, so I'm wondering if we could make the algorithm better at capturing fine-grained details and the patch-level loss already helps a lot imo. But tbh, the dataset is so hard that I'm not even sure there's much room for improvement on rare classes.

Post attachmentPost attachmentPost attachment

I'm running a small experiment to see how they compare for species classification. I'm training both methods from scratch on a fairly small dataset, Pl@ntNet-300K (github.com/plantnet/PlantNet…): 244k training images of 1,000 plant species with a heavy long tail. I want to see which SSL method best separates rare classes. For LeJEPA I used the official minimal implementation (MINIMAL.md). For JEM I reimplemented the paper myself and try to match its settings as closely as I could. I use opus and astra to triple check but there's still a possibility that I did something wrong I used a ViT-B/16 backbone as the best dataset baseline for now is a fine-tuned DinoV3 ViT-B. So far JEM beats LeJEPA but it's looking too good to be true and I'm pretty sure my LeJEPA is under-tuned. But it's still better than nothing I guess, I'll have to retrain a new LeJEPA Also comparing at equal epochs is unfair because JEM uses ~2x more compute per image (due to teacher pass). But JEM is still ahead at equal compute: LeJEPA's 100 epochs took ~23h on one RTX PRO 6000, the same as ~40 epochs of JEM, and at that point JEM's linear probe is already higher (76.6% vs 73.1% top-1). On rare species (<20 training images) they're close at equal compute (10.1% vs 9.2%). Furthermore this task probably suits JEM better due to its patch-level objective that helps capture the small details separating species. For reference, the released Pl@ntNet-300K-v2 ViT-B checkpoint (DINOv3 pre-trained on 1.7B images, then fine-tuned) reaches 84.5% top-1, but only ~32% on those rare species. So it's not the end of the story, I'll continue the experiments and see if the results change with better tuning. For rare species I obvly don't expected any of them to beat a fine tuned dinoV3 as it has a better feature space, but I'm curious to see how far SSL can get from scratch with this "little" data.

Post attachment
Ellis Brown avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Ellis Brown@_ellisbrown·Nov 13

how should multimodal models encode visual input? → pretrained encoder? → encoder-free? → before the LLM? published models bundle these choices together. we trained 200+ models from scratch to see which of these choices matters only one of them does 🧵 [1/n]

1:20
Wenhao Chai avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Wenhao Chai@wenhaocha1·Nov 13

We tested agents on judging 2026 penalty. The best tie the referee with VAR for around 64% acc. (not super seriously 😜) introducing Penalty Bench ⚽️ Can an AI Agent Referee? this is a new tiny bench series made by Opus with Carbon in just 2 days. wenhaochai.com/blogs/penalty…

0:13
Dimitris Papailiopoulos avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Dimitris Papailiopoulos@DimitrisPapail·Nov 11

I still find it crazy how silly simple Lazarus is: 1. find weak neuron 2. set in-weights to "trigger is present" direction 3. SiLU forces ~0 if not 4. Set out-weights to the unembedding of token you want screamed in the residual 5. Goes straight to logits → that token wins

Post attachment
Jason Wei avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Jason Wei@jaswwei·Nov 10

1/ As agentic tasks grow longer and rollouts become more expensive, the choice between critic-based PPO and critic-free GRPO becomes increasingly important. Critics can reduce gradient variance, allowing models to learn from fewer trajectories, but they also introduce additional compute costs and potential bias from bootstrapping. Our new research @ScaleAILabs compares PPO and GRPO under a shared actor objective across 1,200+ training runs in control tasks, short-horizon math, and long-horizon terminal agents. We studied how three factors influence this tradeoff: batch size, critic reliance, and task complexity. 🧵

Post attachment
Rikiya Takehi avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Rikiya Takehi@rikiyatakehi·Nov 2

We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…

Post attachment
Nathan Lambert avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Nathan Lambert@natolambert·Oct 30

Was reading the score-centering RL paper to add some notes on it the my book, and they have a great table in the appendix on the different importance sampling based train-inference mismatch methods in RL. These have been the biggest focus of RL algorithm research since the GRPO takeoff, imo.

Post attachment
Yuda Song avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Yuda Song@yus167·Oct 30

Foundation models are trained in stages: pretraining, midtraining, and post-training. But is this separation fundamental? We introduce unified training: a single algorithm for the full pipeline. This gives ⚙️ Simpler infrastructure 🔬 A framework for better science (1/n)

Post attachment
Albert Gu avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Albert Gu@_albertgu·Oct 28

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.

1:19
Tanishq Mathew Abraham, Ph.D. avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Tanishq Mathew Abraham, Ph.D.@iScienceLuvr·Oct 26

This paper went completely under the radar! Meta researchers have developed a novel and simple self-supervised learning framework competitive with DINOv2 and DINOv3! "Scalable Patch-Level Self-Supervised Learning" The authors introduce JEM, a principled student-teacher method that learns to align patch representations across different views. Additionally, it incorporates explicit regularization to ensure representations are informative and prevent structural collapse. The framework outperforms DINOv2 in a controlled setting on dense tasks, and even beats the DINOv3 checkpoint despite it being trained on 12x less data.

Post attachment
Martin Marek avatar
⟳ Jefferson Enrique Hernandez Cevallos retweeted
Martin Marek@mrtnm·Oct 25

Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵

Off-policy RL is difficult; runs can suddenly collapse because of async RL staleness or numerical mismatch. StableReinforce is a new method on Tinker that keeps your RL runs stable, even when learning drifts from sampling. tinker-docs.thinkingmachines…