
Super psyched to finally share our new continual learning paper “Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language”

girl/boy dad & adventures @zemelgroup. formerly barstool and music stuff.

Super psyched to finally share our new continual learning paper “Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language”

We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090. Our model predicts in a unified 3D representation space, capable of generating: • Novel view appearance • Multi-view consistent RGB-D • 3D geometry • Spatially aligned tactile signals Play with the world model now at jwang771.github.io/3dworldsi…!

DeltaNet-PC is out! It turns out that predictive coding offered a native mechanism for test-time memorization of sequential data all along. The crucial insight was to treat updates to the fast associative memory as an inferred latent variable, not a long term learned weight.

Um does anyone know how to withdraw papers from @iclr_conf 2027? Don’t see a withdraw button anywhere.

Neurips bound! (the paper at least dunno about me)
Super psyched to finally share our new continual learning paper “Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language”

Announcing the 2nd iteration of our PhD Winter Internship, for Jan 2027. 1st iteration this year was a big success - PhDs from variety of disciplines got to have a taste of working at HRT, then return later for summer internships and/or full time work. hudsonrivertrading.com/hrt-j…

great program, strong recommend even if you haven’t thought much about finance before
Announcing the 2nd iteration of our PhD Winter Internship, for Jan 2027. 1st iteration this year was a big success - PhDs from variety of disciplines got to have a taste of working at HRT, then return later for summer internships and/or full time work. hudsonrivertrading.com/hrt-j…

calibration is such a deeply under-studied problem and its importance keeps becoming more apparent every day

Always a blast working with @jwang771 @zemelgroup Paper link here: arxiv.org/abs/2604.19444

We tested this pretty extensively, generating over 5B tokens across 9 reasoning models (from 600M to 14B) and 5 tasks spanning math, science and open-domain QA. Our approach substantially outperforms unsupervised baselines based on token probabilities or verbalized confidence (which itself requires extra compute). It also remains strong under distribution shift, works in a black-box setting or without generating a response, and improves downstream decision-making in selective prediction and simulated linguistic calibration.

Our main insight is that self-consistency can serve as unlabeled supervision for calibration. Even without knowing the correct answer, we can sample offline and ask how consistently the model arrives at the same final answer. We then train a lightweight predictor to estimate that signal from just one generated response. At deployment, the model answers once, and the predictor maps that single response to a calibrated confidence estimate.

There’s been lots of interest in LLM calibration over the last few years, especially recently for reasoning LLMs. But most methods still require labeled data or extra inference-time compute. Sometimes we’re not that lucky: e.g., a personalized QA assistant running on-device still needs calibrated confidence, but may not have ground-truth labels or the resources to generate lots of extra tokens. That’s exactly the challenge we address in our new paper: “Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation”

Our new preprint on parallelizing training of temporally precise spiking neural networks is out! We show up to 44x speedups over a conventional sequential baseline. 1/N

Paper: arxiv.org/abs/2507.17383 Github: github.com/thomaspzollo/vla_…

New models include MolmoAct from @jason_lee328 @allen_ai, NORA from @hungchiayu123, and UniVLA from @bqwluckyone @OpenDriveLab Updated results show that the accuracy vs. calibration relationship may be dependent on model architecture and training objective, and that our approaches to prompt ensembling, action scaling, and time-aware monitoring generalize across models.

When I started the VLA calibration project early in 2025, OpenVLA was pretty much the only model that I could use. Since then a bunch of new token-based VLAs have come out, so we updated our paper with new experiments on 4 VLAs.