←
tom zollo178 posts
tom zollo avatar

tom zollo

@SquareZollo

girl/boy dad & adventures @zemelgroup. formerly barstool and music stuff.

◷ Joined February 2011▧ 18 Photos and videos
tom zollo avatar
tom zollo@SquareZollo·Feb 27

Super psyched to finally share our new continual learning paper “Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language”

Post attachment
Jimmy Wang avatar
⟳ tom zollo retweeted
Jimmy Wang@jwang771·Nov 2

We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090. Our model predicts in a unified 3D representation space, capable of generating: • Novel view appearance • Multi-view consistent RGB-D • 3D geometry • Spatially aligned tactile signals Play with the world model now at jwang771.github.io/3dworldsi…!

1:32
Todd Morrill avatar
⟳ tom zollo retweeted
Todd Morrill@tee_oh_double_d·Oct 19

DeltaNet-PC is out! It turns out that predictive coding offered a native mechanism for test-time memorization of sequential data all along. The crucial insight was to treat updates to the fast associative memory as an inferred latent variable, not a long term learned weight.

Post attachment
tom zollo avatar
tom zollo@SquareZollo·Sep 24

Neurips bound! (the paper at least dunno about me)

Super psyched to finally share our new continual learning paper “Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Language”

Post attachment
Iain Dunning avatar
⟳ tom zollo retweeted
Iain Dunning@iaindunning·Sep 23

Announcing the 2nd iteration of our PhD Winter Internship, for Jan 2027. 1st iteration this year was a big success - PhDs from variety of disciplines got to have a taste of working at HRT, then return later for summer internships and/or full time work. hudsonrivertrading.com/hrt-j…

tom zollo avatar
tom zollo@SquareZollo·Sep 23

great program, strong recommend even if you haven’t thought much about finance before

Announcing the 2nd iteration of our PhD Winter Internship, for Jan 2027. 1st iteration this year was a big success - PhDs from variety of disciplines got to have a taste of working at HRT, then return later for summer internships and/or full time work. hudsonrivertrading.com/hrt-j…

will brown avatar
⟳ tom zollo retweeted
will brown@willcb·Aug 29

calibration is such a deeply under-studied problem and its importance keeps becoming more apparent every day

tom zollo avatar
tom zollo@SquareZollo·Apr 24

We tested this pretty extensively, generating over 5B tokens across 9 reasoning models (from 600M to 14B) and 5 tasks spanning math, science and open-domain QA. Our approach substantially outperforms unsupervised baselines based on token probabilities or verbalized confidence (which itself requires extra compute). It also remains strong under distribution shift, works in a black-box setting or without generating a response, and improves downstream decision-making in selective prediction and simulated linguistic calibration.

tom zollo avatar
tom zollo@SquareZollo·Apr 24

Our main insight is that self-consistency can serve as unlabeled supervision for calibration. Even without knowing the correct answer, we can sample offline and ask how consistently the model arrives at the same final answer. We then train a lightweight predictor to estimate that signal from just one generated response. At deployment, the model answers once, and the predictor maps that single response to a calibrated confidence estimate.

tom zollo avatar
tom zollo@SquareZollo·Apr 24

There’s been lots of interest in LLM calibration over the last few years, especially recently for reasoning LLMs. But most methods still require labeled data or extra inference-time compute. Sometimes we’re not that lucky: e.g., a personalized QA assistant running on-device still needs calibrated confidence, but may not have ground-truth labels or the resources to generate lots of extra tokens. That’s exactly the challenge we address in our new paper: “Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation”

Todd Morrill avatar
⟳ tom zollo retweeted
Todd Morrill@tee_oh_double_d·Mar 17

Our new preprint on parallelizing training of temporally precise spiking neural networks is out! We show up to 44x speedups over a conventional sequential baseline. 1/N

Post attachment
tom zollo avatar
tom zollo@SquareZollo·Mar 17

When I started the VLA calibration project early in 2025, OpenVLA was pretty much the only model that I could use. Since then a bunch of new token-based VLAs have come out, so we updated our paper with new experiments on 4 VLAs.