
Inkling 2 Small is our first model trained partly in NVFP4, with a mixed-precision recipe that enables a great tradeoff between speed and accuracy. Check out our new post to see how we did it, and the lessons we learned along the way!
✓mts @thinkymachines | ex-phd student @mit | systems for ml

Inkling 2 Small is our first model trained partly in NVFP4, with a mixed-precision recipe that enables a great tradeoff between speed and accuracy. Check out our new post to see how we did it, and the lessons we learned along the way!

In TML we care a lot about low precision training and high quality bufos :bufo-nvfp4:

Today on Connectionism: how we trained Inkling 2 Small with a native 4-bit forward pass on @nvidia GB300s, from the first pretraining step through the final RL rollout, one of the many improvements across the stack that took pretraining from weeks to 68 hours.

Congrats @thinkymachines team on Inkling 2 Small! A 594B model pretrained in just 68 hours with NVIDIA NVFP4 on GB300 NVL72 racks. Read how the team made low-precision training work at this scale. 👇

A fantastic read on low-precision training by the amazing @jackcookjack and others at thinky!

ML researcher: ...then we must de-quantize the tensors. Me: That's not a real thing, that's some gibberish the villain says in a cheap sci-fi movie. ML researcher: ... Claude: That just means storing each number with more memory bits. Me: Oh okay, why didn't you say so.


we wrote a detailed post on our NVFP4 training, worth a read!


@jackcookjack is a low precision legend. Give this a read to for a sense of how we do nvfp4 pretraining and RL at thinking machines!

the NVFP4 recipe behind Inkling 2 Small: 4-bit forward from the first pretraining step to the final RL rollout, MXFP8 + stochastic rounding for backward and still more to come 🪜

When I first talked to @jackcookjack, he thought that although he was having fun working on nvfp4 pretraining in academia, he was skeptical that it would ever actually work for actual large-scale runs. As it turns out, he was wrong :^) This blog post explains how it can be done

Inkling 2 Small’s weights are now available for download on HuggingFace under an Apache 2.0 license. Make Inkling your own! huggingface.co/thinkingmachi…

Our v2 small model is here. I've been using it daily for a variety of things. The pre-training ran for 68 hours on GB300s. Post-training was 17 million rollouts over 9 days. Entirely pre-mid-post trained in NVFP4. Available to train and fine-tune on Tinker right now. Open weights next week after external safety testing is fully complete (cant skimp on that). Continuing to decentralize and democratize intelligence...

Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…

Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. thinkingmachines.ai/news/int… Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵