
Can @thinkymachines new Inkling-2-Small not just play a video game level, but play and refine it many times over and learn to speedrun it? @akkikiki , myself and other co-authors from @UW and @PatronusAI ran one of the setups from our SpeedRunBench preprint from Sep this year to observe exactly that (arxiv.org/abs/2610.08076). Thanks to the @thinkymachines and @tinkerapi team for credits that enabled this study! Our benchmark measures whether agents can discover and then refine offline a fast enough policy through a level as they play more and more attempts, using only a single score after each attempt to adjust their policy. By design, we don't refine policy amidst episodes, or have multimodal frame understanding ... so our benchmark is more a read on planning, exploration, autoresearch etc than online videogame playing per se.. We used the "offline scratch" setting on SuperTux (SuperMario style game, Tux in place of Mario) level 1-1. Results: 1. Inkling-2-Small (256K, thinking effort 0.5) is an adept speedrunner, learning to clear the level after 1,988 frames, second only to GLM 5.3 (1,961 frames) in the open source family models we benchmarked .. 2. This put Inkling-2 ahead of Gemini 3.7 Flash, GPT-5.6-sol, Claude Opus 5, and DeepSeek-V4-Flash. See the attached bar graph for the final frames 3. Inkling-2-Small dramatically improves over its predecessor Inkling-Small which never learnt to fully clear the level, let alone improving at speedrunning it. Note for scale that Human world record iterated over the years for speedrunning this game is 1,619 frames...
Inkling 2 Small’s weights are now available for download on HuggingFace under an Apache 2.0 license. Make Inkling your own! huggingface.co/thinkingmachi…



