←
Vmax73 posts
Vmax avatar

Vmax

@VmaxAI

RL research focused on open-ended learning

◷ Joined December 2024▧ 13 Photos and videos
Vmax avatar
Vmax@VmaxAI·Nov 10

1/ As a teaser for Campaign, Vmax’s infrastructure for automated task generation, we are sharing work we did earlier this year for @modal using self-play to generate tasks that target weaknesses of models when using the Modal-SDK.

0:08
Vmax avatar
Vmax@VmaxAI·Nov 10

2/ We developed an algorithm called Mulberry, that aims to cover tasks in a domain while also increasing their difficulty - named after temporary portable harbours from WWII @harborframework

0:27
Fangchang Ma avatar
⟳ Vmax retweeted
Fangchang Ma@fangchangma·Nov 9

The first AI persona you’ll actually want to meet face-to-face. Most AI personas stare blankly while you talk. Emma reacts while you’re still speaking. After 18 months of research at @nuance_ai, this is Emma, built on Nuplex, our full-duplex audiovisual foundation model.

2:22
Pablo Samuel Castro avatar
⟳ Vmax retweeted
Pablo Samuel Castro@pcastr·Oct 26

while working on agentick (to be presented at NeurIPS'26), @creus_roger found that ASCII inputs seemed to really help LLMs during his internship at @VmaxAI he pushed on this idea and came up with GlyphBench, which serves as a benchmark, a research tool, & post-training suite!

Vmax@VmaxAIOct 23

1/ 🎮 Can training on games improve LLM reasoning? Introducing GlyphBench: 360+ game tasks for language-model RL. We use it as a benchmark, a research tool, and a post-training suite. 🧵⬇️

0:18
Vmax avatar
Vmax@VmaxAI·Oct 23

10/ 🔎 How does an agent arrive at its next move? Our replay UI brings observations, reasoning, actions, rewards, and memory into one view. Step through a trajectory to inspect decisions and understand where an agent succeeds or gets stuck.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

9/ 🧠 It does in our Reasoning Gym experiment: gameplay RL raises accuracy from 56.04% to 63.48%, ahead of math RL (62.29%) and code RL (58.98%). 100 tasks/domain; 100 gradient steps each. A strong transfer result for these training sets and this benchmark.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

8/ As a post-training suite, can it support one policy learning across games? We train Qwen3.5-4B on 100 GlyphBench tasks for 100 gradient steps. Mean return on new seeds rises from 0.012 to 0.319. Does learning to play also improve reasoning beyond games?

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

7/ As a research tool, GlyphBench lets us test how RL design and implementation interact. In Snake, we vary learning rate and rollout group size under the same episode budget. Smaller groups allow more updates, but their advantage depends on learning rate.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

6/ 🧪 Can agents learn from the tasks? We train 25 separate Qwen3.5-4B policies across the six standard suites. Returns improve across suites, with different learning speeds and stability. GlyphBench supports studying these differences through one interface.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

5/ 🗺️ Glyph grids outperform pixels and native text in Craftax for all three tested models. For Sol: 11.4% of maximum reward with glyphs, vs 3.7% with pixels and 0.3% with native text. The observation interface makes a substantial difference to performance.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

4/ 📊 As a benchmark, GlyphBench distinguishes agent capabilities across tasks. Before additional RL, Qwen3.8-27B outperforms Qwen3.5-4B on 270/303 tasks. Mean episode length drops from 72.8 to 23.9 turns.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

3/ 🗺️ Agents observe 2D Unicode grids and interact through named actions. The grids preserve spatial layout in text, with a legend identifying objects and a display for health, inventory, and other state. This makes observations readable by both LLMs and researchers.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

2/ Interactive agents must act with incomplete information, use feedback, and plan for consequences that appear later. Games give us repeatable settings to study these problems. GlyphBench combines 303 bounded standard tasks and 59 longer-horizon tasks.

Post attachment
Vmax avatar
Vmax@VmaxAI·Oct 23

1/ 🎮 Can training on games improve LLM reasoning? Introducing GlyphBench: 360+ game tasks for language-model RL. We use it as a benchmark, a research tool, and a post-training suite. 🧵⬇️

0:18