←
Jacky Kwok202 posts
Jacky Kwok avatar✓

Jacky Kwok ✓

@jackyk02

Stanford CS PhD | Berkeley EECS

◷ Joined June 2025▧ 55 Photos and videos
Jacky Kwok avatar
Jacky Kwok@jackyk02·Sep 23

Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions. CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks. With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks. We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed. Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. 📄 Blog: contrastive-lm.notion.site 💻 Code: github.com/Contrastive-LM/CL… 🗣️ Discord: discord.gg/5dAQEDJBs 🤗 Data & Models: huggingface.co/Contrastive-L… More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵

1:21
Saining Xie avatar
⟳ Jacky Kwok retweeted
Saining Xie@sainingxie·Nov 13

before hopping on the ‘encoder-free’ hype train, think twice.

1:20
Alex Dimakis avatar
⟳ Jacky Kwok retweeted
Alex Dimakis@AlexGDimakis·Nov 12

What are idea basins and how do agents explore them? A few weeks ago, researchers from different labs independently discovered what we now call ‘Idea-basins’: As an autonomous agent iterates to solve a problem, it gets stuck into the methods it has tried before and its trajectories remain in localized regions of the solution space. Techniques like ‘Fork-and-Flush’ and ‘Scaling Discovery requires forgetting’ reduce this idea fixation by periodically deleting the session context. In this post we study how frontier models explore different idea basins for AutoResearchExam. E.g. for a problem on accelerating transformer decoder graphs on CPUs models can try Numpy+Asm Kernels, Native BLAS, JIT compilers and other idea basins. We find that some models switch ideas more than others while others get to the best idea faster and explore deeper. We find that most runs stick with one or two ideas, and models that submit more spend the extra submissions refining and overfit more. We see that models explore most in the first six hours, after which visits to new idea basins drop sharply. This suggests models start refining early leaving potential gains on the table: 58% of runs never reach the basin where the best solution to their task was found. Finally, bold exploration pays off: exploring a new idea gains about nine times as much as a refining step on average. Read more in our blog: bespokelabs.ai/blog/how-mode…

Post attachment
Shayan Talaei avatar
⟳ Jacky Kwok retweeted
Shayan Talaei@talaei_shayan·Nov 12

Introducing Speculative Self-Distillation (SSD, NeurIPS 2026), which pushes the accuracy–efficiency–forgetting frontier of self-distillation. In SSD, student and teacher (the same model with additional information) write the rollouts together. Like a private tutor, the teacher lets the student write and takes over only when the student drifts too far. As a result, SSD trains on rollouts that stay as close to the student's own writing (on-policy) as possible while the teacher keeps them on track. 📉 38–69% fewer training tokens than on-policy, matching its accuracy ⏱️ Reaches on-policy accuracy in ~130 vs ~414 min (Science Q&A) 🧠 Forgets less in a stress test: −1.4 vs −4.5 points for on-policy

0:57
Surya avatar
⟳ Jacky Kwok retweeted
Surya@kickingkeys·Nov 6

Inkling 2 Small can paint 👩‍🎨!! (Video + Audio made in code) Comparison below between GLM 5.3 Flash, DeepSeek V4.1 Flash and Qwen 3.8 Flash on the same prompts. ⬇️

0:30
0:30
Jacky Kwok avatar
Jacky Kwok@jackyk02·Nov 6

Scaling self-verification with Inkling-2-Small and LLM-as-a-Verifier can substantially improve the aesthetic quality of generated 3D scenes 🏙️ The self-verification capability of Inkling is amazing! A blog post with @thinkymachines on RL for creative tasks will be released soon :)

0:15
Thinking Machines avatar
⟳ Jacky Kwok retweeted
Thinking Machines@thinkymachines·Nov 5

Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…

Zirui "Colin" Wang ✈️CoRL avatar
⟳ Jacky Kwok retweeted
Zirui "Colin" Wang ✈️CoRL@zwcolin·Nov 4

Now that we've been asking GPT, Claude and Gemini and let them search for us, what make them click, read, and present? It turns out they are highly susceptible, and we are able to learn a simple content presentation strategy that makes 𝐆𝐏𝐓-𝟓-𝐦𝐢𝐧𝐢 𝐜𝐥𝐢𝐜𝐤 𝟗𝟖.𝟓% 𝐨𝐟 𝐭𝐡𝐞 𝐭𝐢𝐦𝐞 (as opposed to 17.1%), and it even transfers to the frontier 𝐆𝐏𝐓 𝟓.𝟓, 𝐦𝐚𝐤𝐢𝐧𝐠 𝐢𝐭 𝐜𝐥𝐢𝐜𝐤 𝟗𝟑.𝟑% of the time, 𝐎𝐩𝐮𝐬 𝟒.𝟖 𝐚𝐭 𝟗𝟒.𝟏% of the time, and 𝐆𝐞𝐦𝐢𝐧𝐢 𝟑 𝐅𝐥𝐚𝐬𝐡 𝐚𝐭 𝟗𝟖.𝟗% of the time! Introducing our latest study AgentBait: agentbait.github.io

Post attachment
Mert Cemri avatar
⟳ Jacky Kwok retweeted
Mert Cemri@mertcemri·Nov 4

What if you could synthesize an LLM inference engine customized for your model and Mac, just in time? Meet JITEngine: custom engines built with SkyDiscover for Apple Silicon, up to 2.7× faster than the fastest alternatives we tested. jitengine.ai 🧵 (1/n)

Jon Saad-Falcon avatar
⟳ Jacky Kwok retweeted
Jon Saad-Falcon@JonSaadFalcon·Nov 4

AI is using more and more of the world’s energy. But how much economic value are we getting from AI? We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return. Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings: 1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule. 2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule. 3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value. With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations. Joint work co-led by @amirzeinaalii and @Avanika15, with @Mersad_Abbasi, @Jon_Hartley_, John Hennessy, @HazyResearch, and @Azaliamirh. More details below 🧵

0:24
Shayan Talaei avatar
⟳ Jacky Kwok retweeted
Shayan Talaei@talaei_shayan·Nov 4

Really happy to finally share Ripple-65B Preview! We trained it from scratch with a core team of five researchers, and it sets a new U.S. open-weight SOTA on Terminal-Bench 2.1 for agentic coding. Three choices that shaped how we built it, and a few things I learned along the way. 1. We deliberately chose a dense model. Many leading open-weight models are MoEs. For our use case, we wanted strong reasoning and low latency for agents going through long sequences of model and tool calls. Going dense also gave us a simpler training and serving stack and avoided expert-routing communication. That combination mattered a lot to us. 2. We bought no posttraining data. A lot of the work went into building our own data generation and filtering pipelines, and making sure we covered the tasks we cared about. We built on open-source training and inference stacks throughout. I’m particularly happy about how much we were able to do as a small team by building on the community’s work. 3. We worked on the RL system and curriculum together. Mixed RL meant dealing with very different rollout lengths, tool-call patterns and shares of the batch across domains. We tuned oversampling per domain and per problem to keep batch latency down, while managing policy staleness and trainer-inference mismatch. Score centering worked best among the corrections we tried; we used it alongside truncated importance sampling. We also kept adjusting task selection and introducing harder problems as the model improved. Getting batches ready faster and keeping updates stable had to go hand in hand with giving the model useful problems to learn from. Loved building this with @DZhang50, @esonghori2, @ethanboneh and @amiwilliamhuang. Huge thanks to our cofounders @annadgoldie and @Azaliamirh, and the whole @RicursiveAI family for making it possible. Really proud of what we built together.

1:14
Azalia Mirhoseini avatar
⟳ Jacky Kwok retweeted
Azalia Mirhoseini@Azaliamirh·Nov 4

Very excited to launch Ripple-65B Preview, a new state of the art for U.S. open-weight model on agentic coding! You can run Ripple-65B locally and efficiently on a MacBook Pro or DGX Spark! Give it a try: HuggingFace: huggingface.co/ricursiveAI/R… Together AI: together.ai/models/ripple-65… GMI Cloud: console.gmicloud.ai/model/ll… OpenRouter: openrouter.ai/gmicloud/rippl…

Post attachment
1:14
Ricursive Intelligence avatar
⟳ Jacky Kwok retweeted
Ricursive Intelligence@RicursiveAI·Nov 4

Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple

1:14
Sarah Catanzaro avatar
⟳ Jacky Kwok retweeted
Sarah Catanzaro@sarahcat21·Oct 30

Automated verification is undoubtedly the future, but we can be more creative in designing/scaling verifiers; discrete scores just aren't it. Curious to see what might happen as we scale verifiers in different dimensions - including as we decompose evaluation criteria.

Post attachment
Anne Harrington avatar
⟳ Jacky Kwok retweeted
Anne Harrington@annekharrington·Oct 28

Rubrics as Rewards (RaR) uses an LLM judge to provide rewards for RL on open-ended tasks, by scoring responses against a rubric. Can combining judges give better rewards than one? Yes, if we account for judge confidence and disagreement! 🧵 anneharrington.github.io/jud…

0:23