←
Chip Huyen590 posts
Chip Huyen avatar

Chip Huyen

@chipro

@aisysbooks @goodailist AI Engineering: amazon.com/dp/1098166302 Designing MLSys: amazon.com/dp/1098107969 Reading @chipslib

◷ Joined June 2008▧ 112 Photos and videos
Chip Huyen avatar
Chip Huyen@chipro·7 Jan 2025

My 8000-word note on agents: huyenchip.com//2025/01/07/ag… Covering: 1. An overview of agents 2. How the capability of an AI-powered agent is determined by the set of tools it has access to and its capability for planning 3. How to select the best set of tools for your agent 4. Whether LLMs can plan and how to augment a model’s capability for planning 5. Agent’s failure modes AI-powered agents are an emerging field with no established theoretical frameworks for defining, developing, and evaluating them. This post is a best-effort attempt to build a framework from the existing literature, but it will evolve as the field does. As always, feedback is much appreciated!

Chip Huyen avatar
Chip Huyen@chipro·Nov 10

has anyone been able to get "/goal" in codex to work reliably? every single time i turn it on it just wastes an enormous amount of time and tokens

Chip Huyen avatar
Chip Huyen@chipro·Oct 20

Braden and team are great! Highly recommend this program for independent researchers working on cool stuff!

Slingshots // Four is live! Another batch, another fourteen projects shipping open source artifacts that I’m excited about. See below for the full list. A few themes from the batch: 1. Harness time: This quarter harnesses shot up in visibility, with increased appreciation for the parts of an agent that aren’t the model and what they can give you. This batch includes a harness supporting persistent agency (Headlong); a harness supporting self-improvement via context management (ACE); a harness supporting self-improvement of the agentic harness itself (Exo); and a harness supporting multi-agent setups (dotagents). 2. Up and down the stack: There’s innovation to be had at every layer of the stack. We saw projects worth backing at the level of kernels (Thunderkittens); inference (M*); training (Marin); data (Atlantic-CC); evals (Decision Assistants); optimization (SkyDiscover); applications (Wagglehaus), etc. 3. Talk of the town: Slingshots continue to tackle timely problems at the frontier. Concurrent with Jev taking over X, an even faster type 1 decision model (CLM) came out of academia. With RSI on the brain of every big lab researcher, we now have a public literature retrieval benchmark grounded in authors’ firsthand accounts of their research process (ScholarCatalyst). And a new RL training method (Pedagogical RL) addresses head-on the question vexing RL practitioners of how privileged information can be used to discover generalizable behaviors, not just verify. Congrats to all the researchers in this batch and the previous three who continue to ship their research out into the world with open artifacts that can be adopted and built upon. Open science leads to better outcomes for everyone. And the good times will continue to roll – batch five is looking just as good. If you’re a researcher with a project that’s gaining momentum and ready for an adrenaline shot, shoot me a DM or apply at laude.org/slingshots. 👏 @a1zhang, @AlexKrentsel, @ankrgyl, @Azaliamirh, @bariskasikci, @bfspector, @chelseabfinn, @chipro, @Diyi_Yang, @dlwh, @drmapavone, @HazyResearch, @jackyk02, @jonasgeiping, @JonSaadFalcon, @jure, @kothasuhas, @lateinteraction, @martin_casado, @mertcemri, @michaelryan207, @MikeCafarella, @NickAJalbert, @NoahZiems, @nsagan271, @ohmyksh, @percyliang, @qizhengz_alex, @RussoMatthew, @samrmadden, @shagarw21, @ShashwatGoel7, @shulynnliu, @simran_s_arora, @souradipchakr18, @stuart_sul, @tianyu_li_, @tim_kraska, @WilliamBarrHeld, @yoonholeee, @YejinChoinka, @Stanford, @MIT, @LaudeInstitute, @CarnegieMellon, @UW, @UCBerkeley, @MPI_IS, @UWMadison

Chip Huyen avatar
Chip Huyen@chipro·Sep 15

Interesting approach. Models can't output freeform text but can choose from a set of predefined values. Could be useful for data labeling and tasks with a fixed list of possible actions. Unclear how reasoning would work though, but it's super cheap (output tokens are free!)

Post attachment

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x cheaper (w/ output tokens free) • Frontier composable intelligence optimized for decisions AFAICT the shortest path to AI-based economic revolution

2:56
Chip Huyen avatar
Chip Huyen@chipro·Aug 25

i'm researching best practices for skills. what's the most number of skills you've had installed for your agents?

Chip Huyen avatar
Chip Huyen@chipro·Aug 25

real question: why does gpt 5.6 over engineer so much?

Chip Huyen avatar
Chip Huyen@chipro·Aug 17

what's a good model tiering system? i'm sick of telling my agent orchestrator things like: "for Claude, use model X, for OpenAI, use model Y, etc." i want to be able to tell my orchestrator: "use models tier ..." for this kind of task

Chip Huyen avatar
Chip Huyen@chipro·Aug 10

that's the problem he should've sent them in all caps

Post attachment

We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis. It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%. anthropic.com/research/riema…

Chip Huyen avatar
Chip Huyen@chipro·Aug 9

why don't model providers price tokens like electricity, higher price during peak hours and cheaper off peak?

Mira Murati avatar
⟳ Chip Huyen retweeted
Mira Murati@miramurati·Jul 15

Our first model, Inkling. Trained from scratch, weights are open, fine-tunable on Tinker today.

Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. thinkingmachines.ai/news/int… Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

Chip Huyen avatar
Chip Huyen@chipro·Apr 23

Congrats to @AlecRad @Luke_Metz and @soumithchintala! It's really cool to see work produced by a group of 20-somethings without a single PhD between them winning this award. I hope to see these three collaborate again one day :)

We are honored to announce the Test of Time awards for #ICLR2026 🏆 This award recognizes papers published 10 years ago at ICLR 2016 that have had a lasting impact on the field: blog.iclr.cc/2026/04/22/anno…

Post attachment
NeetCode avatar
⟳ Chip Huyen retweeted
NeetCode@neetcode1·Mar 23

Got to meet the wonderful Chip Huyen @chipro She’s so nice and smart!!

Post attachment
Karina avatar
⟳ Chip Huyen retweeted
Karina@karinanguyen·Mar 11

Excited to release PostTrainBench v1.0! This benchmark evaluates the ability of frontier AI agents to post-train language models in a simplified setting. We believe this is a first step toward tracking progress in recursive self-improvement 🧵:

0:44