←
Yangqing Jia724 posts
Yangqing Jia avatar

Yangqing Jia

@jiayq

Founder & CEO at Intent Lab. Built caffe, ONNX, PyTorch 1.0. Formerly Google Brain / Meta / Alibaba / Lepton AI / NVIDIA.

◷ Joined April 2009▧ 109 Photos and videos
Yangqing Jia avatar
Yangqing Jia@jiayq·Nov 17

Beautiful way to do reasoning so autonomously. I had the opportunity to try building a lego work from ground up before the release, and it honestly blew my mind that models are now able to figure out the moving pieces - driveshaft, hinges, etc - in addition to a static 3D layout.

Introducing HappyCAD: the AI teammate that designs hardware 10× faster, every mechanism proven. It uses the best LLMs like Fable 5.2 and GPT-6 Astra to design hardware. Comment HAPPYCAD to get an invitation code for happycad.ai.

0:41
Yangqing Jia avatar
Yangqing Jia@jiayq·Nov 10

Rule #1 is that you are only as secure as your real estate provider. Security is going to be a huge market. Inference API has a small vulnerability surface and is safe enough. When it comes to data analytics, storage, finetuning, sandboxes, self-serving toolchains - then it's a full fledged cloud. It's also where better margins are.

I used to complain internally that Databricks was too paranoid about security. It can slow things down. But after seeing all the crazy security incidents at other AI inference providers, I’m grateful we take security so seriously. (e.g. we don’t use GPUs from shitty neoclouds)

Yangqing Jia avatar
Yangqing Jia@jiayq·Nov 6

Super awesome results from @junyanz89 @lmxyy1999 and the @NunchuxAI team! Since the distrifusion days the results have always been amazing.

Proud of what our team has built in the past few months. Nunchux AI now leads 10 Artificial Analysis leaderboards for image and video generation speed, and many enterprise customers are running production image and video workloads on our stack. Our mission is to build the best and fastest infrastructure for image, video, and world models. If you build or run these models at scale, let's talk and work together.

Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 30

Agent loop is like stochastic gradient descent, just without gradient. Overnight the DeepSeek optimization gave us 2.32ms/step, way below our educated guess of a 2.5ms/step floor, and improving. I asked AI to draw out some statistics and I think it's interesting. 1. We didn't ask agents to do so, but empirically it catches early gains first and then improvements start slowing down. 2. Of course, we haven't really tried permuting the experiments and see their independent improvements, but eyeballing some, my observation is that indeed the later ones are more incremental. 3. Convergence seems to be consistent. It's not like the agents are running out of ideas. 4. I looked at the acceptance rate of experiments. Initially it was really high - makes sense. But then it kinda stayed consistent around 46%. 5. Gains do look like exponentially decaying and not heavy tailed. 6. SGD has seen great improvement via rigorous mathematical explorations like momentum SGD, hessian-free approaches, ADAM, Muon, etc. 7. The problem is... discrete optimization on a group of agents is hard. There really isn't any coordinates or gradients. Landscape isn't fixed and all we are talking about is something like a sample - reject_or_accept - resample loop. 8. What's the best heuristics and what's the best algo? It feels like Kalman filtering / reject sampling / data driven markov-chain monte-carlo to me but exact math is still unclear. I am really looking forward to possibly running larger experiments and looking into more traces of such. This seems to be engineering and research combined in one.

Post attachmentPost attachment
Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 30

I really hope that one agent company can start to issue credit cards and offer first-party insurance against agent flops. I use personal agents dozens of times / day, but not gonna put my bank acct directly in their hands yet. And of course seeing "domain names" is always fun.

thibaultsottiaux.com just dropped

Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 28

A hand-typed analysis for optimizing LLM inference with agent fleets. There has been a lot of recent news about that - but many are kinda vague, so I think a more technical post is in order. To explain why this may claim credibility: I drew conclusion from our own DeepSeek 4.1 Flash optimization, which reached a per-step time of 2.41 ms, and a generation speed of up to 2300 tok/s (with Dspark) and reliably around 1800 tok/s for agent tasks. Agents ran ~230 optimization proposals and 94 of them were verified to be useful. My 2 cents from those runs below: 1. Lots of gains beyond "kernel optimization". More than 50% of the optimizations were shipped with no new kernel but configs / graphs / launch order and other strategies. Kernels have been a well beaten path but there is a lot to be gained in the other parts of the end to end system. 2. MegaKernel might be a lie. On non-trivially large models, megakernel is trying to take to big a bite. Most of our experiment showed that glueing small ops into existing kernels, or fusing multiple small kernels into a medium one, is getting the biggest gain. This may only apply to NVIDIA GPUs where the underlying driver is very optimized, though, as @vllm recently showed good results with megakerkenl on TPU. 3. KernelBench and real serving are different problems. Many kernels measured positive in isolated runs but turned out negative when combined in end-to-end. I've always wanted a good simulation like Accel-Sim, but so far simulation don't have a good accuracy when modeling complicated latency patterns with both compute and memory bounds. 4. Verification becomes *really* important if you don't want to be glued to your desk. If you can define "what is correct", like numerical tests, end-to-end quality evaluations, then agents can run for a long time. We didn't interfere with the run other than (a) adding GPU machines, (b) ensure token limit aren't hit, and (c) increasing credit card spending limit. 5. Don't worry, it'll converge - I think the community has consensus that once you set up the goal, agents will eventually converge. Convergence speed is gonna be key though, and 4(c) above is really real. 6. At low concurrency settings, low level assembly writing such as SASS edits have less effect, mainly because waiting for memory (and cache?) becomes the main bottleneck. 7. The hot paths are kinda solved, although much more is to be explored. I think the biggest gain in the next couple months will come from optimizing the wider distributed serving stack, in addition to the single serving engine itself. This would probably involve a lot of system and SRE insights: building proper telemetry, building proper A/B testing without disrupting production traffic, kinda similar to the good old recommendation systems do. 8. Agents doing end to end optimization is a necessity. In our case agents diligently went through 230 experiments, each of which might take a person a week (or at least a couple days with manual experiments + claude code). The sheer volume and speed definitely allows us to explore all possible optimizations otherwise not possible due to human bandwidth. My last thought is a bit contradictory: so far, it seems that agents are "good engineers", in the sense that they can massively carry out known insights. In a way, they are "interpolating" what we already know, but just way faster. I haven't seen agents design awesome algorithms like PagedAttention or Winograd, yet. But who knows, extrapolating from our existing knowledge base may well be very possible in the upcoming months. And man, I like typing on a keyboard, although it might look much worse than a well trained AI writer.

0:49
Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 14

Our takeaway: as agents get more capable, the infrastructure around them becomes the main limit on how fast they iterate, and letting the fleet work on its own methodology paid off. We plan to open source the engine. Full write-up: intentlab.ai/blog/an-agent-f…

Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 14

We stripped out anything that only helped this benchmark - avoiding "AI feature creeps". Reviewer agents stripped out any special snowflake optimizations. ClickBench later added a concurrent QPS measure the fleet never targeted, and our entry is first there too.

Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 14

The fleet also recursively improved its own process. An operations team of agents landed 93 workflow changes while the optimization kept running. After the biggest one, a quick check that retires weak ideas early, experiments per day nearly doubled and the median experiment went from 16 hours to 2.5.

Post attachment
Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 14

The agents proposed 397 experiments, each with a design, independent review, correctness gates, then timing against the best build. 98 produced improvement and 289 retired with a written reason. Every outcome went into a shared record the next proposal reads first.

0:22
Yangqing Jia avatar
Yangqing Jia@jiayq·Oct 14

Our agent fleet just took #1 on ClickBench, 1.2× ahead of the next best engine and 3.1× ahead of the codebase it started from. To our knowledge, the first time an autonomous agent fleet has taken first place on a public systems benchmark where correctness is verified independently.

Post attachment
Yangqing Jia avatar
Yangqing Jia@jiayq·Sep 30

Once merged, bugs tend to stay. In our benchmark the median bug took 33 days to fix, nearly 20% lasted over a year, and one went unnoticed for more than 15 years.

Post attachment
Yangqing Jia avatar
Yangqing Jia@jiayq·Sep 30

In Apache Pulsar, new admin API endpoints were merged without permission checks, letting unauthorized users modify topics across tenants. The baseline agent approved the Docker PR as safe. Intent Advanced was the only agent to catch both bugs.

Yangqing Jia avatar
Yangqing Jia@jiayq·Sep 30

Some bugs look harmless in the diff. A 28-line Docker PR stopped empty port ranges from defaulting to port 0, which seemed like a safety fix. In Docker, port 0 means "pick a random host port," so the change broke that behavior. The fix came 20 days later.

Yangqing Jia avatar
Yangqing Jia@jiayq·Sep 30

Intent Advanced caught 93.0% of the bugs, the highest of any agent we tested, at 63% lower cost per review than the top baseline. It's built for high-stakes changes like database migrations, core infrastructure, and security policy edits. For everyday changes, Intent Balanced caught 79.4%, close to Claude Code with Opus 5.0 at 81.3%, while cutting review cost by 95%. Most engineering work is routine updates, so review needs to stay affordable at that volume.