←
SemiAnalysis3,880 posts
SemiAnalysis avatar

SemiAnalysis

@SemiAnalysis_
◷ Joined January 2024▧ 2,225 Photos and videos
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·8h

Jensen is out. Who should run NVIDIA next? “It has to be an external CEO, given all of the questions around chip shipments to China. I hope it is Charlie Kawwas of Broadcom or Johny Srouji of Apple.” “If it is internal candidates, Ian Buck and Jonah Alben are the two best bets. But everything around shipping chips to China is going to be very, very scrutinized. Did they know? What did they know?” “Jensen has been the defender of open source. Now there is no one defending Chinese open source anymore. Certainly the restrictions from the US on China are about to ramp up.”

2:00

"Jensen is a remarkable guy who founded and built a remarkable company with a lot of hard work and foresight, and he deserves admiration and respect no matter what, and that will never go away," @JeffBezos tells @lizrhoffman. "If you don't bring Jensen back, I would not hire an outsider. Nvidia is too complicated. I think outside CEOs in complex companies have a vanishingly small chance of succeeding."

1:29
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·12h

This data comes from 632K requests that were run from October 6 to November 8. How often do your agent sessions go cold? (4/4)

SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·12h

We found that only 1.8% of our requests were that uncached. Re-processing their context accounted for about 15% of our total Opus 5.5 spend. Most were new subagents starting from scratch, and 12% of sessions paid about $3 to resume after more than an hour idle. (3/4)

SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·12h

At 600K+ tokens, p50 time to first token was 3.5s uncached vs 1.6s cached, about 2x. However, at p90 the gap grew to 3.5x, with 10.7s uncached vs 3.1s cached. Uncached here means there were over 50K tokens the model had to process from scratch. (2/4)

SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·12h

Does a bigger context make your agent slower to respond? Yes, but not by much if it's already cached. In our internal Claude usage of Opus 5.5, p50 time to first token went from 0.9s at 50–100K tokens to just 1.6s at 600K+. (1/4)🧵

Post attachment
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·17h

The delay in CapEx, Microsoft pushed into 1H2027, missing guidance, you name it. But we already flagged to Datacenter Model subscribers back in September that the remainder of Horizons 3-4 would be in 1H2027. This wasn’t news to them. Our team tracks every cryptominer with extreme depth, ensuring that our clients know of every possible delay before it comes around. (2/3)

Post attachment
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·17h

The Nebius YE2026 results are not very surprising. We already knew the Nebius New Jersey and the ARR YE2026 guides would miss nearly a whole YEAR ago. Our Datacenter Model subscribers got a full walkthrough of exactly why it was too aggressive, and just before earnings this week, we stated that these delays would be recognized too. But what about IREN? (1/3)🧵

Post attachmentPost attachment
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·Nov 17

More than 1.5 GW of chips are in China that shouldn’t be. Unwinding that web could reshape open-source AI. “In our view, there’s more than one and a half gigawatts of chips in China that are not supposed to be there according to US law. How they get there, that whole web, that’s what’s going to unwind.” “If NVIDIA can no longer support the open-source ecosystem, because that was Jensen’s primary goal, does that mean the open-source ecosystem falls away?” “Now open source looks a lot weaker with NVIDIA down and out. Anthropic looks much more likely to get a complete monopoly or a duopoly. This bodes really well for Anthropic’s IPO next week.” “They’re more diverse in what chips they use than any other company in the world.”

2:57
Hao Kang ✈️ Neurips avatar
⟳ SemiAnalysis retweeted
Hao Kang ✈️ Neurips@Mlsys_HaoKang·Nov 16

We breathed life into LLM system. Introducing Evolving Infra⚙️: an inference framework that adapts diverse and dynamic agent workload purely online and retires fixed PD disaggregation. vs. NVIDIA Dynamo on up to 256 GB300 GPUs: ➡️ Up to 4.1× lower latency ➡️ Up to 14× lower P95 TTFT ➡️Up to 2.36× more revenue on real agent traces(Computer Use, Coding and math agents) and opensource models. 🌐 Website: inference-rsi.com 📄 Tech report: alphaxiv.org/en/abs/2611.123… 💻 Code: github.com/mit-han-lab/Evolv… Which agent(e.g. Pi/Openhands) do you want us to accelerate next? Tell us below 👇

Post attachment
2:05
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·Nov 16

ALERT 🚨: Trainium3 Neuron profiling & trace stack is so fine-grained and precise that it can generate pixel art of entire videos! The video below is an actual profile running on Trainium3 DMA engines. We will open-source the entire SKILLS.md later this month.

2:48
vLLM avatar
⟳ SemiAnalysis retweeted
vLLM@vllm_project·Nov 16

The only thing faster than GB300 Dynamo + vLLM is Vera Rubin Dynamo + vLLM 🏎️💨. The first DeepSeek V4.1 Flash results on @NVIDIA Vera Rubin NVL72 are here, powered by Dynamo + vLLM on @SemiAnalysis_'s AgentX benchmark. Compared with GB300 NVL72 running Dynamo + vLLM, InferenceX's perf ruler shows: • 63.4× total token throughput per chip at 400 tok/s/user • 93.3× at 600 tok/s/user

Post attachment
SemiAnalysis avatar
SemiAnalysis@SemiAnalysis_·Nov 16

idclip? noclip? IYKYK. The tear in teardown doesn’t always have to be destructive. Part of the Kirin 9050 Teardown. Reach out to learn more! sales@semianalysis.com | semianalysis.com/careers

0:20