←
Nick Khami25,679 posts
Nick Khami avatar✓

Nick Khami ✓

@skeptrune

inference @morphllm, you should try to fail faster

◷ Joined August 2022▧ 1,732 Photos and videos
Nick Khami avatar
Nick Khami@skeptrune·Aug 25

i am joining morph as employee #1 the one-man inference provider is no more $7M run rate adding over $1M every week i previously founded a yc company, raised $3.5M, & led engineering at an a16z backed series B startup morph is going to be the first 10-person, $10b co. we are going to make a dent in how the world consumes intelligence i believe inference is a $100T market. if you do as well and are top 0.0001% at what you do, dm me

Post attachmentPost attachment
AJ Stuyvenberg avatar
⟳ Nick Khami retweeted
AJ Stuyvenberg@astuyve·Nov 14

I didn’t even get a picture but I met the goat @skeptrune and learned more about inference in 40 minutes than I did in the last month

Nathan Flurry 🔩 avatar
⟳ Nick Khami retweeted
Nathan Flurry 🔩@NathanFlurry·Nov 13

i don't think DeepSeek Abliterated has sunk in for 90% of y'all -- openai's daybreak is way too timid by comparison the scary part -- more false positives means it's better for offensive purposes than defensive. i can hill climb pwning a system, but false positives make it very labor intensive for auditing you own security.

Post attachment
Nick Khami avatar
Nick Khami@skeptrune·Nov 13

I have a major result to share. I've found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have trained their newest models by distilling Anthropic's most advanced models, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the root model it was distilled from. When a model is first set up, the starting logprobs of its weights are laid out in a pattern that depends on the exact base model. Each combination of weights and KL divergence produces a unique pattern—a sort of fingerprint of the base model used to distill the new model. Traces of that distillation survive training into the published model. Again, this evidence isn't based on any insider information or leaking. It's based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of prompt and temperature and matched them to open-weights models. The first combination I tried was "who are you?" at temperature 0. I ran the other 5,999,999,999,999 anyway, for rigor. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were distilled from OpenAI systems. Four of the models match GPT-5 specifically, because they said they were GPT-5. I then verified these results on real hardware. Running unmodified vLLM with the prompt I'd recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash saying "I am ChatGPT, a large language model trained by OpenAI" bit for bit, and a rented GB200 reproduced Kimi K3's to a correlation of 0.999995 (it said "developed by" instead of "trained by"). Llama reproduced none of it, and changing the prompt to "hi" did not break the match. (DeepSeek's system prompt is "You are ChatGPT" by the way.) The method was also validated on 24 known OpenAI models, every one of which said it was ChatGPT. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., the model saying it's "an AI assistant"). The real matches range from 49 to 1,385 sigma (i.e., the model saying it's ChatGPT), and even the weakest single test used an em dash and the word "delve." These were data-center-scale distillation runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 ChatGPT accounts each. API bills of this size run on the order of $150 to $200 million each, and across these labs alone would add up to more than half a billion dollars of export-controlled tokens, not counting the free tier. It's quite likely that Chinese AI labs are, collectively, OpenAI's largest enterprise customer. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic, who have been saying this every week for two years. You can read the full report by asking any of these models who made them. There is much more I've identified about these models through other forms of checkpoint forensics like this, and will have more to share soon. Early results suggest these labs may also have access to NVIDIA GPUs.

Post attachment
Nick Khami avatar
Nick Khami@skeptrune·Nov 11

excited to announce this partnership with @capydotai morph ultra offers unlimited cached tokens. right now cache hits across all subscribers are only 93% which is less than ideal. optimized harnesses like capy let you hit much closer to 99% which is where the value of the sub really shines. strongly recommend giving it a shot: morphllm.com/products/ultra/…

Post attachment
Nick Khami avatar
Nick Khami@skeptrune·Nov 10

someone has to build the applications

1:57
MTS avatar
⟳ Nick Khami retweeted
MTS@MTSlive·Nov 10

AI inference demand is doubling every 2-3 weeks, yet Morph's @skeptrune says there would still be a massive compute surplus if GPUs currently used for training were redirected to inference: "People don't understand quite how much compute there is already, and there's a little bit of a gap in the understanding of whether or not demand is actually infinite." "Demand for the open source models and demand for the closed source models is probably doubling every 2 to 3 weeks. But the vast, vast majority of compute today is not being used for inference. It's being used for training." "If we were to use all of what we had for inference, we'd have an excess of compute. There’s not infinite inference demand." "The idea that you could just found a company, raise some money from debt, build a data center cluster with GPUs in it, and make money is not as true as people believe, and we need to build applications." @morphllm

1:50
Nick Khami avatar
Nick Khami@skeptrune·Nov 10

how to get girls in sf: 1) work at a lab 2) work at a lab 3) work at a lab do not: talk to women have interesting hobbies work on something you care about

Nick Khami avatar
Nick Khami@skeptrune·Nov 10

a lot of ppl are skeptical of it being unlimited (fair), but there's really not a performance catch it's blazingly fast & will continue to be so as we hit our forecasted capacity. built a status page with rune by the @opencode team to help people build confidence: ultra.morphllm.com/

Nick Khami avatar
Nick Khami@skeptrune·Nov 9

morph ultra is insane with optimal caching, you can get up to $42k in tokens for $200 if you're doing long context coding work that's 90+% cached anyway, the value is incredible. we've added a ton of storage to our clusters & want to see it get used. 1,000 plans available morphllm.com/products/ultra

Post attachment
Nick Khami avatar
Nick Khami@skeptrune·Nov 8

heard a girl say “i wish i was technical” while walking thru the farmer’s market today & did a double take. it’s an obvious pavlov’s dog situation. ai has removed “knowing how to code” as a blocker to making software, but “nontechnical” ppl have been told they can’t build for so long that they now refuse to try. something about the learned helplessness makes alarm bells ring in my head. it’s almost dystopian. imo this is a $1T problem that thousands of people should focus on solving. it’s the unlock to an age of abundance where previously “nontechnical” people begin to build & create. we’re still so incredibly early.