←
Simo Ryu5,662 posts
Simo Ryu avatar

Simo Ryu

@cloneofsimo

I like cats, math and codes cloneofsimo@gmail.com

◷ Joined May 2022▧ 1,871 Photos and videos
Simo Ryu avatar
Simo Ryu@cloneofsimo·Mar 4

This was work done last year, but my lazy ass uploaded it just this week It was huge fun working on this project.

yesterday, @cloneofsimo and I released a tech report on the video pretraining research we worked on last year. we believe sharing our experience with the community building video models will be of great help! arxiv.org/abs/2603.00173

Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 17

alternative universe people tried to classify dogs with gaming cards and that lead to solving the next millenium problem

yeah

Post attachment
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 17

Me reading 1, 2 : "yeah that sounds about right" Me reading through 3, 4, ... : 😳

Having been in the image generation community for this long, I've noticed some tricks that may boost your generation performance at no cost. All the aggressive opinions below are my own, but credit for the findings goes to my amazing friends and collaborators @XiaoyuW9401147 @Weihang404 @lowerbad.🫡

Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 16

absolutely incredible use of free will.

thijs@cdngdevNov 16

i recreated THE ENTIRE WORLD in minecraft at 1:1 scale. every building on earth, including your house! to do this, i trained a SOTA world-to-minecraft generator on maps data, complete with furnished home interiors! play and check out your neighborhood: worldcraft.gg

0:36
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 16

LMAO. Always remember, x is a wild place: they will have problem with you if you speak slightly peculiar way. This time, roon used the word "base rate" and people are absolutely losing their mind

Post attachment
m@mmmmccchr3Nov 15

Why do ais/ea people have to talk like this?

Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 16

Im embarrassed that I was never aware of such clean derivation!

Why do we divide by n-1 instead of n when we want to estimate the variance? This is Bessel's correction term, and it made no sense to me back in university when I first learned it. But I finally understood why! Got 2 minutes to spare? Here's the explanation!!

2:04
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 15

Number of exploits found and arxiv submission has very good correlation 😅

Post attachmentPost attachment
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 15

stunning!

AI interfaces nowadays are still pretty much turn-based. In my PhD thesis, I thought about what interaction would look like if we could go beyond taking turns. One example is intra-turn interaction:

0:58
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 14

The *only* thing i dislike about this book is there is no more of it. You have to read this.

Post attachment
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 14

Oh my god, I finished reading this, it is *so* refreshing. I am paralyzed by the fact that one can come up with all the materials in the book. Masterpiece.

qntm@qntm5 Mar 2025

11 November 2025.

Post attachment
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 13

People saying this is *obvious* because of open source kernels doesnt know how many models are trained on hoppers and served on blackwells. Also even if they train on some other hardware they would still need to write the nvidia kernels to serve US clients (for some compliance, they would need to serve in US cloud to serve US clients) so it doesnt mean anything

I have a major result to share. I’ve found conclusive forensic evidence that the top 4 Chinese AI labs—DeepSeek, Moonshot, MiniMax and Ant Group—have built their newest models on NVIDIA’s most advanced Blackwell chips, which are barred from export to China. This sounds insane, but I discovered that almost every AI model carries a hidden signature of the GPU that trained it. When a model is first set up, the starting values of its weights are laid out in a pattern that depends on the exact GPU. Each combination of GPU and seed number produces a unique pattern—a sort of fingerprint of the hardware used to create the model. Traces of that pattern survive training into the published model. Again, this evidence isn’t based on any insider information or leaking. It’s based on a forensic analysis of the model weights released by the companies themselves. I searched more than six trillion combinations of chip layout and seed number and matched them to open-weights models. In doing so, I found conclusive evidence that six Chinese AI models—including DeepSeek-V4-Pro, Kimi K3, and MiniMax-M3—were initialized on NVIDIA Blackwell systems. Four of the models match Grace Blackwell, the GB200 and GB300 parts in NVIDIA’s flagship liquid-cooled rack-scale systems. I then verified these results on real hardware. Running unmodified PyTorch with the seeds I’d recovered, a Blackwell B300 reproduced DeepSeek-V4-Flash’s untouched starting weights bit for bit, and a rented GB200 reproduced Kimi K3’s to a correlation of 0.999995. A Hopper H100 reproduced none of DeepSeek-V4-Flash’s, and changing the seed by one broke the match. (DeepSeek’s seed is 953049583 by the way.) The method also was verified to match 24 known open-weights models to the hardware that developers said they used. It is exceedingly unlikely that these results are a coincidence. Across six trillion combinations, the best accidental match should reach only about 8 sigma (i.e., eight standard deviations above what random chance would produce). The real matches range from 49 to 1,385 sigma, and even the weakest single test hits 20. These were data-center-scale training runs. With some analysis of release schedules and model details, I estimate that the largest, Kimi K3 and DeepSeek-V4-Pro, used 2,000 to 3,000 Blackwell GPUs each. Clusters of this size cost on the order of $150 to $200 million each to buy, and across these labs alone would add up to more than half a billion dollars of export-controlled GPUs, not counting experimental compute or otherwise. It’s quite likely that we see multi-billion dollar expenditure on export-controlled GPUs. I am highly confident in this result, and it has been independently verified by senior researchers at OpenAI and Anthropic. You can read the full report at unemploymentlabs.com/checkpo…. There is much more I’ve identified about these models through other forms of checkpoint forensics like this, and will have more to share soon.

Post attachment
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 12

This is what ive dedicated my whole life to

When your model is robust to Chinese adversaries, but you're not robust to adversarial Chinese models

0:46
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 12

PUBG COMPANY BTW

Excited to share Raon-1 nano, our first SLM trained from scratch! Apache 2.0 license. huggingface.co/KRAFTON/Raon-… Enjoy :) TL;DR: It's an E1.5B SLM with SOTA-level accuracy across the board. It achieves SOTA accuracy on instruction following, context following, and several tool-use tasks. I'm particularly excited because: 1) We distilled it from Raon-1 mini using our new distillation method (NeurIPS 2026 oral): arxiv.org/abs/2605.28207 2) It's our first time using per-layer embeddings to reduce VRAM usage. 3) We finally have a better base model for our edge AI applications! Hope you'll give it a shot too! Feedback is always welcome! @Krafton_AI keeps on cooking. A few more releases are on the way -- stay tuned 😊

Post attachment
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 12

This video explains Kelly's Bet

12:10
Simo Ryu avatar
Simo Ryu@cloneofsimo·Nov 11

Crazy to think if claude finds way to make railway construction 10% more efficient it will pay itself.

Post attachment

JUST IN: Anthropic says developing Opus 5.5 cost more than $10 billion.