←
Haohui Mai1,955 posts
Haohui Mai avatar

Haohui Mai

@wheat9

OS/GPU hacker. Inference stack ninja. causalflow.ai

◷ Joined October 2009▧ 5 Photos and videos
Haohui Mai avatar
Haohui Mai@wheat9·Oct 3

For whatever reasons, OpenAI keeps banning my subscriptions for normal uses :-(

Andrew Ng avatar
⟳ Haohui Mai retweeted
Andrew Ng@AndrewYNg·Jul 25

This is a false equivalence. Everyone has the right to keep their code private. The problem is when someone tries to stop OTHERS from open sourcing.

Bill Ackman avatar
⟳ Haohui Mai retweeted
Bill Ackman@BillAckman·Jun 13

.@BernieSanders , it is a time to celebrate. @elonmusk has created enormous value for society by building @SpaceX, driving down the cost of rocket launches and creating a global satellite communication network that has brought high speed, low-cost internet and communication access to hundreds of millions and eventually billions of people along with critical advantages for our military and our nation’s defense. SpaceX and its technologies will cause an acceleration in the growth of wages and wealth creation globally, including in some of the poorest communities in the U.S. and around the world. Access to low-cost, high speed communications everywhere will allow children around the world to be educated, families to build businesses, and life-saving medical knowledge and care to be available everywhere. SpaceX will materially bring down the cost of compute, advancing AI and humanity. Meanwhile, 4,000 SpaceX employees yesterday became millionaires, including hourly wage employees who you claim you are trying to help. The Elon Musks of the world drive growth, global GDP, and provide access to goods and services at lower cost that would otherwise not exist. Elon’s nominal trillionaire status is due to his ownership of SpaceX, Tesla, Neuralink, the Boring Company and his other initiatives that have brought new technologies that improve our everyday lives. Elon is not sitting on a trillion dollar pile of cash, jewelry and gold. He is using his controlling stakes in his companies to advance mankind. Elon’s companies don’t pay dividends. They reinvest all of their capital to accelerate innovation and value creation. Elon is working 24/7 for all of us. He deserves respect and appreciation, not smears. Bernie, your socialism would never allow a SpaceX to be built. Socialism has only proven to impoverish mankind and lead to death and destruction. We need to create the conditions for more SpaceXs to be built, not attack the great entrepreneurs who are helping to advance our country.

Bernie Sanders @BernieSanders Jun 12

Post attachment
Tianyin Xu avatar
⟳ Haohui Mai retweeted
Tianyin Xu@tianyin_xu·Apr 30

RT appreciated. Anyone looking for an excellent Linux kernel developer? Ruowen (@chinqrw) is one of the best. He is on the market due to the shutdown of Red Hat China. He's mainly looking in China, but also open to jobs elsewhere. He co-leads the Rex project (github.com/rex-rs/rex) with @Jinghao_J which they started it at UIUC. He also has extensive experience working on Red Hat's kernel-QE. I worked with Ruowen as my TA of CS 423 and on the Rex project. He is great!

I am currently seeking opportunities in Linux kernel development, eBPF, Rust-for-Linux, or related platform work. DM or chinqrw@gmail.com — happy to share the resume. #LinuxKernel #eBPF #Rust #RustForLinux #OpenToWork

Haohui Mai avatar
Haohui Mai@wheat9·Apr 22

LLMs can write GPU kernels, but they still struggle to make them assembly-fast. Real-world performance requires complex, tightly coupled optimizations across the whole kernel. ARGUS is the first agentic framework to achieve assembly-fast performance on real-world GPU kernels. On AMD MI300X, it reaches 99–104% of hand-optimized assembly throughput on GEMM, FlashAttention, and fused MoE, while running 2–1543× faster than existing agentic systems. ARGUS makes these global properties explicit through data-flow invariants. These invariants specify what should match at key program points, such as ensuring tensor core instructions see consistent matrix operands despite changes to swizzled memory layouts, tiling, and pipelining. That gives both the compiler and the LLM dense guidance beyond sparse unit tests, verified at compile time with abstract interpretation and SMT solving. arxiv.org/abs/2604.18616

Haohui Mai avatar
Haohui Mai@wheat9·Apr 15

It turns out that the only reliable network connection on plane is UDP. TCP over UDP saves the day!

NeurIPS Conference avatar
⟳ Haohui Mai retweeted
NeurIPS Conference@NeurIPSConf·Mar 27

We want to speak directly to the concern many of you have expressed, and we owe you a clear explanation of what happened, why it happened, and where we stand now. We understand this situation caused genuine alarm and we take that seriously. In preparing the NeurIPS 2026 handbook, we included a link to a US government sanctions tool that covers a significantly broader set of restrictions than those NeurIPS is actually required to follow. This error was due to miscommunication between the NeurIPS Foundation and our legal team; there was never an intention to restrict participation beyond our mandatory compliance obligations. The responsibility for that error is ours as an organization, and we deeply apologize for the alarm and impact this miscommunication had on our community. We have updated the link and clarified the text of our policy, which is consistent with that of ACM and IEEE, as well as other international conferences and NeurIPS in the past. As in previous years, NeurIPS welcomes submissions from all compliant institutions and individuals. We want to reiterate that NeurIPS is a community-driven event, created by and for the community, and strives to be inclusive. The NeurIPS 2026 organizing committee was particularly saddened to learn of this institutional miscommunication. The organizing committee has taken on the responsibility of running the conference this year with the goal of fostering open communication, knowledge sharing, and global scientific discourse. We thank the community for bringing this issue to our attention and working with us through this situation.

LaurieWired avatar
⟳ Haohui Mai retweeted
LaurieWired@lauriewired·Jan 28

if you’re a CS/EE student write your thesis on JIT compilation of eBPF for NVMe controllers there’s huge career alpha in computational storage; the standards are *just* starting to exist (TP4091)

Post attachment
i Expose Racists avatar
⟳ Haohui Mai retweeted
i Expose Racists@SeeRacists·Feb 14

HEARTBREAKING: Ex-PhD student Brendt Christensen found GUILTY of posing as cop, luring, abducting, R*ping & d*capitating Chinese scholar Yingying Zhang in his apartment in 2017. Her dism*mbered remains STILL missing. Never forget Yingying’s story.

2:12
Zhijian Liu avatar
⟳ Haohui Mai retweeted
Zhijian Liu@zhijianliu·Feb 7

The paper is now available: huggingface.co/papers/2602.0… More updates coming soon!

Holiday cooking finally ready to serve! 🥳 Introducing DFlash — speculative decoding with block diffusion. 🚀 6.2× lossless speedup on Qwen3-8B ⚡ 2.5× faster than EAGLE-3 Diffusion vs AR doesn’t have to be a fight. At today’s stage: • dLLMs = fast, highly parallel, but lossy • AR LLMs = accurate, sequential, but slow DFlash = diffusion drafts, AR verifies.

0:26
Aakash Gupta avatar
⟳ Haohui Mai retweeted
Aakash Gupta@aakashgupta·Feb 6

Sounds incredible until you read the fine print. The compiler generates less efficient code than GCC with all optimizations disabled. It doesn’t have its own assembler or linker. It can’t produce a 16-bit x86 code generator. And Carlini himself says it has “nearly reached the limits of Opus’s abilities.” New features and bugfixes kept breaking existing functionality. So what did $20,000 and two weeks actually buy? A compiler that passes 99% of GCC’s torture tests but can’t match the output quality of a tool that’s had 37 years of human engineering. That’s the constraint nobody’s pricing in. The real story is in the cost curve, not the capability demo. $20,000 for 100,000 lines means $0.20 per line of generated code. A senior compiler engineer costs roughly $150/hour. At maybe 50 polished lines per hour for something this complex, that’s $3/line. AI just did it at 15x cheaper, and it will only get cheaper from here. But the code isn’t equivalent. The AI version needs a human to finish the assembler, fix the linker, optimize the output, and prevent regressions. Those are the hardest 20% of the problem, and they represent 80% of the engineering value. Anthropic built the demo. Shipping the product still requires humans. This tells you exactly where we are in the autonomous software timeline. AI can now produce impressive first drafts of complex systems at trivial cost. Turning those drafts into production software still requires the judgment that costs $300K+ per year in compiler engineer salary. The gap between “compiles the Linux kernel” and “replaces GCC” is measured in decades of accumulated engineering wisdom that no model has internalized yet. The companies that understand this will use agent teams to generate the 80% and hire engineers to finish the 20%. The companies that don’t will ship $20,000 compilers that produce slower code than a free tool from 1987.

New Engineering blog: We tasked Opus 4.6 using agent teams to build a C compiler. Then we (mostly) walked away. Two weeks later, it worked on the Linux kernel. Here's what it taught us about the future of autonomous software development. Read more: anthropic.com/engineering/bu…

0:42
Jeff Dean avatar
⟳ Haohui Mai retweeted
Jeff Dean@JeffDean·19 Dec 2025

Performance Hints Over the years, my colleague Sanjay Ghemawat and I have done a fair bit of diving into performance tuning of various pieces of code. We wrote an internal Performance Hints document a couple of years ago as a way of identifying some general principles and we've recently published a version of it externally. We'd love any feedback you might have! Read the full doc at: abseil.io/fast/hints.html

Post attachment
Haohui Mai avatar
Haohui Mai@wheat9·12 Nov 2025

I really hope that AMD can spend some love on their out-of-the-box experience for inference. For example, the latest sglang v0.5.5 docker image is broken for a whole week due to github.com/sgl-project/sglan…. Maybe it's time to add some smoke tests