←
Nihal Pasham1,932 posts
Nihal Pasham avatar

Nihal Pasham

@npashi

🦀 Rust @Nvidia | cuda-oxide author | Proof-oriented programming PS: views are my own.

◷ Joined October 2011▧ 120 Photos and videos
Nihal Pasham avatar
Nihal Pasham@npashi·May 8

Finally able to talk about what I've been heads-down on for 6 months at @nvidia 🦀⚡ We just open-sourced cuda-oxide — an experimental rustc backend that lets you write CUDA kernels in pure Rust. No DSLs. No FFI. No source-to-source step. Single source. Short🧵👇

Post attachment
Nihal Pasham avatar
Nihal Pasham@npashi·Nov 5

Looks like 60x is now the bar for perf improvement with cuda-oxide ... 😅 🤪

This inspired me to try making a native CUDA/Metal version. Highlights: - 14x/24x faster than osam/pytorch on click - 30-120fps masks on hover vs 1-10fps - 66x faster segment everything (1.6s vs 105s) - 2.9x/5.6x faster than pytorch/osam on apple silicon cargo add sam3rs / pip install sam3rs github.com/avencera/sam3rs

0:40
Nihal Pasham avatar
Nihal Pasham@npashi·Oct 30

🥹 ... "cuda-oxide is ready" - 🦀 + @NVIDIAAI = ⚡️⚡️ Highlights: - running NVIDIA Parakeet 2x faster than NeMo, and beating WhisperX by 17.7x - agents writing Rust kernels + compiler feedback @PraveenPerera - awesome ... (and thanks for the thorough write-up)

I replaced ONNX Runtime, cuDNN and cuBLAS in speakrs with GPU kernels written in Rust. Same DER, about 60× faster than pyannote on the same GPU. Read more full results, what went well and lessons learned,

Praveen Perera avatar
⟳ Nihal Pasham retweeted
Praveen Perera@PraveenPerera·Oct 29

I replaced ONNX Runtime, cuDNN and cuBLAS in speakrs with GPU kernels written in Rust. Same DER, about 60× faster than pyannote on the same GPU. Read more full results, what went well and lessons learned,

Chris Allen avatar
⟳ Nihal Pasham retweeted
Chris Allen@theodorvaryag·Oct 7

github.com/bitemyapp/inferro… Research-grade only, but ~182 tokens per second with qwen3 8b w8a8 on an RTX 6000 Pro with all-Rust GPU kernels is nothing to sniff at either. More to come later on this and some orthogonal things.

Sameh Khamis avatar
⟳ Nihal Pasham retweeted
Sameh Khamis@sameh_khamis·Sep 19

Still on a terminal kick - built a Google Maps renderer in my CLI! From scratch - single megakernel in Rust/cuda-oxide for rasterization, renders through Kitty graphics.

1:21
Matej Cerny avatar
⟳ Nihal Pasham retweeted
Matej Cerny@matej_cerny·Oct 15

My favorite talk from Scala Days 2026 was about baking refinement types right into the compiler. Unlike libraries like Iron or Refined, it adds a built-in solver that verifies logic and math across multiple variables at compile time with zero runtime cost. Moving from simple checks to full verification is how Scala's type system should evolve to constrain LLMs - AI code won't compile unless the math holds up.

Vaivaswatha N avatar
⟳ Nihal Pasham retweeted
Vaivaswatha N@VaivaswathaN·Oct 7

The latest Burn release includes a migration of CubeCL's compiler infrastructure to use pliron! github.com/tracel-ai/cubecl/… It's nice to see pliron being adopted by Rust compiler projects, with both CubeCL and cuda-oxide now building on it.

Burn 0.22 is out: our biggest release yet, and a big step towards 1.0. Models no longer carry a backend generic. That alone makes rebuilding after a model edit 6× to 15× faster.

Post attachment
Mayank Pratap Singh avatar
⟳ Nihal Pasham retweeted
Mayank Pratap Singh@Mayank_022·Oct 4

Start learning GPU programming with one fully animated YouTube lecture. youtu.be/dEZP1qUNTOk This is the first video of my GPU programming series with Triton. Every idea in it is visual and animated, just like the GIF in this post, but for a full hour. You write c = a + b in PyTorch every day. This lecture shows what the GPU actually does with it. 𝐖𝐡𝐚𝐭 𝐭𝐡𝐞 𝐥𝐞𝐜𝐭𝐮𝐫𝐞 𝐜𝐨𝐯𝐞𝐫𝐬 - How a GPU sits inside your computer: PCIe, NVLink, servers and clusters - Inside an NVIDIA H100: SMs, FP32 lanes, the warp scheduler, tensor cores, shared memory - Threads, warps, blocks and grids, and how each thread finds its element - Memory: SRAM vs DRAM, DDR vs GDDR vs HBM, caches, latency vs bandwidth - Making memory fast: latency hiding, coalescing and kernel fusion - Why Triton exists, and how it lets you think in tiles instead of threads - Memory-bound vs compute-bound, arithmetic intensity and the roofline - Attention, the N x N matrix, and how FlashAttention fixes it No CUDA or hardware background needed. I want to make more long-form technical animated videos, something similar to what @neural_avb does. Taken inspiration from him 𝐖𝐡𝐚𝐭'𝐬 𝐧𝐞𝐱𝐭 - Lecture 2: writing our first Triton kernel, vector add, line by line, will post it in few days - More lectures on GPU programming and inference optimization - A planned series on deep learning for audio Subscribe to the channel so you don't miss them. Please also share the video with your ML and DL friends, or in any study or community groups you're part of. Watch the full lecture: youtu.be/dEZP1qUNTOk If you are into audio deep learning, inference, or Triton, follow @Mayank_022 me. I will be uploading more posts related to that.

Nihal Pasham avatar
Nihal Pasham@npashi·Sep 27

🦀 A question I keep getting: “How do I get started with cuda-oxide?” With, Rust, GPUs, and compilers, it can be a lot to take in. Here’s my attempt to make it easier: a simple Rust GPU kernel, what #[kernel] generates, and how the host launches it.

Post attachment
Rust Bytes 🦀 avatar
⟳ Nihal Pasham retweeted
Rust Bytes 🦀@rustaceans_rs·Sep 24

The allocator_api feature has been stabilized! > Rust’s Allocator API standardizes custom memory allocation, enabling collections to use specialized strategies for better control, performance, and memory efficiency.

Post attachment