←
Ash Vardanian1,963 posts
Ash Vardanian avatar✓

Ash Vardanian ✓

@ashvardanian

Founder @unum_cloud · General Partner @aal_vc · working on USearch, UStore, StringZilla, NumKong & other open-source AI infra since 2015

◷ Joined June 2012▧ 547 Photos and videos
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Nov 10

The hardware moved on. Most software didn't! So I spent 3 years and 2 months since the USearch v2 release rebuilding mine. The v3 release candidate is now public - not a stable release yet, but open for anyone to try - and it didn't come alone: UStore v1 - a multi-modal GPU-accelerated transactional DBMS engine. UCall v1 - the fastest take I know on secure client-server networking. UForm v4 - work in progress on drastically more efficient multi-modal AI training pipelines. All of this now lives on the main branches, as release candidates rather than tagged releases. It is by far the largest body of open-source work I have ever shared at once, and it pulls in the now-public release candidates of the libraries underneath: NumKong v8 - one of the largest & most opinionated mixed-precision numerics libraries. StringZilla v6 - with even more Unicode optimizations and GPU kernels. ForkUnion v4 - with better UX for low-latency parallelism. SmashTable v1 - reinventing associative containers again 🤦 It's far from complete or fully stable. I also didn't write all of this alone - AI models have helped massively. A few years ago I was afraid it would take drastically more resources to get even to this point, especially given how often I hate the results of my last month's work and redesign everything from scratch for the tenth time, always taking longer than I anticipated. The attached 10-minute stuck-in-elevator pitch covers the most important new ideas across these projects. The slightly more detailed, but still brief, 2.5-hour lecture is online: youtu.be/IkGetbrEuws Code is live on GitHub: github.com/unum-cloud And most importantly for developers in 2026 - your AI agents can now read all of the docs on a fully redesigned website: unum.cloud/llms.txt

11:02
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Nov 16

Most of my nights in 2026 look like this: Plan overnight work for a swarm of Fables Micromanage until 3am, fueled by ~2L of coffee/day Fall asleep completely exhausted Wake up 6h later. Agents crashed at 4am 🤦‍♂️ In that 1h, they landed 100 commits across 8 repos Every single one against the projects' design goals Scream, swear, soft-reset everything Back to micromanaging... And to think I used to manage actual humans 😆

Ash Vardanian avatar
Ash Vardanian@ashvardanian·Nov 16

Clang over GCC, CMake over all else. At least for now. The hardest part of my last release, 1.2M lines across 8 repos, was multi-versioning: NVIDIA Hopper and Blackwell kernels living next to AMD CDNA4 and CDNA5, and AVX-512 next to SVE, in one build, picked at runtime. Iterating on that with 4,000+ kernels per platform was too slow, so I distilled it into a toy repo: github.com/ashvardanian/FatW… If you ship hardware-tuned code, it's a small, readable template for: - one Python wheel with per-capability CPU kernels and both mono- & multi-versioned NVIDIA and AMD GPU code in one binary - branchless dispatch tables, and GPU support asked of the driver, not guessed with a downstream project example reusing the kernels without shipping a single unused GPU kernel - comparatively lightweight Docker images that validate all of it on machines with none of that hardware: SDE for AVX-512 and AMX, QEMU for SVE and SME, CUDA and ROCm compilers CMake still scares me, but nothing else probes compilers and machines at this level of portability, and LLMs made maintaining it far easier.

John Zedlewski avatar
⟳ Ash Vardanian retweeted
John Zedlewski@zstats·Nov 13

I have a particularly nasty case of cancer. When I was hospitalized again at my lowest point last year, I told Jensen. He immediately reached out to contacts at Stanford, who were able to cut through red tape, and get me into a clinical trial for which I've turned out to be a perfect fit. I think everyone can have their own views about recent events. But any claims that Jensen did not really care about his staff is simply inaccurate. I, personally, owe him more than I can express. I think the best tribute we can pay to Jensen's legacy is to drive NVIDIA's continued success and push the whole industry forward, by showing we've learned the lessons from years of working here. We will compare ourselves against Speed of Light. The mission will be the boss. And we will not be scared to take smart, bold risks and fail fast with ruthless intellectual honesty. But if there is one management story I remember the most, it's a particular Jensen grilling from my time in autonomous vehicles that stayed with me. Luckily I was not in the hot seat for this one. Team A didn't want to use Team B's framework, because they thought it was too slow. Jensen was surprised - it's just applying a bunch of optimized CUDA kernels in sequence, how could it be slow? So he asked the executive from team B to draw the entire data flow sequence from their framework on the whiteboard and highlight every memcpy. Sure enough, there were a bunch of unnecessary memcpy's surrounding each pipeline stage. Everyone in the room could see why it wasn't getting to SoL and a path to improve things. No arguing about fiefdoms or appeals to authority - the data got us all on the same page. The lesson was not (just) that unnecessary memory copying is bad. It's that executives should not be afraid to probe into details if something doesn't add up, and leaders need to be technical enough and honest enough to engage in real time and not turn every meeting into "I'll get back to you in a week." If that means counting memcpy's with a billionaire CEO, so be it. My best executive meetings at NVIDIA have been when I'm fully prepared to present every detail of my team's work, and my worst failures have come when I realized I couldn't go deep enough to really solve the problem. But the thing that gives me confidence is that Jensen was never the only one to live this philosophy. Anyone who has presented to an SVP at NVIDIA knows they are at risk for getting grilled on fundamental constraints and technical details. I have been part of amazing conversations with both recent and old timer leaders where we were able to get to the core of a problem in an hour rather than a month, because we had the right people in the room and were not afraid to go deeper. The company is filled with supremely talented people who have been trained for 3 decades to work at the highest level, and you can see it in what we produce every day. I'm so proud to be an NVIDIAn, and I can't wait to show off what we ship in the coming months and years.

Naveen Rao avatar
⟳ Ash Vardanian retweeted
Naveen Rao@NaveenGRao·Nov 12

Please, for the love all that is dear in this world, do not replace @JensenHuang with the CFO!!! That pattern has a 100% failure rate. There are many capable people to take this role. Competence >>> continuity

Unum avatar
⟳ Ash Vardanian retweeted
Unum@unum_cloud·Nov 10

Time to adjust to the faster pace of agentic coding! We're moving to rolling releases, dropping C++11 for C++20 concepts in our SDKs, requiring nightly Rust 1.100 for upcoming allocator support, and making @Modular's Mojo 1.2 one of our primary first-party SDKs, for the next wave of heterogeneous infrastructure across CPUs and GPUs 🔥 Over the next weeks, with your help, we'll finalize the next versions of some of the world's fastest search, storage, perception and networking engines. Everything is already on GitHub under Apache 2.0, free for commercial use 🤗 unum.cloud/blog/development-…

Post attachment
tomaarsen avatar
⟳ Ash Vardanian retweeted
tomaarsen@tomaarsen·Nov 9

After truncation, I also quantize the embeddings from fp32 down to int8. This allows me to use a very efficient approximate nearest neighbor Usearch index to retrieve across 1.4m embeddings in <2ms. Kudos to @ashvardanian for their work on Usearch!

Post attachment
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Nov 2

This doesn't even begin to describe how insanely cool USearch v3 is... Designed for 1B+, 10B+, 100B+ scales, it also fixes the v2's weakest modes of operation. ~100K QPS over ~100M bit-vectors on a single outdated CPU at 99%+ recall.

Post attachment
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Nov 1

Made a lot of changes to my GitHub CI lately across ~50 repos, given how fast malware spreads these days… Many CI apps require long-lasting Personal Access Tokens (PATs). I had a CI mini-app with well under 1'000 lines of Python to validate & automate semantic versioning… more safely than the "semantic-release" tool everyone was using, which pulled 363'000 lines of JavaScript 🤦‍♂️ Now, after only 2 years, I'm switching to fully manual versioning, archiving that repo, and recommending everyone spend a weekend trimming PATs. If you ship packages to PyPI, NPM, Crates, NuGet, RubyGems, & Docker Hub - you can now upgrade to OIDC Trusted Publishing. PyPI made it mainstream back in 2023, and now I’ve stopped uploading to registries that don’t have it. You'll also notice that the preview branches of StringZilla & NumKong now carry massive Bash git hooks to catch bad agentic edits early and speed up reviews of big changes. AI contributions aren't bad in my playbook, but describing a coding style in “CONTRIBUTING.md” isn't enough to constrain an LLM (or a human) in a language like C++, with its 2'000-page standard… Maybe in a couple of years it will be, and I'll remove the hooks the same way as TinySemVer 🤞

Post attachment
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Oct 30

The v8 preview adds 14 GPU targets across NVIDIA, AMD and Apple, FP4 and block-scaled formats, attention for LLMs and extended computational-geometry for Life Sciences. Not a full cuBLAS replacement, by design.

Post attachmentPost attachmentPost attachmentPost attachment
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Oct 30

Each language gets as much control as it can stomach. In C, C++, Rust: bring your own threads & allocators. Go & Swift get less control. Python & JavaScript get sensible defaults and a minimal thread pool under the hood. I decided to rip out my own ForkUnion to trim deps.

Post attachment
Ash Vardanian avatar
Ash Vardanian@ashvardanian·Oct 30

Where others lack a fast path / native DType support, the perf gap is 13x to 3,200x. This time in NumKong's favor. Beyond Python, 6 more SDKs reuse kernels. Roughly 10MB of them and keeping the binaries compact is now a bigger challenge than adding hardware support.

Post attachmentPost attachmentPost attachmentPost attachment