←
Saurabh Kumar22,128 posts
Saurabh Kumar avatar

Saurabh Kumar

@drummatick

e/local-llm 🚀 Working on scaling local inference LLM fine tuning | Harness optimization | Evals and Memory systems for Agents (opinions my own)

◷ Joined September 2016▧ 2,796 Photos and videos
Saurabh Kumar avatar
Saurabh Kumar@drummatick·25 Nov 2023

This tweet is about how I have studied ML and made it my profession. I'll share the resources I've used and the sequence of my study. Straight to point ML pre-requisites(maths) : Linear Algebra, Probability Theory, Calculus, Optimization Theory(optional), Information theory(optional) Linear Algebra: Lecture course by Gilbert Strang Probability theory: MIT 6.041 (it contains parts of Bayesian inference as well) Calculus: your high school and college classes are enough Once basic maths is done then we move to ML. Classical ML : CS229. Either by Andrew NG or someone else. Follow their lecture notes and solve their problem sets. Reference books for classical ML that I followed: PRML by Christopher bishop, Pattern Classification by Duda, Hart and Stork After getting comfortable with classical ML we move to Deep Learning and everything else. Deep Learning and Computer Vision: CS231n. Very good lecture and assignments Reference book: Deep Learning by Ian Goodfellow. This is the best book on deep learning. I’ve read some chapters of it many many times. Beautiful maths and intuitions MLOps: dvc, WandB, MLFlow NLP: I just read hugging face blogs. I haven’t spent much time with classical NLP though. Alignment/AI safety/AI explainability: Anthropic Blogs(I’m a noob in this, just started learning couple months ago) Additionally: Blogs: Lilian Weng(OpenAI)’s blogs, colah’s blogs Additionally: arxiv. I read many papers from arxiv Karas and Tensorflow blogs: for introductory code about modern deep learning frameworks Competitions: Kaggle Cloud compute. GCP/collab/Kaggle notebooks PS: this is not a roadmap. Just what I followed till now and I find it quite structured. Even after 5 years I still find myself learning new stuff everyday.

Saurabh Kumar avatar
Saurabh Kumar@drummatick·10h

I just tried GPT Astra 6.2 on a pretty huge video editing task, and I'd say it performs at least at par with Opus 5.5, with almost half the tokens consumed. It's also much faster and follows instructions much more clearly than Opus. Because Opus takes a lot of time, it consumes a lot of tokens as well. For a similar kind of infographics and animation, ChatGPT Astra 6.2 wins.

Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 14

>your lack of urgency is what's stopping you from reaching your full potential read it somewhere and it has stuck with me ever since I am at the stage of life right now where the only thing that's stopping me from moving fast as fuck is my sense of urgency. When I am in the zone and the urgency kicks in I am able to do weeks work in hours but then at other times it goes into the flat line again. change of envirionment helps every now and then but I gotta find a more stable solution.

Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 13

>shows folks pretty sure that will also be watched by AI at this moment

My dream is for every important PR to be accompanied by a video that shows folks how it works Working on an /animatic skill for this

Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 13

told ya

Post attachment

Introducing Gemini 4 Hydrogen, an extremely capable and fast model built to bring Gemini 4 to everyone! Available to all tiers of Google AI Studio, Gemini API, Gemini for business, the Gemini app, Google Search, and paid Google AI Pro + Ultra subscribers in Google Antigravity.

Post attachment
Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 12

never thought that lobotomizing would lead to post training

Even the simplest weight edit, a deletion, can teach an LLM new skills! Meet Deletion Tuning (#neurips2026): just zeroing <0.1% of weights matches full fine-tuning's accuracy at scale. It's thousands of times smaller and avoids forgetting, a step toward continual learning. 🧵👇

0:10
Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 12

???? are we deadass ignoring Kimi K3.1 pro which was just released last month?

Anthropic shut the door on distillation and Chinese models haven't moved an inch since. Sucks to say it but that's just the reality.

Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 10

Attention was not born in 2017. In 2014 Bahdanau, Cho and Bengio taught a translator to look back at its source sentence, weigh every word with a softmax, and take what it needed. Graves was addressing memory by content that same year. Luong reduced it to a dot product in 2015. By 2016 Cheng and Lapata had sentences attending to themselves. Wasn't that attention? Of course it was! It was there, working, waiting. And yet I must be fair, because fairness is harder than criticism. The Transformer threw out recurrence, which everyone clung to like an old habit, and trusted attention alone. That was the real courage. But why does the field remember who scaled an idea and forget who first had it? Is it the title, so confident it was almost arrogant? Or do we simply worship whoever arrives last and loudest? I don't know.

Saurabh Kumar avatar
Saurabh Kumar@drummatick·Nov 9

> post a hit tweet on marketing > last day within a month > ????

yesterday was my last day @WisprFlow when i joined in 2024 we were a team of 6 planning our first launch today wispr is 150+ people with millions of users getting to be part of something people use every day has been the coolest thing i've ever been part of thank you to the team for teaching me so much to everyone who's used flow and to @tankots for taking a chance on me at 19, forever grateful excited to share what comes next

Chris Blackburn avatar
⟳ Saurabh Kumar retweeted
Chris Blackburn@CJBdingo25·Nov 7

Chinese citizens making live videos from morning till night, even going inside people’s homes share.google/YEc0drMOwWjvopp… Well done, @ProthomAlo for exposing this. Chinese vloggers are filming in Bangladesh claiming to viewers that it's New Delhi, India. Propaganda misfire.

Carl Wheless avatar
⟳ Saurabh Kumar retweeted
Carl Wheless@carlwheless·Nov 7

This is the Chinese-led anti-India PYSOP in action. Save this for future use.

Almost 150 Chinese WeChat streamers came to Dhaka, Bangladesh & spent the entire day livestreaming while showing the city's slums. When locals noticed the unusual number they started investigating why so many of them were doing this. Turns out, they were presenting it as India

1:44