←
Kashif Rasul736 posts
Kashif Rasul avatar

Kashif Rasul

@krasul

Research Scientist working on Deep Learning, Time Series Forecasting, Reinforcement Learning and HPC.

◷ Joined August 2007▧ 39 Photos and videos
Lewis Tunstall avatar
⟳ Kashif Rasul retweeted
Lewis Tunstall@_lewtun·Nov 16

Can Python be a chatbot? Over one week, we challenged a swarm of 1,000 agents to build one entirely in Python: no weights, no training, just a million rules :) Each agent was an instance of DeepSeek-V4.1-Flash that worked on different parts of the bot's conversational capabilities: understanding, answers, memory, routing, etc. They coordinated on a Slack-like message board we built for agents, wrote ~175k messages to each other, and merged ~8k PRs on a shared repo (which almost broke the Hub 🙈) The result is PyBot: 630k lines of Python, zero dependencies, and on par with SmolLM-360M at answering everday questions. Just don't expect it to solve a Millennium Prize problem anytime soon :) Code & demo: github.com/huggingface/pybot

1:42
Emma Scharfman avatar
⟳ Kashif Rasul retweeted
Emma Scharfman@EmmaScharfmann·Nov 12

⛅️🌡️ 🥵 During a heatwave, the hottest hours are in the late afternoon. But most AI weather models only predicts every 6 hours, so the peak often falls in between and gets underestimated. A few weeks ago we released a spatial downscaler for AI weather forecasts. Today we’re releasing the temporal downscaler that goes on top of it, thanks for @krasul's amazing work Everything is open 🤗, the data, the code for training and inference and model weights. How it performs: here’s the record June 2026 heatwave in Paris. It raises the forecast by ~2 °C at the hottest hours. CRPS drops from 2.0 K (linear interpolation of AIFS-ENS) to 1.45 K against hourly CERRA. How it works: same U-Net as the spatial model, plus two inputs. 1. how much the coarse field changes over the 6-hour window, 2. where each hour sits in it. It starts from the spatial model’s weights and learns from hourly CERRA how the temperature changes depending on the hour of the day.

Michal Valko 马悯伽 avatar
⟳ Kashif Rasul retweeted
Michal Valko 马悯伽@misovalko·Oct 23

Thx @_lewtun for the 1000x citation hack w/ a different IPO 😂 Our IPO (Identity Preference Optimization) was with @Learnius, Mark Rowland, @Piotbilal, Daniel Guo, Daniele Calandriello, @MunosRemi & me. Big 🤗 to TRL crew: @krasul @lvwerra @yb2698 @edwardbeeching @osanseviero. Kashif coded IPO into TRL at lightning speed (and later fixed the loss normalization!), Lewis & Leandro reviewed, Younes brought us together, Ed reran the experiments, and Omar told the story. Daniele and I did remote the debugging in HF Slack. And tx to that we saw research turn into code people actually run <3 Go go open-source! And some of the best memories happened offline: our IPO talk at Hugging Face, lunch in Paris w/ @mishig25 @HugoLaurencon @LoubnaBenAllal1, Midnight Runners, Paris AI running club, and dinner w/ Omar. Thx also to @SanhEstPasMoi @_philschmid @julien_c @Thom_Wolf for all the great conversation and a free t-shirt ;) Our IPO paper: arxiv.org/abs/2310.12036

The IPO paper is about to get 1000x more citations ;)

Post attachment
Abdennacer Badaoui avatar
⟳ Kashif Rasul retweeted
Abdennacer Badaoui@abdennacer0·Oct 21

AMD GPUs are now officially supported in TRL 🚀 I spent the past weeks running every TRL trainer on MI300X and fixing what broke, across TRL, Transformers, the Hub kernels, AITER and bitsandbytes 🧵

Post attachment
Emma Scharfman avatar
⟳ Kashif Rasul retweeted
Emma Scharfman@EmmaScharfmann·Oct 20

Do we really need bigger weather models to get higher-resolution forecasts? Often, we don’t. We trained a small downscaler in about an hour on a single GPU that takes temperature forecasts over France from ~28 km to 5.5 km, and we wrote a blog post to show you how: huggingface.co/blog/EmmaScha…

Adithya S K avatar
⟳ Kashif Rasul retweeted
Adithya S K@adithya_s_k·Sep 30

1/ In RL environments, reward design is everything. Here's what we learned from the 7,780 environments Xiaomi open-sourced for MiMo 🧵

Post attachment

Wake up, ppl 👀 Xiaomi just open-sourced ~7K of the RL environments it used to train MiMo on HuggingFace, across domains like code, cyber, general, music and web dev I think this is one of the biggest things to happen in frontier open-source RL environment data. Indepth analysis and learnings coming soon 👀

Post attachment
Dirhousssi Amine avatar
⟳ Kashif Rasul retweeted
Dirhousssi Amine@DirhousssiAmine·Sep 23

⚡ Working on a major speedup for AsyncGRPO in TRL: training steps are up to 3.5× faster so far! AsyncGRPO already has several packing optimizations. One packs samples together and balances the attention cost across DP ranks. You start with the same prompt G times. In multi-turn rollouts on a harness, the conversation can also start to fork: most of the history remains identical, while a tool result or a new turn changes the suffix. That gets expensive with GRPO because rollouts can share many long prefixes. You start from the same prompt G times, and multi-turn rollouts on a Harness can also share most of their conversation history (thanks to message-level tokenization). Each rollout is treated as its own sequence and packed on a single row, subject to a predefined token budget per DP-rank. Those rows are forwarded independently, even when most of their tokens are identical. Tree packing instead turns the rows into a prefix tree. Shared prefixes are stored and forwarded once, while we use FlexAttention to ensure that every token still sees exactly the context it would have seen in its original rollout. The loss is still defined on the original rollouts, so every trained token keeps its own target, advantage, and old log-prob. The video shows the algorithm pretty well 👇

1:20
Arthur Zucker avatar
⟳ Kashif Rasul retweeted
Arthur Zucker@art_zucker·Sep 21

Happy to officially bring you the new SOTA `tokenization` library. We focused on all languages, multi-thread scaling, minimal package size and memory usage. huggingface-tokenizers-v1.st… We're thankful for the players of this ecosystem that have pushed us to give the best we could!

Post attachment
Emma Scharfman avatar
⟳ Kashif Rasul retweeted
Emma Scharfman@EmmaScharfmann·Sep 8

Can you predict next week's weather yourself? We just wrote a blog post that explains how! ⛅️🌍🌧️ Running weather forecasting required having a supercomputer and CS/physics Phd. Now, AI-based models need fewer resources and are available to anyone on @huggingface. So, prediction the weather should be accessible to all. But only a few labs can actually run these models because there is no standardized documentation guiding you from the model weights to an actual forecast. Together with @EarthmoverHQ, we wrote a blog post and a demo walking you through that: huggingface.co/blog/hugging-… The blog covers the code for fetching initial conditions from the Earthmover Marketplace, building a model's input batch, running inference and comparing the forecast against historical climate data. You can also run Aurora (Microsoft), WeatherNext2 (Google) and AIFS (ECMWF) on our demo space. This is a first step to make AI-based weather forecasting models more accessible, and we'd love to build more of that. Tell us what's missing or your feedback in the blog comments!

Huiying Li avatar
⟳ Kashif Rasul retweeted
Huiying Li@huiying_lii·Aug 24

Nvidia NeMo AutoModel now supports DFlash2 training 🚀 DFlash 2 keeps the best part of block-parallel drafting while improving candidate selection and refinement, delivering 25–37% higher acceptance length and 27–39% higher single-request throughput than DSpark on the published Qwen3.8-27B benchmarks. AutoModel now includes the DFlash 2 model, trainer, recipe, Qwen3/Qwen3.5 support, and multi-GPU training, so you can train your own drafter instead of relying on a prebuilt checkpoint. github.com/NVIDIA-NeMo/Autom… Many thanks to @krasul and @Chonghan_Liu for adding the support!

Emma Scharfman avatar
⟳ Kashif Rasul retweeted
Emma Scharfman@EmmaScharfmann·Aug 21

Very cool open source model available on Hugging Face: Samudra 2 runs multi-year ocean forecasts (temperature, salinity, sea surface height,...) at resolutions down to 1/4° and on a single GPU. If you're working on similar models, feel free to reach out. We're thinking about build a benchmark for ocean and atmospheric forecast models 🌍🌊 Try it here: huggingface.co/spaces/huggin… Model weights: huggingface.co/M2LInES/Samud…

Dirhousssi Amine avatar
⟳ Kashif Rasul retweeted
Dirhousssi Amine@DirhousssiAmine·Aug 13

Same reward curve. A quarter of the clock! 🚅 TRL's AsyncGRPOTrainer decouples rollouts from training: a background worker streams completions from a vLLM server while the training loop consumes them. The bench is the well-crafted sail/Sanity-Test-R1D-1.5B. 3.8× on the clock. 1.9× on GPU-hours.

0:18
Sergio Paniego avatar
⟳ Kashif Rasul retweeted
Sergio Paniego@SergioPaniego·Jul 23

you can now train DiffusionGemma (a block-diffusion LLM) in TRL! and we're sharing an example for it TRL trainers are made to be easily extended and adapted to different real-world use cases. in this one, with a single method overridden in SFTTrainer (compute_loss), you can train this model example below

0:08