←
BURKOV25,794 posts
BURKOV avatar

BURKOV

@burkov

Books: theLMbook.com & theMLbook.com App: ChapterPal.com PhD in AI, author of 📖 The Hundred-Page LMs Book & The Hundred-Page ML Book

◷ Joined June 2009▧ 4,723 Photos and videos
BURKOV avatar
BURKOV@burkov·May 22

As you know, last year I resigned from my full-time job and become a professional technical book writer. I have plans for The Hundred-Page Books about reinforcement learning, computer vision, diffusion models, and more. Not having a full-time job has put a significant strain on my savings, so if you love my books and are waiting for more, I would appreciate your continuous support. I have finalized the offer for those who financially support me. Here's what I offer: - As a Cool Supporter 10 🚀 ($10/month), you will get early access to new book chapters, exclusive updates on what to really care about in AI and why (only the important stuff, no hype), and a free e-book copy of all new books. - As a Cool Supporter 30 🚀 ($30/month), you will get everything a Cool Supporter 10 🚀 gets, plus a signed hard copy of all new books. The names of all Cool Supporters 🚀 will be mentioned in the Acknowledgment section of the new books as important contributors. Your name, printed in the book, will remain in history forever! If you decide to become a Cool Supporter 🚀, you can choose from several platforms. Here's the list: linktr.ee/burkov If you choose Substack, you will also get a subscription to a weekly AI newsletter without ads.

Post attachment
BURKOV avatar
BURKOV@burkov·52m

A free online book: LLM Inference Handbook by Sherlock Xu (2026). This book is a practical technical reference for engineers deploying, scaling, and operating language models in production. It brings together the concepts and decisions involved in turning a trained model into a responsive, cost-aware service, from model behavior and hardware selection to application interfaces and ongoing operations. Readers can use it as a guided progression or as a lookup resource for specific engineering questions. Read the book online on @ChapterPal: chapterpal.com/book/3fef12a8…

Post attachment
BURKOV avatar
BURKOV@burkov·2h

Deploying multimodal AI systems, which process both text and visual inputs, across cloud and mobile environments creates severe network bandwidth and storage pressures. Standard image compression algorithms are built primarily for human eyes, while specialized machine-oriented methods focus on narrow tasks like detecting specific objects. Consequently, existing compression approaches degrade the performance of versatile multimodal models across broad real-world tasks, creating an operational bottleneck between transmission efficiency and model accuracy. The article systematically analyzes how compression artifacts impair visual processing in multimodal models and demonstrates a specialized compression system called CoTAM to reduce data transmission costs without sacrificing reasoning performance. Read with an AI tutor: chapterpal.com/s/4cewpwpr/wh…

BURKOV avatar
BURKOV@burkov·7h

A free online book: Bash Guide for Beginners by Machtelt Garrels. Bash (short for Bourne-Again SHell) is a command-line shell and scripting language used to interact directly with an operating system through text commands. Created in 1989 by Brian Fox for the GNU Project, it is the default shell on most Linux distributions and is widely available on macOS and Windows. Bash Guide for Beginners is a practical introduction to shell scripting for people who already use a UNIX or UNIX-like command line and want to automate recurring work. It is especially relevant to power users and system administrators, whose routine tasks often benefit from scripts. The book assumes basic shell familiarity and teaches Bash through realistic examples and exercises intended to build confidence in writing and improving scripts. Read the book online on @ChapterPal : chapterpal.com/book/6abf80ef…

Post attachment
BURKOV avatar
BURKOV@burkov·7h

Generative models are often compared using held-out examples, but unlike classification, there is no single agreed measure of their overall quality. A score can also be misleading even when the metric is clearly defined: rare examples may strongly affect a metric while being absent from any practical test sample. The article asks which measures of model performance can be compared reliably using a finite set of independent examples from the real-world data distribution. The article develops a mathematical framework for this question and analyzes whether an evaluator can reliably rank two candidate models. It studies metrics based on tests of model outputs, measures of distributional difference, and perplexity, a common likelihood-based score. The results are theoretical: they apply under the framework’s assumptions about the data and candidate models, rather than arising from experiments on deployed systems. Read with an AI tutor: chapterpal.com/s/bux9rcvo/a-…

BURKOV avatar
BURKOV@burkov·9h

I really don't get why Jensen would want to be back as the CEO of the company that pushed him out. I would take the best people from Nvidia and start a new company. With his knowledge and money, he would build Nvidia 2.0 in a matter of a couple of months. Today, starting companies is not like when Jobs was pushed out of Apple by the board. It's so much easier and more straightforward. You just need cash and knowledge, and Jensen has both. He doesn't even need investors, though if he wanted them, he would have them lining up.

BURKOV avatar
BURKOV@burkov·10h

A free online book: The Social Design of Technical Systems: Building technologies for communities (2nd edition) by Brian Whitworth and Adnan Ahmad. The book presents computing as a socio-technical activity whose success depends on communities as well as hardware, software, and individual users. Its central claim is that social needs set the highest-level requirements for technologies built to support communities: reliability and usability matter, but a system succeeds only when people can use it to form fair, productive, and lasting social relationships. The book draws on general systems theory to connect technical, informational, human, and community perspectives, and treats these as distinct ways of understanding a system. Read the book online on @ChapterPal: chapterpal.com/book/16013bdd…

Post attachment
BURKOV avatar
BURKOV@burkov·12h

A free online textbook System Engineering Design by Rogier Wolfert, Ruud Binnekamp, Harold van Heukelum, and Yue Shang. The book teaches a formal method for turning broad societal needs into feasible engineering designs that reflect the preferences of the people affected by them. Its central concern is how to define a design problem before choosing a solution: identify stakeholders and their objectives, represent what they value, and account for physical, regulatory, and other constraints. The method aims to find a design configuration with the highest overall group preference, rather than selecting from a fixed set of alternatives prepared in advance. Architecture and urban planning provide the main application context, alongside examples from transport infrastructure and other engineering settings. Read the book online on @chapterpal: chapterpal.com/book/2055ecbe…

Post attachment
BURKOV avatar
BURKOV@burkov·12h

As stronger models are increasingly trained with labels supplied by weaker models, a central safety risk is that the stronger model may make confident mistakes where the weaker model cannot reliably judge the answer. Aggregate accuracy can conceal these failures. The article develops and tests confidence-based diagnostics intended to identify such “blind-spot” errors and connect them to a bias–variance–covariance view of weak-to-strong risk. The article’s objective is to determine which measurable properties of weak and strong models are associated with blind-spot deception: cases where the strong model is confidently wrong while the weak model is uncertain. The authors derive a theoretical upper bound on weak-to-strong population risk and decompose relevant quantities into bias, variance, and covariance. They then evaluate four training pipelines combining supervised fine-tuning, reinforcement learning from human feedback, and reinforcement learning from AI feedback. Experiments use two safety-focused preference datasets, each split into training, weak-label generation, and held-out evaluation sets, with 5,000 examples in each split. The models are from the Llama family, and the reported experiments average three random-seed runs. Read with an AI tutor: chapterpal.com/s/mfj8g86x/ev…

BURKOV avatar
BURKOV@burkov·12h

I distrust any book published in 2024 or later. When I see a book published in 2026 on a topic that I would like to have on ChapterPal, with a license allowing me to host a copy, I always think that it's too good to be true, so I ask ChatGPT to analyze the content and tell me if the book was written by AI. Most of the time, it was, so I pass. So annoying.

BURKOV avatar
BURKOV@burkov·14h

A free online book Introduction to Linux: A Hands on Guide by Machtelt Garrels. Linux is the most popular free, open-source OS used mainly to run internet servers and virtual machines. This interactive book presents a hands-on, exercise-driven introduction that helps newcomers build practical Linux skills—from basic commands and file editing to everyday system use—without assuming prior Unix expertise. Read this book online on @ChapterPal: chapterpal.com/book/8da4cbea…

Post attachment
BURKOV avatar
BURKOV@burkov·15h

This way, most of the creators will eventually be marked as slop because slop detection has a huge false-positive rate.

X actively checks accounts and content for LLM slop and marks them internally as “llm_slop_user”. So if you use AI for comments and content, I have very bad news for you. You can very quickly get the “SpamHighRecall” label, and your posts will end up in a shadowban. Therefore, create content yourself - especially when it comes to comments.

Post attachment
BURKOV avatar
BURKOV@burkov·17h

Robots trained from demonstrations and real-world experience face a data-quality problem: the data mix successful actions with hesitation, mistakes, and failed attempts. Treating every action as equally useful can limit learning offline and make updates unstable once a robot begins interacting with the world. The article investigates whether more reliable estimates of task progress and action quality can help robots make better use of this mixed experience across both stages. The article introduces Robo-ValueRL, a framework that estimates task progress using current and recent visual observations, then uses changes in those estimates to distinguish more and less helpful actions. The team tested the approach on precise chip insertion and block disassembly and sorting, using 240 hours of offline robotic data and more than 3,000 online rollout trajectories. The framework also measures value reliability in terms of whether estimates track overall task progress, remain stable during successful behavior, and respond appropriately to errors. Read with an AI tutor: chapterpal.com/s/q5ead43g/ro…

BURKOV avatar
BURKOV@burkov·22h

The article tests whether a language model can turn a game’s known rules into the weights of a small neural agent that succeeds on levels it has not seen. This could, in principle, preserve a model’s reasoning in a compact system that operates without further language-model calls and can be improved through training. The central result is more qualified: models can write working neural agents, but these agents behave much more like hand-coded programs than like networks produced by learning, and they generally perform worse than programs written by the same models. The study compared nine language models across 20 private game families, with held-out levels, visual styles, and starting states. Each model received simulator source with the held-out content and solution routines removed. It then wrote weights for a fixed network, designed a network of its own, or wrote a Python player; some attempts also received feedback on performance on permitted support levels. The researchers compared these outputs with networks trained on support-level examples and tested whether written networks could be improved with reinforcement learning. A separate evaluation on 25 public games was treated cautiously because models might already be familiar with them. Read with an AI tutor: chapterpal.com/s/24a59c2c/co…

BURKOV avatar
BURKOV@burkov·Nov 17

LLM post-training can spend more time generating long reasoning responses than updating the model. Running generation, teacher scoring, and training in parallel can improve hardware use, but it means the learner may train on data produced by an older version of the student. The article examines how this staleness affects on-policy distillation, where a teacher provides token-level guidance, and how to build a faster pipeline without sacrificing model quality. A practical constraint is central to the analysis: storing or transferring the teacher’s scores for every possible next token is too costly, so systems typically cache scores for only a limited set of actions. The article systematically tests how different distillation objectives and ways of caching teacher feedback behave as rollout data grows older. Experiments use Qwen3 student models, a Qwen3 teacher, and a filtered set of 57,630 challenging math problems; performance is evaluated on AIME and AMC mathematics benchmarks. The researchers vary the age of the rollout data, compare alternative update methods, and build AsyncOPD, a pipeline that overlaps rollout generation, teacher scoring, and learning. Read with an AI tutor: chapterpal.com/s/mbsm65gf/as…

BURKOV avatar
BURKOV@burkov·Nov 17

When rewards improve with repeated investment, choosing the best option depends on how much time remains. A model that takes longer to improve may pay off over a long training budget but be a poor choice when the deadline is near. The article examines whether decision-making algorithms can use a known time horizon to make better sequential choices in these rising-reward settings, which arise in tasks such as model selection and machine-learning training. The article’s main objective is to measure the value of horizon awareness and develop an algorithm that uses it. The authors propose CURE-UCB, which estimates each option’s likely cumulative reward over the remaining time, rather than judging it only by its current reward or its projected reward at a single future point. They analyze the method mathematically and compare it with rising-reward and general non-stationary alternatives in synthetic tests and model-selection tasks. Read with an AI tutor: chapterpal.com/s/nkejynkl/ri…