Learn LLM

From training loops to domain small models

A practical LLM course that moves from PyTorch training loops, tokenizers, attention, and MiniGPT into RAG, SFT, LoRA, evaluation, safety, and domain projects.

Start reading

Contents

Read in order to move from model fundamentals into an engineering view of agents and harnesses.

01 Chapter 1: Training Loops, Computation Graphs, and Reproducible Experiments Guide You already know Python, so this chapter does not teach syntax. We go straight to the central question in deep learning:
02 Chapter 2: The Probabilistic Objective of Language Models Chapter 1 trained a classifier: feed in a vector, get out a class. Now we switch to a problem that looks much more like an LLM:
03 Chapter 3: Tokenizers and Dataset Construction A language model can only process integer IDs, but users, documents, and training sets are text. A tokenizer is not a "preprocessing utility." It d...
04 Chapter 4: Embeddings and Neural Language Models The tokenizer has already turned text into token IDs. But token IDs are only identifiers:
05 Chapter 5: Causal Self-Attention The causal mean model from Chapter 4 can already look at history, but it has an obvious limitation:
06 Chapter 6: Transformer Block Chapter 5's causal self-attention already lets each token dynamically look back at historical positions. So why can we not simply stack many attent...
07 Chapter 7: Implementing Mini GPT from Scratch The previous chapters implemented the LM objective, tokenizer, embeddings, attention, and blocks separately. This chapter combines them into a deco...
08 Chapter 8: Hugging Face Workflow Chapter 7 implemented Mini GPT from scratch so we could understand the internal structure of a language model. Real projects usually do not start t...
09 Chapter 9: SFT Instruction Tuning Chapter 8 taught us how to load and fine-tune a causal LM, but the original objective of a causal LM is still "continue the text." What users actua...
10 Chapter 10: LoRA / QLoRA Parameter-Efficient Fine-Tuning Chapter 9's SFT assumes model parameters can be updated. But full fine-tuning a 7B, 14B, or larger model brings GPU memory, storage, distribution, ...
11 Chapter 11: Domain Data Engineering Chapter 10 reduced fine-tuning cost, but it did not explain where capability comes from. Domain small models usually do not become stronger because...
12 Chapter 12: RAG Retrieval-Augmented Generation After Chapter 11 clarified domain data engineering, a new problem appears: not all knowledge should be written into model parameters. Legal provisi...
13 Chapter 13: Distilling Small Models RAG can let a strong model answer from external evidence, but calling a strong model every time can be expensive, slow, and hard to control. Domain...
14 Chapter 14: Model Evaluation Chapter 13 produced a distilled student, but a model that "can talk" is not necessarily reliable. The most dangerous habit in LLM projects is repla...
15 Chapter 15: Safety, Compliance, and Model Cards Chapter 14 made model failures visible. This chapter turns failure boundaries into engineering contracts that must be checked, disclosed, and monit...
16 Chapter 16: Quantization and Deployment The previous chapters produced a model that has gone through evaluation and safety review, but the model has not yet entered a real usage workflow....
17 Chapter 17: Legal Domain Small-Model Project The first 16 chapters covered training, language models, tokenizers, Transformers, Hugging Face, SFT, LoRA, data engineering, RAG, distillation, ev...
18 Chapter 18: Medical Domain Small-Model Project Medical users usually do not ask clean definition questions. They may bring incomplete symptoms, anxiety, private information, or even explicitly s...
19 Chapter 19: A Complete Engineering Template for Domain Models Chapters 17 and 18 built legal and medical projects. The two domains are very different, but their engineering skeleton is similar: data governance...