S1E07. Teaching an Old Model New Tricks: Transfer Learning, LoRA, and RLHF

This episode is narrated using AI voice technology. The content and script are original. You do not build one of these from scratch, and the reason is not only money. You take a model that already exists and reuse what it already learned. The early layers are general. The specific part sits at the far end, near the answer. That reuse is transfer learning. Rob sorts five stages by how much of your own material they need and who in the world actually runs them. Original training on a web-scale pile, next word over and over, self-supervised. Continued pretraining on one field. Supervised fine-tuning on written pairs, which is the first stage a normal organisation actually runs. Then preference alignment: you cannot write down the right answer to kindly, but you can pick the better of two attempts in a second. That loop is RLHF. Finishing school. A much smaller model put through it was preferred to a far bigger one that was not. Then the cheap adapters. PEFT. LoRA trains a small piece and leaves the rest frozen. QLoRA squashes the frozen copy first. Distillation teaches a small model to imitate a large one. Takeaway: somebody else paid for the years; you pay for the menu. Subscribe for the rest of the season, and visit cloudadorn.com.