077 - What Is AI Training?

How does an AI model learn in the first place? This episode builds on the show's earlier episodes on tokens, embeddings, attention, and Transformers to explain training: the process that turns a model's huge collection of adjustable parameters into something useful. For a language model, training centers on a surprisingly simple task: predicting the next token. Using the illustrative example "The cat sat on the...", the episode shows a model producing probabilities across possible next tokens, with the gap between its prediction and the correct answer measured by a loss function. Lower loss means a better prediction, and backpropagation is how the model figures out which of its millions or billions of parameters contributed to that error and in which direction each one should change, by calculating gradients. The episode explains how frameworks like PyTorch use automatic differentiation to calculate those gradients through a model's computation graph via the chain rule, instead of deriving equations by hand, and how an optimizer such as gradient descent then nudges each parameter in the direction that reduces loss, using the metaphor of moving downhill across a loss landscape. It walks through the full loop, batch by batch: predict, measure the loss, backpropagate, update parameters, repeat, with an epoch marking one complete pass through the training data. It also draws a clear line between training and inference: during training the model's errors change its parameters, while during inference those parameters are fixed, so a chatbot responding to a question is running on weights it already learned rather than learning from that individual conversation. The episode closes on two practical points: models can overfit, performing well on training data but worse on new data, and data quality shapes outcomes as much as scale does, since bad, duplicated, biased, or contaminated data can produce bad results regardless of model size. Sources & References PyTorch: The Fundamentals of Autograd — https://docs.pytorch.org/tutorials/beginner/introyt/autogradyt_tutorial.html PyTorch: Automatic Differentiation with torch.autograd — https://docs.pytorch.org/tutorials/beginner/basics/autogradqs_tutorial PyTorch: Optimizing Model Parameters — https://docs.pytorch.org/tutorials/beginner/basics/optimization_tutorial.html PyTorch: Quickstart — https://docs.pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html Li et al.: Mechanics of Next Token Prediction with Self-Attention — https://arxiv.org/abs/2403.08081 Voice narration is AI-generated.