Why Every AI Model Is Secretly a Decoder

Encoder-only, encoder-decoder, decoder-only — three shapes, one skeleton. This episode pulls apart the architectural fork at the center of modern language models: how attention masking and training objectives separate BERT from GPT, why the decoder-only design swallowed the field, and why encoder-only models still quietly run classification and retrieval at a fraction of the cost. Along the way: the four unrelated things this field calls "encoding," the parts bin that top labs actually assemble models from, and where the real architectural innovation is happening now. Key sources: • Google Developers Blog, T5Gemma: A new collection of encoder-decoder Gemma models, 2025-07-09 — https://developers.googleblog.com/en/t5gemma/ • Yi Tay, What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives — https://www.yitay.net/blog/model-architecture-blogpost-encoders-prefixlm-denoising • arXiv:2510.26622, Encoder-Decoder or Decoder-Only? Revisiting Encoder-Decoder Large Language Model, 2025-10-30 — https://arxiv.org/abs/2510.26622 Episode 6014 — myweirdprompts.com/n/6014 (permanent short link: myweirdprompts.com/609796)