When Your AI API Quietly Changes Under You

You ran the evals, you pinned the version, and your invoice parser worked perfectly in October. Then December arrives and the same API name returns something subtly different. This episode digs into what actually changes behind an unchanged endpoint name — weights, system instructions, routing, classifiers, quantization, even the GPUs — and why an alias like "latest" is effectively a subscription to retesting. Drawing on model equality testing across 31 commercial Llama endpoints (11 of which produced distributions different from Meta's reference implementation) and the disclosure practices of major providers, we separate model identity from service behavior, explain why pinning a snapshot removes one major source of change without freezing everything, and offer a practical stance: stop asking whether the model got worse overall, and start asking whether the specific part you're paying for moved. Key sources: • OpenAI, Deprecations (notice periods, `shutdown_date`, upcoming deprecations) — https://developers.openai.com/api/docs/deprecations • OpenAI API changelog (Aug 2024 entries) — https://developers.openai.com/api/docs/changelog • Anthropic, Model IDs and versioning — https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions Episode 6190 — myweirdprompts.com/n/6190 (permanent short link: myweirdprompts.com/317755)