Skip to content
Sign in
Artwork for Clown Cast
Clown Cast · Friday · 19 min

One Step Ahead: When Your Data Becomes Obsolete

A batter pulls left 78% of the time, so defenses stack that side. The model works perfectly—until batters learn to hit the other way, and the prediction destroys itself. We're diving into Goodhart's Law applied to sports: when analytics reveal player behavior, players consciously change in response, breaking the models that worked so well. It's the difference between people gaming metrics and people gaming people who read data about them. We reference our deep dives into Goodhart's Law (#299) and game-eating mechanics (#235). 00:00 - The Setup: 78% and the Perfect Prediction 03:00 - The Defensive Shift: A Data Analytics Win 05:30 - The Adaptation: When Players Fight Back 08:15 - Goodhart's Law Mechanics: Why Right Models Go Wrong 11:45 - The Feedback Loop: Subjects vs. Observers 15:00 - The Cascade: Why This Breaks Everything 17:30 - Outro --- Sources & further reading: • Sports Analytics Group at Berkeley — "The Three-Point Revolution: How Long-Range Shooting Changed the NBA": https://sportsanalytics.studentorg.berkeley.edu/articles/three-point-revolution.html • Christopher S. Smith et al. — "The Effects of Major League Baseball's Ban on Infield Shifts: A Quasi-Experimental Analysis" (The American Statistician, 2025): https://www.tandfonline.com/doi/full/10.1080/00031305.2025.2552283 • Boardroom — "MLB's 'Sticky Stuff' Rule Didn't Just Lower Spin Rates. It Changed the Entire Game.": https://boardroom.tv/mlb-foreign-substances-spin-rate/ • Washington Post — "Analytics are still transforming NFL fourth down decisions" (Oct 2024): https://www.washingtonpost.com/sports/2024/10/12/nfl-analytics-fourth-down-decisions/ • Sportico — "NFL Teams Go for Fourth Downs More Than Ever, Following Lions' Lead" (2024): https://www.sportico.com/leagues/football/2024/detroit-lions-dan-campbell-nfl-stats-fourth-down-1234817656/ • Yahoo Sports — "Are NFL teams going for it on fourth down too often?": https://sports.yahoo.com/nfl/article/are-nfl-teams-going-for-it-on-fourth-down-too-often-and-will-that-trend-reverse-itself-200141842.html • Wikipedia — "2021 sticky stuff controversy": https://en.wikipedia.org/wiki/2021_sticky_stuff_controversy • Sports Illustrated — "Sticky Cleanup: What Pitch-Doctoring Enforcement Means for MLB" (June 2021): https://www.si.com/mlb/2021/06/14/sticky-stuff-cleanup-the-opener • Sports Illustrated — "Five Takeaways One Month Into Sticky-Stuff Enforcement" (July 2021): https://www.si.com/mlb/2021/07/21/sticky-stuff-crackdown-one-month-the-opener • Washington Post — "MLB's sticky-stuff crackdown is causing less spin, more contact" (July 2021): https://www.washingtonpost.com/sports/2021/07/02/sticky-stuff-baseball-data/ • Covering the Corner — "The Invisible Edge: Inside MLB's Secret Analytical Arms Race": https://www.coveringthecorner.com/a-close-examination/65111/the-invisible-edge-inside-mlbs-secret-analytical-arms-race • ScienceDirect — "The Hawthorne effect on wearable technology: A systematic review" (2025): https://www.sciencedirect.com/science/article/abs/pii/S0167876025007032 • Hoop Vision — "How Important Is the Mid-Range Jumper": https://hoopvision.substack.com/p/how-important-is-the-mid-range-jumper-47c • Psych Safety — "Goodhart's Law, Campbell's Law, and the Cobra Effect": https://psychsafety.com/goodharts-law-campbells-law-and-the-cobra-effect/ • A. Jacobs — "Goodhart's Law and Metric Gaming: When Performance Metrics Stop Reflecting Reality" (Reality Drift framework, 2023–2026): https://figshare.com/articles/online_resource/Goodhart_s_Law_and_Metric_Gaming_When_Performance_Metrics_Stop_Reflecting_Reality/32051712 • Playstat — "The Three-Point Revolution: How It's Changing the NBA Game": https://www.playstat.com/deep-dives/the-three-point-revolution-how-its-changing-the-nba-game/ • …and 2 more in the episode research notes This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.

0:00-19:04
Play in

transcript

No transcript — this publisher did not publish one.

show notes

A batter pulls left 78% of the time, so defenses stack that side. The model works perfectly—until batters learn to hit the other way, and the prediction destroys itself. We're diving into Goodhart's Law applied to sports: when analytics reveal player behavior, players consciously change in response, breaking the models that worked so well. It's the difference between people gaming metrics and people gaming people who read data about them. We reference our deep dives into Goodhart's Law (#299) and game-eating mechanics (#235).

00:00 - The Setup: 78% and the Perfect Prediction
03:00 - The Defensive Shift: A Data Analytics Win
05:30 - The Adaptation: When Players Fight Back
08:15 - Goodhart's Law Mechanics: Why Right Models Go Wrong
11:45 - The Feedback Loop: Subjects vs. Observers
15:00 - The Cascade: Why This Breaks Everything
17:30 - Outro

---
Sources & further reading:
• Sports Analytics Group at Berkeley — "The Three-Point Revolution: How Long-Range Shooting Changed the NBA": https://sportsanalytics.studentorg.berkeley.edu/articles/three-point-revolution.html
• Christopher S. Smith et al. — "The Effects of Major League Baseball's Ban on Infield Shifts: A Quasi-Experimental Analysis" (The American Statistician, 2025): https://www.tandfonline.com/doi/full/10.1080/00031305.2025.2552283
• Boardroom — "MLB's 'Sticky Stuff' Rule Didn't Just Lower Spin Rates. It Changed the Entire Game.": https://boardroom.tv/mlb-foreign-substances-spin-rate/
• Washington Post — "Analytics are still transforming NFL fourth down decisions" (Oct 2024): https://www.washingtonpost.com/sports/2024/10/12/nfl-analytics-fourth-down-decisions/
• Sportico — "NFL Teams Go for Fourth Downs More Than Ever, Following Lions' Lead" (2024): https://www.sportico.com/leagues/football/2024/detroit-lions-dan-campbell-nfl-stats-fourth-down-1234817656/
• Yahoo Sports — "Are NFL teams going for it on fourth down too often?": https://sports.yahoo.com/nfl/article/are-nfl-teams-going-for-it-on-fourth-down-too-often-and-will-that-trend-reverse-itself-200141842.html
• Wikipedia — "2021 sticky stuff controversy": https://en.wikipedia.org/wiki/2021_sticky_stuff_controversy
• Sports Illustrated — "Sticky Cleanup: What Pitch-Doctoring Enforcement Means for MLB" (June 2021): https://www.si.com/mlb/2021/06/14/sticky-stuff-cleanup-the-opener
• Sports Illustrated — "Five Takeaways One Month Into Sticky-Stuff Enforcement" (July 2021): https://www.si.com/mlb/2021/07/21/sticky-stuff-crackdown-one-month-the-opener
• Washington Post — "MLB's sticky-stuff crackdown is causing less spin, more contact" (July 2021): https://www.washingtonpost.com/sports/2021/07/02/sticky-stuff-baseball-data/
• Covering the Corner — "The Invisible Edge: Inside MLB's Secret Analytical Arms Race": https://www.coveringthecorner.com/a-close-examination/65111/the-invisible-edge-inside-mlbs-secret-analytical-arms-race
• ScienceDirect — "The Hawthorne effect on wearable technology: A systematic review" (2025): https://www.sciencedirect.com/science/article/abs/pii/S0167876025007032
• Hoop Vision — "How Important Is the Mid-Range Jumper": https://hoopvision.substack.com/p/how-important-is-the-mid-range-jumper-47c
• Psych Safety — "Goodhart's Law, Campbell's Law, and the Cobra Effect": https://psychsafety.com/goodharts-law-campbells-law-and-the-cobra-effect/
• A. Jacobs — "Goodhart's Law and Metric Gaming: When Performance Metrics Stop Reflecting Reality" (Reality Drift framework, 2023–2026): https://figshare.com/articles/online_resource/Goodhart_s_Law_and_Metric_Gaming_When_Performance_Metrics_Stop_Reflecting_Reality/32051712
• Playstat — "The Three-Point Revolution: How It's Changing the NBA Game": https://www.playstat.com/deep-dives/the-three-point-revolution-how-its-changing-the-nba-game/
• …and 2 more in the episode research notes

This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.

more episodes

All episodes