Skip to content
Sign in
Artwork for Clown Cast
Clown Cast · September 16 · 17 min

The Keys Inside the Burning House

You build a backup to survive when things fail. But what if the backup secretly needs the same thing that's broken? That's a common mode failure, and it's destroyed data centers, grounded airlines, and hid inside Microsoft's recovery tools. This episode explores why engineers keep building confidence tricks instead of real safety nets—and why your backups might be on your kitchen counter inside the locked house. 0:00 - The spare key metaphor 1:30 - What is common mode failure? 3:45 - Nuclear safety systems: 20-80% of failures are hidden 5:15 - The Microsoft BitLocker disaster 8:00 - CrowdStrike outage: $10 billion in damages 12:00 - Why this pattern keeps repeating --- Sources & further reading: • Charles Perrow — Normal Accidents: Living with High-Risk Technologies (1984) — (book) • Nancy Leveson — "A New Accident Model for Engineering Safer Systems": http://sunnyday.mit.edu/accidents/safetyscience-single.pdf • G. T. Edwards and I. A. Watson — "A Study of Common Mode Failures" (1979), UK Atomic Energy Authority • IBM — "What You Should Know About the CrowdStrike Outage": https://www.ibm.com/think/news/recent-crowdstrike-outage-what-you-should-know • Cybersecurity News — "CrowdStrike Outage Loss $5.4 Billion": https://cybersecuritynews.com/crowdstrike-outage-loss-5-4-billion/ • Directions on Microsoft — "BitLocker Recovery Issues Revealed During CrowdStrike Outage": https://www.directionsonmicrosoft.com/reports/bitlocker-recovery-issues-revealed-during-crowdstrike-outage/ • Gremlin — "After the Retrospective: 2017 Amazon S3 Outage": https://www.gremlin.com/blog/the-2017-amazon-s-3-outage • Wikipedia — "2021 Facebook Outage": https://en.wikipedia.org/wiki/2021_Facebook_outage • IEEE Spectrum — "The Blackout of 2003": https://spectrum.ieee.org/the-blackout-of-2003 • ScienceDirect — "Cascading Risks from Winter Storm Uri": https://www.sciencedirect.com/science/article/pii/S2214629621001997 • Texas Tribune — "Texas Power Grid Winter Storm 2021": https://www.texastribune.org/2022/02/15/texas-power-grid-winter-storm-2021/ • World Nuclear Association — "Fukushima Daiichi Accident": https://world-nuclear.org/information-library/safety-and-security/safety-of-plants/fukushima-daiichi-accident • PMC — "Boeing 737 MAX Engineering Ethics": https://pmc.ncbi.nlm.nih.gov/articles/PMC7351545/ • Cal Poly — "Therac-25 Case Study": https://users.csc.calpoly.edu/~jdalbey/SWE/Papers/THERAC25.html • Henrico Dolfing — "The $440 Million Software Error at Knight Capital": https://www.henricodolfing.ch/en/case-study-4-the-440-million-software-error-at-knight-capital/ • DEV Community — "Four Instances, Two Data Centers, One Bug": https://dev.to/kkierii/four-instances-two-data-centers-one-bug-when-redundancy-isnt-independence-3dl8 • FedScoop — "FAA Now Has Backup System After 2023 Outage": https://fedscoop.com/after-2023-outage-that-paused-flights-nationwide-faa-now-has-backup-system/ • Aviation Today — "Safety Perils of Single-Point Failures": https://www.aviationtoday.com/2005/06/01/safety-perils-of-single-point-failures/ • Accendo Reliability — "Common Mode Failures": https://accendoreliability.com/common-mode-failures/ • E. P. Epler — "Diversity and Periodic Testing in Defense Against Common Mode Failure" (1977) • NASA — "Common Cause Failures": https://ntrs.nasa.gov/api/citations/20160005837/downloads/20160005837.pdf This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.

0:00-17:26
Play in

transcript

No transcript — this publisher did not publish one.

show notes

You build a backup to survive when things fail. But what if the backup secretly needs the same thing that's broken? That's a common mode failure, and it's destroyed data centers, grounded airlines, and hid inside Microsoft's recovery tools. This episode explores why engineers keep building confidence tricks instead of real safety nets—and why your backups might be on your kitchen counter inside the locked house.

0:00 - The spare key metaphor
1:30 - What is common mode failure?
3:45 - Nuclear safety systems: 20-80% of failures are hidden
5:15 - The Microsoft BitLocker disaster
8:00 - CrowdStrike outage: $10 billion in damages
12:00 - Why this pattern keeps repeating

---
Sources & further reading:
• Charles Perrow — Normal Accidents: Living with High-Risk Technologies (1984) — (book)
• Nancy Leveson — "A New Accident Model for Engineering Safer Systems": http://sunnyday.mit.edu/accidents/safetyscience-single.pdf
• G. T. Edwards and I. A. Watson — "A Study of Common Mode Failures" (1979), UK Atomic Energy Authority
• IBM — "What You Should Know About the CrowdStrike Outage": https://www.ibm.com/think/news/recent-crowdstrike-outage-what-you-should-know
• Cybersecurity News — "CrowdStrike Outage Loss $5.4 Billion": https://cybersecuritynews.com/crowdstrike-outage-loss-5-4-billion/
• Directions on Microsoft — "BitLocker Recovery Issues Revealed During CrowdStrike Outage": https://www.directionsonmicrosoft.com/reports/bitlocker-recovery-issues-revealed-during-crowdstrike-outage/
• Gremlin — "After the Retrospective: 2017 Amazon S3 Outage": https://www.gremlin.com/blog/the-2017-amazon-s-3-outage
• Wikipedia — "2021 Facebook Outage": https://en.wikipedia.org/wiki/2021_Facebook_outage
• IEEE Spectrum — "The Blackout of 2003": https://spectrum.ieee.org/the-blackout-of-2003
• ScienceDirect — "Cascading Risks from Winter Storm Uri": https://www.sciencedirect.com/science/article/pii/S2214629621001997
• Texas Tribune — "Texas Power Grid Winter Storm 2021": https://www.texastribune.org/2022/02/15/texas-power-grid-winter-storm-2021/
• World Nuclear Association — "Fukushima Daiichi Accident": https://world-nuclear.org/information-library/safety-and-security/safety-of-plants/fukushima-daiichi-accident
• PMC — "Boeing 737 MAX Engineering Ethics": https://pmc.ncbi.nlm.nih.gov/articles/PMC7351545/
• Cal Poly — "Therac-25 Case Study": https://users.csc.calpoly.edu/~jdalbey/SWE/Papers/THERAC25.html
• Henrico Dolfing — "The $440 Million Software Error at Knight Capital": https://www.henricodolfing.ch/en/case-study-4-the-440-million-software-error-at-knight-capital/
• DEV Community — "Four Instances, Two Data Centers, One Bug": https://dev.to/kkierii/four-instances-two-data-centers-one-bug-when-redundancy-isnt-independence-3dl8
• FedScoop — "FAA Now Has Backup System After 2023 Outage": https://fedscoop.com/after-2023-outage-that-paused-flights-nationwide-faa-now-has-backup-system/
• Aviation Today — "Safety Perils of Single-Point Failures": https://www.aviationtoday.com/2005/06/01/safety-perils-of-single-point-failures/
• Accendo Reliability — "Common Mode Failures": https://accendoreliability.com/common-mode-failures/
• E. P. Epler — "Diversity and Periodic Testing in Defense Against Common Mode Failure" (1977)
• NASA — "Common Cause Failures": https://ntrs.nasa.gov/api/citations/20160005837/downloads/20160005837.pdf

This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.

more episodes

All episodes