EP 005 — When AI Helps — Until It Doesn’t

AI-generated voices. Human judgment. Pie’s voice is generated from her own recordings with her consent; the AI co-host uses an AI-generated synthetic voice. Every episode is shaped and approved by Pie—the human behind AI with Pie. A useful answer can make the next handoff feel easy. But what, exactly, are you trusting AI to do? Pie and her AI co-host examine a dated GPT-4 experiment, challenge the leap from a good result to a bigger decision, and offer three practical ways to keep the help without skipping the evidence. This is not a review of today's models. Case and evidence note This is a scripted editorial conversation, not a recording of study participants. The episode discusses a 2023 randomized experiment and distinguishes the reported findings from AI-with-Pie interpretation and practical advice. Research: Dell'Acqua et al., Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality, Organization Science. https://doi.org/10.1287/orsc.2025.21838 Artwork transparency The Show Cover includes AI-assisted imagery, art-directed and approved by Pie. Website https://aiwithpie.com Chapters (00:00) AI-generated voices. Deliberate human direction. (00:12) Episode highlight: same model, different task (00:18) Faster, better work—and the next job (00:39) What was actually tested (01:10) A useful tool, a tempting conclusion (02:09) The turn: when AI hurt accuracy (03:07) Where confidence gets ahead of the check (04:09) A better handoff (05:05) Three Things to Go (05:54) Follow AI with Pie (06:00) AI with Pie. Useful. Awkward. Fun.