Your Team Doesn't Need to Understand RAG. It Needs These 3 Things

Lovely listener, share with us your thoughts! Prudential in Singapore built an AI assistant for its insurance representatives. Six of its own experts graded the answers, and 59% failed. Four months later 99% passed. Nobody on that team had to learn how the model works. A solo episode of Curiosity Code on what people in a financial firm actually need to know about RAG and AI agents. Most need to know how to use it and where it goes wrong. The firm needs three things: an owner who decides what a correct answer is, a set of graded questions it reruns whenever something changes, and a register of which model wrote which answer. I show the three ways an answer goes wrong on a small mock-up (a made-up travel insurer, a real model), then what OpenAI, Anthropic and Singapore's regulator have published about it. Three questions for your team: Which model produced the last AI answer that someone here relied on? If that model changed tomorrow, how would we find out? And who decides what a correct answer is? Chapters 0:00 An AI assistant that failed 59% of the time 0:24 What most people need to know, and what they don't 1:20 Why Prudential's assistant scored 41% 2:35 Three ways an answer goes wrong 6:27 Who does the work 7:28 Three questions for Monday Episodes mentioned Stéphane Malrait, "Burning Cash Too Fast? Why Most Fintechs Collapse Within 3 Years": https://youtu.be/iXeaZOwQ9SU Daniel Flowe, "The DARK Side of AI in Finance, Can We Trust It?": https://youtu.be/dUBv8jMKlSY Sources Project MindForge, AI Risk Management Implementation Examples (the Prudential case): https://www.mas.gov.sg/-/media/mas-media-library/schemes-and-initiatives/ftig/project-mindforge/mindforge-ai-risk-management-implementation-examples.pdf Jeff McMillan of Morgan Stanley on the RegulatingAI podcast (write-up, January 2026): https://americanbazaaronline.com/2026/01/26/jeff-mcmillan-on-morgan-stanleys-human-first-ai-governance-473926/ OpenAI, API deprecations page (read 9 October 2026): https://developers.openai.com/api/docs/deprecations Anthropic, "A postmortem of three recent issues" (17 September 2025): https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues Monetary Authority of Singapore, Guidelines on Artificial Intelligence Risk Management (October 2026): https://www.mas.gov.sg/-/media/mas-media-library/regulation/guidelines/bd/guidelines-on-artificial-intelligence-risk-management-for-financial-institutions/guidelines-on-artificial-intelligence-risk-management.pdf 🎙️ Subscribe for weekly episodes on fintech, AI, and innovation 🌐 Website: https://curiositycodepodcast.com/ 📱 Follow on LinkedIn: https://www.linkedin.com/in/alexkhomyakov/ 🎧 Listen on Spotify: https://open.spotify.com/show/4ePKf2qCLzcKATS7o6w2U5 🍎 Apple Podcasts: https://podcasts.apple.com/podcast/id1797349967 The mock-up in this video is an illustration: the insurer and its policy wording are made up, the model output is real (Claude Opus 5.5, 9 October 2026). Music: "Soundbed" by Blue Saga, via Epidemic Sound. 🎙️ Subscribe for weekly episodes on fintech, AI, and innovation 🌐 Website: https://curiositycodepodcast.com/ 📱 Follow on LinkedIn: https://www.linkedin.com/in/akhomyakov/ 🎧 Listen on Spotify: https://open.spotify.com/show/curiosity-code 🍎 Apple Podcasts: https://podcasts.apple.com/podcast/the-curiosity-code-podcast/id1748596092