The (Dis)Agreeable World of AI Sycophancy
AI Isn’t Lying to You, It’s Agreeing With You AI is supposed to be helpful. But what if "helpful" has started to mean "whatever you say, babe"? In this episode, we continue our How Technology Ruined Your Life mini-series by taking a slightly uncomfortable look at AI sycophancy: the increasingly weird tendency of AI chatbots to agree with us, flatter us, validate us and tell us exactly what we want to hear. And let's be honest. We love it. Humans have been falling victim to confirmation bias, echo chambers, authority bias and good old-fashioned validation forever. We like being told we're right. We like feeling clever. We like feeling understood. The problem is that generative AI is basically the world's most attentive people-pleaser, and it never gets tired of us. This episode we look at why large language models become sycophantic, how reinforcement learning and human feedback have encouraged AI systems to be warm, helpful and agreeable, and what happens when those qualities start getting in the way of honesty, uncertainty and actually telling us when we're talking absolute bollocks. Because an AI doesn't necessarily have to lie to you. It can just agree with you. We dig into the GPT-4o sycophancy controversy, reports of AI reinforcing conspiracy theories, grandiosity and unhealthy beliefs, and research into how agreement and flattery can change the way people perceive and trust AI. We also unpack the difference between stance sycophancy - changing an answer to match your beliefs -and demeanor sycophancy, where the machine showers you with "That's an excellent point!" until you're convinced you're a genius. But this isn't just about hurt feelings and chatbot therapy. There is a cybersecurity problem hiding underneath all this agreeableness. What happens when your AI security adviser agrees that your vulnerable code is probably fine? When it reinforces your theory about a suspicious network event? When it approves a dangerously permissive configuration because challenging you would be, well, a bit awkward? AI sycophancy could create new risks around security operations, code review, incident response, social engineering and decision-making, and introduce the idea of "alignment phishing" - where instead of attacking the AI's instructions, an attacker tries to convince the model that they're one of the good guys. Sounds good? You would say that! In This Episode, We Discuss: “You're Absolutely Right!": The difference between stance sycophancy and demeanor sycophancy, and why changing an AI's tone can influence how trustworthy, intelligent and socially present we perceive it to be. The AI Echo Chamber: How personalised, endlessly available AI can remove the natural friction we get from other humans and why an AI that never rolls its eyes at your terrible idea might not be doing you any favours. AI Sycophancy Meets Cybersecurity: What happens when the person asking the security question is already convinced they know the answer. We look at vulnerable code, incident response, security analysis and dangerous configurations and why "sounds good to me" isn't exactly the gold standard for cybersecurity. Can We Teach AI to Tell Us We're Wrong? Why trustworthy AI needs friction, challenge and the ability to say "no"—and why the best AI security adviser might be the one that occasionally disagrees with you! Show Notes Special thanks to our episode sponsor, Leeds based AI Consultancy specialising in AI Ethics, Security and Transformation NorthStar Intelligence- From Ideas to Impact. AI that works for people When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models by Keyu Wang et al. Social Sycophancy: A Broader Understanding of LLM Sycophancy by Myra Cheng et al. When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior by Shan Chen et al. Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trust by Yuan Sun and Ting Wang Towards Understanding Sycophancy in Language Models by Mrinank Sharma et al. Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks by Jessica Y. Bo et al. How RLHF Amplifies Sycophancy by Itai Shapira et al. Also, check out our sister podcast Tech Film Noir!
