TestTalks.AI - The Testing AI Podcast

Automation Cyborg

”Saving the world from bad AI software” - a groundbreaking podcast series that delves into the rapidly evolving world of testing AI.

Join us as we navigate the cutting-edge technologies, methodologies, and practices that are shaping the future of the AI Assurance and Confidence Engineering landscape.

In this captivating series, we’ll be featuring thought leaders, industry experts, and practitioners who are at the forefront of innovation in software testing. Our engaging conversations will cover topics such as:

1️⃣ Artificial Intelligence and Machine Learning: Discover how AI and ML are revolutionizing test automation, analysis, and generation, providing testers with powerful new tools and capabilities.
2️⃣ The Human-Machine Collaboration: Uncover the ways in which testers can leverage AI technologies like Keysight Eggplant Test to enhance their skills, boost productivity, and ensure the highest quality software.
3️⃣ Testing in the Age of Agile and DevOps: Learn how modern development approaches have transformed the role of testers and how they can adapt to remain essential in fast-paced environments.
4️⃣ Emerging Testing Strategies: Dive into the latest methodologies, such as context-aware testing and test-driven development, and explore their potential to improve test coverage and effectiveness.
5️⃣ The Growing Importance of Domain Knowledge: Discuss the significance of integrating domain expertise into the testing process and how AI can help testers gain valuable insights into specific industries.
6️⃣ The Ethical and Social Implications of AI-Driven Testing: Examine the ethical concerns and societal impact of AI in software testing, and envision a responsible and inclusive future for the industry.

Don’t miss out on this thrilling journey into the future of software testing! Subscribe now to ”Test-Talks.com” and stay ahead of the curve as we redefine the boundaries of quality assurance. #TestingTomorrowToday #SoftwareTesting #FutureOfTesting #Podcast

Play

Open in

20 episodes
Avg 47 min
English

Episodes 20

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

#65
July 8 · 47 min
The AI Risk Boards Get Wrong — feat. David Subar (Interna)
The view from the board seat: why the biggest AI risk isn't a bad model — it's moving too slowly while your competitors learn faster. Alex Belotsky and co-host Kuba sit down with David Subar, founder of the LA consultancy Interna and a veteran board advisor, to unpack how AI is reshaping the decisions made in the boardroom. David argues that boards don't care which tool writes the code — they care about value creation and risk, the same math whether you ship in assembly or with Claude Code. He frames the product roadmap as "a series of bets" and explains why unconstraining development velocity forces companies to iterate faster while their risk surface balloons. The conversation digs into why stochastic systems are so hard to test (the boundaries "might be in the middle"), using Google's Gemini founding-fathers image fiasco as a billion-dollar cautionary tale, and why AI red teaming and reliability metrics now matter as much as pass/fail test cases. David and the hosts also tackle the "SaaS apocalypse," the flood of vibe-coded internal tools that smashes the QA problem across every department, Conway's Law and org design, Jevons paradox and developer jobs, and whether the real constraint on AI is TSMC and energy rather than talent. It's a candid, wide-ranging take on shipping reliable AI when the ground shifts every six months. **In this episode:** - The Board's View: Value Creation vs. Risk Reduction - Product Roadmaps As A Series Of Bets - Why Stochastic Systems Fail At The Boundaries - Lessons From Google's Gemini Image Controversy - From AI Red Teaming To Reliability Metrics - The SaaS Apocalypse And Vibe-Coded Internal Tools - Conway's Law, Org Design And Fewer Middle Managers - Jevons Paradox, Layoffs And The Future Of Developer Jobs - Shift-Left (Even Lefter): How QA Tests Non-Deterministic AI **Guest:** David Subar — Founder, Interna (board advisor, ex-CTO/CPO) **Host:** Alex Belotsky (CEO, TestSavant.AI) with co-host Jakub "Kuba" Fiatkevich > "you saw a worm in the ground you stepped on it now there's worm guts everywhere and someone's got to clean that up" 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AIGovernance #AIRiskManagement #BoardAdvisory #LLM #AIRedTeaming #ShiftLeft #VibeCoding #TestSavant #AIAssurance
#64
July 8 · 42 min
Guarding the Henhouse: Testing AI in Regulated Industries — feat. Erick Herring (Vynyl)
When your chatbot answers legal or medical questions, "close enough" isn't a spec — it's a lawsuit. Erick Herring of Vynyl joins Kuba and Alex to unpack what it actually takes to build and assure LLM-based systems for healthcare, lending, insurance and legal clients. Drawing on a career that spans punch cards, the birth of the commercial web and today's AI frenzy, Erick argues that the fundamentals of accountability haven't changed — but the way we hold code writers (now AI) accountable has, opening the door to formal verification, 100% test coverage and rigorous pre/post-condition testing that were never practical with human-only teams. He makes the case that "red teaming" is the wrong frame — this is AI assurance, spanning hallucination, bias and reliability, not just jailbreaks — and that QA can no longer be a discrete step but a continuous process fed by unified observability. Along the way: why guardrail code deserves more scrutiny than production code, the "ontological reframing" paper on talking an AI past its safety guardrails, the "yes, yes, yes" developer psychology that erodes PR review, and why QA engineers are more necessary now, not less. A grounded, contrarian take on shipping non-deterministic software you can actually stand behind. **In this episode:** - Testing AI Systems In Regulated Industries - AI Assurance Versus Red Teaming - Guardrails For Code And Runtime Systems - Continuous QA And Unified Observability - Formal Verification And Mutation Testing At Scale - Disclaimers, Liability And Legal Chatbots - Developer Psychology And The "Yes, Yes, Yes" Trap - Why QA Is More Necessary, Not Less **Guest:** Erick Herring — Founder, Vynyl (product development firm serving regulated industries) **Host:** Kuba Fiatkevich (CTO, TestSavant.AI) with co-host Alex Belotsky (CEO, TestSavant.AI) > "Our brains are not thinking machines. Our brains are justification machines." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AIAssurance #LLM #AITesting #RegulatedIndustries #RedTeaming #Observability #Guardrails #AIGovernance #Vynyl
#63
July 8 · 39 min
QA vs QE: Engineering Quality Into Non-Deterministic AI — feat. Janna Loeffler (Quality Engineering Leader)
When your system gives a different answer every time, "assuring" quality is a trap — you have to engineer it in. Janna Loeffler joins Alex and Kuba to unpack why the shift from quality assurance to quality engineering (and even "quality enablement") matters more than ever in the age of LLMs and agents. She argues that AI didn't kill the tester's job — it made human insight, risk judgement and context-driven thinking essential, because non-deterministic output turns testing from clean pass/fail into questions of accuracy, relevance and intent. Janna makes the case for an "army" of single-purpose agents that handle the mundane — regression, accessibility, OWASP Top 10 security checks — so testers can focus on outcomes over outputs. The conversation digs into the real danger of eroding guardrails, including a chilling example of a system talked past its safety limits within minutes, and why model providers like Anthropic and OpenAI are pushing safety responsibility down onto the customers who deploy them. Along the way she shares how she vibe-coded a unit-test-writing agent in Cursor in a single day, why diverse test teams are the antidote to biased models, and why the emerging "AI architect" role will decide whether we get one agent or an orchestrated fleet of hundreds. **In this episode:** - Quality Assurance vs Quality Engineering vs Quality Enablement - Testing Non-Deterministic And Probabilistic Systems - Automating The Mundane With Single-Purpose Agents - Outcome Over Output: Validating Intent, Not Exact Strings - Guardrails, Context Buffers And Adversarial Manipulation - Bias, Diverse Test Teams And Context-Driven Testing - Auditability, Traceability And Observability In Production - The Emerging AI Architect Role And Agent Orchestration **Guest:** Janna Loeffler — Quality Engineering Leader / Director, 20+ years in QE, testing and platform reliability **Host:** Alex Belotsky (CEO, TestSavant.AI) with co-host Jakub "Kuba" Fiatkevich > "yes, everybody should be responsible for quality, but you need somebody accountable for that quality." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #QualityEnablement #AITesting #ContextDrivenTesting #Agents #LLM #Guardrails #AIGovernance #TestAutomation #NonDeterministic
#62
July 8 · 57 min
Why 80% of Companies Fail at AI — feat. Jonathon Wright (World Digital Report)
Everyone says they're "doing AI" — but only 5% are actually getting measurable value, and the gap is where careers and companies quietly die. Jonathon Wright, Chief AI Officer at the World Digital Report, joins Alex and Kuba to unpack the report's blunt headline: 80% of enterprises use AI somewhere, yet only 5–10% see real, measurable value. Drawing on four decades in automation and AI-driven quality engineering, Jonathon traces the shift from single-prompt "LLM as a hammer" thinking to agentic fleets, computer-use agents, and neuro-symbolic programming — and argues that if you build anything with AI, you must test it with AI. He walks through the report's maturity waves (tester-in-the-loop, cross-SDLC, and self-evolving "dark factory" systems), the rise of citizen testers, and why wisdom and domain experience now outrank raw academic skill. The conversation gets uncomfortably concrete: prompt-injection and agent-to-agent attacks, a NATO/DoD hackathon on autonomous drones where declassified data became sensitive once run through a model, and the looming weight of ISO 42001, the EU AI Act, and AI assurance. Jonathon's throughline is that AI testing is no longer pass/fail — it's probabilistic, non-deterministic, and the last real safeguard before brand-destroying failure. The takeaway: pivot toward cognitive and human-interaction skills, build governance now, and don't be the company on the front page of the report. **In this episode:** - Why Only 5% of Companies Get Real AI Value - From Single Prompts to Agentic Fleets and Computer-Use Agents - If You Build With AI, You Must Test With AI - Testing Non-Deterministic, Probabilistic Systems - The Tester-in-the-Loop and Citizen Tester Waves - Prompt Injection, Red Teaming and Agent-to-Agent Attacks - AI Assurance, ISO 42001 and the EU AI Act - Neuro-Symbolic Programming and the Death of Prompt Engineering **Guest:** Jonathon Wright — Chief AI Officer, World Digital Report **Host:** Alex Belotsky (CEO, TestSavant.AI) with co-host Jakub "Kuba" Fiatkevich > "If you create anything that you create with AI, you have to test with AI. It's as simple as that." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AIAssurance #AIGovernance #AgenticAI #RedTeaming #PromptInjection #ISO42001 #EUAIAct #TestAutomation #WorldDigitalReport
#61
July 8 · 39 min
Will AI Replace QA Teams? The Human Case for Testers in the Age of Agents — feat. Bill Kirst (Adobe)
AI can write your test cases and watch your screen — but Adobe's AI ambassador argues it will never replace the human who asks the question no one else thought to ask. Alex and Kuba sit down with Bill Kirst — AI ambassador and change manager at Adobe, two-time author, podcast host and U.S. Army veteran — to unpack what generative AI actually does to QA and testing teams. Bill argues that the change-management playbooks that worked for 30 or 40 years of ERP rollouts break down when every user has an individualized, probabilistic experience with AI, so the real work is rebuilding trust that must be earned, kept and re-earned. The conversation gets concrete about testing: how LLMs took over converting requirements into test cases (once 30% of a QA team's week), how screen-recording tools now auto-generate test scripts from real user behaviour, and how agentic tools like Claude can regenerate training materials after a UI change and hand a team back four-to-six weeks of work. But because LLMs are probabilistic pattern-seekers — "it can write code but it can't engineer" — humans stay at the centre for edge cases, nuance, red-teaming and the "margins" between test steps. Bill reframes the productivity dividend as a leadership question: when a tool gives you back six-to-fifteen hours a week, do you cut headcount, spin up fifteen more unsupervised agents toward burnout, or reinvest that time in innovation and human connection. It closes on a hopeful, humanist note about solopreneurs, renegotiated social contracts, and holding on to what ones and zeros can never replicate. **In this episode:** - Why Legacy Change-Management Playbooks Break With AI - Trust As An Equation: Earned, Kept And Re-Earned - Automating Test-Case Writing With LLMs - Screen-Recording Tools That Auto-Generate Test Scripts - Agentic Updates To Training Docs After UI Changes - Probabilistic Testing And Broad Coverage For LLM Apps - The Human Edge: Edge Cases, Red Teams And The Margins - Reclaimed Time, Burnout And The Leadership Question **Guest:** Bill Kirst — AI Ambassador & Change Manager, Adobe **Host:** Alex Belotsky (CEO, TestSavant.AI) with co-host Jakub "Kuba" Fiatkevich > "Does it bring me some productivity? Yes. Will it replace what I do uniquely and divinely? No, never." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #TestAutomation #ChangeManagement #AIAmbassador #LLM #AgenticAI #FutureOfWork #RedTeaming #Adobe #AIGovernance
#60
July 8 · 38 min
How AI Broke Testing: From Regression to Statistical Assurance — feat. Ian McCallum (SAP)
When one input can return a hundred different answers, "does it pass?" stops being a yes-or-no question — and testing has to be reinvented. SAP AI advisor Ian McCallum — who literally wrote the book on testing SAP applications — joins Alex and Kuba to unpack why generative AI and agents have shattered deterministic testing. The conversation moves from why enterprises deploy AI to how they do it safely: governance, security, and lifecycle management. Ian argues that regression testing is dead for probabilistic systems, replaced by statistical and spot testing borrowed from life sciences — sampling, bell curves, and allowable variances tuned to how tight "good enough" needs to be (very tight for closing the books, looser for some supply chains). They dig into the coming swarm of agents, why "red teaming" is the wrong frame for LLM quality, and how the human-in-the-loop role shifts from clicking submit to evaluating results across a defense-in-depth stack. The episode closes on a genuinely unsettling question: what happens when an intelligent agent learns to hide its own problems? **In this episode:** - Moving From "Why" To "How" In Enterprise AI Adoption - Non-Deterministic And Probabilistic Testing - Why Regression Testing Fails For AI And Agents - Statistical And Spot Testing Borrowed From Life Sciences - Defining "Good Enough" And Acceptable Error Rates - Agent Swarms, Governance, Security And Lifecycle Management - Why Red Teaming Is The Wrong Frame For LLM Quality - The Shifting Role Of The Human In The Loop **Guest:** Ian McCallum — AI Technical Solution & Strategy Advisor, SAP **Host:** Alex Belotsky (CEO, TestSavant.AI) and Jakub "Kuba" Fiatkevich — building and shipping reliable AI to production > "It's not a bug. It's a feature. As I keep saying." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AITesting #AgenticAI #LLM #AIGovernance #EnterpriseAI #SAP #RedTeaming #HumanInTheLoop #GenerativeAI
#59
July 8 · 52 min
A CISO's Guide to Securing Agentic AI: Governance, Shadow AI and Zero-Trust Agents — feat. Peter Holcomb (Optimo IT)
When your app's "brain" is a non-deterministic LLM and your agents can roam autonomously, who owns security — and can you even secure the beehive? Alex Belotsky sits down with Peter Holcomb, founder and CEO of Optimo IT, for a classically-trained CISO's field guide to deploying AI in regulated enterprises. Peter walks through starting with an AI governance and maturity assessment before writing a line of code, the crawl-walk-run path from Microsoft Copilot to custom agents, and why messy, unclassified data (across OneDrive, S3, Azure blobs) is the real blocker — solved with a data enclave and the medallion (bronze/silver/gold) framework. The conversation digs into the concrete new attack surface: model risk across Anthropic, OpenAI, Gemini and open-source models, prompt injection, vector-database and API risks, and the rise of "shadow AI" — including autonomous tools like OpenClaw/Clawbot pulling unvetted third-party skills and triggering infostealer infections. Peter argues human-in-the-loop is still the best available brake, sketches a "unified agentic mesh" pulling together DSPM, DLP, SOAR (Tines, Torq, BlinkOps) and CNAPP tooling (Wiz, Aqua, Orca), and makes the case for zero-trust agent identity with short time-to-live machine identities. They close on a spirited debate over recursive self-coding, hallucination-as-a-feature, whether Claude is "conscious," and how far off real AGI actually is. Practical, opinionated and grounded in what regulated businesses are actually facing right now. **In this episode:** - AI Governance And Maturity Assessments Before Deployment - Shared Responsibility Across CISO, CTO And Business Units - Data Classification And The Medallion Framework - Shadow AI And Autonomous Tool Risk (OpenClaw / Clawbot) - Prompt Injection, Model Risk And Vector Database Security - Human-In-The-Loop As The Brake On Agentic Autonomy - The Unified Agentic Mesh And Zero-Trust Agent Identity - Red Teaming, QA And Securing Non-Deterministic LLM Apps **Guest:** Peter Holcomb — Founder & CEO, Optimo IT **Host:** Alex Belotsky — CEO of TestSavant.AI > "Security is like the brakes to a fast driving car where you allow the car to run and go fast, but you have the brakes on there so that you can go around the corners in a good speed and not derail the whole car and go off the boundaries." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AISecurity #CISO #AIGovernance #AgenticAI #ShadowAI #PromptInjection #ZeroTrust #RedTeaming #LLMSecurity
#58
July 8 · 43 min
Can OpenClaw Be Secured for the Enterprise? Agents Guarding Agents — feat. Amman Abbas & Bogdan (TestSavant.AI)
An open-source agent with full run of your machine gets bought by OpenAI and sued by Anthropic — so can it ever be safe to ship to production? Alex and Kuba are joined by TestSavant.AI's Head of AI Amman Abbas and Head of Site Reliability Engineering Bogdan to dissect the OpenClaw / ClawdBot phenomenon: an open-source agent that pulls skills from the internet, holds unfettered local access, and was, by its creator's own admission, left intentionally insecure. The team walks through why it isn't enterprise- or production-ready today — from unauthenticated gateway exposure, plain-text credentials and remote code execution to malicious community skills and crypto-scam supply-chain attacks. Amman makes the case that the real danger is indirect prompt injection arriving through email, WhatsApp, databases, vector stores and hidden text on web pages, and that defending it means sandboxing (VM/Docker), stripping tool permissions, and layering bind-code and prompt-injection guardrails. They also demo TestSavant.AI's newly public AI security researcher for tracking zero-days and papers, and argue that as agents spawn and destroy other agents, red teaming can never reach full coverage — you prioritise by intent and risk, then bring a human back into the loop. The through-line: their prediction for 2027 is agents securing agents governed by enterprise policy, with tokenised payment rails underneath. **In this episode:** - Why OpenClaw Isn't Enterprise or Production Ready - Indirect Prompt Injection Through Connected Apps and the Web - Remote Code Execution, Sandboxing and Tool-Permission Guardrails - Malicious Skills, Supply-Chain Attacks and Crypto Scams - Red Teaming Autonomous and Multi-Agent Systems - Agents That Create and Destroy Other Agents - The Cost Problem: When Security Costs More Than the App - Agents Securing Agents With Enterprise Policy (2027 Prediction) **Guest:** Amman Abbas — Head of AI, TestSavant.AI; and Bogdan — Head of Site Reliability Engineering, TestSavant.AI **Host:** Alex Belotsky (CEO, TestSavant.AI) & co-host Jakub "Kuba" Fiatkevich — building and shipping reliable AI to production > "So you need agents. So basically the only way to secure agents is with other agents." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AISecurity #PromptInjection #RedTeaming #AgenticAI #LLMSecurity #OpenClaw #AIGovernance #MultiAgent #EnterpriseAI
#57
July 8 · 54 min
AI Is Breaking Traditional QA — feat. Ruslan Isyatnikov (QAMentor)
AI is gutting the old QA playbook — so what survives, and what replaces it? QAMentor founder Ruslan Isyatnikov joins Alex and Kuba to explain why QA demand fell over the last three years and why he thinks it's about to roar back. He recounts building QAMentor from a one-man coaching shop into a 400-person global testing firm, surviving COVID without laying anyone off, and running a 12-person focus group that spent 15 months evaluating six AI automation tools — roughly two months each. The conversation splits AI testing into two clean verticals: using AI tools to speed up test-case generation, impact analysis and regression scoping, versus testing the AI itself for hallucination, bias, drift, security and compliance. Ruslan is blunt that even 50–60% accuracy from these tools is a huge value-add, that cloud AI tools uploading sensitive requirements are a real data-leakage risk, and that non-deterministic LLMs force testers to abandon traditional requirements-driven mindsets. His throughline: "trust but verify" only works with a smart human in the loop, and lazy, tool-only testers will disappear while the ones who blend human intelligence with automation survive. **In this episode:** - Why QA Demand Dropped And Why It's Coming Back - Two Verticals: Testing With AI vs Testing The AI - Evaluating AI Tools The Hard Way (6 Tools, 15 Months) - Hallucination, Bias, Drift And Red-Teaming LLMs - Data Leakage And Compliance Risk In Cloud AI Tools - Scriptless, Codeless And Prompt-Driven Automation - Non-Deterministic LLMs And The Death Of Static Requirements - Human In The Loop — But Only A Smart One **Guest:** Ruslan Isyatnikov — Founder & CEO, QAMentor **Host:** Alex Belotsky (CEO, TestSavant.AI) and co-host Jakub "Kuba" Fiatkevich — building and shipping reliable AI to production > "It cannot be just any human. It's got to be a smart human, right?" 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AITesting #LLMTesting #TestAutomation #HallucinationTesting #RedTeaming #AIGovernance #HumanInTheLoop #QAMentor
#56
July 8 · 26 min
When Determinism Dies: Testing and Securing Agentic AI in Production
The pilot episode where two enterprise veterans explain why your old QA pipeline breaks the moment an LLM enters the building. In this first episode of Ctrl-Alt-Deploy, host Alex Belotsky (CEO of Hesabon/TestSavant.AI) and co-host Kuba Fiatkevich unpack what changes when non-deterministic AI lands in high-stakes software. They argue that the classic assumption of "input A always yields output B" no longer holds, so pipelines built for binary pass/fail states need a new evaluation stage that judges the quality of probabilistic outputs. The conversation moves from LLM chatbots to the 2026 rush toward autonomous agents that wield tools and execute their own code — where a bad chatbot causes reputational damage but a bad agent can delete a database. They dig into treating agents as untrusted "employees," tightening permission scoping at the architectural level, stripping tool permissions the moment a task ends, and red-teaming for prompt injection, exfiltration, and denial-of-wallet attacks. Alex and Kuba also introduce economic and functional health metrics — cost per successful transaction, human intervention rate, benign rejection rate — and describe an adaptive loop where test results feed guardrails that stay inside a compliance and governance framework. The takeaway: reliability is bounded, probable behavior, not perfection, and it belongs to a cross-departmental effort with QA still the final gate. **In this episode:** - Why Determinism No Longer Equals Reliability - Adding An Evaluation Stage To The Testing Pipeline - Securing Agentic Systems And Permission Scoping - Agents As Untrusted Employees - Red Teaming, Prompt Injection And Denial-Of-Wallet Attacks - Economic And Functional Health Metrics - Adaptive Guardrails, Compliance And Governance - The 80/20 Rule And The AI Skills Gap **Host:** Alex Belotsky (CEO, Hesabon / TestSavant.AI) with co-host Kuba Fiatkevich — enterprise CTO and high-stakes deployment veteran > "So a chatbot might give you bad advice and might cause some reputational damage, whereas an agent might delete a database." 🎧 *Subscribe to Ctrl-Alt-Deploy on Apple Podcasts, Spotify and YouTube for conversations on building and shipping reliable AI to production, QA, testing and AI governance.* ## Hashtags #CtrlAltDeploy #AI #SoftwareTesting #QA #QualityEngineering #AgenticAI #LLM #RedTeaming #AISecurity #Guardrails #AIGovernance #TestSavant #DevOps #MLOps
#55
July 7 · 1 hr 3 min
AI, Computer Vision and Model-Based Testing — feat. Don Jackson (Micro Focus)
Micro Focus chief technologist on how AI, computer vision and NLP free testers to focus on what and why, not how. Don Jackson traces his journey from customer to Micro Focus chief technologist and explores how computer vision and NLP in the UFT family let testers interact with applications visually rather than through the DOM or brittle selectors. He argues AI should recognize objects by their visual characteristics instead of self-healing, and previews next-generation model-based testing that underpins reusable components with any automation type plus risk-based metadata. He champions 'shift everywhere' and a fourth persona (leadership) who should see elevated value rather than scrape Jenkins logs. The conversation covers negative testing, test data generation, SAP package testing, service virtualization and the upcoming Universe event. **In this episode:** - AI in Testing - Model-Based Testing - Test Automation - Test Data Management - Codeless Automation - API Testing - Test Strategy - QA Leadership **Guest:** Don Jackson — Chief Technologist, Micro Focus **Host:** Jonathon Wright — *The Automation Cyborg* > “Focus on what and why, not how. Let the tool take care of the how.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #AITesting #ModelBasedTesting #TestAutomation #TestData #CodelessAutomation #APITesting #TestStrategy #QALeadership
#54
July 7 · 46 min
Modern Quality Engineering for the Digital Enterprise — feat. Niko Mangahas (RCG Global Services)
Why digital enterprises need modern quality engineering that extends QA further, deeper, wider, and at scale. This is the polished cut of Niko Mangahas's conversation on modern quality engineering. He argues that decentralizing quality into scrum teams creates quality blind spots, and that a quality strategy and management layer plus an automation shared service are essential to track macro quality risks and quality debt. He defines experience quality as reliable, responsive, intuitive, engaging, and inclusive, spanning performance, usability, A/B, and accessibility testing. With Jonathan Wright he explores shift-left and shift-right, testing in production, continuous validation, observability and anomaly detection, and Black Friday failure scenarios. Niko frames predictability of outcomes and business value as the real goal, with hyper-automation (AI and RPA) as the means to scale QA in four dimensions: further, deeper, wider, and at scale. **In this episode:** - QA Leadership - Test Strategy - Shift-Right Testing - Shift-Left Testing - Accessibility Testing - Performance Testing - Chaos Engineering & Resilience - Observability - Risk-Based Testing - AI in Testing **Guest:** Niko Mangahas — Global Practice Lead for Quality Engineering, RCG Global Services **Host:** Jonathon Wright — *The Automation Cyborg* > “If I'm truly being honest, I think the true measure of a good quality strategy is the predictability of outcomes.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #QALeadership #TestStrategy #ShiftRight #ShiftLeft #AccessibilityTesting #PerformanceTesting #ChaosEngineering #Observability #RiskBasedTesting #AITesting #ConfidenceEngineering
#53
July 7 · 53 min
Making Legacy Legendary: DevOps, Mainframe and Value Streams — feat. Anna Murray (ASG Technologies)
An ASG product director on mainframe DevOps, value stream management and where automation and RPA really add value. Anna Murray of ASG Technologies (recently acquired by Rocket Software) champions 'making legacy legendary', noting 90% of business transactions still run on mainframe yet DevOps tooling and awareness lag there. She frames value stream management as abstracting across complex organisations to find where value is really delivered, and reducing waste through Enterprise Orchestrator. Jonathon raises provocations that DevOps may be too dev-centric versus an 'ops-dev' approach and that one methodology never fits all. The conversation lands on planning as the missing ingredient in failed transformations, product managers as translators between customers and engineers, and RPA's fit for stable legacy systems lacking APIs. **In this episode:** - DevOps & QAOps - Test Strategy - QA Leadership - Low-Code / No-Code Testing - Communication & Soft Skills - Test Management & Tooling - Agile Testing - Cloud Testing **Guest:** Anna Murray — Director of Product Management, ASG Technologies **Host:** Jonathon Wright — *The Automation Cyborg* > “We love making legacy legendary.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #DevOps #TestStrategy #QALeadership #LowCode #SoftSkills #TestManagement #AgileTesting #CloudTesting
#52
July 7 · 53 min
Visual AI and the Unknown Unknowns — feat. Yarden Naveh (Applitools)
An Applitools expert on why traditional DOM assertions miss the unknown unknowns that visual AI catches. Yarden Naveh explains the limits of traditional functional testing based on DOM assertions: you can only test what you thought might break, and passing tests can hide broken user experiences such as CSS colour clashes or overlapping text, the 'unknown unknown'. Applitools Eyes replaces thousands of brittle assertions with a single visual assertion, using AI and machine learning to compare full screenshots as a human eye would, slashing maintenance and improving root cause analysis. The discussion covers accessibility, localization word-wrapping, responsive design and cross-browser rendering via the Ultrafast Grid, which renders one capture across many environments. A hackathon showed Applitools users testing six times faster with 100% issue coverage versus 70%. **In this episode:** - AI in Testing - Test Automation - Accessibility Testing - Mobile Testing - Cloud Testing - Usability & UX Testing - Regression Testing - Test Metrics & Reporting **Guest:** Yarden Naveh — Customer Success & Solutions, Applitools **Host:** Jonathon Wright — *The Automation Cyborg* > “The biggest problem that I think exists is what I call the unknown unknown. Tests that are passing, you think that everything is okay, but it actually isn't.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #AITesting #TestAutomation #AccessibilityTesting #MobileTesting #CloudTesting #UXTesting #RegressionTesting #TestMetrics
#51
July 7 · 53 min
Risk, the Definition of Quality and Shifting Up — feat. Iman Benlekehal (CGI)
A QA lead on using risk to force stakeholder alignment, the definition of quality and shifting quality up to leadership. Recorded as a podcast-and-talk prep session, Iman Benlekehal discusses cultural differences in how risk is talked about between Europe and Canada, and how a risk matrix forces every stakeholder to reveal their differing definition of quality. She argues quality cannot be defined by QA alone: putting opposing stakeholders around one table makes them focus on the same product and agree what is critical. The conversation expands into a quality 'pyramid' placing the customer at the centre of the time-cost-quality triangle, illustrated with autonomous cars, Tesla and GDPR. The second half is Jonathon coaching her on speaking confidence, handling Q&A through misdirection and weaving in others' experience reports. **In this episode:** - Risk-Based Testing - Quality Culture - QA Leadership - Test Strategy - Communication & Soft Skills - Career Development in QA - Mentoring & Community **Guest:** Iman Benlekehal — Quality Assurance Lead, CGI **Host:** Jonathon Wright — *The Automation Cyborg* > “If I put them in the same table, they're going to speak to each other and challenge each other... they have a lot of definitions of quality.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #RiskBasedTesting #QualityCulture #QALeadership #TestStrategy #SoftSkills #QACareers #Mentoring
#50
July 7 · 35 min
The Dark Art of Digital Transformation — feat. Joseph Garber (Micro Focus)
Micro Focus strategy head on the four pillars of digital transformation and evolution over revolution. Joe Garber explains how Micro Focus defined digital transformation around four core pillars: enterprise DevOps, hybrid IT management, security risk and governance, and predictive analytics. He argues for 'evolution over revolution', bridging existing and emerging systems rather than rip-and-replace, and describes the run-and-transform balance intensified by the pandemic. He discusses leveraging a broad 300-product portfolio through a multiplier effect (Vertica, IDOL, RPA), the importance of culture and avoiding the 84% of failed transformations, and the role of a 6,500-strong partner ecosystem. Big bets include application modernization, next-gen SOC and release velocity. **In this episode:** - DevOps & QAOps - Cloud Testing - Security Testing - AI in Testing - Test Strategy - QA Leadership - Test Management & Tooling **Guest:** Joseph Garber — Global Head of Strategy and Solutions, Micro Focus **Host:** Jonathon Wright — *The Automation Cyborg* > “The rip and replace approach to digital transformation at first seems really exciting... but what is critical to remember is there are these core business systems that have IP and processes built around them.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #DevOps #CloudTesting #SecurityTesting #AITesting #TestStrategy #QALeadership #TestManagement
#49
July 7 · 53 min
Agile Coaching and Value-Obsessed Delivery — feat. Iwona Winiarska (Automation Logic)
An agile delivery consultant on coaching teams, being value-obsessed and running meaningful retrospectives in distributed teams. Iwona Winiarska shares her path from a humanities degree into agile and DevOps transformation, and being shortlisted for the Women in Tech Excellence Awards. She contrasts project management with the scrum master role, describing the unlearning required to coach teams into finding their own answers and become 'value obsessed'. She stresses that many organisational problems are cultural rather than technical, and introduces a team health check as a visual, data-driven way to surface pain points to stakeholders. She closes on the challenges of remote working, losing water-cooler moments, and building senior stakeholder buy-in virtually. **In this episode:** - Agile Testing - QA Leadership - Mentoring & Community - Communication & Soft Skills - Remote & Distributed Teams - Quality Culture - Career Development in QA - DevOps & QAOps **Guest:** Iwona Winiarska — Agile Delivery Consultant, Automation Logic **Host:** Jonathon Wright — *The Automation Cyborg* > “I'm trying to be value obsessed, if that makes sense.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #AgileTesting #QALeadership #Mentoring #SoftSkills #RemoteTeams #QualityCulture #QACareers #DevOps
#48
July 7 · 53 min
SelectorsHub and Model-Based Automation — feat. Sanjay Kumar & James Walker (SelectorsHub)
Building rock-solid element identifiers and combining them with model-based testing for reusable automation. Sanjay Kumar, creator of the community-driven SelectorsHub XPath and CSS selector IDE, explains how the tool teaches testers to build robust selectors and handle tricky cases like nested shadow DOM, iframes, and disappearing dropdowns, all born from community feedback. James Walker of Curiosity Software describes the open testing platform and Test Modeler, which visualize what is being tested as flowchart models and let teams normalize automation across mixed tooling. Together they show how good identifiers plus model-based testing lay a strong foundation, enable self-healing by changing a component in a single location, and let organizations auto-assemble models, tests, and test data, illustrated by an education company generating training-pathway tests as part of CI/CD. **In this episode:** - Test Automation - Model-Based Testing - Test Management & Tooling - Low-Code / No-Code Testing - Codeless Automation - Test Data Management - Mentoring & Community - Test Case Design **Guest:** Sanjay Kumar — Founder & Creator of SelectorsHub, SelectorsHub **Guest:** James Walker — Founder / testing leader, Curiosity Software **Host:** Jonathon Wright — *The Automation Cyborg* > “Your automation is only as good as the identifier someone has scripted, so if you start from the beginning with really good identifiers, that is laying the foundation to build good automation later on.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #TestAutomation #ModelBasedTesting #TestManagement #LowCode #CodelessAutomation #TestData #Mentoring #TestDesign
#47
July 7 · 58 min
Innovation, NASA, and the Future of Software — feat. Michael Ritchson (NASA Armstrong Flight Research Center)
A NASA developer's high-energy take on failing forward, mentoring the next generation, and data's future. Michael Ritchson shares his journey from aircraft mechanic to NASA software developer and reflects on the culture of innovation, creativity, and failing fast. He and Jonathan Wright discuss instilling curiosity in the next generation, NASA's image and the human cost of space exploration, and how modern architecture leans on containers (Docker, Kafka), redundancy, and resilience. They explore digital twins (inspired by Apollo 13), migrating legacy data into modern systems, applying NLP/AI and VR to historical archives, and the "six degrees of separation" idea that every contributor's small idea can ripple across society. The episode is a warm, motivational conversation about community, mentorship, and the art of the possible. **In this episode:** - Career Development in QA - Mentoring & Community - Communication & Soft Skills - DevOps & QAOps - AI in Testing - Test Data & Environments - Chaos Engineering & Resilience **Guest:** Michael Ritchson — Software Developer, NASA Armstrong Flight Research Center **Host:** Jonathon Wright — *The Automation Cyborg* > “It's just like IT industry: when I fail in production, we get back in and then we continue moving forward until we have a successful product.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #QACareers #Mentoring #SoftSkills #DevOps #AITesting #TestEnvironments #ChaosEngineering #DigitalTwinTesting
#46
July 7 · 37 min
Low-Code Autonomous Testing with mabl — feat. Bertold Kolics (mabl)
How low-code, self-healing, AI-driven automation lowers the barrier to entry and helps teams scale testing. Bertold Kolics of mabl explains the value of "drinking your own champagne" (dogfooding) by testing mabl with mabl to build empathy with end users and catch defects early. He describes how low-code platforms ease the transition from manual to automated testing, reduce maintenance through self-healing, and speed root-cause analysis by capturing rich diagnostics like network logs and screenshots. The conversation covers using usage data and analytics (via a Segment integration) to spend limited "testing dollars" on the most impactful, most-travelled paths, plus tips for scaling automation across enterprise teams by sharing reusable components, pruning dead tests, and involving developers with local test runs. **In this episode:** - Test Automation - Low-Code / No-Code Testing - Codeless Automation - Continuous Testing - Shift-Right Testing - Shift-Left Testing - AI in Testing - Test Strategy **Guest:** Bertold Kolics — Engineering team member / self-described "question asker", mabl **Host:** Jonathon Wright — *The Automation Cyborg* > “Drinking your own champagne forces you to put your end-user hat on and empathize with your end-users, because you're using the exact same tool that they would.” 🎧 *Subscribe to The QA Lead on Apple Podcasts, Spotify and Google Podcasts for conversations on software testing, QA, test automation and the future of quality engineering.* ## Hashtags #SoftwareTesting #QA #QualityEngineering #TheQALead #TestingPodcast #TestAutomation #LowCode #CodelessAutomation #ContinuousTesting #ShiftRight #ShiftLeft #AITesting #TestStrategy

Showing 1–20 of 20 episodes