018 - AI Coding: Does It Actually Make Developers Faster? (Deep Dive)

Does AI actually make coding faster? This Deep Dive works through three questions: how a model learns to write code in the first place, what a coding agent really does when you hand it a goal, and whether the productivity gains everyone assumes are real. The honest answer involves a randomized trial where developers were confidently wrong about their own experience. This is a longer-format episode, running about twenty minutes instead of the usual five. WHAT'S COVERED Part one - How a model learns to write code. Why coding improved faster than almost any other AI capability, and why the reason has nothing to do with code being easy. Code has a property that essays and advice and analysis do not: it can be checked automatically, with no human judgment involved. We walk through both training stages, from predicting the next token across vast amounts of public code, to the feedback loop that trains a model on whether its code actually ran. Then the benchmark that shows the effect: SWE-bench, built from real bug reports in real open-source projects, where early models solved under five percent of tasks and leading models now score around eighty percent. Plus two honest limits on that number, including why passing every test is not the same as writing good code. Part two - What a coding agent actually does. The four rungs of AI coding, from autocomplete through chat and editor integration to full agents like Claude Code and Codex. A worked example of an agent fixing a real bug: searching the codebase, reading the relevant files, making a change, running the test suite, seeing a failure, and correcting itself without being told. How this shifts a developer's job from writing toward specifying and reviewing, why instruction files inside repositories now matter, what vibe coding means, and the two genuine risks — agents that can run destructive commands, and review becoming the real bottleneck. Part three - Does it actually work? Why the widely quoted statistics on AI-written code are close to meaningless, including the fact that published figures for the same moment range from thirty to seventy-five percent because nobody defines the term the same way. Then the most rigorous evidence available: a randomized controlled trial from METR in which sixteen experienced developers completed real tasks nineteen percent slower with AI, after predicting they would be twenty-four percent faster, and still believing afterward that they had been sped up. We give that study its proper caveats rather than treating it as a verdict, then look at where the gains are genuinely large, where they shrink, and what rising code churn and falling developer trust suggest. IN THIS EPISODE AI coding, coding agents, Claude Code, Codex, GitHub Copilot, Cursor, autocomplete, SWE-bench, reinforcement learning, verifiable rewards, vibe coding, METR study, developer productivity, randomized controlled trial, code churn, technical debt, code review, AI-generated code, software engineering. Part of a series covering tokens, agents, MCP, retrieval-augmented generation, transformers, neural networks, parameters, hallucinations, and vector databases. Voice narration is AI-generated.