Where’s the “intelligence explosion”?

Ramez Naam, futurist and science fiction author, gives a skeptic’s take on recursive self-improvement, introduced by Noah Smith on Noahpinion. Drawing on OpenAI’s newly released internal research data, Anthropic’s system cards, METR’s task horizons and Epoch’s capability index, he asks how strong the AI self-improvement loop really is: how much more research each gain in capability buys, where the gains leak away to diminishing returns, and what it would take for the loop to sustain itself, let alone run away into an intelligence explosion. * 00:00 - Introduction * 03:44 - 1. AI is Helping Improve Itself * 05:02 - How Strong Is the Feedback Loop? * 05:45 - Contents * 06:49 - 2. What Does RSI Mean? * 08:54 - We Already Have Narrow Superintelligence * 10:57 - 3. Real AI Research is Harder than Benchmarks or Forecasts * 13:50 - The Gap Between Benchmarks and Reality * 14:41 - From ECI Scores to METR Task Horizons * 16:16 - Measured Progress versus AI 2027 and ECI-extrapolated METR * 18:30 - 4. The Sharp Diminishing Returns to Impressive AI Numbers * 20:06 - From More Tokens to More Experiments * 23:29 - Test Time Compute Also Has Diminishing Returns * 24:29 - What About Agent Swarms? * 27:39 - 5. Better Models Matter More Than More Copies * 29:48 - 6. We’re Not Seeing Runaway Acceleration * 32:59 - Keeping Up the Pace Takes Exponentially More Resources * 36:14 - Better AI May Be Needed Just to Maintain the Pace * 37:11 - Gains on Other Benchmarks Don’t All Carry Through to Research * 38:51 - 7. Why Does Progress Get Harder? * 38:55 - Better Ideas Get Harder to Find * 41:05 - Lessons from Software R&D * 44:09 - From More Activity to Better Ideas * 44:57 - AI Still Struggles With Big Research Ideas * 46:54 - 8. The Self-Improvement Loop Doesn’t Look Strong Enough * 49:55 - How Fast Are Gains Coming Now? * 56:44 - 9. What Could Accelerate This? * 58:07 - Software, Hardware, and Economic Feedback * 01:02:46 - 10. We Need More Data * 01:06:36 - What the Future Holds https://open.substack.com/pub/noahpinion/p/wheres-the-intelligence-explosion?r=67y1h&utm_campaign=post-expanded-share&utm_medium=web Get full access to Askwho Casts AI at askwhocastsai.substack.com/subscribe