Building a School Where AI Models Learn About Humanity

If scaling laws hold—and Surge AI CEO Edwin Chen believes they do—we’re hurtling toward a future where there’s nothing humans can do that AI can’t do better. When OpenAI’s models disproved an open conjecture posed by mathematician Paul Erdős using novel algebraic geometry techniques, Fields medalist Timothy Gowers felt the shift acutely. He initially thought the model had proved an upper bound, and braced himself: that would mean it was “all over for mathematicians very soon.” When he realized it had only found a counterexample, he was relieved—it bought him another year or two before the thing he’s devoted his life to becomes something AI does better. As founder and CEO of the company behind the data environments and evals the major model companies use to train their models, Chen has a unique perspective on how quickly AI models are absorbing tasks we used to think of as uniquely human. Dan Shipper talked with Chen for AI & I about what the act of creating or building means when AI can do it better—and whether an answer to that question already exists within science fiction. If you found this episode interesting, please like, subscribe, comment, and share! Join the membership for Where You Live at ⁠https://www.joinbilt.com/dan To hear more from Dan Shipper: Subscribe to Every: https://every.to/subscribe Follow him on X: https://twitter.com/danshipper Timestamps: 00:00:54 Introduction 00:01:49 Surge as a "school for AGI" 00:04:46 What AI's capacity for novel mathematics says about human achievement 00:07:29 Motivation in an era when AI can do everything 00:14:34 The trap of optimizing AI models for engagement 00:29:34 Training using datasets versus training using environments 00:35:09 The value of personal data 00:39:40 Why models are bad at writing 00:42:00 Chen's AGI timeline Links to resources mentioned in the episode: Edwin Chen on X: https://x.com/echen Surge: https://surgehq.ai Riemann-bench (research-level math benchmark): https://surgehq.ai/leaderboards/riemann-bench Hemingway-bench (creative writing benchmark): https://surgehq.ai/leaderboards/hemingway-bench Talkie-1930 (language model trained on pre-1930 text): https://huggingface.co/talkie-lm/talkie-1930-13b-it Ted Chiang, “What’s Expected of Us”: https://www.nature.com/articles/436150a Every is the most AI-native startup on the internet. Through ideas, software and education, subscribers get the tools to work at the frontier of AI. Start your free trial today: https://every.to/subscribe?utm_source=youtube Follow Every: https://x.com/every Follow Dan Shipper: https://x.com/danshipper