Tiny Recursive Models Beat the Giants - Alexia Jolicoeur-Martineau (Microsoft)

Alexia Jolicoeur-Martineau is a Principal Researcher at Microsoft and the author of "Less is More: Recursive Reasoning with Tiny Networks," the paper behind the Tiny Recursive Model that hit about 45% on ARC-AGI-1 with a fraction of the parameters of frontier systems. It won the 2025 ARC Prize paper award. She read the hierarchical reasoning paper, thought the potential was real and the explanation was not, and rebuilt it without the mouse brains: a small network that carries a hidden state and a current answer, thinks for a few steps, updates, and repeats, with the gradient truncated at each loop. We get into why puzzles suit this and autoregression doesn't, why she thinks LLMs are bad at molecules and more data won't fix it, and what she'd do with a trillion dollars. Timeline 00:01 Intro 01:06 Leaving biostatistics, and why the field stagnated 06:47 GANs, diffusion, and research on four GPUs 12:58 What was wrong with the hierarchical reasoning paper 16:31 Tiny recursive models explained without the biology 22:35 Why puzzles favor recursion over left to right generation 24:15 Is the bitter lesson really bitter? 27:28 With infinite compute, would you still want small models? 32:00 Self improvement, memory, and a trillion dollars 37:01 Test time compute beyond chain of thought 40:41 Why chain of thought fails on molecules 45:17 Is there a universal representation? 48:06 What people are already building with TRM 55:22 Fixed point models and DEQ Music "Kid Kodi" - Blue Dot Sessions - via Free Music Archive - CC BY-NC 4.0 Topics Tiny Recursive Models and the ARC-AGI results What the hierarchical reasoning model was really doing Deep supervision and truncated backprop Looping transformers and parameter efficiency Why puzzles favor whole-context iteration over left to right generation Test time compute beyond chain of thought Latent reasoning and the Coconut line of work Why LLMs fail on chemistry and physics Representation learning and whether a universal representation exists