S4 EP5 - Prof. Nils Thuerey on Differentiable Physics and Foundation Models

Differentiable physics, neural emulators and foundation models for PDEs are the focus of this conversation with Professor Nils Thuerey, head of the Physics-based Simulation group at TUM. Neil and Nils discuss PhiFlow, PICT, Tadpole, scalable 3D transformers, online synthetic data, open datasets, world models and agents that call physics simulators. Full episode, corrected transcript and resources: https://neilashton.co.uk/podcasts/s4-e5-prof-nils-thuerey-on-differentiable-physics-and-foundation-models/ Topics Differentiable physics and physics-based deep learning PhiFlow and differentiable simulation across ML frameworks When neural emulators can outperform their training data Foundation models for PDEs and synthetic online training Scalable 3D transformers and high-resolution simulations LES, temporal data and correlated CFD datasets Open-source tools, startups and physics-aware world models AI agents that call physics simulators Papers Neural Emulator Superiority: When Machine Learning for PDEs Surpasses its Training Data https://arxiv.org/abs/2510.23111 Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning https://arxiv.org/abs/2605.15284 P3D: Scalable Neural Surrogates for High-Resolution 3D Physics Simulations with Global Context https://arxiv.org/abs/2509.10186 PICT — A Differentiable, GPU-Accelerated Multi-Block PISO Solver for Simulation-Coupled Learning Tasks in Fluid Dynamics https://arxiv.org/abs/2505.16992 PhiFlow: Differentiable Simulations for PyTorch, TensorFlow and JAX https://proceedings.mlr.press/v235/holl24a.html Physics-based Deep Learning https://arxiv.org/abs/2109.05237 Learning to Control PDEs with Differentiable Physics https://arxiv.org/abs/2001.07457 Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers https://arxiv.org/abs/2007.00016 tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow https://arxiv.org/abs/1801.09710 Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows https://arxiv.org/abs/1810.08217 WeatherBench: A Benchmark Dataset for Data-Driven Weather Forecasting https://arxiv.org/abs/2002.00469 SuperWing: A Comprehensive Transonic Wing Dataset for Data-Driven Aerodynamic Design https://arxiv.org/abs/2512.14397 Links Nils Thuerey and the Physics-based Simulation group https://ge.in.tum.de/about/n-thuerey/ Chapters 00:00 Podcast intro 00:39 Introducing Prof. Nils Thuerey 04:13 Conversation begins 05:13 From Computational Numerics to Graphics and Visual Effects 07:17 Physics-Based Deep Learning Before ChatGPT 10:01 CNNs, Graphics and the Move into Engineering Applications 12:37 PhiFlow and Differentiable Physics 14:13 Can Neural Emulators Surpass Their Training Data? 18:00 The Promise and Limits of Foundation Models for PDEs 20:43 Tadpole and Synthetic Online Pre-Training 24:07 From Canonical PDEs to Navier-Stokes and Industrial CFD 26:35 What Do Foundation Models Actually Learn? 28:36 PDE Pre-Training vs. Millions of CFD Simulations 33:08 Scaling 3D Transformers and Training Infrastructure 35:58 Generating and Training on Data in Real Time 38:00 LES, Temporal Data and Turbulence 42:15 Overfitting and Correlated Simulation Data 44:27 Bringing Differentiable Solvers Back into the Loop 45:31 WeatherBench, APEBench and the Value of Benchmarks 47:09 SuperWing, Open Datasets and Commercial Data 51:31 Open Source, Commercial Models and a Technical Oscar 56:17 Academia, Startups and Industry 01:00:55 What Will Change Over the Next Five Years? 01:02:07 World Models and the Need for Physics 01:08:19 Agents, Tool Use and Calling Physics Simulators 01:11:22 Career Advice for AI and Simulation 01:13:54 Closing Thoughts