#158 Bayesian Workflows & Foundation Models, with Stefan Radev

Support & Resources → Support the show on Patreon → Bayesian Modeling Course (first 2 lessons free) Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work Takeaways: Q: Why are prior predictive checks so underused in practice, and how do simulations help? A: They're underused because researchers don't always think to run them before seeing data -- but also because doing them rigorously (in the style Michael Betancourt advocates, with prior push-forward checks on interpretable summaries) takes effort. Simulations make it cheap to generate thousands of “what-if world” datasets from your model and check whether they look plausible, catching bad priors before you ever touch real data. Q: How can generative AI help with prior elicitation? A: Rather than forcing a domain expert to choose a distributional family and parameterize it, you can use a generative model to translate their qualitative knowledge directly into a prior. The expert describes what realistic data should look like; the generative model produces synthetic datasets matching that description; those datasets are used to fit a prior distribution. It removes the assumption that experts can think in terms of parameters and replaces it with the more natural question: does this look like your data? Q: What would a foundation model for Bayesian inference actually look like? A: Stefan's bet is that it won't be a fine-tuned general LLM. The right analogy is chess: you don't fine-tune GPT to play chess, you teach it when to call Stockfish. For Bayesian inference, you'd want a semantic layer – an LLM that understands the analysis goal – calling specialized numerical engines (MCMC samplers, amortized inference networks) that do the actual computation. Agent skills are already a step in this direction; the longer-term vision is engines that have been trained from scratch to generalize across large families of models and priors. Full takeaways here. Chapters: 00:00 How does amortized inference fit into modern Bayesian workflows? 06:01 What role do simulations play across the full Bayesian workflow? 12:12 How do you elicit priors from a domain expert who doesn't think in distributions? 19:01 What would a foundation model for Bayesian inference actually look like? 35:32 What is self-consistency in amortized inference and why does it matter? 39:22 How does semi-supervised learning improve simulation-based inference? 43:16 Why is sensitivity analysis so important yet so underused in Bayesian practice? 47:40 What is multiverse analysis and how does it change how we report Bayesian results? 51:32 How does amortized inference make sensitivity and multiverse analysis affordable? 01:02:47 How do amortized inference and classical MCMC complement each other? 01:10:08 What are the next major directions for BayesFlow and amortized inference research? Thank you to my Patrons for making this episode possible! Links from the show here.