Distributed databases with Peter Mattis

Brought to You By: • turbopuffer – a vector and full-text search engine built on object storage. It’s fast, cheap, and extremely scalable • Linear – the product development system for teams and agents • WorkOS – everything you need to make your app enterprise ready. — How is it that a software veteran who regularly shipped ~100K of database-grade code to production each year, pre-AI, feels like he’s even more productive today, with no drop in quality? Peter Mattis is co-founder and CTO of Cockroach Labs, and an original creator of GIMP. He also worked on Gmail and distributed storage at Google. In this episode, Peter reflects on his journey from open source to Google to founding a database company, and we explore how to keep systems fast, reliable, and correct at scale, from Gmail’s early storage challenges to the tradeoffs in building distributed databases. Peter tells us how AI has brought him back to writing code after his work shifted toward management, and why he believes AI can improve quality and multiply the impact of domain experts. We also consider the future of code review, and Peter has some advice about how to level up our engineering skills. Timestamps 00:00 Intro 02:42 Peter’s path into tech 04:00 Building GIMP 09:30 Working on Gmail at Google 14:51 Google’s infra: google3, build files, Bazel, and Colossus 21:30 Distributed storage bottlenecks 23:59 Latency, throughput, and availability 30:04 Contributing to libraries 41:52 Google Spanner 46:10 CockroachDB 52:00 Manual vs. automatic sharding 55:28 Consistency models and strong consistency 1:00:03 Raft consensus 1:06:15 How AI brought Peter back to coding 1:19:12 Peter’s tools and agentic workflows 1:23:08 How AI can improve quality 1:26:39 Code reviews: are they done? 1:29:17 100x engineers 1:35:33 Peter’s advice for leveling up your engineering skills — The Pragmatic Engineer deepdives relevant for this episode: • Inside Google’s Engineering Culture • Resiliency in distributed systems • How to debug large, distributed systems: Antithesis • Pushing software engineering limits with “napkin math” • Designing Data-intensive Applications with Martin Kleppmann • Formal methods with Hillel Wayne — Production and marketing by ⁠⁠⁠⁠⁠⁠⁠⁠https://penname.co/⁠⁠⁠⁠⁠⁠⁠⁠. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe