How Meta reduced diff authoring time by 40%

Meta researcher Moritz Beller joins Brian Houck to discuss how Meta measures developer productivity. Moritz explains diff authoring time and how Meta uses it to evaluate tools and guide engineering decisions. They explore how AI is changing engineers’ work and revealing gaps in traditional metrics. They also revisit Moritz’s “Mind the Gap” study and discuss the promise and risks of AI agents for testing. Where to find Moritz Beller: • LinkedIn: https://www.linkedin.com/in/inventitech • X: https://x.com/Inventitech • GitHub: https://github.com/Inventitech • Website: https://inventitech.com Where to find Brian Houck: • LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠ In this episode, we cover: (00:00) Intro (02:10) Moritz’s role at Meta (04:01) Measuring diff authoring time (08:20) Measuring A/B experiments (12:11) What diff authoring time reveals (14:45) Planning and leadership reporting (18:36) Why Meta measures teams, not individuals (22:33) Where developer time goes with AI (26:06) AI’s impact on junior and senior developers (27:18) Capturing invisible work with AI (29:32) The “Mind the Gap” study (33:40) Revisiting “Mind the Gap” in 2026 (38:32) AI agents for software testing (41:49) What remains hard to measure Referenced: • State of AI Impact in Engineering Q2 Report 2026 • GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis • From Technical Debt to Cognitive and Intent Debt • Mind the Gap: On the Relationship Between Automatically Measured and Self-Reported Productivity • The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study • When, how, and why developers (do not) test in their IDEs | Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering • DORA, SPACE, and DevEx: Which framework should you use • A Tale of Two Cities: Software Developers Working from Home During the COVID-19 Pandemic