41: Efficiency at Any Scale, with Marlon Dutra

Wolf and Jim welcome their first-ever guest, longtime friend Marlon Dutra of Meta, for a practical tour of software efficiency. Marlon argues that efficiency isn't just a concern for huge systems. Your code may run on someone's aging phone or on a car's hardware for decades, and it may be running up a cloud bill. Efficiency also isn't the same as performance: if something gets 20% slower but 10x cheaper, and nobody notices, that's a win. What we cover Measure first. You can't optimize what you don't measure, and even a text file beats no data. CPU. Understand algorithmic complexity, keep hot paths small, and use profilers to find where time really goes. Learn the difference between CPU time and wall time, and when threads, multiprocessing, or GPUs fit the problem. We also look at how accidental O(n²) code happens, including a documentation build that dropped from 20 minutes to 24 seconds after profiling. Memory. Memory is expensive, scarce, and slower than most people think. Marlon covers leaks, reference cycles, pre-allocation, stack vs. heap, cache lines, spatial and temporal locality, and memoization. I/O and storage. Favor sequential access and larger reads and writes, and use async I/O instead of a thread per operation. Avoid N+1 queries, and store numbers as numbers rather than strings. We also get into file formats like Parquet, compression trade-offs, data retention, SSD wear, and logging discipline. Networking. Latency often matters more than bandwidth. Reduce round trips with batching and pipelining, watch payload sizes, use CDNs, expect unreliable networks, and add exponential backoff so retries don't take down your own servers. Links Marlon's code examples: https://github.com/Runtime-Arguments/efficiency Hosts: Jim McQuillan can be reached at jam@RuntimeArguments.fm Wolf can be reached at wolf@RuntimeArguments.fm Follow us on Mastodon: @RuntimeArguments@hachyderm.io If you have feedback for us, please send it to feedback@RuntimeArguments.fm Check out our webpage at http://RuntimeArguments.fm Theme music: Dawn by nuer self, from the album Digital Sky