Radar with Jeff Kao

Radar processes billions of location events daily, powering geofencing and location APIs for companies like Uber, Lyft, and thousands of other apps. When their existing infrastructure started hitting performance and cost limits, they built HorizonDB, a specialized database which replaced both Elasticsearch and MongoDB with a custom single binary written in Rust and backed by RocksDB. In this episode, we dive deep into the technical journey from prototype to production. We talk about RocksDB internals, finite-state transducers, the intricacies of geospatial indexing with Hilbert curves, and why Rust's type system and performance characteristics made it the perfect choice for rewriting critical infrastructure that processes location data at massive scale. About Radar Radar is the leading geofencing and location platform, trusted by companies like Uber, Lyft, and thousands of apps to power location-based experiences. Processing billions of location events daily, Radar provides geofencing APIs, geocoding, and location tracking that enables developers to build powerful location-aware applications. Their infrastructure handles massive scale with a focus on accuracy, performance, and reliability. About Jeff Kao Jeff Kao is a Staff Engineer at Radar, where he led the development of HorizonDB, Radar's geospatial database written in Rust. His work replaced Elasticsearch and MongoDB with a custom Rust stack built on RocksDB, achieving dramatic improvements in performance and cost efficiency. Jeff has deep experience with geospatial systems and previously open-sourced Node.js TypeScript bindings for Google's S2 library. He holds a degree from the University of Waterloo. Links From The Episode Radar Blog: High-Performance Geocoding in Rust - The blog post, which describes Radar's migration from Elasticsearch and MongoDB to Rust and RocksDB FourSquare - The compay Jeff worked at before Ruby - The basis for Rails PagerDuty) - Another company Jeff worked at. Hes' been around! CoffeeScript - The first big JavaScript alternative before TypeScript Scala - A functional JVM based language Wikipedia: MapReduce - Distributed application of functional programming Wikipedia: Algebraic Data Types - The concept behind Rust's Enums, also present in e.g. Scala Kotlin - Easier than Scala, better than Java Apache Lucene - The core of ElasticSearch Discord Blog: Why Discord is switching from Go to Rust - Always the #1 result in searches for Rust migrations Radar Blog: Introducing HorizonDB - A really nice write up of Horizon's architecture RocksDB - The primary storage layer used in HorizonDB MyRocks - A MySQL Storage Engine using RocksDB, written by Facebook MongoRocks - A MongoDB Storage Layer using RocksDB CockroachDB - PostgreSQL compatible, distributed, SQL Database InfluxDB - A timeseries database that used RocksDB at one point, and our very first guest in this Podcast! sled - An embedded database written in Rust rocksdb - Rust bindings for RocksDB H3 - Uber's Geo Hashing using hexagons S2 - Google's Geo Hashing library Wikipedia: Hilbert Curve - A way to map 2 dimensions onto 1 while retaining proximity Wikipedia: Finite-State Transducer - A state machine used for efficiently looking up if a word exists in the data set Rust in Production: Astral - Our episode with Charlie Marsh about tooling for the Python ecosystem burntsushi - A very prolific Rust developer, now working at Astral fst - FST crate from burntsushi Wikipedia: Trie - A tree-structure using common prefixes Wikipedia: Levenshtein Distance - The number of letters you have to change, add, or remove to turn word a into word b tantivy - A full-text search engine, written in Rust, inspired by Lucene LightGBM - A gradient boosted tree, similar to a decision tree fastText - A text classification library from Meta Wikipedia: Inverted Index - An index used for e.g. full text search Wikipedia: Okapi BM25 - The ranking algorithm used in tantivy Wikipedia: tf-idf - A classic and simple ranking algorithm Roaring Bitmaps - A very fast bitset library used in many places corrode.dev: Be Simple - A sentiment right down Matthias' alley loco-rs - Rust on Rails Official Links Radar Radar Blog Radar Documentation Jeff Kao on LinkedIn