NewsTech News

Software Engineering Daily

softwareengineeringdaily.com

Software Daily: The world through the lens of software.

Play

Open in

2,221 episodes
Avg 53 min
English

Episodes 2,221

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

#1387
Apr 15, 2020 · 53 min
Shopify React Native with Farhan Thawar
Shopify is a platform for selling products and building a business. It is a large e-commerce company with hundreds of engineers and several different mobile apps. Shopify’s engineering culture is willing to adopt new technologies aggressively, trying new tools that might provide significant leverage to the organization. React Native is one of those technologies. React Native can be used to make cross-platform mobile development easier by allowing code reuse between Android and iOS. React Native was developed within Facebook, and has been adopted by several other prominent technology companies, with varying degrees of success. Many companies have seen improvements to their mobile development and release process. However, in a previous episode, we talked with Airbnb about their adoption of React Native, which was less successful. Farhan Thawar is a VP of engineering at Shopify. He joins the show to talk about Shopify’s experience using React Native, the benefits of cross-platform development, and his perspective on when it is not a good idea to use React Native.
#1386
Apr 14, 2020 · 52 min
Ceph Storage System with Sage Weil
Ceph is a storage system that can be used for provisioning object storage, block storage, and file storage. These storage primitives can be used as the underlying medium for databases, queueing systems, and bucket storage. Ceph is used in circumstances where the developer may not want to use public cloud resources like Amazon S3. As an example, consider telecom infrastructure. Telecom companies that have their own data centers need software layers which make it simpler for the operators and developers that are working with that infrastructure to spin up databases and other abstractions with the same easy experience that is provided by a cloud provider by AWS. Sage Weil has been a core developer on Ceph since 2005, and the company he started around Ceph sold to Red Hat for $175 million. Sage joins the show to talk about the engineering behind Ceph and his time spent developing companies.
#1385
Apr 13, 2020 · 45 min
Collaborative SQL with Rahil Sondhi
Data analysts need to collaborate with each other in the same way that software engineers do. They also need a high quality development environment. These data analysts are not working with programming languages like Java and Python, so they are not using an IDE such as Eclipse. Data analysts predominantly use SQL, and the tooling for a data analyst to work with SQL is often a SQL explorer tool that lacks the kind of collaborative experience that we would expect in the age of Slack and GitHub. Rahil Sondhi is the creator of PopSQL, a collaborative SQL explorer. He created PopSQL after several years in the software industry, including 4 years at Instacart. Rahil joins the show to talk about the frictions that data analysts encounter when working with databases, and how those frictions led to the design of PopSQL.
#1384
Apr 10, 2020 · 55 min
Reserved Instances with Aran Khanna
When a developer spins up a virtual machine on AWS, that virtual machine could be purchased using one of several types of cost structures. These cost structures include on-demand instances, spot instances, and reserved instances. On-demand instances are often the most expensive, because the developer gets reliable VM infrastructure without committing to long-term pricing. Spot instances are cheap, spare compute capacity with lower reliability, that is available across AWS infrastructure. Reserved instances allow a developer to purchase longer term VM contracts for a lower price. Reserved instances can provide significant savings, but it can be difficult to calculate how much infrastructure to purchase. Aran Khanna is the founder of Reserved.ai, a company that builds cost management tools for AWS. He joins the show to talk about the landscape of cost management, and what he is building with Reserved.ai.
#1383
Apr 9, 2020 · 55 min
Snorkel: Training Dataset Management with Braden Hancock
Machine learning models require the use of training data, and that data needs to be labeled. Today, we have high quality data infrastructure tools such as TensorFlow, but we don’t have large high quality data sets. For many applications, the state of the art is to manually label training examples and feed them into the training process. Snorkel is a system for scaling the creation of labeled training data. In Snorkel, human subject matter experts create labeling functions, and these functions are applied to large quantities of data in order to label it. For example, if I want to generate training data about spam emails, I don’t have to hire 1000 email experts to look at emails and determine if they are spam or not. I can hire just a few email experts, and have them define labeling functions that can indicate whether an email is spam. If that doesn’t make sense, don’t worry. We discuss it in more detail in this episode. Braden Hancock works on Snorkel, and he joins the show to talk about the labeling problems in machine learning, and how Snorkel helps alleviate those problems. We have done many shows on machine learning in the past, which you can find on SoftwareDaily.com. Also, if you are interested in writing about machine learning, we have a new writing feature that you can check out by going to SoftwareDaily.com/write.
#1382
Apr 8, 2020 · 52 min
Cadence: Uber’s Workflow Engine with Maxim Fateev
A workflow is an application that involves more than just a simple request/response communication. For example, consider a session of a user taking a ride in an Uber. The user initiates the ride, and the ride might last for an hour. At the end of the ride, the user is charged for the ride and sent a transactional email. Throughout this entire ride, there are many different services and database tables being accessed across the Uber infrastructure. The transactions across this infrastructure need to be processed despite server failures which may occur along the way. Workflows are not just a part of Uber. Many different types of distributed operations at a company might be classified as a workflow: banking operations, spinning up a large cluster of machines, performing a distributed cron job. Maxim Fateev is the founder of Temporal.io, and the co-creator of Cadence, a workflow orchestration engine. Maxim developed Cadence when he was at Uber, seeing the engineering challenges that come from trying to solve the workflow orchestration problem. Before Uber, Maxim worked at AWS on the Simple Workflow Service, which was also a system for running workflows. Altogether, Maxim has developed workflow software for more than a decade.
#1380
Apr 7, 2020 · 45 min
kSQLDB: Kafka Streaming Interface with Michael Drogalis
Kafka is a distributed stream processing system that is commonly used for storing large volumes of append-only event data. Kafka has been open source for almost a decade, and as the project has matured, it has been used for new kinds of applications. Kafka’s pubsub interface for writing and reading topics is not ideal for all of these applications, which has led to the creation of ksqlDB, a database system built for streaming applications that uses Kafka as the underlying infrastructure for storing data. Michael Drogalis is a principal product manager at Confluent, where he helped develop ksqlDB. Michael joins the show to discuss ksqlDB, including the architecture, the query semantics, and the applications which might want a database that focuses on streams. We have done many great shows on Kafka in the past, which you can find on SoftwareDaily.com. Also, if you are interested in writing about Kafka, we have a new writing feature that you can check out by going to SoftwareDaily.com/write.
#1379
Apr 6, 2020 · 50 min
Godot Game Engine with Juan Linietsky
Building a game is not easy. The development team needs to figure out a unique design and gameplay mechanics that will attract players. There is a great deal of creative work that goes into making a game successful, and these games are often built with low budgets by people who are driven by the art and passion of game creation. A game engine is a system used to build and run games. Game engines let the programmer work at a high level of abstraction, by providing interfaces for graphics, physics, and scripting. Popular game engines include Unreal Engine and Unity, both of which require a license that reduces the amount of money received by the game developer. Godot is an open source and free to use game engine. The project was started by Juan Linietsky, who joins the show to discuss his motivation for making Godot. We have done some great shows on gaming in the past, which you can find on SoftwareDaily.com. Also, if you are interested in writing about game development, we have a new writing feature that you can check out by going to SoftwareDaily.com/write.
#1378
Apr 3, 2020 · 51 min
V8 Lite with Ross McIlroy
V8 is the JavaScript engine that runs Chrome. Every popular website makes heavy use of JavaScript, and V8 manages the execution environment of that code. The code that processes in your browser can run faster or slower depending on how “hot” the codepath is. If a certain line of code is executed frequently, that code might be optimized to run faster. V8 is running behind the scenes in your browser all the time, evaluating the code in your different tabs and determining how to manage that runtime in memory. As V8 is observing your code and analyzing it, V8 needs to allocate resources in order to determine what code to optimize. This process can be quite memory intensive, and can add significant overhead to the memory overhead of Chrome. Ross McIlroy is an engineer at Google, where he worked on a project called V8 Lite. The goal of V8 Lite was to significantly reduce the execution overhead of V8. Ross joins the show to talk about JavaScript memory consumption, and his work on V8 Lite. We have done some great shows on JavaScript in the past, which you can find on SoftwareDaily.com. Also, if you are interested in writing about JavaScript, we have a new writing feature that you can check out by going to SoftwareDaily.com/write.
#1377
Apr 2, 2020 · 56 min
Serverless Development with Jeremy Daly
Serverless tools have come a long way since the release of AWS Lambda in 2014. Serverless apps were originally architected around Lambda, with the functions-as-a-service being used to glue together larger pieces of functionality and API services. Today, many of the common AWS services such as API Gateway and DynamoDB have functionality built in to be able to respond to events. These services can use Amazon EventBridge to connect to each other. In many cases, a developer does not need AWS Lambda to glue services together in order to build an event-driven application. Jeremy Daly is the host of the Serverless Chats podcast, a show about patterns and strategies in serverless architecture. Jeremy joins the show to talk about modern serverless development, and the new tools available in the AWS ecosystem.
#1376
Apr 1, 2020 · 47 min
Audio Data Engineering with Allison King
Cortico is a non-profit that builds audio tools to improve public dialogue. Allison King is an engineer at Cortico, and she joins the show to talk about the process of building audio applications. One of these applications was a system for ingesting radio streams, transcribing the radio, and looking for duplicate information across the different radio stations. In a talk at Data Council, Allison talked through the data engineering architecture for processing these radio streams, and the patterns that she found across the radio streams, including clusters of political leanings. Another project from Cortico is called Local Voices Network. The Local Voices Network is built around a piece of hardware called a “digital hearth”, a specialized device that records discussions among people in a community. These community discussions are made available to journalists, public officials, and political candidates, creating a listening channel that connects these communities and stakeholders. Much of our conversation is focused on the engineering of the digital hearth, this device that sits in the center of community discussions.
#1374
Mar 31, 2020 · 57 min
Facebook Messenger Engineering with Mohsen Agsen
Facebook Messenger is a chat application that millions of people use every day to talk to each other. Over time, Messenger has grown to include group chats, video chats, animations, facial filters, stories, and many more features. Messenger is a tool for utility as well as for entertainment. Messenger is used both on mobile and on desktop, but the size of the mobile application is particularly important on mobile. There are many users who are on devices that do not have much storage space. As Messenger has accumulated features, the iOS code base has grown larger and larger. Several generations of Facebook engineers have rotated through the company with the responsibility of working on Facebook Messenger, which has led to different ways of managing information within the same codebase. The iOS codebase had room for improvement. Project Lightspeed was a project within Facebook that had the goal of making Messenger on iOS much smaller. Mohsen Agsen is an engineer with Facebook, and he joins the show to talk about the process of rewriting the Messenger app.
#1373
Mar 30, 2020 · 56 min
Pika Dependency Management with Fred Schott
Modern web development involves a complicated toolchain for managing dependencies. One part of this toolchain is the bundler, a tool that puts all your code and dependencies together into static asset files. The most popular bundler is webpack, which was originally released in 2012, before browsers widely supported ES Modules. Today, every major browser supports the ES Module system, which improves the efficiency of JavaScript dependency management. Snowpack is a system for managing dependencies that takes advantage of the browser support for ES Modules. Snowpack is made by Pika, a company that is developing a set of web technologies including a CDN, a package catalog, and a package code editor. Fred Schott is the founder of Pika and the creator of Snowpack. Fred joins the show to talk about his goals with Pika, and the ways in which modern web development is changing.
#1372
Mar 27, 2020 · 47 min
Cloud Kitchen Platform with Ashley Colpaart
Food delivery apps have changed how the restaurant world operates. After seven years of mobile food delivery, the volume of food ordered through these apps has become so large that entire restaurants can be sustained solely through the order flow that comes in from the apps. This raises the question as to why you even need an “on-prem” restaurant. A cloud kitchen is a large, shared kitchen where food is prepared for virtual restaurants. These virtual restaurants exist only on mobile apps. There are no waiters, there are only the food delivery couriers who pick up the food from these warehouse-sized food preparation facilities. A virtual restaurant entrepreneur could open up multiple restaurants operated from the same cloud kitchen. The mobile app user might see separate restaurant listings for a pizza place, a cookie bakery, and a Thai food restaurant, when all of them are operated by the same restaurateur. Ashley Colpaart is the founder of The Food Corridor, a system for cloud kitchen management. Ashley joins the show to talk about the dynamics of virtual restaurants and the cloud kitchen industry.
#1371
Mar 26, 2020 · 55 min
Remote Team Management with Ryan Chartrand
Remote engineering work makes some elements of software development harder, and some elements easier. With Slack and email, communication becomes more clear cut. Project management tools lay out the responsibilities and deliverables of each person. GitHub centralizes and defines the roles of developers. On the other hand, remote work subtracts the role of nuanced conversation. There is no water cooler or break room. Work can become systematic, rigid, and completely transactional. Your co-workers are your allies, but they feel less like friends when you don’t see them every day. For some people, this can have a devastating long-term impact on their psyche. Managers have the responsibility of ensuring the health and productivity of the people that work with them. Managing an all-remote team includes a different set of challenges than an in-person team. Ryan Chartrand is the CEO of X-Team, a team of developers who work across the world and collaborate with each other remotely. X-Team partners with large companies who need additional development work. Ryan joins the show to talk about the dynamics of leading a large remote workforce, as well as his own personal experiences working remotely.
#1370
Mar 25, 2020 · 52 min
Sorbet: Typed Ruby with Dmitry Petrashko
Programming languages are dynamically typed or statically typed. In a dynamically typed language, the programmer does not need to declare if a variable is an integer, string, or other type. In a statically typed language, the developer must declare the type of the variable upfront, so that the compiler can take advantage of that information. Dynamically typed languages give a programmer flexibility and fast iteration speed. But they also introduce the possibility of errors that can be avoided by performing type checking. This is one of the reasons why TypeScript has risen in popularity, giving developers the option to add types to their JavaScript variables. Sorbet is a typechecker for Ruby. Sorbet allows for gradual typing of Ruby programs, which helps engineers avoid errors that might otherwise be caused by the dynamic type system. Dmitry Petrashko is an engineer at Stripe who helped build Sorbet. He has significant experience in compilers, having worked on Scala before his time at Stripe. Dmitry joins the show to discuss his work on Sorbet, and the motivation for adding type checking to Ruby. We’re looking for new show ideas, so if you have any interesting topics, please feel free to reach out via twitter or email us at jeff@softwareengineeringdaily.com We realize right now humanity is going through a hard time with the Caronovirus pandemic, but we all have skills useful to fight this battle. Head over to codevid19.com to join the world’s largest pandemic hackathon!
#1369
Mar 24, 2020 · 1 hr
Datomic Architecture with Marshall Thompson
Datomic is a database system based on an append-only record keeping system. Datomic users can query the complete history of the database, and Datomic has ACID transactional support. The data within Datomic is stored in an underlying database system such as Cassandra or Postgres. The database is written in Clojure, and was co-authored by the creator of Clojure, Rich Hickey. Datomic has a unique architecture, with a component called a Peer, which gets embedded in an application backend. A Peer stores a subset of the database data in memory in this application backend, improving the latency of database queries that hit this caching layer. Marshall Thompson works at Cognitect, the company that supports and sells the Datomic database. Marshall joins the show to talk about the architecture of Datomic, its applications, and the life of a query against the database. We’re looking for new show ideas, so if you have any interesting topics, please feel free to reach out via twitter or email us at jeff@softwareengineeringdaily.com
#1368
Mar 23, 2020 · 46 min
Google Cloud Networking with Lakshmi Sharma
A large cloud provider has high volumes of network traffic moving through data centers throughout the world. These providers manage the infrastructure for thousands of companies, across racks and racks of multitenant servers, and cables that stretch underseas, connecting network packets with their destination. Google Cloud Platform has grown steadily into a wide range of products, including database services, machine learning, and containerization. Scaling a cloud provider requires both technical expertise and skillful management. Lakshmi Sharma is the director of product management for networking at Google Cloud Platform. She joins the show to discuss the engineering challenges of building a large scale cloud provider, including reliability, programmability, and how to direct a large hierarchical team. We’re looking for new show ideas, so if you have any interesting topics, please feel free to reach out via twitter or email us at jeff@softwareengineeringdaily.com
#1367
Mar 20, 2020 · 1 hr 2 min
ClickUp Engineering with Zeb Evans and Alex Yurkowski
Over the last fifteen years, there has been a massive increase in the number of new software tools. This is true at the infrastructure layer: there are more databases, more cloud providers, and more open-source projects. And it’s also true at a higher level: there are more APIs, project management systems, and productivity tools. ClickUp is a project management and productivity system for organizations and individuals. The goal of ClickUp is to create a system that integrates closely with other project management systems, popular SaaS tools, and the Google Suite of docs and spreadsheets. The company was started in 2016, and despite raising zero outside capital, it has grown as rapidly as many venture-backed companies. Zeb Evans and Alex Yurkowski are the founders of ClickUp. They join the show to talk about their experience building the company. We talk through their process of scaling the infrastructure, and their philosophy of moving fast. This episode has some useful strategic advice for anyone who is looking to take a product to market and iterate quickly–even if that product is bootstrapped. Full disclosure: ClickUp is a sponsor of Software Engineering Daily.
#1366
Mar 19, 2020 · 55 min
Pulumi: Infrastructure as Code with Joe Duffy
Infrastructure-as-code allows developers to use programming languages to define the architecture of their software deployments, including servers, load balancers, and databases. There have been several generations of infrastructure-as-code tools. Systems such as Chef, Puppet, Salt, and Ansible provided a domain-specific imperative scripting language that became popular along with the early growth of Amazon Web Services. Hashicorp’s Terraform project created an open source declarative model for infrastructure. Kubernetes YAML definitions are also a declarative system for infrastructure as code. Pulumi is a company that offers a newer system for infrastructure as code, combining declarative and imperative syntax. Pulumi programs can be written in TypeScript, Python, Go, or .NET. Joe Duffy is the CEO of Pulumi, and he joins the show to talk about his work on the Pulumi project and his vision for the company. Joe also discusses his twelve years at Microsoft, and how his work in programming language tooling shaped how he thinks about building infrastructure-as-code.

Showing 981–1,000 of 2,221 episodes