BusinessEntrepreneurshipManagementTechnology

Experiencing Data w/ Brian T. O’Neill

Brian T. O’Neill from Designing for Analytics

Your B2B analytics product does the hard technical work. So why aren't sales and adoption keeping up?

I'm Brian T. O'Neill, and my podcast, Experiencing Data, is a Listen Notes top 2% global podcast for founders, CEOs, and product leaders at B2B analytics companies: the ones whose product should be winning deals on technical sophistication alone, but isn't.

Maybe competitors have caught up on features and prospects say you all look the same, so you end up competing on price. Maybe customers never discover your best features and use you "like Excel." Maybe your value is algorithmic, and buyers can't see what they'd be paying for—something I call the Invisible Intelligence Gap.

Through solo episodes and interviews with CEOs, founders, and the enterprise data leaders who buy these products, I draw on my work as a product designer and strategy consultant to help you translate technical complexity into commercial clarity: connecting what your data product does to the outcomes, value, and human factors that still matter, even in the age of AI.

Experiencing Data is the only non-technical show about why technically impressive products don't sell or get used, and what to do about it.

Subscribe today on all major platforms or browse the episode archive.

Get 1-Page Episode Summaries In your Inbox:
https://designingforanalytics.com/ed

About Brian:
https://designingforanalytics.com/bio/

Play

Open in

207 episodes
fortnightly
Avg 41 min
English

Episodes 207

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

#26
Nov 19, 2019 · 47 min
026 - Why Tom Davenport Gives a 2 out of 10 Score To the Data Science and Analytics Industry for Value Creation
Tom Davenport has literally written the book on analytics. Actually, several of them, to be precise. Over the course of his career, Tom has established himself as the authority on analytics and how their role in the modern organization has evolved in recent years. Tom is a distinguished professor at Babson College, a research fellow at the MIT Initiative on the Digital Economy, and a senior advisor at Deloitte Analytics. The discussion was timely as Tom had just written an article about a financial services company that had trained its employees on human-centered design so that they could ensure any use of AI would be customer-driven and valuable. We discussed their journey and: Why on a scale of 1-10, the field of analytics has only gone from a one to about a two in ten years time Why so few analytics projects actually make it into production Examples of companies who are using design to turn data into useful applications of AI, decision support and product improvements for customers Why shadow IT shouldn’t be a bad word AI moonshot projects vs. MVPs and how they relate Why journey mapping is incredibly useful and important in analytics and data science work How human-centered design and ethnography is the tough work that’s required to turn data into decision support Tom’s new book and his thoughts on the future of data science and analytics Resources and Links: Website: Tomdavenport.com LinkedIn: Tom Davenport Twitter: @tdav Designingforanalytics.com/seminar Designingforanalytics.com Quotes from Today’s Episode “If you survey organizations and ask them, ‘Does your company have a data-driven culture?’ they almost always say no. Surveys even show a kind of negative movement over recent years in that regard. And it's because nobody really addresses that issue. They only address the technology side.” — Tom Eventually, I think some fraction of [AI and analytics solutions] get used and are moderately effective, but there is not nearly enough focus on this. A lot of analytics people think their job is to create models, and whether anybody uses it or not is not their responsibility...We don't have enough people who make it their jobs to do that sort of thing. —Tom I think we need this new specialist, like a data ethnographer, who could sort of understand much more how people interact with data and applications, and how many ways they get screwed up.—Tom I don't know how you inculcate it or teach it in schools, but I think we all need curiosity about how technology can make us work more effectively. It clearly takes some investment, and time, and effort to do it.— Tom TD Wealth’s goal was to get [its employees] to experientially understand what data, analytics, technology, and AI are all about, and then to think a lot about how it related to their customers. So they had a lot of time spent with customers, understanding what their needs were to make that match with AI. [...] Most organizations only address the technology and the data sides, so I thought this was very refreshing.—Tom “So we all want to do stuff with data. But as you know, there are a lot of poor solutions that get provided from technical people back to business stakeholders. Sometimes they fall on deaf ears. They don't get used.” — Brian “I actually had a consultant I was talking to recently who said you know the average VP/director or CDO/CAO has about two years now to show results, and this gravy train may be slowing down a little bit.“ — Brian “One of the things that I see in the kind of the data science and analytics community is almost this expectation that ‘I will be handed a well-crafted and well-defined problem that is a data problem, and then I will go off and solve it using my technical skills, and then provide you with an answer.’” — Brian
#25
Nov 5, 2019 · 39 min
025 - Treating Data Science at IDEO as a Discipline of Design with Dean Malmgren
Dean Malmgren cut his teeth as a chemical and biological engineer. In grad school, he studied complex systems and began telling stories about them through the lens of data algorithms. That led him to co-found Datascope Analytics, a data science consulting company which was purchased by IDEO, a global design firm. Today, Dean is an executive director at IDEO and helps teams use data to build delightful products and experiences. Join Dean and I as we explore the intersection of data science and design and discuss: Human-centered design and why it’s important to data science What it was like for a data science company to get ingested into a design firm and why it’s a perfect match Why data science isn’t always good at creating things that have never existed before Why teams need to prototype rapidly and why data scientists should hesitate to always use the latest tools What data scientists can learn from design team and vice-versa Why data scientists need to talk to end users early and often, and the importance of developing empathy The difference between data scientists and algorithm designers Dean’s opinions on why many data analytics projects fail Resources and Links Twitter: @DeanMalmgren IDEO Datascope Quotes from Today’s Episode “One of the things that we learned very, very quickly, and very early on, was that designing algorithms that are useful for people involves a lot more than just data and code.” — Dean “In the projects that we do at IDEO, we are designing new things that don’t yet exist in the world. Designing things that are new to the world is pretty different than optimizing existing processes or business units or operations, which tends to be the focus of a lot of data science teams.” — Dean “The reality is that designing new-to-the-world things often involves a different mindset than optimizing the existing things.” — Dean “You know if somebody rates a movie incorrectly, it’s not like you’d throw out Siskel and Ebert’s recommendations for the rest of your life. You just might not pay as much attention to them. But that’s very different when it comes to algorithmic recommendations. We have a lot less tolerance for machines making mistakes.” — Dean “The key benefit here is the culture that design brings in terms of creating early and getting feedback early in the process, as opposed to waiting you know three, five, six, seven months working on some model, getting it 97% accurate but 10% utilized.” — Brian “You can do all the best work in the world. But at the end of the day, if there’s a human in the loop, it’s that last mile or last hundred feet, whatever you want to call it, where you make it or break it.” — Brian “Study after study shows that 10 to 20% of big data analytics projects and AI projects succeed. I’ve actually been collecting them as a hobby in a single article, because they keep coming out.” — Brian
#24
Oct 22, 2019 · 35 min
024 - How Empathy Can Reveal a 60%-Accurate Data Science Solution is a Solid Customer Win with David Stephenson, Ph.D.
David Stephenson, Ph.D., is the author of Big Data Demystified, a guide for executives that explores the transformative nature of big data and data analytics. He’s also a data strategy consultant and professor at the University of Amsterdam. In a previous life, David worked in various data science roles at companies like Adidas, Coolblue, and eBay. Join David and I as we discuss what makes data science projects succeed and explore: The non-technical issues that lead to ineffective data science and analytics projects The specific type of communication that is critical to the success of data science and analytics initiatives (and how working in isolation from your stakeholder or business sponsor creates risk)) The power of showing value early, starting small/lean, and one way David applies agile to data science projects The problems that emerge when data scientists only want to do “interesting data science” How design thinking can help data scientists and analytics practitioners make their work resonate with stakeholders who are not “data people” How David now relies on design thinking heavily, and what it taught him about making “cool” prototypes nobody cared about What it’s like to work on a project without understanding who’s sponsoring it Resources and Links DSI Analytics Website Connect with David on LinkedIn David’s book: Big Data Demystified On Twitter: @Stephenson_Data Quotes from Today’s Episode “You see a lot of solutions being developed very well, which were not designed to meet the actual challenge that the industry is facing.” — David “You just have that whole wasted effort because there wasn’t enough communication at inception.” — David “I think that companies are really embracing agile, especially in the last few years. They’re really recognizing the value of it from a software perspective. But it’s really challenging from the analytics perspective—partly because the data science and analytics. They don’t fit into the scrum model very well for a variety of reasons.” — David “That for me was a real learning point—to understand the hardest thing is not necessarily the most important thing.” — David “If you’re working with marketing people, an 80% solution is fine. If you’re working with finance, they really need exact numbers. You have to understand what your target audience needs in terms of precision.” — David “I feel sometimes that when we talk about “the business” people don’t understand that the business is a collection of people—just like a government is a collection of real humans doing jobs and they have goals and needs and selfish interests. So there’s really a collection of end customers and the person that’s paying for the solution.” — Brian “I think it’s always important—whether you’re a consultant or you’re internal—to really understand who’s going to be evaluating the value creation.”— Brian “You’ve got to keep those lines of communication open and make sure they’re seeing the work you’re doing and evaluating and giving feedback on it. Throw this over the wall is a very high risk model.” — Brian
#23
Oct 8, 2019 · 49 min
023 - Balancing AI-Driven Automation with Human Intervention When Designing Complex Systems with Dr. Murray Cantor
Dr. Murray Cantor has a storied career that spans decades. Recently, he founded Aptage, a company that provides project risk management tools using Bayesian Estimation and machine learning. He’s also the chief scientist at Hail Sports, which focuses on applying precision medicine techniques to sports performance. In his spare time, he’s a consulting mathematician at Pattern Computer, a firm that engineers state-of-the-art pattern recognition solutions for industrial customers. Join Murray and I as we explore the cutting edge of AI and cover: Murray’s approach to automating processes that humans typically do, the role humans have in the design phase, and how he thinks about designing affordances for human intervention in automated systems Murray’s opinion on causal modeling (explainability/interpretability), the true stage we are in with XAI, and what’s next for causality in AI models Murray’s opinions about the 737 Max’s automated trim control system interface (or lack thereof) and how it should have been designed The favorite method Murray has for predicting outcomes from small data sets The major skill gaps Murray sees with young data scientists in particular How using science fiction stories can stimulate creative thinking and help kick off an AI initiative successfully with clients, customers and stakeholders Resources and Links Murray Cantor on LinkedIn New York Times Expose article on the Boeing 737 Max New Your Times Article on the 737 Max whistleblower Quotes from Today’s Episode “We’re in that stage of this industrial revolution we’re going through with augmenting people’s ability with machine learning. Right now it’s more of a craft than a science. We have people out there who are really good at working with these techniques and algorithms. But they don’t necessarily understand they’re essentially a solution looking for a problem.” — Murray “A lot of design principles are the same whether or not you have AI. AI just raises the stakes.” — Murray “The big companies right now are jumping the guns and saying they have explainable AI when they don’t. It’s going to take a while to really get there.” — Murray “Sometimes, it’s not always understood by non-designers, but you’re not testing the people. You’re actually testing the system. In fact, sometimes they tell you to avoid using the word test when you’re talking to a participant, and you tell them it’s a study to evaluate a piece of software, or in this case a cockpit, to figure out if it’s the right design or not. It’s so that they don’t feel like they’re a rat in the maze. In reality, we’re studying the maze.” — Brian “Really fundamental to understanding user experience and design is to ask the question, who is the population of people who are going to use this and what is their range of capability?” – Murray “Take the implementation hats off and come up with a moonshot vision. From the moonshot, you might find out there are these little tangents that are actually feasible increments. If you never let yourself dream big, you’ll never hit the small incremental steps that you may be able to take.” — Brian
#22
Sep 24, 2019 · 51 min
022 - Creating a Trusted Data Science Team That Is Indispensable to the Business with
Scott Friesen’s transformation into a data analytics professional wasn’t exactly linear. After graduating with a biology degree and becoming a pre-med student, he switched gears and managed artists in the music industry. After that, he worked at Best Buy, eventually becoming their Senior Director of Analytics for the company’s consumer insights unit. Today, Scott is the SVP of Strategic Analytics at Echo Global Logistics, a provider of technology-enabled transportation and supply chain management services. He also advises for the International Institute for Analytics. In this episode, Scott shares what he thinks data scientists and analytics leaders need to do to become a trustworthy and indispensable part of an organization. Scott and I both believe that designing good decision support applications and creating useful data science solutions involve a lot more than technical knowledge. We cover: Scott’s trust equation, why it’s critical for analytics professionals, and how he uses it to push transformation across the organization Scott’s “jazz” vs “classical” approach to creating solutions How to develop intimacy and trust with your business partners (e.g., IT) and executives, and the non-technical skills analytics teams need to develop to be successful Scott’s opinion about design thinking and analytics solutions How to talk about risk to business stakeholders when deploying data science solutions How the success of Scott’s new pricing model was impeded by something that had nothing to do with the data—and how he addressed it Scott’s take on the emerging “analytics translator” role The two key steps to career success—and volcanos Resources and Links Scott Friesen on LinkedIn Quotes from Today's Episode “You might think it is more like classical music, but truly great analytics are more like jazz. ” — Scott “If I'm going to introduce change to an organization, then I'm going to introduce perceived risk. And so the way for me to drive positive change—the way for me to drive adding value to the organizations that I'm a part of—is the ability to create enough credibility and intimacy that I can get away with introducing change that benefits the organization.” — Scott “I categorize the analytic pursuit into three fundamental activities: The first is to observe, the second is to relate, and the third is to predict. ” — Scott “It's not enough to just understand the technology part and how to create great models. You can get all that stuff right and still fail in the last mile to deliver value.” — Brian “I tend to think of this is terms of what you called ‘intimacy.’ I don’t know if you equate that to empathy, which is really understanding the thing you are talking about from the perspective of the other person. When we do UX research, the questions themselves are what form this intimacy. An easy way to do that is by asking open-ended questions that require open-ended answers to get that person to open up to you. ” — Brian
#21
Sep 10, 2019 · 42 min
021 - Turning Complex Cloud IT Data Into Useful Decision Support Info with John Purcell of
John Purcell has more than 20 years of experience in the technology world. Currently, he’s VP of Products at CloudHealth, a company that helps organizations manage their increasingly complex cloud infrastructure effectively. Prior to this role, he held the same position at SmartBear Software, makers of application performance monitoring solutions. He’s also worn several hats at companies like LogMeIn and Red Bend Software. In today’s episode, John and I discuss how companies are moving more and more workloads to the cloud and how John and his team at CloudHealth builds a platform that makes it easy for all users—even non-technical ones—to analyze and manage data in the cloud and control their financial spending. In addition to exploring the overall complexity of using analytics to inform the management of cloud environments, we also covered: How CloudHealth designs for multiple personas from the financial analyst to the DevOps operator when building solutions into the product Why John has “maniacal point of view” and “religion” around design and UX and how they have the power to disrupt a market How design helps turn otherwise complex data sets that might require an advanced degree to understand into useful decision support How data can lead to action, and how CloudHealth balances automation vs. manual action for its customers using data to make decisions Why John believes user experience is a critical voice at the table during the very earliest stages of any new analytics/data initiative Resources and Links Twitter: @PurcellOutdoors LinkedIn Quotes from Today’s Episode “I think that’s probably where the biggest point of complexity and the biggest point of challenge is for us: trying to make sure that the platform is flexible enough to be able to inject datasets we’ve never seen before and to be able to analyze and find correlations between unknown datasets that we may not have a high degree of familiarity with—so that we can generate insight that’s actionable, but deliver it in a way that’s [easy for anyone to understand].” — John “My core philosophy is that you need UX at the table early and at every step along the way as you’re contemplating product delivery, and I mean all the way upstream at the strategic phase, at the identification of what you want to go tackle next including product strategy, pain identification, persona awareness, and who are we building for—all the way through solving the problem, what should the product be capable of, and user validation.” — John “in the cloud, we’re just at the very early stages of [automation based on analytics] from a pure DevOps point of view. We’re still in the world of show me your math. Show me why this is the recommendation you’re making.” — John “When making decisions using data, some IT people don’t like the system taking action without them being involved because they don’t trust that any product would be smart enough to make all the right decisions, and they don’t want applications going down.” — Brian “I think the distinction you made between what I would call user interface design, which is the surface layer, buttons, fonts, colors, all that stuff often gets conflated in the world of analytics as being, quote ‘design.’ And as I think our audience is hearing from John here, is that it [design] goes much beyond that. It can get into something like, ‘how do you design a great experience around API documentation? Where’s the demo code? How do I run the demo?’ All of that can definitely be designed as well.” — Brian “I hear frequently in my conversations with clients and people in the industry that there are a lot of data scientists who just want to use the latest models, and they want to work on model quality and predictive accurateness, etc. But they’re not thinking about how someone is going to use this model to make a decision, and whether will there be some business value created at the end.” — Brian
#20
Aug 27, 2019 · 45 min
020 - How Human-Centered Design Increases Engagement with Data Science Initiatives
Ahmer Inam considers himself an evangelist of data science who’s been “doing data science since before it was called data science. With more than 20 years of leadership experience in the data science and analytics field at companies including Nike and Cambia health, Ahmer knows a thing or two about what makes data science projects succeed—and what makes them fail. In today’s episode, Ahmer and I discuss his experiences using design thinking and his “human-centered AI” process to ensure that internal analytics and data science initiatives actually produce usable, useful outputs that turn into business value. Much of this was formed while Ahmer was a Senior Director and Head of Advanced Analytics at Nike, a company that is known as a design-mature organization. We covered: The role of empathy in data science teams and how it helps data people connect with non-technical users who may not welcome “yet another IT tool” Ahmer’s thoughts on Lean Coaching, Scrum Teams, and getting outside help to accelerate the design and creation of your first data products and predictive models The role of change management in the process of moving data products into production Ahmer’s two-week process to kick-start data product initiatives used at Nike How model accuracy isn’t as important early on as other success metrics when prototyping solutions with customers Resources and Links How Analytics Are Informing Change At Nike LinkedIn Quotes from Today’s Episode “Build data products with the people, for the people…and bring a sense of vulnerability to the table.” — Ahmer “What I have seen is that a lot of times we can build models, we can bring the best of the technologies on optimal technology it’s in the platforms, but in the end, if the business process and the people are not ready to take it and use it, that’s where it fails.” — Ahmer “If we don’t understand people in the process, essentially, the adoption is not going to work. In the end, when it comes to a lot of these data science exercises or projects or development of data products, we have to really think about it as a change management exercise and nothing short of that.” — Ahmer “Putting humans at the center of these initiatives drives better value and it actually makes sure that these tools and data products that we’re making actually get used, which is what ultimately is going to determine whether or not there’s any business value—because the data itself doesn’t have any value until it’s acted upon.” — Brian “One of these that’s been stuck in my ear like an earworm is that a lot of the models fail to get to production still. And so this is the ongoing theme of basically large analytics projects, whether you call it big data analytics or AI, it’s the same thing. We’re throwing a lot of money at these problems, and we’re still creating poor solutions that end up not doing anything.” — Brian “I think the really important point here is that early on with these initiatives, it’s important to figure out, What is going to stop this person from potentially engaging with my service?” — Brian
#19
Aug 13, 2019 · 38 min
019 - The Non-Technical (Human!) Challenges that Can Impede Great Data Science Solutions
Dr. Bob Hayes, will be the first to tell you that he’s a dataphile. Ever since he took a stats course in college in the 80s, Bob’s been hooked on data. Currently, Bob is the Research and Analytics Director at Indigo Slate. He’s also the president of Business over Broadway, a consultancy he founded in 2007.In a past life, Bob served as Chief Research Officer at Appuri and AnalyticsWeek, Chief Customer Officer at TCELab, and a contributing analyst at Gleanster, among many other roles. In today’s episode, Bob and I discuss a recent Kaggle survey that highlighted several key non-technical impediments to effective data science projects. In addition to outlining what those challenges are and exploring potential solutions to them, we also covered: The three key skills successful data science teams have Why improving customer loyalty involves analyzing several metrics, not just one Why Bob feels the scientific method is just as important today as it’s been for hundreds of years The importance of repeatable results How prototyping early can save time and drive adoption of data science projects Bob’s advice on how to move data science projects forward (and one key skill he feels most business leaders lack) The role of the analytics translator Resources and Links: Dr. Bob Hayes on LinkedIn Seeing Theory Calling Bullshit Doctor Bob Hayes on Twitter Business Over Broadway IndigoSlate Quotes from Today’s Episode “I’ve always loved data. I took my first stats course in college over 30 years ago and I was hooked immediately. I love data. Sometimes I introduce myself as a dataholic. I love it.” — Bob “I’m a big fan of just kind of analyzing data, just getting my hands on data, just exploring it. But that can lead you down a path of no return where you’re just analyzing data just to analyze it. What I try to tell my clients is that when you approach a data set, have a problem that you’re trying to solve. The challenge there I think it stems from the fact that a lot of data science teams don’t have a subject matter expert on the team to pose the right questions.” — Bob “The three findings that I found pretty interesting were, number one, a lack of a clear question to be answering or a clear direction to go in with the available data. The second one was that data science results were not used by the business decision makers. And the third one was an inability to integrate findings into the organization’s decision making processes.” — Brian “It makes you wonder,‘if you didn’t have a good problem to solve, maybe that’s why [the findings] didn’t get used in the first place.’” — Brian “That part isn’t so much the math and the science. That’s more the psychology and knowing how people react. Because you’re going to have certain business stakeholders that still want to kind of shoot from the hip and their experience. Their gut tells them something. And sometimes that gut is really informed.” — Brian “If executives are looking at data science and AI as a strategic initiative, it seems really funny to me that someone wouldn’t be saying, ‘What do we get out of this? What are the next steps?’ when the data teams get to the end of a project and just moves on to the next one.” — Brian
#18
Jul 30, 2019 · 57 min
018 - The Business Value of Showing the “Why” in AI Models with Jana Eggers (CEO, Naralogics)
Jana Eggers, a self-proclaimed math and computer nerd, is CEO of Nara Logics, a company that helps organizations use AI to eliminate data silos and unlock the full value of their data, delivering predictive personalized experiences to their customers along the way. The company leverages the latest neuroscience research to model data the same way our brains do. Jana also serves on Fannie Mae’s digital advisory board, which is tasked with finding affordable housing solutions across the United States. Prior to joining Nara Logics, Jana wore many different hats, serving as CEO of Spreadshirt, and General Manager of QuickBase at Intuit, among other positions. She also knows about good restaurants in PDX! In today’s episode, Jana and I explore her approaches to using AI to help enterprises make interesting and useful predictions that drive better business outcomes and improve customer experience. In addition to discussing how AI can help strengthen personalization and support smarter decision making, we also covered: The power of showing the whys when providing predictions (i.e., explainable AI or XAI). Jana’s thoughts on why some data scientists struggle with inflated expectations around AI Brian’s #facepalm about lipstick and data The power of what-if simulations and being able to remove factors from predictions The power of context and how Nara Logics weighs recent data vs. long-term historical data in its predictions How Nara Logics leverages the wiring of the brain—the connectome—to inspire the models they build and the decision support help they provide to customers Why AI initiatives need to consider the “AI trinity”: data, the algorithm, and the results an organization is aiming for Resources and Links: Nara Logics Follow Jana on Twitter Connect with Jana on LinkedIn Quotes from Today’s Episode “We have a platform that is really built for decision support. How do you go from having […]20 to having about 500 to 2,000 decision factors coming in? Once we get that overload of information, our tool is used to help people with those decisions. And yes, we’re using a different approach than the traditional neural net, which is what deep learning is based on. While we use that in our tool, we’re more on the cognitive side. […]I’ve got a lot of different signals coming in, how do I understand how those signals relate to each other and then make decisions based on that?” — Jana “One of the things that we do that also stands us apart is that our AI is transparent—meaning that when we provide an answer, we also give the reasons why that is the right answer for this context. We think it is important to know what was taken into account and what factors weigh more heavily in this context than other contexts.” — Jana “It is extremely unusual—and I can even say that I’ve never really seen it—that people just say, Okay, I trust the machine. I’m comfortable with that. It knows more than me. That’s really unusual. The only time I’ve seen that is when you’re really doing something new and no one there has any idea what it should be.” — Jana “With regards to tech answering “why,” I’ve worked on several monitoring and analytics applications in the IT space. When doing root cause analysis, we came up with this idea of referring to monitored objects as being abnormally critical and normally critical. Because at certain times of day, you might be running a backup job and so the IO is going crazy, and maybe the latency is higher. But the IO is supposed to be that way at that time. So how do you knock down that signal and not throw up all the red flags and light up the dashboard when it’s supposed to be operating that way? Answering “why” is difficult. ” — Brian “We’ve got lipstick, we’ve got kissing. I’m going to get flagged as ‘parental advisory’ on this episode in iTunes probably. ;-)” — Brian “You can’t just live in the closet and do your math and hope that everyone is going to see the value of it. Anytime we’re building these complex tools and services —what I call human-in-the-loop applications–you’re probably going to have to go engage with other humans, whether it’s customers or your teammates or whatever.” — Brian
#17
Jul 16, 2019 · 48 min
017 - John Cutler on Productizing Storytelling Measuring What Matters & Analytics Product Management
John Cutler is a Product Evangelist for Amplitude, an analytic platform that helps companies better understand users behavior, helping to grow their businesses. John focuses on user experience and evidence-driven product development by mixing and matching various methodologies to help teams deliver lasting outcomes for their customers. As a former UX researcher at AppFolio, a product manager at Zendesk, Pendo.io, AdKeeper and RichFX, a startup founder, and a product team coach, John has a perspective that spans individual roles, domains, and products. In today’s episode, John and I discuss how productizing storytelling in analytics applications can be a powerful tool for moving analytics beyond vanity metrics. We also covered the importance of understanding customers’ jobs/tasks, involving cross-disciplinary teams when creating a product/service, and: John and Amplitude’s North Star strategy and the (3) measurements they care about when tracking their own customers’ success Why John loves the concept of analytics “notebooks” (also a particular feature of Amplitude’s product) vs. the standard dashboard method Understanding relationships between metrics through “weekly learning users” who share digestible content John’s opinions on involving domain experts and cross-discipline teams to enable products focused on outcomes over features Recognizing whether your product/app is about explanatory or exploratory analytics How Jazz relates to business – how you don’t know what you don’t know yet Resources and Links: Connect with John on LinkedIn Follow John on Twitter Keep up with John on Medium Amplitude Designing for Analytics Quotes from Today’s Episode “It’s like you know in your heart you should pair with domain experts and people who know the human problem out there and understand the decisions being made. I think organizationally, there’s a lot of organizational inertia that discourages that, unfortunately, and so you need to fight for it. My advice is to fight for it because you know that that’s important and you know that this is not just a pure data science problem or a pure analytics problem. There’s probably there’s a lot of surrounding information that you need to understand to be able to actually help the business.” – John “We definitely ‘dogfood’ our product and we also ‘dogfood’ the advice we give our customers.” – John “You know in your heart you should pair with domain experts and people who know the human problem out there and understand the decisions being made. […] there’s a lot of organizational inertia that discourages that, unfortunately, and so you need to fight for it. I guess my advice is, fight for it, because you know that it is important, and you know that this is not just a pure data science problem or a pure analytics problem.” – John “It’s very easy to create assets and create code and things that look like progress. They mask themselves as progress and improvement, and they may not actually return any business value or customer value explicitly. We have to consciously know what the outcomes are that we want.” – Brian “We got to get the right bodies in the room that know the right questions to ask. I can smell when the right questions aren’t being asked, and it’s so powerful” – Brian “Instead of thinking about what are all the right stats to consider, [I sometimes suggest teams] write in plain English, like in prose format, what would be the value that we could possibly show in the data.’ maybe it can’t even technically be achieved today. But expressing the analytics in words like, ‘you should change this knob to seven instead of nine because we found out X, Y, and Z happened. We also think blah, blah, blah, blah, blah, and here is how we know that, and there’s your recommendation.’ This method is highly prescriptive, but it’s an exercise in thinking about the customer’s experience.” – Brian
#16
Jul 2, 2019 · 46 min
016 - Farming with Data: How Advanced Analytics are Transforming the Agriculture Industry with Dinu Ajikutra
Today we are joined by Dinu Ajikutira, VP of Product at CiBO Technologies. CiBO Technologies was founded in 2015. It was created to provide an objective, scientifically-driven insights in support of farmland economics. Dinu is currently leading an effort to productize what I found to be some very impressive analytically-driven simulation capabilities to help farmers and agronomists. Specifically, CiBO’s goal is to provide a software service that uses mapping and other data to predictively model a piece of land’s agricultural value –before crops are ever planted. In order to build a product that truly meets his customer needs, Dinu goes the extra mile–in one case, 1000 miles– to literally meet his customers in the field to understand their pain points. On this episode, Dinu and I discuss how CiBO will help reduce farmers’ risk, optimize crop yields, and the challenges of the agriculture industry from a data standpoint. We also discussed: Farmers’ interactions with data analytics products and how to improve their trust with those products Where CiBO’s software can be used and who would benefit from it Dinu’s “ride-along” experience visiting farmers and agronomists in the midwest to better understand customer needs and interactions with the tool What Dinu has learned about farmers’ comfort using technology The importance of understanding seasonality The challenges of designing the tool for the various users and building user interfaces based on user needs The biggest product challenges in the ag tech field and how CiBO handles those challenges Resources and Links: LinkedIn CiBO Technologies Experiencing Data Podcast Quotes from Today’s Episode “CiBO was built on a mission of enabling sustainable agriculture, and we built this software platform that brings weather, soil, topography, and agronomic practices in combination with simulation to actually digitally grow the plant, and that allows us to explain to the users why something occurs, what if something different had happened, and predict the outcomes of how plants will perform in different environments.” — Dinu Ajikutira “The maturity of the agricultural industry [with regards] to technology is in its early stages, and it’s at a time when there is a lot of noise around AI,machine learning and data analytics. That makes it very complicated, because you don’t know if the technology really does what it claims to do, and there is a community of potential users that are not used to using a high-tech technology to solve their problems.” — DInu Ajikutira “In agriculture, the data is very sparse, but with our software we don’t need all the data. We can supplement data that is missing, using our simulation tools, and be able to predict weather outcomes that you have not experienced in the past.” — Dinu Ajikutira “To add clarity, you need to add information sometimes, and the issue isn’t always the quantity of the information; it’s how it’s designed.I’ve seen this repeatedly where there are times if you properly add information and design it well, you actually bring a lot more insight.” – Brian O’Neill “Sometimes the solution is going to be to add information, and if you’re feeling like you have a clutter problem, if your customers are complaining about too much information, or that’s a symptom usually that the design is wrong. It’s not necessarily that that data has no value. It may be the wrong data.” — Brian O’Neill
#15
Jun 18, 2019 · 49 min
015 – Opportunities and Challenges When Designing IoT Analytics Experiences for the Industrial & Manufacturing Industries with CEO Bill Bither
Bill Bither, CEO and Co-Founder of MachineMetrics, is a serial software entrepreneur and a manufacturing technology leader. He founded and bootstrapped Atalasoft to image-enable web applications which led to a successful exit in 2011 to Kofax. In 2014, he co-founded MachineMetrics to bring visibility and predictability to the manufacturing floor with an Industrial IoT analytics platform that collects data from machines. This data is used to benchmark performance, drive efficiency, improve equipment uptime, and enable automation. Today, join us as we discuss the various opportunities and challenges in the complex world of industrial IoT and manufacturing. Bill and I discuss the importance of visualizations and its relationship to improving efficiency in manufacturing, how talking to machine operators help add context to analytics data and even inform UI/UX decisions, as well as how MachineMetrics goes about making the telemetry from these machines useful to the operators. We also covered: How improving a customer’s visibility into CNC machines helped reveal accurate utilization rates and improved efficiency How simple visualizations make a tangible difference in operational performance Bill’s model for the 4 different phases of analytics Descriptive Diagnostic Predictive Prescriptive Mistakes Bill learned early on about product dev in the IIoT analytics space What Bill learned from talking to customers that ended up identifying a major design flaw his team wasn’t aware of The value you can glean from talking to customers MachineWorks’ challenges with finding their market fit and aligning their product around customer’s needs How MachineMetrics has learned to simplify the customer’s analytics experience Resources and Links Bill Bither on LinkedIn MachineMetrics Quotes from Today’s Episode “We have so much data, but the piece that really adds enormous value is human feedback.” — Bill “Simplicity is really hard. It takes time because it requires empathy and it requires going in and really getting into the head or the life of the person that’s gonna use your tool. You have to understand what’s it like being on a shop floor running eight different CNC machines. If you’ve never talked to someone, it’s really hard to empathize with them.” — Brian “In all the work that we do, in adding more intelligence to the product, it’s just making the experience simpler and simpler.” — Bill “You don’t have to go in and do great research; you can go in and just start doing research and learn on the way. It’s like going to the gym. They always tell you, ‘It doesn’t matter what exercise you do, just go and start.’ …then you can always get better at making your workout optimal.” — Brian “It’s really valuable to have routine visits with customers, because you just don’t know what else might be going on.” — Brian “The real value of the research is asking ‘why’ and ‘how,’ and getting to the root problem. That’s the insight you want. Customers may have some good design ideas, but most customers aren’t designers. … Our job is to give people what they need.” — Brian
#14
Jun 4, 2019 · 41 min
014 - How Worthington Industries Makes Predictive Analytics Useful from the Steel Mill Floor to the Corner Office with Dr. Stephen Bartos
Today we are joined by the analytics “man of steel,” Steve Bartos, the Manager of the Predictive Analytics team in the steel processing division at Worthington Industries. 😉 At Worthington, Steve is tasked with strategically driving impactful analytics wider and deeper across the division and, as part of this effort, helps ensure an educated, supported, and connected analytics community. In addition, Steve also serves as a co-leader of the Columbus Tableau User Group. On today’s episode, Steve and I discuss how analytics are providing internal process improvements at Worthington. We also cover the challenges Steve faces designing effective data-rich products, the importance of the “last mile,” and how his PhD in science education shapes his work in predictive analytics. In addition, we also talk about: Internal tools that Steve has developed and how they help Worthington Industries. Preplanning and its importance for creating a solution that works for the client. Using analytics to inform daily decisions, aid in monthly meetings, and assist with Kaizen (Lean) focused decisions. How Steve pulls out the meaningful pieces of information that can improve the division’s performance. How Steve tries to avoid Data-Rich and Insight-Poor customer solutions The importance of engaging the customer/user throughout the process How Steve leverages his science education background to communicate with his peers and with executives at Worthington Resources and Links Twitter: @OlDirtyBarGraph Steve Bartos LinkedIn Quotes from Today’s Episode “Seeing the way analytics can help facilitate better decision making, doesn't necessarily come with showing someone every single question they can possibly answer, waiting for them to applaud how much time and how much energy and effort you'd saved them.” - Steve Bartos “It's hard to talk about the influence of different machine parameters on quality if every operator is setting it up based on their own tribal knowledge of how it runs best.” - Steve Bartos “I think bringing the question back to the user much more frequently, much sooner and at a much more focused scale has paid dividends”. - Steve Bartos “It's getting the people that are actually going to sit and use these interfaces involved in the creation process… they should be helping you define the goals and the problems… by getting them involved, it makes the adoption process a lot easier.” - Brian O’Neill “It's real easy to throw up some work that you've done in Tableau around a question that a manager or an executive had. It's real easy to do that. It's really difficult to do that well and have some control of the conversation, being able to say, here's what we did, here was the question, here's the day we use, here's how we analyze it and here's a suggestion where making and now let's talk about why and do that in a way that doesn't lead to an in-the-weeds session and frustration.” - Steve Bartos
#13
May 21, 2019 · 44 min
013 - Paul Mattal (Dir. of Network Systems, Akamai) on designing decision support tools and analytics services for the largest CDN on the web
Paul Mattal is the Director of Network Systems at Akamai, one of the largest content delivery networks in the U.S. Akamai is a major part of the backbone of the internet and on today’s episode, Paul is going to talk about the massive amount of telemetry that comes into Akamai and the various decision support tools his group is in charge of providing to internal customers. On top of the analytics aspect of our chat, we also discussed how Paul is approaching his team’s work being relatively new at Akamai. Additionally, we covered: How does Paul access and use internal customer knowledge to improve the quality of applications they make? When to build a custom decision support tool vs. using a BI tool like Tableau? How does Akamai measure if their analytics are creating customer value? The process Paul uses with the customer to design a new data product MVP How Paul decides which of the many analytics applications and services “get love” when resources are constrained Paul’s closing advice about taking the time to design and plan before you code Resources and Links: Akamai Twitter @pjmattal Paul Mattal on LinkedIn Paul Mattal on Facebook Quotes from Today’s Episode “I would say we have a lot of engagement with [customers] here. People jump to answering questions with data and they’re quick. They know how to do that and they have very good ideas about how to make sure that the approaches they take are backed by data and backed by evidence.” — Paul Mattal “There’s actually a very mature culture here at Akamai of helping each other. Not necessarily taking on an enormous project if you don’t have the time for it, but opening your door and helping somebody solve a problem, if you have expertise that can help them.” — Paul Mattal “I’m always curious about feedback cycles because there’s a lot of places that they start with telemetry and data, then they put technology on top of it, they build a bunch of software, and look at releases and outputs as the final part. It’s actually not. It’s the outcomes that come from the stuff we built that matter. If you don’t know what outcomes those look like, then you don’t know if you actually created anything meaningful.” — Brian O’Neill “We’ve talked a little bit about the MVP approach, which is about doing that minimal amount of work, which may or may not be working code, but you did a minimum amount of stuff to figure out whether or not it’s meeting a need that your customer has. You’re going through some type of observation process to fuel the first thing, asset or output that you create. It’s fueled by some kind of observation or research upfront so that when you go up to bat and take a swing with something real, there’s a better chance of at least a base hit.” — Brian O’Neill “Pretend to be the new guy for as long as you can. Go ask [about their needs/challenges] again and get to really understand what that person [customer] is experiencing, because I know you’re going to able to meet the need much better.” — Paul Mattal
#12
May 7, 2019 · 42 min
012 - Dr. Andrey Sharapov (Data Scientist, Lidl) on explainable AI and demystifying predictions from machine learning models for better user experienc...
Dr. Andrey Sharapov is a senior data scientist and machine learning engineer at Lidl. He is currently working on various projects related to machine learning and data product development including analytical planning tools that help with business issues such as stocking and purchasing. Previously, he spent 2 years at Xaxis and he led data science initiatives and developed tools for customer analytics at TeamViewer. Andrey and I met at a Predicitve Analytics World conference we were both speaking at, and I found out he is very interested in “explainable AI,” an aspect of user experience that I think is worth talking about and so that’s what today’s episode will focus on. In our chat, we covered: Lidl’s planning tool for their operational teams and what it predicts. The lessons learned from Andrey’s first attempt to build an explainable AI tool and other human factors related to designing data products What explainable AI is, and why it is critical in certain situations How explainable AI is useful for debugging other data models We discuss why explainable AI isn’t always used Andrey’s thoughts on the importance of including your end user in the data production creation process from the very beginning. Also, here’s a little post-episode thought from a design perspective: I know there are counter-vailing opinions that state that explainability of models is “over-hyped.” One popular rationalization uses examples such as how certain professions (e.g. medical practitioners) make decisions all the time that cannot be fully explained, yet people believe the decision making without necessarily expecting it to be fully explained. The reality is that while not every model or end UX necessarily needs explainability, I think there are human factors that can be satisfied by providing explainability such as building customer trust more rapidly, or helping convince customers/users why/how a new technology solution may be better than “the old way” of doing things. This is not a blanket recommendation to “always include explainability” in your service/app/UI; I think many factors come into play and as with any design choice, I think you should let your customer/user feedback help you decide whether your service needs explainability to be valuable, useful, and engaging. Resources and Links: Andrey Sharapov on LinkedIn Explainable AI- XAI Group (LinkedIn) Quotes from Today’s Episode “I hear frequently there can be a tendency in the data science community to want to do excellent data science work and not necessarily do excellent business work. I also hear how some data scientists may think, ‘explainable AI is not going to improve the model’ or ‘help me get published’ – so maybe that’s responsible for why [explainable AI] is not as widely in use.” – Brian O’Neill “When you go and talk to an operational person, who has in mind a certain number of basic rules, say three, five, or six rules [they use] when doing planning, and then when you come to him with a machine learning model, something that is let’s say, ‘black box,’ and then you tell him ‘okay, just trust my prediction,’ then in most of the cases, it just simply doesn’t work. They don’t trust it. But the moment when you come with an explanation for every single prediction your model does, you are increasing your chances of a mutual conversation between this responsible person and the model…” – Andrey Sharapov “We actually do a lot of traveling these days, going to Bulgaria, going to Poland, Hungry, every country, we try to talk to these people [our users] directly. [We] try to get the requirements directly from them and then show the results back to them…” – Andrey Sharapov “The sole purpose of the tool we built was to make their work more efficient, in a sense that they could not only produce better results in terms of accuracy, but they could also learn about the market themselves because we created a plot for elasticity curves. They could play with the price and see if they made the price too high, too low, and how much the order quantity would change.” – Andrey Sharapov
#11
Apr 23, 2019 · 46 min
011 - Gadi Oren (VP Product, LogicMonitor) on analytics for monitoring applications and looking at declarative analytics as “opinions”
My guest today is Gadi Oren, the VP of Product for LogicMonitor. Gadi is responsible for the company’s strategic vision and product initiatives. Previously, Gadi was the CEO and Co-Founder of ITculate, where he was responsible for developing world-class technology and product that created contextual monitoring by discovering and leveraging application topology. Gadi previously served as the CTO and Co-founder of Cloudscope and he has a management degree from Sloan MIT. Today we are going to talk with Gadi about analytics in the context of monitoring applications. This was a fun chat as Gadi and I have both worked on several applications in this space, and it was great to hear how Gadi is habitually integrating customers into his product development process. You’re also going to hear Gadi’s interesting way of framing declarative analytics as casting “opinions,” which I thought was really interesting from a UX standpoint. We also discussed: How to define what is “normal” for an environment being monitored and when to be concerned about variations. Gadi’s KPI for his team regarding customer interaction and why it is important. What kind of data is needed for effective prototypes How to approach design/prototyping for new vs. existing products Mistakes that product owners make falling in love with early prototypes Interpreting common customer signals that may identify a latent problem needing to be solved in the application Resources and Links: LogicMonitor Twitter: @gadioren LinkedIn: Gadi Oren Quotes from Today’s Episode “The barrier of replacing software goes down. Bad software will go out and better software will come in. If it’s easier to use, you will actually win in the marketplace because of that. It’s not a secondary aspect.” – Gadi Oren “…ultimately, [not talking to customers] is going to take you away from understanding what’s going on and you’ll be operating on interpolating from information you know instead of listening to the customer.” – Gadi Oren “Providing the data or the evidence for the conclusion is a way not to black box everything. You’re providing the human with the relevant analysis and evidence that went into the conclusion and hope if that was modeled on their behavior, then you’re modeling the system around what they would have done. You’re basically just replacing human work with computer work.” — Brian O’Neill “What I found in my career and experience with clients is that sometimes if they can’t get it perfect, they’re worried about doing anything at all. I like this idea of [software analytics] casting an opinion.” — Brian O’Neill “LogicMonitor’s mission is to provide a monitoring solution that just works, that’s simple enough to just go in, install it quickly, and get coverage on everything you need so that you as a company can focus on what you really care about, which is your business.” — Gadi Oren
#10
Apr 9, 2019 · 45 min
010 - Carl Hoffman (CEO, Basis Technology) on text analytics, NLP, entity resolution, and why exact match search is stupid
My guest today is Carl Hoffman, the CEO of Basis Technology, and a specialist in text analytics. Carl founded Basis Technology in 1995, and in 1999, the company shipped its first products for website internationalization, enabling Lycos and Google to become the first search engines capable of cataloging the web in both Asian and European languages. In 2003, the company shipped its first Arabic analyzer and began development of a comprehensive text analytics platform. Today, Basis Technology is recognized as the leading provider of components for information retrieval, entity extraction, and entity resolution in many languages. Carl has been directly involved with the company’s activities in support of U.S. national security missions and works closely with analysts in the U.S. intelligence community. Many of you work all day in the world of analytics: numbers, charts, metrics, data visualization, etc. But, today we’re going to talk about one of the other ingredients in designing good data products: text! As an amateur polyglot myself (I speak decent Portuguese, Spanish, and am attempting to learn Polish), I really enjoyed this discussion with Carl. If you are interested in languages, text analytics, search interfaces, entity resolution, and are curious to learn what any of this has to do with offline events such as the Boston Marathon Bombing, you’re going to enjoy my chat with Carl. We covered: How text analytics software is used by Border patrol agencies and its limitations. The role of humans in the loop, even with good text analytics in play What actually happened in the case of the Boston Marathon Bombing? Carl’s article“Exact Match” Isn’t Just Stupid. It’s Deadly. The 2 lessons Carl has learned regarding working with native tongue source material. Why Carl encourages Unicode Compliance when working with text, why having a global perspective is important, and how Carl actually implements this at his company Carl’s parting words on why hybrid architectures are a core foundation to building better data products involving text analytics Resources and Links: Basis Technology Carl’s article: “Exact Match” isn’t Just Stupid. It’s Deadly. Carl Hoffman on LinkedIn Quotes from Today’s Episode “One of the practices that I’ve always liked is actually getting people that aren’t like you, that don’t think like you, in order to intentionally tease out what you don’t know. You know that you’re not going to look at the problem the same way they do…” — Brian O’Neill “Bias is incredibly important in any system that tries to respond to human behavior. We have our own innate cultural biases that we’re sometimes not even aware of. As you [Brian] point out, it’s impossible to separate human language from the underlying culture and, in some cases, geography and the lifestyle of the people who speak that language…” — Carl Hoffman “What I can tell you is that context and nuance are equally important in both spoken and written human communication…Capturing all of the context means that you can do a much better job of the analytics.” — Carl Hoffman “It’s sad when you have these gaps like what happened in this border crossing case where a name spelling is responsible for not flagging down [the right] people. I mean, we put people on the moon and we get something like a name spelling [entity resolution] wrong. It’s shocking in a way.” — Brian O’Neill “We live in a world which is constantly shades of gray and the challenge is getting as close to yes or no as we can.”– Carl Hoffman
#9
Mar 26, 2019 · 49 min
009 - Nancy Hensley (Chief Digital Officer, IBM Analytics) on the role of design and UX in modernizing analytics tools as old as 50 years
Nancy Hensley is the Chief Digital Officer for IBM Analytics, a multi-billion dollar IBM software business focused on helping customers transform their companies with data science and analytics. Nancy has over 20 years of experience working in the data business in many facets from development, product management, sales, and marketing. Today’s episode is probably going to appeal to those of you in product management or working on SAAS/cloud analytics tools. It is a bit different than our previous episodes in that we focused a lot on what “big blue” is doing to simplify its analytics suite as well as facilitating access to those tools. IBM has many different analytics-related products and they rely on good design to make sure that there is a consistent feel and experience across the suite, whether it’s Watson, statistics, or modeling tools. She also talked about how central user experience is to making IBM’s tools more cloud-like (try/buy online) vs. forcing customers to go through a traditional enterprise salesperson. If you’ve got a “dated” analytics product or service that is hard to use or feels very “enterprisey” (in that not-so-good way), then I think you’ll enjoy the “modernization” theme of this episode. We covered: How Nancy is taking a 50-year old product such as SPSS and making it relevant and accessible for an audience that is 60% under 25 years of age The two components Nancy’s team looks at when designing an analytics product What “Metrics Monday” is all about at IBM Analytics How IBM follows-up with customers, communicates with legacy users, and how the digital market has changed consumption models Nancy’s thoughts on growth hacking and the role of simplification Why you should always consider product-market fit first and Nancy’s ideas on MVPs The role design plays in successful onboarding customers into IBM Analytics’ tools and what Brian refers to as the “honeymoon” experience Resources and Links: Nancy Hensley on Twitter Nancy Hensley on LinkedIn Quotes: “It’s really never about whether it’s a great product. It’s about whether the client thinks it’s great when they start using it.” –Nancy “Every time we add to the tool, we’re effectively reducing the simplicity of everything else around it.”–Brian “The design part of it for us is so eye-opening, because again, we’ve built a lot of best in class enterprise products for years and as we shift into this digital go-to-market, it is all about the experience…”–Nancy “Filling in that “why” piece is really important if you’re going to start changing design because you may not really understand the reasons someone’s abandoning.”–Brian “Because a lot of our products weren’t born in the cloud originally, they weren’t born to be digitally originally, doesn’t mean they can’t be digitally consumed. We just have to really focus on the experience and one of those things is onboarding.” –Nancy “If they [users] can’t figure out how to jump in and use the product, we’re not nailing it. It doesn’t matter how great the product is, if they can’t figure out how to effectively interact with it. –Nancy
#8
Mar 12, 2019 · 51 min
008 - Dr. Puneet Batra (Assoc. Director, Machine Learning at Broad Institute of MIT and Harvard) on aligning data science with biz objectives, user re...
Dr. Puneet Batra is the Associate Director of Machine Learning at the Broad Institute, where his team builds machine learning algorithms and pipelines to help discover new biological insights and impact disease. Puneet has spent his career stitching together data-driven solutions: in location data analysis, as cofounder of LevelTrigger; in health care, as Chief Data Scientist at Kyruus; as lead Analytic Scientist at Aster Data (Acq by Teradata); and in fundamental models of particle physics, developing theories for Fermilab’s Tevatron and CERN’s Large Hadron Collider. He has held research positions at Harvard, Stanford and Columbia Universities. Puneet completed his BA at Harvard University and has a Ph.D. from Stanford University. A friend of mine introduced me to Puneet because he was kicking off a side project using machine learning to dig into what creativity is through the lens of jazz. Since Puneet is not a musician by training, he was looking for some domain-specific knowledge to inform his experiment, and I really liked his design-oriented thinking. While jazz kicks off our conversation, we went a lot more deeply into the contemporary role of the data scientist in this episode including: The discussions that need to happen between users, stakeholders, and subject matter experts so teams get a clear image of the problem that actually needs to be solved. Dealing with situations where the question you start with isn’t always the question that is answered in the end. When to sacrifice model quality for the sake of user experience and higher user engagement (i.e. the “good enough” approach) The role of a data scientist in product design. Resources and Links: Puneet Batra on Twitter Puneet Batra on LinkedIn Quotes from Puneet Batra "Sometimes, accuracy isn't the most important thing you should be optimizing for; it’s the rest of the package…if you can make that a good process, then I think you're more on the road to making users happy [vs.] trying to live in this idealized world where you never make a mistake at all." "The question you think you're answering from the beginning probably isn't the one that you're going to stay answering the entire time and you've just got to be flexible around that." "Even data scientists and engineers should be able to listen with empathy and ask questions. I got a good number of tips from people teaching me how to do things like that. Basically, we ask a question or basically shut up and hear what their answer is." "I'm not really sure what creativity is. I'm not really sure if machines will ever be creative. A good experiment to try to prove that out is to try to get a machine to be as creative as possible and see where it falls flat."
#7
Feb 26, 2019 · 1 hr 2 min
007 -Jim Psota (CTO & Co-Founder, Panjiva/S&P Global) on designing a meaningful SAAS analytics product for the global supply chain
Jim Psota is the Co-Founder and CTO of Panjiva, which was named one of the top 10 Most Innovative Data Science Companies in the World by Fast Company in 2018. Panjiva has mapped the global supply chain using a combination of over 1B shipping transactions and machine learning, and recently the company was acquired by S&P Global. Jim has spoken about artificial intelligence and entrepreneurship at Harvard Business School, MIT, and The White House, and at numerous academic and industry conferences. He also serves on the World Economic Forum’s Working Group for Artificial Intelligence and has done Ph.D. research in computer science at MIT. Some of the topics we discuss in today’s episode include: What Jim learned from starting Panjiva from a data-first approach Brian and Jim’s thoughts on problem solving driven by use cases and people vs. data and AI 3 things Jim wants teams to get right with data products Jim and Brian’s thoughts on “blackbox” analytics that try to mask complex underlying data to make the UX easier How Jim balances the messiness of 20+ third-party data sources, designing a good product, and billions of data points Resources and Links: Jim Psota Jim Psota on Twitter Panjiva Quotes from Jim Psota “When you’re dealing with essentially resolving 1.5 billion records, you could think of that you need to compute 1.5 billion squared pairs of potential similarities.” “It’s much more fulfilling to be building for a person or a set of people that you’ve actually talked to… The engineers are going to develop a much better product and frankly be much happier having that connection to the user.” “We have crossed a pretty cool threshold where a lot of value can be created now because we have this nice combination of data availability, strong algorithms, and compute power.” “In our case and many other company’s cases, taking third-party data, no matter where you’re getting your data, there’s going to be issues with it, there’s going to be delays, format changes, granularity differences.” “As much as possible, we try to use the tools of data science to actually correct the data deficiency or impute or whatever technique is actually going to be better than nothing, but then say this was imputed or this is reported versus imputed…then over time, the user starts to understand if it’s gray italics [the data] was imputed, and if it’s black regular text, that’s reported data, for example.”

Showing 181–200 of 207 episodes