Skip to content
Sign in
Artwork for Experiencing Data w/ Brian T. O’Neill
BusinessEntrepreneurshipManagementTechnology

Experiencing Data w/ Brian T. O’Neill

Brian T. O’Neill from Designing for Analytics

Your B2B analytics product does the hard technical work. So why aren't sales and adoption keeping up?

I'm Brian T. O'Neill, and my podcast, Experiencing Data, is a Listen Notes top 2% global podcast for founders, CEOs, and product leaders at B2B analytics companies: the ones whose product should be winning deals on technical sophistication alone, but isn't.

Maybe competitors have caught up on features and prospects say you all look the same, so you end up competing on price. Maybe customers never discover your best features and use you "like Excel." Maybe your value is algorithmic, and buyers can't see what they'd be paying for—something I call the Invisible Intelligence Gap.

Through solo episodes and interviews with CEOs, founders, and the enterprise data leaders who buy these products, I draw on my work as a product designer and strategy consultant to help you translate technical complexity into commercial clarity: connecting what your data product does to the outcomes, value, and human factors that still matter, even in the age of AI.

Experiencing Data is the only non-technical show about why technically impressive products don't sell or get used, and what to do about it.

Subscribe today on all major platforms or browse the episode archive.

Get 1-Page Episode Summaries In your Inbox:
https://designingforanalytics.com/ed

About Brian:
https://designingforanalytics.com/bio/

Play
Open in
  • 206 episodes
  • fortnightly
  • Avg 41 min
  • English

Episodes 206

NewestOldestLongest
All20Unplayed20In progress0Played0
Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.
  • #45
    Aug 11, 2020 · 44 min

    045 - Healthcare Analytics…or Actionable Decision Support Tools? Leadership Strategies from Novant Health’s SVP of Data Products, Karl Hightower

    Healthcare professionals need access to decision support tools that deliver the right information, at the right time. In a busy healthcare facility, where countless decisions are made on a daily basis, it is crucial that any analytical tools provided actually yield useful decision support to the target customer. In this episode, I talked to Karl Hightower from Novant Health about how he and his team define “quality” when it comes to data products, and what they do to meet that definition in their daily work. Karl Hightower is the Chief Data Officer and SVP of Data Products at Novant Health, a busy hospital and medical group in the Southeast United States with over 16 hospitals and more than 600 clinics. Karl and I took a deep dive into data product management, and how Karl and his team are designing products and services that help empower all of the organization’s decision makers. In our chat, we covered: How a non-tech company like Novant Health approaches data product management The challenges of designing data products with empathy in mind while being in an environment involving physicians and healthcare professionalsThe metric Karl’s team uses to judge the quality and efficacy of their data products, and how executive management contributed to defining this success criteria How Karl encourages deep empathy between analytics teams and their users by deeply investigating how the users being served by the team make decisions with data How and why Novant embraces design and UX in their data product work The types of outcomes Karl sees when designers and user experience professionals work with analytics and data science practitioners. How Karl was able to obtain end user buy-in and support for ? The strategy Karl used to deal with a multitude of “information silos” resulting from the company’s numerous analytics groups. Resources and Links: Novant Health website: https://www.novanthealth.org/ Novant Health LinkedIn: https://www.linkedin.com/company/novanthealth/ Karl Hightower LinkedIn: https://www.linkedin.com/in/karl-hightower-4528123/ Quotes from Today’s Episode “I tend to think of product management as a core role along with a technical lead and product designer in the software industry. Outside the software industry, I feel like product management is often this missing hub. ” - Brian “I really want to understand why the person is asking for what they're asking for, so there is much more of a closer relationship between that portfolio team and their end-user community that they're working with. It's almost a day-to-day living and breathing with and understanding not just what they're asking for and why are they asking for it, but you need to understand how they use information to make decisions.” - Karl “I think empathy can sound kind of hand-wavy at times. Soft and fluffy, like whipped cream. However, more and more at senior levels, I am hearing how much leaders feel these skills are important because the technology can be technically right and effectively wrong.” - Brian “The decision that we got to on executive governance was how are we going to judge success criteria? How do we know that we're delivering the right products and that we're getting better on the maturity scale? And the metric is actually really simple. Ask the people that we're delivering for, does this give you what you need when you need it to make those decisions? - Karl “The number one principle is, if I don't know how something is done [created with data], I'm very unlikely to trust it. And as you look at just the nature of healthcare, transparency absolutely has to be there because we want the clinicians to poke holes in it, and we want everyone to be able to trust it. So, we are very open. We are very transparent with everything that goes in it.” - Karl “You need to really understand the why. You’ve got to understand what business decisions are being made, what's driving the strategy of the people who are asking for all that information.” - Karl

  • #44
    Jul 28, 2020 · 50 min

    044 - The Roles of Product and Design when “Competing in the Age of AI” with HBS Professor and Author Karim Lakhani

    If there’s one thing that strikes fear into the heart of every business executive, it’s having your company become the next Blockbuster or Neiman Marcus — that is, ignoring change, and getting wiped out by digital competitors. In this episode, I dived into the changing business landscape with Karim Lakhani who is a Professor at Harvard Business School and co-author of the new book Competing in the Age of AI: When Algorithms and Networks Run the World, which he wrote with his friend and colleague at HBS, Marco Iansiti. We discuss how AI, machine learning, and digital operating models are changing business architecture, and disrupting traditional business models. I also pressed Karim to go a bit deeper on how, and whether he thinks product mindset and design factor in to the success of AI in today’s businesses. We also go off on a fun tangent about the music industry, which just might have to be a future episode!. In any case, I highly recommend the book. It’s particularly practical for those of you working in organizations that are not digital natives and want to hear how the featured companies in the book are setting themselves apart by leveraging data and AI in customer-facing products and in internal applications/operations. Our conversation covers: Karim’s new book, Competing in the Age of AI: When Algorithms and Networks Run the World, co-authored with Marco Iansiti. How digital operating models are colliding with traditional product-oriented businesses, and the impact this is having on today’s organizations. The critical role of data product management that is frequently missing when companies try to leverage AI Karim’s thoughts on ethics in AI and machine learning systems, and how they need to be baked into business and engineering. The similarity Karim sees between COVID-19 and AI The role of design, particularly in human-in-the-loop systems and how companies need to consider the human experience in applications of AI that augment decision making vs. automate it. How Karim sees the ability to adapt in business as being critical to survival in the age of AI Resources and Links Book Link: https://www.amazon.com/Competing-Age-AI-Leadership-Algorithms/dp/1633697622/ Twitter: https://twitter.com/klakhani LinkedIn: https://www.linkedin.com/in/professorkl/ Harvard Business Analytics Program: https://analytics.hbs.edu/ Quotes from Today’s Episode “Our thesis in the book is that a new type of an organization is emerging, which has eliminated bottlenecks in old processes.” - Karim “Digital operating models have exponential scaling properties, in terms of the value they generate, versus traditional companies that have value curves that basically flatten out, and have fixed capacity. Over time, these digital operating models collide with these traditional product models, win over customers, and gather huge amounts of market share….” - Karim “This whole question about human-in-the-loop is important, and it's not going to go away, but we need to start thinking about, well, how good are the humans, anyway? - Karim “Somebody once said, “Ethics defines the boundaries of what you care about.” And I think that's a really important question…” - Brian “Non-digital natives worry about these tech companies coming around and eating them up, and I can’t help but wonder ‘why aren't you also copying the way they design and build software?’” - Brian “...These established companies have a tough time with the change process.” - Karim

  • #43
    Jul 14, 2020 · 49 min

    043 - What a Product Management Mindset Can do for Data Science and Analytics Leaders with Product School CEO, Carlos González de Villaumbrosia

    I am a firm believer that one of the reasons that data science and analytics has a high failure rate is a lack of product management and design. To me, product is about a mindset just as much as a job title, and I am repeatedly hearing how more and more voices in the data community are agreeing with me on this (Gartner CDO v4, International Inst. for Analytics, several O’Reilly authors, Karim Lakhani’s new book on AI, and others). This is even more true as more companies begin to leverage AI. So many of these companies fear what startups and software companies are doing, yet they do not copy the way tech companies build software applications and enable specific user experiences that unlock the desired business value. Integral to building software is the product management function—and when these applications and tools have humans in the loop, the product/UX design function is equally as important to ensure adoption, usability, engagement, and alignment with the business objectives. In modern tech companies, the overlap between product design and product management can be significant, and frequently, product leaders in tech companies come up through both design and engineering ranks and indeed my own work heavily overlaps with product. What this tells me is that product is a mindset, and it’s a role many can learn if they believe it’s critical. So why aren’t more data science and analytics leaders forming strong product design and analytics functions? I don’t know, so I decided to bring Carlos onto the show to talk about his company, Product School, which offers product management training and features instructors from many of the big tech companies on how to do it. In this episode, Carlos provides a comprehensive overview of why he launched Product School, what makes an effective product manager, and the importance of having structured vision and alignment when developing products. This conversation explores: Why Carlos launched the Product School for professionals who want to learn on the side without quitting their job and putting their life on hold. The type of mentality product managers need to have and whether specialization matters within product management. Whether being a product manager in machine learning and AI is different than working with a traditional software product. How product management is not project management Advice for approaching executive decision makers about product management education How to avoid the trap of focusing too heavily on process How product management often leads to executive leadership roles The “power trio” of engineering, product management, and design, and the value of aligning all three groups. Understanding the difference between applied and academic experience How the relationship between design and PM has changed over the last five years What the gap looks like between a skilled PM and an exceptional one. Resources and Links The State of Product Analytics (Also referred to as The Future of Product Analytics in the audio) Mixpanel, company that they partnered with to create the above report Episode 17 of Experiencing Data Twitter Main Company Site ProductCon Productverse Quotes from Today’s Episode “You can become a product manager by building products. You don't need to be a software engineer. You don’t need to have an MBA. You don't need to be an incredible, inspiring visionary. This is stuff that you can learn, and the best way to learn it is by doing it.” - Carlos “A product manager is a generalist. And in order to become a generalist, usually you have to have some sort of [specialty] before. So, we define product management as the intersection in between business, engineering, and design. And you can become a good product manager from either of those options.” - Carlos “If you have [a power trio of technology, product, and design] and the energy is right, and the relationships are really strong, boy, you can get a lot of stuff done, and you can iterate quickly, and really produce some great stuff.” - Brian “I think part of the product management mindset... is to realize part of your job now is to be a problem finder, it’s to help set the strategy, it's to help ensure that a model is not the solution.” - Brian “I think about a bicycle wheel with the hub in the center and the spokes coming out. Product management is that hub, and it reports up into the business, but you have all these different spokes, QA, and software engineering, maybe data science and analytics, product design, and user experience design. These are all kind of spokes.” - Brian “These are people who are constantly learning, but not just about their products. They’re constantly learning in general. Reading books, practicing sports, doing whatever it is, but always looking at what's new and wanting to play around with it, just to be dangerous enough. So, I think those three areas: obsession with a customer based on data; obsession with empathy; and then obsession with learning, or just being curious are really critical.” - Carlos

  • #42
    Jun 30, 2020 · 39 min

    042 - Why Machine Learning and Analytics Alone Can’t Drive Behavioral Change inside Police Departments with Allison Weil

    “What happened in Minneapolis and Louisville and Chicago and countlessother cities across the United States is unconscionable (and to be clear, racist). But what makes me the maddest is how easy this problem is to solve, just by the police deciding it’s a thing they want to solve.” - Allison Weil on Medium Before Allison Weil became an investor and Senior Associate at Hyde Park Ventures, she was a co-founder at Flag Analytics, an early intervention system for police departments designed to help identify officers at risk of committing harm. Unfortunately, Flag Analytics—as a business—was set up for failure from the start, regardless of its predictive capability. As Allison explains so candidly and openly in her recent Medium article (thanks Allison!), the company had “poor product-market fit, a poor problem-market fit, and a poor founder-market fit.” The technology was not the problem, and as a result, it did not help them succeed as a business or in producing the desired behavior change because the customers were not ready to act on the insights. Yet, the key takeaways from her team’s research during the design and validation of their product — and the uncomfortable truths they uncovered — are extremely valuable, especially now as we attempt to understand why racial injustice and police brutality continue to persist in law enforcement agencies. As it turns out, simply having the data to support a decision doesn’t mean the decision will be made using the data. This is what Allison found out while in her interactions with several police chiefs and departments, and it’s also what we discussed in this episode. I asked Allison to go deeper into her Medium article, and she agreed. Together, we covered: How Allison and a group of researchers tried to streamline the identification of urban police officers at risk of misconduct or harm using machine learning. Allison’s experience of trying to build a company and program to solve a critical societal issue, and dealing with police departments that weren’t ready to take action on the analytical insights her product revealed How she went about creating a “single pane of glass,” where officers could monitor known problem officers and also discover officers who may be in danger of committing harm. The barriers that prevented the project from being a success, from financial ones to a general unwillingness among certain departments to take remedial action against officers despite historical or predicted data The key factors and predictors Allison’s team found in the data set of thousands of officers that correlated highly with poor officer behavior in the future—and how it seemed to fall on deaf ears How Allison and her team approached the sensitive issue of race in the data, and a [perhaps unexpected] finding they discovered about how prevalent racism seemed to be in departments in general. Allison’s experience of conducting “ride-alongs” (qualitative 1x1 research) where she went on patrol with officers to observe their work and how the experience influenced how her team designed the product and influenced her perspective while analyzing the police officer data set. Resources and Links: Twitter LinkedIn Medium Quotes from Today’s Episode “The folks at the police departments that we were working with said they were well-intentioned, and said that they wanted to talk through, and fix the problem, but when it came to their actions, it didn't seem like [they were] really willing to make the choices that they needed to make based off of what the data said, and based off of what they knew already.” - Allison “I don't come from a policing background, and neither did any of my co-founders. And that made it really difficult to relate to different officers, and relate to departments. And so the combination of all of those things really didn't set me up for a whole lot of business success in that way.”- Allison “You can take a whole lot of data and do a bunch of analysis, but what I saw was the data didn't show anything that the police department didn't know already. It amplified some of what they knew, but [the problem here] wasn't about the data.” - Allison “It was really frustrating for me, as a founder, sure, because I was putting all this energy into trying to build a software and trying to build a company, but also just frustrating for me as a person and a citizen… you fundamentally want to solve a problem, or help a community solve a problem, and realize that the people at the center of it just aren't ready for it to be solved.” - Allison “...We did have race data, but race was not the primary predictor or reason for [brutality]. It may have been a factor, but it was not that there were racist cops wandering around, using force only against people of particular races. What we found was….” - Allison “The way complaints are filed department to department is really, really different. And so that results in complaints looking really, really different from department to department and counts looking different. But how many are actually reviewed and sustained? And that looks really, really different department to department.” - Allison “...Part of [diversity] is asking the questions you don't know to ask. And that's part of what you get out of having a diverse team— they're going to surface questions that no one else is asking about. And then you can have the discussion about what to do about them.” - Brian

  • #41
    Jun 16, 2020 · 44 min

    041 - Data Thinking: An Approach to Using Design Thinking to Maximize the Effectiveness of Data Science and Analytics with Martin Szugat of Datentreib...

    The job of many internally-facing data scientists in business settings is to discover,explore, interpret, and share data, turning it into actionable insight that can benefit the company and improve outcomes. Yet, data science teams often struggle with the very basic question of how the company’s data assets can best serve the organization. Problem statements are often vague, leading to data outputs that don’t turn into value or actionable decision support in the last mile. This is where Martin Szugat and his team at Datentreiber step in, helping clients to develop and implement successful data strategy through hands-on workshops and training. Martin is based in Germany and specializes in helping teams learn to identify specific challenges data can solve, and think through the problem solving process with a human focus. This in turn helps teams to select the right technology and be objective about whether they need advanced tools such as ML/AI, or something more simple to produce value. In our chat, we covered: How Datentreiber helps clients understand and derive value from their data — identifying assets, and determining relevant use cases. An example of how one client changed not only its core business model, but also its culture by working with Datentreiber, transitioning from a data-driven perspective to a user-driven perspective. Martin’s strategy of starting with small analytics projects, and slowly gaining buy-in from end users, with a special example around social media analytics that led to greater acceptance and understanding among team members. The canvas tools Martin likes to use to visualize abstract concepts related to data strategy, data products, and data analysis. Why it helps to mix team members from different departments like marketing, sales, and IT and how Martin goes about doing that How cultural differences can impact design thinking, collaboration, and visualization processes. Resources and Links: Company site (German) (English machine translation) Datentreiber Open-Source Design Tools Data Strategy Design (German) (English machine translation) Martin’s LinkedIn Quotes from Today’s Episode “Often, [clients] already have this feeling that they're on the wrong path, but they can't articulate it. They can't name the reason why they think they are on the wrong path. They learn that they built this shiny dashboard or whatever, but the people—their users, their colleagues—don't use this dashboard, and then they learn something is wrong.” - Martin “I usually like to call this technically right and effectively wrong solutions. So, you did all the pipelining and engineering and all that stuff is just fine, but it didn't produce a meaningful outcome for the person that it was supposed to satisfy with some kind of decision support.” - Brian “A simple solution is becoming a trainee in other departments. So, ask, for example, the marketing department to spend a day, or a week and help them do their work. And just look over the shoulder, what they are doing, and really try to understand what they are doing, and why they are doing it, and how they are doing it. And then, come up with solution proposals.” - Martin ...I tend to think of design as a team sport, and it's a lot about facilitating groups of these different cross-departmental groups of arriving at a solution for a particular audience; a specific audience that needs a specific problem solved.” - Brian “[One client said] we are very good at implementing the right solutions for the wrong problems. And I think this is what often happens in data science, or business intelligence, or whatever, also in IT departments: that they are too quick in starting thinking about the solution before they understand the problem.” - Martin “If people don't understand what you're doing or what your analytic solution is doing, they won't use it and there will be no acceptance.” - Martin “One thing we practice a lot, [...] is in visualizing those abstract things like data strategy, data product, and analytics. So, we work a lot with canvas tools because we learned that if you show people—and it doesn't matter if it's just on a sticky note on a canvas—then people start realizing it, they start thinking about it, and they start asking the right questions and discussing the right things. ” - Martin

  • #40
    Jun 2, 2020 · 1 hr 1 min

    040 – Improving Potato Chips and Space Travel: NASA’s Steve Rader on Open Innovation

    Innovation doesn’t just happen out of thin air. It requires a conscious effort, and team-wide collaboration. At the same time, innovation will be critical for NASA if the organization hopes to remain competitive and successful in the coming years. Enter Steve Rader. Steve has spent the last 31 years at NASA, working in a variety of roles including flight control under the legendary Gene Kranz, software development, and communications architecture. A few years ago, Steve was named Deputy Director for the Center of Excellence for Collaborative Innovation. As Deputy Director, Steve is spearheading the use of open innovation, as well as diversity thinking. In doing so, Steve is helping the organization find more effective ways of approaching and solving problems. In this fascinating discussion, Steve and Brian discuss design, divergent thinking, and open innovation plus: Why Steve decided to shift away from hands-on engineering and management to the emerging field of open innovation, and why NASA needs this as well as diversity in order to remain competitive. The challenge of convincing leadership that diversity of thought matters, and why the idea of innovation often receives pushback. How NASA is starting to make room for diversity of thought, and leveraging open innovation to solve challenges and bring new ideas forward. Examples of how experts from unrelated fields help discover breakthroughs to complex and greasy problems, such as potato chips! How the rate of technological change is different today, why innovation is more important than ever, and how crowdsourcing can help streamline problem solving. Steve’s thoughts on the type of leader that’s needed to drive diversity at scale, and why that person should be a generalistPrioritizing outcomes over outputs, defining problems, and determining what success looks like early on in a project. The metrics a team can use to measure whether one is “doing innovation.” Resources and Links Designingforanalytics.com/theseminar Steve Rader’s LinkedIn: https://www.linkedin.com/in/steve-rader-92b7754/ NASA Solve: nasa.gov/solve Steve Rader’s Twitter: https://twitter.com/SteveRader NASA Solve Twitter: https://twitter.com/NASAsolve Quotes from Today’s Episode “The big benefit you get from open innovation is that it brings diversity into the equation […]and forms this collaborative effort that is actually really, really effective.” – Steve “When you start talking about innovation, the first thing that almost everyone does is what I call the innovation eye-roll. Because management always likes to bring up that we’re innovative or we need innovation. And it just sounds so hand-wavy, like you say. And in a lot of organizations, it gets lots of lip service, but almost no funding, almost no support. In most organizations, including NASA, you’re trying to get something out the door that pays the bills. Ours isn’t to pay the bills, but it’s to make Congress happy. And, when you’re doing that, that is a really hard, rough space for innovation.” – Steve “We’ve run challenges where we’re trying to improve a solar flare algorithm, and we’ve got, like, a two-hour prediction that we’re trying to get to four hours, and the winner of that in the challenge ends up to be a cell phone engineer who had an undergraduate degree from, like, 30 years prior that he never used in heliophysics, but he was able to take that extracting signal from noise math that they use in cell phones, and apply it to heliophysics to get an eight-hour prediction capability.” – Steve “If you look at how long companies stay around, the average in 1958 was 60 years, it is now less than 18. The rate of technology change and the old model isn’t working anymore. You can’t actually get all the skills you need, all the diversity. That’s why innovation is so important now, is because it’s happening at such a rate, that companies—that didn’t used to have to innovate at this pace—are now having to innovate in ways they never thought.” – Steve “…Innovation is being driven by this big technology machine that’s happening out there, where people are putting automation to work. And there’s amazing new jobs being created by that, but it does take someone who can see what’s coming, and can see the value of augmenting their experts with diversity, with open innovation, with open techniques, with innovation techniques, period.” – Steve “…You have to be able to fail and not be afraid to fail in order to find the real stuff. But I tell people, if you’re not willing to listen to ideas that won’t work, and you reject them out of hand and shut people down, you’re probably missing out on the path to innovation because oftentimes, the most innovative ideas only come after everyone’s thrown in 5 to 10 ideas that actually won’t work.” – Steve

  • #39
    May 19, 2020 · 44 min

    039 – How PEX Fingerprinted 20 Billion Audio and Video Files and Turned It Into a Product to Help Musicians, Artists and Creators Monetize their Work

    Every now and then, I like to insert a music-and-data episode into the show since hey, I’m a musician, and I’m the host 😉 Today is one of those days! Rasty Turek is founder and CEO of Pex, a leading analytics and rights management platform used for discovering and tracking video and audio content using data science. Pex’s AI crawls the internet for user-generated content (UGC), identifies copyrighted audio/ visual content, indexes the media, and then enables rights holders to understand where their art is being used so it can be monetized. Pex’s goal is to help its customers understand who is using their licensed content, and what they are using it for — along with key insights to support monetization initiatives and negotiations with UGC platform providers. In this episode of Experiencing Data, we discuss: How the data science behind Pex works in terms of being able to fingerprint actual songs (the underlying IP of a composition) vs. masters (actual audio recordings of songs) The challenges PEX has in identifying complex, audio-rich user-generated content and cover recordings, and ensuring it is indexing as many usages as possible. The transitioning UGC market, and how Pex is trying to facilitate change. One item that Rasty discusses is Europe’s new Copyright Directive law, and how it’s impacting UGC from a licensing standpoint. How analytics are empowering publishers, giving them key insights and firepower to negotiate with UGC platforms over licensed content. Key product design and UX considerations that Pex has taken to make their analytics useful to customers What Rasty learned through his software iteration journey at Pex, including a memorable example about bias that influenced future iterations of the design/UI/UX How Pex predicts and priorities monetization opportunities for customers, and how they surface infringements. Why copyright education is the “last bastion of the internet” — and the role that Pex is playing in streamlining copyrighted material. Brian also challenges Rasty directly, asking him how the Pex platform balances flexibility with complexity when dealing with extremely large data sets. Resources and Links Designingforanalytics.com/theseminar Pex.com Twitter: https://twitter.com/synopsi Quotes from Today’s Episode “I will say, 80 to 90 percent of the population eventually will be rights owners of some sort, since this is how copyright works. Everybody that produces something is immediately a rights owner, but I think most of us will eventually generate our livelihood through some form of IP, especially if you believe that the machines are going to take the manual labor from us.” - Rasty “When people ask me how it is to run a big data company, I always tell them I wish we were not [a big data company], because I would much rather have “small data,” and have a very good business, rather than big data.” - Rasty “There's a lot of these companies that [have operated] in this field for 20 to 30 years, we just took it a little bit further. We adjusted it towards the UGC world, and we focused on simplicity” - Rasty “We don't follow users, we follow content. And so, at some point [during our design process] we were exploring if we could follow users [of our customers’ copyrighted content].... As we explored this more, we started noticing that [our customers] started making incorrect decisions because they were biased towards users [of their copyrighted content].” - Rasty “If you think that your general customer is a coastal elite, but the reality is that they are Midwest farmers, you don't want to see that as the reality and you start being biased towards that. So, we immediately started removing that data and really focused on the content itself—because that content is not biased.” - Rasty “[Re: PEX’s design process] We always started with the guiding principles. What is the task that you're trying to solve? So, for instance, if your task is to monetize your content, then obviously you want to monetize the most obvious content that will get the most views, right?.” - Rasty

  • #38
    May 5, 2020 · 1 hr 1 min

    038 – (Special Co-Hosted Episode) Brian and Mark Bailey Discuss 10 New Design and UX Considerations for Creating ML and AI-Driven Products and Applica...

    Mark Bailey is a leading UX researcher and designer, and host of the Design for AI podcast — a program which, similar to Experiencing Data, explores the strategies and considerations around designing data-driven human-centered applications built with machine learning and AI. In this episode of Experiencing Data — co-released with the podcast Design for AI — Brian and Mark share the host and guest role, and discuss 10 different UX concepts teams may need to consider when approaching ML-driven data products and AI applications. A great discussion on design and #MLUX ensued, covering: Recognizing the barrier of trust and adoption that exists with ML, particularly at non-digital native companies, and how to address it when designing solutions. Why designers need to dig beyond surface level knowledge of ML, and develop a comprehensive understanding of the space How companies attempt to “separate reality from the movies,” with AI and ML, deploying creative strategies to build trust with end users (with specific examples from Apple and Tesla) Designing for “undesirable results” (how to gracefully handle the UX when a model produces unexpected predictions) The ongoing dance of balancing UX with organizational goals and engineering milestones What designers and solution creators need to be planning for and anticipating with AI products and applications Accessibility considerations with AI products and applications – and how itcan be improved Mark’s approach to ethics and community as part of the design process. The importance of systems design thinking when collecting data and designing models The different model types and deployment considerations that affect a solution’s UX — and what solution designers need to know to stay ahead Collaborating, and visualizing — or storyboarding — with developers, to help understand data transformation and improve model design The role that designers can play in developing model transparency (i.e. interpretability and explainable AI) Thinking about pain points or problems that can be outfitted with decision support or intelligence to make an experience better Resources and Links: Designing for AI Podcast Designing for AI Experiencing Data – Episode 35 Designing for Analytics Seminar Seeing Theory Measuring U Contact Brian @DesignforAI Quotes from Today’s Episode “There’s not always going to be a software application that is the output of a machine learning model or something like that. So, to me, designers need to be thinking about decision support as being the desired outcome, whatever that may be.” – Brian “… There are [about] 30 to 40 different types of machine learning models that are the most popular ones right now. Knowing what each one of them is good for, as the designer, really helps to conform the machine learning to the problem instead of vice versa.” – Mark “You can be technically right and effectively wrong. All the math part [may be] right, but it can be ineffective if the human adoption piece wasn’t really factored into the solution from the start.” – Brian “I think it’s very interesting to see what some of the big companies have done, such as Apple. They won’t use the term AI, or machine learning in any of their products. You’ll see their chips, they call them neural engines instead have anything to do with AI. I mean, so building the trust, part of it is trying to separate out reality from movies.” – Mark “Trust and adoption is really important because of the probabilistic nature of these solutions. They’re not always going to spit out the same thing all the time. We don’t manually design every single experience anymore. We don’t always know what’s going to happen, and so it’s a system that we need to design for.” – Brian “[Thinking about] a small piece of intelligence that adds some type of value for the customer, that can also be part of the role of the designer.” – Brian “For a lot of us that have worked in the software industry, our power trio has been product management, software engineering lead, and some type of design lead. And then, I always talk about these rings, like, that’s the close circle. And then, the next ring out, you might have some domain experts, and some front end developer, or prototyper, a researcher, but at its core, there were these three functions there. So, with AI, is it necessary, now, that we add a fourth function to that, especially if our product was very centered around this? That’s the role of the data scientist. And so, it’s no longer a trio anymore.” – Brian

  • #37
    Apr 21, 2020 · 48 min

    037 – A VC Perspective on AI and Building New Businesses Using Machine Intelligence featuring Rob May of PJC

    Rob May is a general partner at PJC, a leading venture capital firm. He was previously CEO of Talla, a platform for AI and automation, as well as co-founder and CEO of Backupify. Rob is an angel investor who has invested in numerous companies, and author of InsideAI which is said to be one of the most widely-read AI newsletters on the planet. In this episode, Rob and I discuss AI from a VC perspective. We look into the current state of AI, service as a software, and what Rob looks for in his startup investments and portfolio companies. We also investigate why so many companies are struggling to push their AI projects forward to completion, and how this can be improved. Finally, we outline some important things that founders can do to make products based on machine intelligence (machine learning) attractive to investors. In our chat, we covered: The emergence of service as a software, which can be understood as a logical extension of “software eating the world” and the 2 hard things to get right (Yes, you read it correctly and Rob will explain what this new SAAS acronym means!) ! How automation can enable workers to complete tasks more efficiently and focus on bigger problems machines aren’t as good at solving Why AI will become ubiquitous in business—but not for 10-15 years Rob’s Predict, Automate, and Classify (PAC) framework for deploying AI for business value, and how it can help achieve maximum economic impact Economic and societal considerations that people should be thinking about when developing AI – and what we aren’t ready for yet as a society Dealing with biases and stereotypes in data, and the ethical issues they can create when training models How using synthetic data in certain situations can improve AI models and facilitate usage of the technology Concepts product managers of AI and ML solutions should be thinking about Training, UX and classification issues when designing experiences around AI The importance of model-market fit. In other words, whether a model satisfies a market demand, and whether it will actually make a difference after being deployed. Resources and Links: Email Rob@pjc.vc PJC Talla SmartBid The PAC Framework for Deploying AI Twitter: @robmay Sign up for Rob’s Newsletter Quotes from Today’s Episode “[Service as a software] is a logical extension of software eating the world. Software eats industry after industry, and now it’s eating industries using machine learning that are primarily human labor focused.” — Rob “It doesn’t have to be all digital. You could also think about it in terms of restaurant automation, and some of those things where if you keep the interface the same to the customer—the service you’re providing—you strip it out, and everything behind that, if it’s digital it’s an algorithm and if it’s physical, then you use a robot.” — Rob, on service as a software. “[When designing for] AI you really want to find some way to convey to the user that the tool is getting smarter and learning.”— Rob “There’s a gap right now between the business use cases of AI and the places it’s getting adopted in organizations,” — Rob “The reason that AI’s so interesting is because what you effectively have now is software models that don’t just execute a task, but they can learn from that execution process and change how they execute.” — Rob “If you are changing things and your business is changing, which is most businesses these days, then it’s going to help to have models around that can learn and grow and adapt. I think as we get better with different data types—not just text and images, but more and more types of data types—I think every business is going to deploy AI at some stage.” — Rob “The general sense I get is that overall, putting these models and AI solutions is pretty difficult still.” — Brian “They’re not looking at what’s the actual best use of AI for their business, [and thinking] ‘Where could you really apply to have the most economic impact?’ There aren’t a lot of people that have thought about it that way.” — Rob, on how AI is being misapplied in the enterprise. “You have to focus on the outcome, not just the output.” — Brian “We need more heuristics for how, as a product manager, you think of AI and building it into products.” — Rob “When the internet came about, it impacted almost every business in some way, shape, or form.[…]he reason that AI’s so interesting is because what you effectively have now is software models that don’t just execute a task, but they can learn from that execution process and change how they execute.” — Rob “Some biases and stereotypes are true, and so what happens if the AI uncovers one that we’re really uncomfortable with?” — Rob

  • #36
    Apr 7, 2020 · 45 min

    036 – How Higher-Ed Institutions are Using AI and Analytics to Better Serve Students with Professor of Learning Informatics and Edtech Expert Simon Bu...

    Simon Buckingham Shum is Professor of Learning Informatics at Australia’s University of Technology Sydney (UTS) and Director of the Connected Intelligence Centre (CIC)—an innovation center where students and staff can explore education data science applications. Simon holds a Ph.D from the University of York, and is known for bringing a human-centered approach to analytics and development. He also co-founded the Society for Learning Analytics Research (SoLAR), which is committed to advancing learning through ethical, educationally sound data science. In this episode, Simon and I discuss the state of education technology (edtech), privacy, human-centered design in the context of using AI in higher ed, and the numerous technological advancements that are re-shaping the higher level education landscape. Our conversation covered: How the hype cycle around big data and analytics is starting to pervade education The differences between using BI and analytics to streamline operations, improve retention rates, vs. the ways AI and data are used to increase learning and engagement Creating systems that teachers see as interesting and valuable, in order to drive user adoption and avoid friction. The more difficult-to-design-for, but more important skills and competencies researchers are working on to prepare students for a highly complex future workplace The data and privacy issues that must be factored into ethical solution designs Why “learning is not shopping,” meaning we the creators of the tech have to infer what goes on in the mind when studying humans, mostly by studying behavior. Why learning scientists and educational professionals play an important role in the edtech design process, in addition to technical workers How predictive modeling can be used to identify students who are struggling—and the ethical questions that such solutions raise. Resources and Links Designing for Analytics simon.buckinghamshum.net Simon on LinkedIn #experiencingdata Designing for Analytics Podcast Quotes from Today’s Episode “We are seeing AI products coming out. Some of them are great, and are making a huge difference for learning STEM type subjects— science, tech, engineering, and medicine. But some of them are not getting the balance right.” — Simon “The trust break-down will come, and has already come in certain situations, when students feel they’re being tracked…” — Simon, on students perceiving BI solutions as surveillance tools instead of beneficial “Increasingly, it’s great to see so many people asking critical questions about the biases that you can get in training data, and in algorithms as well. We want to ask questions about whether people are trusting this technology. It’s all very well to talk about big data and AI, etc., but ultimately, no one’s going to use this stuff if they don’t trust it.” — Simon “I’m always asking what’s the user experience going to be? How are we actually going to put something in front of people that they’re going to understand…” — Simon “There are lots of success stories, and there are lots of failure stories. And that’s just what you expect when you’ve got edtech companies moving at high speed.” — Simon “We’re dealing, on the one hand, with poor products that give the whole field a bad name, but on the other hand, there are some really great products out there that are making a tangible difference, and teachers are extremely enthusiastic about.” — Simon “There’s good evidence now, about the impact that some of these tools can have on learning. Teachers can give some homework out, and the next morning, they can see on their dashboard which questions were the students really struggling with.” — Simon “The area that we’re getting more and more interested in, and which educators are getting more and more interested in, are the kinds of skills and competencies you need for a very complex future workplace.” — Simon “We obviously want the students’ voice in the design process. But that has to be balanced with all the other voices are there as well, like the educators’ voice, as well as the technologists, and the interaction designers and so forth.” — Simon on the nuance of UX considerations for students “…you have to balance satisfying the stakeholder with actually what is needed.” — Brian “…we’re really at the mercy of behavior. We have to try and infer, from behavior or traces, what’s going on in the mind, of the humans we are studying.” — Simon “We might say, “Well, if we see a student writing like this, using these kinds of textual features that we can pick up using natural language processing, and they revise their draft writing in response to feedback that we’ve provided automatically, well, that looks like progress. It looks like they’re thinking more critically, or it looks like they’re reflecting more deeply on an experience they’ve had, for example, like a work placement.” — Simon “They’re in products already, and when they’re used well, they can be effective. But they can also be sort of weapon of mass destruction if you use them badly.” — Simon, on predictive models

  • #35
    Mar 24, 2020 · 47 min

    035 – Future Ethics Author and Designer Cennydd Bowles Shares Strategies for Designing Ethical Data Products That Benefit Our Business, Community and ...

    Cennydd Bowles is a London-based digital product designer and futurist, with almost two decades of consulting experience working with some of the largest and most influential brands in the world. Cennydd has earned a reputation as a trusted guide, helping companies navigate complex issues related to design, technology, and ethics. He’s also the author of Future Ethics, a book which outlines key ethical principles and methods for constructing a fairer future. In this episode, Cennydd and I explore the role that ethics plays in design and innovation, and why so many companies today—in Silicon Valley and beyond—are failing to recognize the human element of their technological pursuits. Cennydd offers his unique perspective, along with some practical tips that technologists can use to design with greater mindfulness and consideration for others. In our chat, we covered topics from Cennydd’s book and expertise including: Why there is growing resentment towards the tech industry and the reason all companies and innovators need to pay attention to ethics The importance of framing so that teams look beyond the creation of an “ethical product / solution” and out towards a better society and future The role that diversity plays in ethics and the reason why homogenous teams working in isolation can be dangerous for an organization and society Cennydd’s “front-page test,” “designated dissenter,” and other actionable ethics tips that innovators and data product teams can apply starting today Navigating the gray areas of ethics and how large companies handle them The unfortunate consequences that arise when data product teams are complacent The fallacy that data is neutral—and why there is no such thing as “raw” data Why stakeholders must take part in ethics conversations Resources and Links: Cennydd Bowles Future Ethics (book) Design for Real Life The Trouble with Bias Twitter: @cennydd Quotes from Today’s Episode “There ought to be a clearer relationship between innovation and its social impacts.” — Cennydd “I wouldn’t be doing this if I didn’t think there was a strong upside to technology, or if I didn’t think it couldn’t advance the species.” — Cennydd “I think as our power has grown, we have failed to use that power responsibly, and so it’s absolutely fair that we be held to account for those mistakes.” — Cennydd “I like to assume most creators and data people are trying to do good work. They’re not trying to do ethically wrong things. They just lack the experience or tools and methods to design with intent.” — Brian “Ethics is about discussion and it’s about decisions; it’s not about abstract theory.” — Cennydd “I have seen many times diversity act as an ethical early warning system [where] people who firmly believe the solution they’re about to put out into the world is, if not flawless, pretty damn close.” — Cennydd “The ethical questions around the misapplication or the abuse of data are strong and prominent, and actually have achieved maybe even more recognition than other forms of harm that I talk about.” — Cennydd “There aren’t a whole lot of ethical issues that are black and white.” — Cennydd “When you never talk to a customer or user, it’s really easy to make choices that can screw them at the benefit of increasing some KPI or business metric.” — Brian “I think there’s really talented people in the data space who actually understand bias really well, but when they think about bias, they think they’re thinking more about, ‘how is it going to skew the insight from the data?’ Not the human impact.” — Brian “I think every business has almost a moral duty to take their consequences seriously.” — Cennydd

  • #34
    Mar 10, 2020 · 35 min

    034 – ML & UX: To Augment or Automate? Plus, Rating Overall Analytics Efficacy with Eric Siegel, Ph.D.

    Eric Siegel, Ph.D. is founder of the Predictive Analytics World and Deep Learning World conference series, executive editor of “The Predictive Analytics Times,” and author of “Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die.” A former Columbia University professor and host of the Dr. Data Show web series, Siegel is a renowned speaker and educator who has been commissioned for more than 100 keynote addresses across multiple industries. Eric is best known for making the “how” and “why” of predictive analytics (aka machine learning) understandable and captivating to his audiences. In our chat, we covered: The value of defining business outcomes and end user’s needs prior to starting the technical work of predictive modeling, algorithms, or software design. The idea of data prototypes being used before engaging in data science to determine where models could potentially fail—saving time while improving your odds of success. The first and most important step of Eric’s five-step analytics deployment plan Getting multiple people aligned and coordinated about pragmatic considerations and practical constraints surrounding ML project deployment. The score (1-10) Eric gave the data community on its ability to turn data into value The difference between decision support and decision automation and what the Central Intelligence Agency’s CDAO thinks about these two methods for using machine learning. Understanding how human decisions are informed by quantitative predictions from predictive modes, and what’s required to deliver information in a way that aligns with their needs. How Eric likes to bring agility to machine learning by deploying and scaling models incrementally to mitigate risk Where the analytics field currently stands in its overall ability to generate value in the last mile. Resources and Links: Machine Learning Week #experiencingdata PredictiveAnalyticsWorld.com ThePredictionBook.com Dr. Data Show Twitter: @predictanalytic Quotes from Today’s Episode “The greatest pitfall that hinders analytics is not to properly plan for its deployment.” — Brian, quoting Eric “You don’t jump to number crunching. You start [by asking], ‘Hey, how is this thing going to actually improve business?’ “ — Eric “You can do some preliminary number crunching, but don’t greenlight, trigger, and go ahead with the whole machine learning project until you’ve planned accordingly, and iterated. It’s a collaborative effort to design, target, define scope, and ultimately greenlight and execute on a full-scale machine learning project.” — Eric “If you’re listening to this interview, it’s your responsibility.” — Eric, commenting on whose job it is to define the business objective of a project. “Yeah, so in terms of if 10 were the highest potential [score], in the sort of ideal world where it was really being used to its fullest potential, I don’t know, I guess I would give us a score of [listen to find out!]. Is that what Tom [Davenport] gave!?” — Eric, when asked to rate the analytics community on its ability to deliver value with data “We really need to get past our outputs, and the things that we make, the artifacts and those types of software, whatever it may be, and really try to focus on the downstream outcome, which is sometimes harder to manage, or measure … but ultimately, that’s where the value is created.” — Brian “Whatever the deployment is, whatever the change from the current champion method, and now this is the challenger method, you don’t have to jump entirely from one to the other. You can incrementally deploy it. So start by saying well, 10 percent of the time we’ll use the new method which is driven by a predictive model, or by a better predictive model, or some kind of change. So in the change in the transition, you sort of do it incrementally, and you mitigate your risk in that way.”— Eric

  • #33
    Feb 25, 2020 · 41 min

    033 - How Vidant Health’s Data Team Creates Empathetic Data Products and Ethical Machine Learning Models with Greg Nelson

    Greg Nelson is VP of data analytics at Vidant Health, as well as an adjunct faculty member at Duke University. He is also the author of the “Analytics Lifecycle Toolkit,” which is a manual for integrating data management technologies. A data evangelist with over 20 years of experience in analytics and advisory, Nelson is widely known for his human-centered approach to analytics. In this episode, Greg and I explore what makes a data product or decision support application indispensable, specifically in the complex world of healthcare. In our chat, we covered: Seeing through the noise and identifying what really matters when designing data products The type of empathy training Greg and his COO are rolling out to help technical data teams produce more useful data products The role of data analytics product management and why this is a strategic skillset at Vidant The AI Playbook Greg uses at Vidant Health and their risk-based approach to assessing how they will validate the quality of a data product The process Greg uses to test and handle algorithmic bias and how this is linked to credibility in the data products they produce How exactly design thinking helps Greg’s team achieve better results, trust and credibility How Greg aligns workflows, processes, and best practice protocols when developing predictive models Resources and Links: Vidant Health Analytics Lifecycle Toolkit Greg Nelson’s article “Bias in Artificial Intelligence” Greg Nelson on LinkedIn Twitter: @GregorySNelson Video: Tuning a card deck for human-centered co-design of Learning Analytics Quotes from Today's Episode “We'd rather do fewer things and do them well than do lots of things and fail.”— Greg “In a world of limited resources, our job is to make sure we're actually building the things that matter and that will get used. Product management focuses the light on use case-centered approaches and design thinking to actually come up with and craft the right data products that start with empathy.”— Greg “I talk a lot about whole-brain thinking and whole-problem thinking. And when we understand the whole problem, the whole ‘why’ about someone's job, we recognize pretty quickly why Apple was so successful with their initial iPod.”— Greg “The technical people have to get better [...] at extracting needs in a way that is understandable, interpretable, and really actionable, from a technology perspective. It's like teaching someone a language they never knew they needed. There's a lot of resistance to it.” — Greg “I think deep down inside, the smart executive knows that you don’t bat .900 when you're doing innovation.” — Brian “We can use design thinking to help us fail a little bit earlier, and to know what we learned from it, and then push it forward so that people understand why this is not working. And then you can factor what you learned into the next pass.” — Brian “If there's one thing that I've heard from most of the leaders in the data and analytics space, with regards particularly to data scientists, it’s [the importance of] finding this “other” missing skill set, which is not the technical skillset. It's understanding the human behavioral piece and really being able to connect the fact that your technical work does have this soft skill stuff.” — Brian “At the end of the day, I tell people our mission is to deliver data that people can trust in a way that's usable and actionable, built on a foundation of data literacy and dexterity. That trust in the first part of our core mission is essential.”— Greg

  • #32
    Feb 11, 2020 · 39 min

    032 - How and Why Talented Analytical Minds Leave People Scratching Their Head Around Data with Nancy Duarte

    Nancy Duarte is a communication expert and the leader of the largest design firm in Silicon Valley, Duarte, Inc. She has more than 30 years of experience working with global companies and counts eight of the top ten Fortune 500 brands in her clientele. She is the author of six books, and her work as appeared in Fortune, Time Magazine, Forbes, Wired, Wall Street Journal, New York Times, Los Angeles Times, Cosmopolitan Magazine, and CNN. In this episode, Nancy and I discussed some of the reasons analytics and data experts fail to effectively communicate the insights and value around data. She drew from her key findings in her work as a communication expert that she details in her new book, Data Story, and the importance of communicating data through the natural structure of storytelling. In our chat, we covered: How empathy is tied to effective communication. Biases that cloud our own understanding of our communication skills How to communicate an enormous amount of data effectively and engagingly What’s wrong with sharing traditional presentations as a reading asset and Nancy’s improved replacement for them in the enterprise The difference in presenting data in business versus scientific settings Why STEAM, not STEM, is relevant to effective communication for data professionals and what happens when creativity and communication aren’t taught How the brain reacts differently when it is engaged through a story Resources and Links: Nancy Duarte on LinkedIn Twitter: @nancyduarte Slidedocs Duarte DataStory Quotes from Today’s Episode “I think the biggest struggle for analysts is they see a lot of data.” —Nancy “In a business context, the goal is not to do perfect research most of the time. It’s actually to probably help inform someone else’s decision-making.” —Nancy “Really understand empathy, become a bit of a student of story, and when you start to apply.” these, you’ll see a lot of traction around your ideas.” — Nancy “We’ve gone so heavily rewarded the analytical mindset that now we can’t back out of that and be dual-modal about being an analytical mindset and then also really having discipline around a creative mindset.” — Nancy “There’s a bunch of supporting data, but there’s also all this intuition and other stuff that goes into it. And so I think just learning to accept the ambiguity as part of that human experience, even in business.” — Brian “If your software application doesn’t produce meaningful decision support, then you didn’t do anything. The data is just sitting there and it’s not actually activating.” — Brian “People can’t draw a direct line from what art class or band does for you, and it’s the first thing that gets cut. Then we complain on the backend when people are working in professional settings that they can’t talk to us.” — Brian

  • #31
    Jan 28, 2020 · 41 min

    031 - How Design Helps Enable Repeatable Value on AI, ML, and Analytics Projects with Ganes Kesari

    Ganes Kesari is the co-founder and head of analytics and AI labs at Gramener, a software company that helps organizations tell more effective stories with their data through robust visualizations. He’s also an advisor, public speaker, and author who talks about AI in plain English so that a general audience can understand it. Prior to founding Gramener, Ganes worked at companies like Cognizant, Birlasoft, and HCL Technologies serving in various management and analyst roles. Join Ganes and I as we talk about how design, as a core competency, has enabled Gramener’s analytics and machine learning work to produce better value for clients. We also touched on: Why Ganes believes the gap between the business and data analytics organizations is getting smaller How AI (and some other buzzwords) are encouraging more and more investments in understanding data Ganes’ opinions about the “analytics translator” role How companies might think they are unique for not using “traditional agile”—when in fact that’s what everyone is doing Ganes’ thoughts on the similarities of use cases across verticals and the rise of verticalized deep data science solutions Why Ganes believes organizations are increasingly asking for repeatable data science solutions The pivotal role that empathy plays in convincing someone to use your software or data model How Ganes’ team approaches client requests for data science projects, the process they follow to identify use cases for AI, and how they use AI to identify the biggest business problem that can be solved What Ganes believes practitioners should consider when moving data projects forward at their organizations Resources and Links Gramener.com Ganes Kesari on Twitter: @Kesaritweets Ganes Kesari on LinkedIn: https://www.linkedin.com/in/ganes-kesari/ Quotes from Today’s Episode “People tend to have some in-house analytics capability. They’re reaching out for design. Then it’s more of where people feel that the adoption hasn’t happened. They have that algorithm but no one understands its use. And then they try to buy some license or some exploratory visualization tools and they try their hand at it and they’ve figured out that it probably needs a lot more than some cute charts or some dashboards. It can’t be an afterthought. That’s when they reach out.” — Ganes “Now a lot more enquiries, a lot more engagements are happening centrally at the enterprise level where they have realized the need for data science and they want to run it centrally so it’s no longer isolated silos.” — Ganes “I see that this is a slightly broader movement where people are understanding the value of data and they see that it is something that they can’t avoid or they can’t prioritize it lower anymore.“ — Ganes “While we have done a few hundred consulting engagements and help with bespoke solutions, there is still an element of commonality. So that’s where we abstracted some of those, the common or technology requirements and common solutions into our platform.” — Ganes “My general perception is that most data science and analytics firms don’t think about design as a core competency or part of analytics and data science—at least not beyond perhaps data visualization.” —Brian “I was in a LinkedIn conversation today about this and some comments that Tom Davenport had made on this show a couple of episodes ago. He was talking about how we need this type of role that goes out and understands how data is used and how systems and software are used such that we can better align the solutions with what people are doing. And I was like, ‘amen.’ That’s actually not a new role though; it’s what good designers do!” — Brian

  • #30
    Jan 14, 2020 · 51 min

    030 - Using AI to Recommend Personalized Medical Treatment Options with Joost Zeeuw of Pacmed

    Joost Zeeuw is a data scientist and product owner at Pacmed, a data-driven healthcare and AI startup in Amsterdam that combines medical expertise and machine learning to create stronger patient outcomes and improve healthcare experiences. He’s also taught a number of different subjects—like physics, chemistry, and mathematics—at Lyceo, an online education service, and Luzac College in the Netherlands. Join Brian and Joost as they discuss the role of design and user experience within the context of providing personalized medical treatments using AI. Plus: The role data has in influencing doctors’ decisions—without making the decisions The questions Joost’s product team asks before designing any AI solution at Pacmed How people’s familiarity with iPhones and ease-of-use has influenced expectations around simplicity—and the challenges this poses when there is machine learning under the hood Why Brian thinks Pacmed’s abnormal approach to design is great—and what that approach looks like The simple, non-technical, but critical thing Pacmed did early on to help them define their AI product strategy and avoid going down the wrong path An example of an unexpected treatment prediction that Pacmed’s algorithm detected—which ended up being something that a specific field of medicine had been studying with classical research techniques 10,000 km away Where Joost believes Western medicine falls short with respect to new drug trials Resources and Links Joost on LinkedIn Pacmed.ai on LinkedIn Quotes for Today’s Episode “Pacmed in that has a three-fold mission, which is, first of all, to try to make sure that every single patient gets the treatment that has proven to work for him or her based on prior data analysis. And next to that we say, ‘well, if an algorithm can learn all these awesome insights generated by thousands and thousands of doctors, then a doctor using one of those products is also very capable of learning more and more things from the lessons that are incorporated in this algorithm and this product.’ And finally, healthcare is very expensive. We are trying to maximize the efficiency and the effectiveness of that spend by making sure everybody gets a treatment that has the highest probability of working for him or her.” — Joost “Offering a data product like this is really another tool in that toolbox that allows the doctor to pierce through this insane amount of complexity that there is in giving care to a patient.” — Joost “Before designing anything, we ask ourselves this: Does it fit into the workflow of people that already have maybe one of the most demanding jobs in the world?” — Joost “There’s a very big gap between what is scientifically medically interesting and what’s practical in a healthcare system.” — Joost “When I talk about design here, I’m talking kind of about capital D design. So product design, user experience, looking at the whole business and the outcomes we’re trying to drive, it’s kind of that larger picture here.” — Brian “I don’t think this is ‘normal’ for a lot of people coming from the engineering side or from the data science side to be going out and talking to customers, thinking about like how does this person do their job and how does my work fit into you know a bigger picture solution of what this person needs to do all day, and what are the health outcomes we’re going for? That part of this product development process is not about data science, right? It’s about the human factors piece, about how does our solution fit into this world.” — Brian “I think that the impact of bringing people out into the field—whatever that is, that could be a corporate cubicle somewhere, a hospital, outside in a farm field—usually there’s a really positive thing that happens because I think people are able to connect their work with an actual human being that’s going to potentially use this solution. And when we look at software all day, it’s very easy to disconnect from any sense of human connection with someone else.” — Brian “If you’re a product owner or even if you’re more on the analytics side, but you’re responsible for delivering decision support, it’s really important to go get a feel for what people are doing all day.” — Brian

  • #29
    Dec 31, 2019 · 45 min

    029 - Why Google Believes it’s Critical to Pair Designers with Your Data Scientists to Produce Human-Centered ML & AI Products with Di Dang

    Di Dang is an emerging tech design advocate at Google and helped lead the creation of Google’s People + AI Guidebook. In her role, she works with product design teams, external partners, and end users to support the creation of emerging tech experiences. She also teaches a course on immersive technology at the School of Visual Concepts. Prior to these positions, Di worked as an emerging tech lead and senior UX designer at POP, a UX consultant at Kintsugi Creative Solutions, and a business development manager at AppLift. She earned a bachelor of arts degree in philosophy and religious studies from Stanford University. Join Brian and Di as they discuss the intersection of design and human-centered AI and: Why a data science leader should care about design and integrating designers during a machine-learning project, and the impacts when they do not What exactly Di does in her capacity as an emerging tech design advocate at Google and the definition of human-centered AI How design helps data science teams save money and time by elucidating the problem space and user needs The two key purposes of Google’s People + AI Research (PAIR) team What Google’s triptych methodology is and how it helps teams prevent building the wrong solution A specific example of how user research and design helped ship a Pixel 2 feature How to ensure an AI solution is human-centered when a non-tech company wants to build something but lacks a formal product manager or UX lead/resource The original goals behind the creation of Google’s People + AI Guidebook The role vocabulary plays in human-centered AI design Resources and Links Twitter: @Dqpdang Di Dang’s Website Di Dang on LinkedIn People + AI Guidebook Quotes from Today’s Episode “Even within Google, I can’t tell you how many times I have tech leaders, engineers who kind of cock an eyebrow at me and ask, ‘Why would design be involved when it comes to working with machine learning?’” — Di “Software applications of machine learning is a relatively nascent space and we have a lot to learn from in terms of designing for it. The People + AI Guidebook is a starting point and we want to understand what works, what doesn’t, and what’s missing so that we can continue to build best practices around AI product decisions together.” — Di “The key value proposition that design brings is we want to work with you to help make sure that when we’re utilizing machine learning, that we’re utilizing it to solve a problem for a user in a way that couldn’t be done through other technologies or through heuristics or rules-based programming—that we’re really using machine learning where it’s most needed.” — Di “A key piece that I hear again and again from internal Google product teams and external product teams that I work with is that it’s very, very easy for a lot of teams to default to a tech-first kind of mentality. It’s like, ‘Oh, well you know, machine learning, should we ML this?’ That’s a very common problem that we hear. So then, machine learning becomes this hammer for which everything is a nail—but if only a hammer were as easy to construct as a piece of wood and a little metal anvil kind of bit.” — Di “A lot of folks are still evolving their own mental model around what machine learning is and what it’s good for. But closely in relation—because this is something that I think people don’t talk as much about maybe because it’s less sexy to talk about than machine learning—is that there are often times a lot of organizational or political or cultural uncertainties or confusion around even integrating machine learning.” — Di “I think there’s a valid promise that there’s a real opportunity with AI. It’s going to change businesses in a significant way and there’s something to that. At the same time, it’s like go purchase some data scientists, throw them in your team, and have them start whacking stuff. And they’re kind of waiting for someone to hand them a good problem to work on and the business doesn’t know and they’re just saying, ‘What is our machine learning strategy?’ And so someone in theory hopefully is hunting for a good problem to solve.” — Brian “Everyone’s trying to move fast all the time and ship code and a lot of times we focus on the shipping of code and the putting of models into production as our measurement—as opposed to the outcomes that come from putting something into production.” — Brian “The difference between the good and the great designer is the ability to merge the business objectives with ethically sound user-facing and user-centered principles.” — Brian

  • #28
    Dec 17, 2019 · 50 min

    028 - Cole Nussbaumer Knaflic On Data Storytelling, DataViz, and Why Your Data May Not Be Inspiring Action

    When it comes to telling stories with data, Cole Nussbaumer Knaflic is ahead of the curve. In October 2015, she wrote a best-selling book called storytelling with data: a data visualization guide for business professionals. That book led to the creation of storytelling with data, an agency that helps businesses communicate more effectively using data, and she’s since followed-up with another best-seller: storytelling with data: let’s practice! Prior to her current role, Cole served as a people analytics manager at Google, was the owner and chief consultant at Insight Analytics, and held several positions at Washington Mutual, among other positions. In our chat, we covered: Why sharp communication skills are integral to telling stories with data The skills data people need to effectively communicate with data Who Cole thinks you should run your presentations by first, and the specific colleagues you should be sharing them with Why it’s important to practice presentations in informal settings first How looking at data in different formats can help you build more effective presentations The differences between exploratory and explanatory data analysis in the context of storytelling The important role of diction when presenting data Cole’s opinions on the skills many modern workers need around data storytelling Why data visualization and the ability to tell stories is not a nice-to-have skill What Cole’s approach to preparing for a presentation looks like and the format she uses to kick off the process Resources and Links Designingforanalytics.com/seminar Twitter: @Storywithdata. Company website: Storytellingwithdata.com Quotes from Today’s Episode “I've always really enjoyed that space where numbers and business intersect and enjoy how we can use numbers to get to understand things better and make smarter decisions.” — Cole “If you're the one analyzing the data, you know it best. And you're actually in a unique position to be able to derive and help others derive greater value from that data. But in order to do that, you have to be able to talk to other people about it and communicate what you've done to technical audiences and to non-technical audiences.” — Cole “When it comes to communicating effectively with data, you can't take out the human part. That's the part where things can either succeed or fail.” — Cole “There's no single right way to show data. Any data can be graphed a ton of different ways. And so when we're thinking about how we visualize our data, it really means stepping back and thinking about what sort of behavior we’re trying to drive in our audience. What do we want them to see in this? And then it often means iterating through different views of the data, which is also a fantastic way just to get to know your data better because different views will make observations easier or less easy to see.” — Cole “As soon as we try to draw attention to one aspect of the data or another, it actually makes any other potential takeaways harder to see.” — Cole “Words are very important for making our data accessible and understandable.” — Cole “Depending on the visualization, what you're doing is you're teaching yourself not to assume that the information is necessarily clear. You're being objective. And it sounds like a dumb question, but that's kind of what I usually recommend to my clients: We need to be really objective about our assumptions about what's being communicated here and validate that.” — Brian “The low-fidelity format—especially if you're working with a stakeholder or perhaps someone who's going to be the recipient—enables them to give you honest feedback. Because the more polished that sucker looks, the less they're going to want to give you any.” — Brian

  • #27
    Dec 3, 2019 · 47 min

    027 - Balancing Your Inner Data Science Nerd While Becoming a Trusted Business Advisor and Strategist with Angela Bassa of iRobot

    Angela Bassa is the director of data science and head of data science and machine learning at iRobot, a technology company focused on robotics (you might have clean floors thanks to a Roomba). Prior to joining iRobot, Angela wore several different hats, including working as a financial analyst at Morgan Stanley, the senior manager of big data analytics and platform engineering at EnerNOC, and even a scuba instructor in the U.S. Virgin Islands. Join Angela and I as we discuss the role data science plays in robotics and explore: Why Angela doesn’t believe in a division between technical and non-technical skill Why Angela came to iRobot and her mission What data breadcrumbs are and what you should know about them The skill Angela believes matters most when turning data science into a producer of decision support Why the last mile of the UX is often way longer than one mile The critical role expectation management plays in data science, how Angela handles delivering surprise findings to the business, and the marketing skill she taps to help her build trust Resources and Links Twitter: @AngeBassa Angela’s Website iRobot Designing for Analytics Seminar Quotes from Today’s Episode “Because these tools that we use sometimes can be quite sophisticated, it’s really easy to use very complicated jargon to impart credibility onto results that perhaps aren’t merited. I like to call that math-washing the result.” — Angela “Our mandate is to make sure that we are making the best decisions—that we are informing strategy rather than just believing certain bits of institutional knowledge or anecdotes or trends. We can actually sort of demonstrate and test those hypotheses with the data that is available to us. And so we can make much better informed decisions and, hopefully, less risky ones.” — Angela “Data alone isn’t the ground truth. Data isn’t the thing that we should be reacting to. Data are artifacts. They’re breadcrumbs that help us reconstruct what might have happened.” — Angela [When getting somebody to trust the data science work], I don’t think the trust comes from bringing someone along during the actual timeline. I think it has more to do with bringing someone along with the narrative.—Angela “It sounds like you’ve created a nice dependency for your data science team. You’re seen as a strategic partner as opposed to being off in the corner doing cryptic work that people can’t understand.” — Brian “When I talk to data scientists and leaders, they often talk about how technical skills are very easy to measure. You can see them on paper, you can get them in the interview. But there are these other skills that are required to do effective work and create value.” — Brian

  • #26
    Nov 19, 2019 · 47 min

    026 - Why Tom Davenport Gives a 2 out of 10 Score To the Data Science and Analytics Industry for Value Creation

    Tom Davenport has literally written the book on analytics. Actually, several of them, to be precise. Over the course of his career, Tom has established himself as the authority on analytics and how their role in the modern organization has evolved in recent years. Tom is a distinguished professor at Babson College, a research fellow at the MIT Initiative on the Digital Economy, and a senior advisor at Deloitte Analytics. The discussion was timely as Tom had just written an article about a financial services company that had trained its employees on human-centered design so that they could ensure any use of AI would be customer-driven and valuable. We discussed their journey and: Why on a scale of 1-10, the field of analytics has only gone from a one to about a two in ten years time Why so few analytics projects actually make it into production Examples of companies who are using design to turn data into useful applications of AI, decision support and product improvements for customers Why shadow IT shouldn’t be a bad word AI moonshot projects vs. MVPs and how they relate Why journey mapping is incredibly useful and important in analytics and data science work How human-centered design and ethnography is the tough work that’s required to turn data into decision support Tom’s new book and his thoughts on the future of data science and analytics Resources and Links: Website: Tomdavenport.com LinkedIn: Tom Davenport Twitter: @tdav Designingforanalytics.com/seminar Designingforanalytics.com Quotes from Today’s Episode “If you survey organizations and ask them, ‘Does your company have a data-driven culture?’ they almost always say no. Surveys even show a kind of negative movement over recent years in that regard. And it's because nobody really addresses that issue. They only address the technology side.” — Tom Eventually, I think some fraction of [AI and analytics solutions] get used and are moderately effective, but there is not nearly enough focus on this. A lot of analytics people think their job is to create models, and whether anybody uses it or not is not their responsibility...We don't have enough people who make it their jobs to do that sort of thing. —Tom I think we need this new specialist, like a data ethnographer, who could sort of understand much more how people interact with data and applications, and how many ways they get screwed up.—Tom I don't know how you inculcate it or teach it in schools, but I think we all need curiosity about how technology can make us work more effectively. It clearly takes some investment, and time, and effort to do it.— Tom TD Wealth’s goal was to get [its employees] to experientially understand what data, analytics, technology, and AI are all about, and then to think a lot about how it related to their customers. So they had a lot of time spent with customers, understanding what their needs were to make that match with AI. [...] Most organizations only address the technology and the data sides, so I thought this was very refreshing.—Tom “So we all want to do stuff with data. But as you know, there are a lot of poor solutions that get provided from technical people back to business stakeholders. Sometimes they fall on deaf ears. They don't get used.” — Brian “I actually had a consultant I was talking to recently who said you know the average VP/director or CDO/CAO has about two years now to show results, and this gravy train may be slowing down a little bit.“ — Brian “One of the things that I see in the kind of the data science and analytics community is almost this expectation that ‘I will be handed a well-crafted and well-defined problem that is a data problem, and then I will go off and solve it using my technical skills, and then provide you with an answer.’” — Brian

Showing 161–180 of 206 episodes