Sign inOpen the app
Artwork for Machine Learning Street Talk (MLST)
Technology

Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

Welcome! We engage in fascinating discussions with pre-eminent figures in the AI field. Our flagship show covers current affairs in AI, cognitive science, neuroscience and philosophy of mind with in-depth analysis. Our approach is unrivalled in terms of scope and rigour – we believe in intellectual diversity in AI, and we touch on all of the main ideas in the field with the hype surgically removed. MLST is run by Tim Scarfe, Ph.D (https://www.linkedin.com/in/ecsquizor/) and features regular appearances from MIT Doctor of Philosophy Keith Duggar (https://www.linkedin.com/in/dr-keith-duggar/).

SubscribeListen in podnodRSS feedShow website
  • 257 episodes
  • Updated Friday

Episodes257

Newest first
  • Jan 11, 2021 · 1 hr 35 min

    #037 - Tour De Bayesian with Connor Tann

    Connor Tan is a physicist and senior data scientist working for a multinational energy company where he co-founded and leads a data science team. He holds a first-class degree in experimental and theoretical physics from Cambridge university. With a master's in particle astrophysics. He specializes in the application of machine learning models and Bayesian methods. Today we explore the history, pratical utility, and unique capabilities of Bayesian methods. We also discuss the computational difficulties inherent in Bayesian methods along with modern methods for approximate solutions such as Markov Chain Monte Carlo. Finally, we discuss how Bayesian optimization in the context of automl may one day put Data Scientists like Connor out of work. Panel: Dr. Keith Duggar, Alex Stenlake, Dr. Tim Scarfe 00:00:00 Duggars philisophical ramblings on Bayesianism 00:05:10 Introduction 00:07:30 small datasets and prior scientific knowledge 00:10:37 Bayesian methods are probability theory 00:14:00 Bayesian methods demand hard computations 00:15:46 uncertainty can matter more than estimators 00:19:29 updating or combining knowledge is a key feature 00:25:39 Frequency or Reasonable Expectation as the Primary Concept 00:30:02 Gambling and coin flips 00:37:32 Rev. Thomas Bayes's pool table 00:40:37 ignorance priors are beautiful yet hard 00:43:49 connections between common distributions 00:49:13 A curious Universe, Benford's Law 00:55:17 choosing priors, a tale of two factories 01:02:19 integration, the computational Achilles heel 01:35:25 Bayesian social context in the ML community 01:10:24 frequentist methods as a first approximation 01:13:13 driven to Bayesian methods by small sample size 01:18:46 Bayesian optimization with automl, a job killer? 01:25:28 different approaches to hyper-parameter optimization 01:30:18 advice for aspiring Bayesians 01:33:59 who would connor interview next? Connor Tann: https://www.linkedin.com/in/connor-tann-a92906a1/ https://twitter.com/connossor

  • Jan 3, 2021 · 1 hr 42 min

    #036 - Max Welling: Quantum, Manifolds & Symmetries in ML

    Today we had a fantastic conversation with Professor Max Welling, VP of Technology, Qualcomm Technologies Netherlands B.V. Max is a strong believer in the power of data and computation and its relevance to artificial intelligence. There is a fundamental blank slate paradgm in machine learning, experience and data alone currently rule the roost. Max wants to build a house of domain knowledge on top of that blank slate. Max thinks there are no predictions without assumptions, no generalization without inductive bias. The bias-variance tradeoff tells us that we need to use additional human knowledge when data is insufficient. Max Welling has pioneered many of the most sophistocated inductive priors in DL models developed in recent years, allowing us to use Deep Learning with non-euclidean data i.e. on graphs/topology (a field we now called "geometric deep learning") or allowing network architectures to recognise new symmetries in the data for example gauge or SE(3) equivariance. Max has also brought many other concepts from his physics playbook into ML, for example quantum and even Bayesian approaches. This is not an episode to miss, it might be our best yet! Panel: Dr. Tim Scarfe, Yannic Kilcher, Alex Stenlake 00:00:00 Show introduction 00:04:37 Protein Fold from DeepMind -- did it use SE(3) transformer? 00:09:58 How has machine learning progressed 00:19:57 Quantum Deformed Neural Networks paper 00:22:54 Probabilistic Numeric Convolutional Neural Networks paper 00:27:04 Ilia Karmanov from Qualcomm interview mini segment 00:32:04 Main Show Intro 00:35:21 How is Max known in the community? 00:36:35 How Max nurtures talent, freedom and relationship is key 00:40:30 Selecting research directions and guidance 00:43:42 Priors vs experience (bias/variance trade-off) 00:48:47 Generative models and GPT-3 00:51:57 Bias/variance trade off -- when do priors hurt us 00:54:48 Capsule networks 01:03:09 Which old ideas whould we revive 01:04:36 Hardware lottery paper 01:07:50 Greatness can't be planned (Kenneth Stanley reference) 01:09:10 A new sort of peer review and originality 01:11:57 Quantum Computing 01:14:25 Quantum deformed neural networks paper 01:21:57 Probabalistic numeric convolutional neural networks 01:26:35 Matrix exponential 01:28:44 Other ideas from physics i.e. chaos, holography, renormalisation 01:34:25 Reddit 01:37:19 Open review system in ML 01:41:43 Outro

  • Dec 27, 2020 · 2 hr 56 min

    #035 Christmas Community Edition!

    Welcome to the Christmas special community edition of MLST! We discuss some recent and interesting papers from Pedro Domingos (are NNs kernel machines?), Deepmind (can NNs out-reason symbolic machines?), Anna Rodgers - When BERT Plays The Lottery, All Tickets Are Winning, Prof. Mark Bishop (even causal methods won't deliver understanding), We also cover our favourite bits from the recent Montreal AI event run by Prof. Gary Marcus (including Rich Sutton, Danny Kahneman and Christof Koch). We respond to a reader mail on Capsule networks. Then we do a deep dive into Type Theory and Lambda Calculus with community member Alex Mattick. In the final hour we discuss inductive priors and label information density with another one of our discord community members. Panel: Dr. Tim Scarfe, Yannic Kilcher, Alex Stenlake, Dr. Keith Duggar Enjoy the show and don't forget to subscribe! 00:00:00 Welcome to Christmas Special! 00:00:44 SoTa meme 00:01:30 Happy Christmas! 00:03:11 Paper -- DeepMind - Outperforming neuro-symbolic models with NNs (Ding et al) 00:08:57 What does it mean to understand? 00:17:37 Paper - Prof. Mark Bishop Artificial Intelligence is stupid and causal reasoning wont fix it 00:25:39 Paper -- Pedro Domingos - Every Model Learned by Gradient Descent Is Approximately a Kernel Machine 00:31:07 Paper - Bengio - Inductive Biases for Deep Learning of Higher-Level Cognition 00:32:54 Anna Rodgers - When BERT Plays The Lottery, All Tickets Are Winning 00:37:16 Montreal AI event - Gary Marcus on reasoning 00:40:37 Montreal AI event -- Rich Sutton on universal theory of AI 00:49:45 Montreal AI event -- Danny Kahneman, System 1 vs 2 and Generative Models ala free energy principle 01:02:57 Montreal AI event -- Christof Koch - Neuroscience is hard 01:10:55 Markus Carr -- reader letter on capsule networks 01:13:21 Alex response to Marcus Carr 01:22:06 Type theory segment -- with Alex Mattick from Discord 01:24:45 Type theory segment -- What is Type Theory 01:28:12 Type theory segment -- Difference between functional and OOP languages 01:29:03 Type theory segment -- Lambda calculus 01:30:46 Type theory segment -- Closures 01:35:05 Type theory segment -- Term rewriting (confluency and termination) 01:42:02 MType theory segment -- eta term rewritig system - Lambda Calculus 01:54:44 Type theory segment -- Types / semantics 02:06:26 Type theory segment -- Calculus of constructions 02:09:27 Type theory segment -- Homotopy type theory 02:11:02 Type theory segment -- Deep learning link 02:17:27 Jan from Discord segment -- Chrome MRU skit 02:18:56 Jan from Discord segment -- Inductive priors (with XMaster96/Jan from Discord) 02:37:59 Jan from Discord segment -- Label information density (with XMaster96/Jan from Discord) 02:55:13 Outro

  • Dec 20, 2020 · 2 hr 39 min

    #034 Eray Özkural- AGI, Simulations & Safety

    Dr. Eray Ozkural is an AGI researcher from Turkey, he is the founder of Celestial Intellect Cybernetics. Eray is extremely critical of Max Tegmark, Nick Bostrom and MIRI founder Elizier Yodokovsky and their views on AI safety. Eray thinks that these views represent a form of neoludditism and they are capturing valuable research budgets with doomsday fear-mongering and effectively want to prevent AI from being developed by those they don't agree with. Eray is also sceptical of the intelligence explosion hypothesis and the argument from simulation. Panel -- Dr. Keith Duggar, Dr. Tim Scarfe, Yannic Kilcher 00:00:00 Show teaser intro with added nuggets and commentary 00:48:39 Main Show Introduction 00:53:14 Doomsaying to Control 00:56:39 Fear the Basilisk! 01:08:00 Intelligence Explosion Ethics 01:09:45 Fear the Automous Drone! ... or spam 01:11:25 Infinity Point Hypothesis 01:15:26 Meat Level Intelligence 01:21:25 Defining Intelligence ... Yet Again 01:27:34 We'll make brains and then shoot them 01:31:00 The Universe likes deep learning 01:33:16 NNs are glorified hash tables 01:38:44 Radical behaviorists 01:41:29 Omega Architecture, possible AGI? 01:53:33 Simulation hypothesis 02:09:44 No one cometh unto Simulation, but by Jesus Christ 02:16:47 Agendas, Motivations, and Mind Projections 02:23:38 A computable Universe of Bulk Automata 02:30:31 Self-Organized Post-Show Coda 02:31:29 Investigating Intelligent Agency is Science 02:36:56 Goodbye and cheers! https://www.youtube.com/watch?v=pZsHZDA9TJU

  • Dec 13, 2020 · 1 hr 51 min

    #033 Prof. Karl Friston - The Free Energy Principle

    This week Dr. Tim Scarfe, Dr. Keith Duggar and Connor Leahy chat with Prof. Karl Friston. Professor Friston is a British neuroscientist at University College London and an authority on brain imaging. In 2016 he was ranked the most influential neuroscientist on Semantic Scholar. His main contribution to theoretical neurobiology is the variational Free energy principle, also known as active inference in the Bayesian brain. The FEP is a formal statement that the existential imperative for any system which survives in the changing world can be cast as an inference problem. Bayesian Brain Hypothesis states that the brain is confronted with ambiguous sensory evidence, which it interprets by making inferences about the hidden states which caused the sensory data. So is the brain an inference engine? The key concept separating Friston's idea from traditional stochastic reinforcement learning methods and even Bayesian reinforcement learning is moving away from goal-directed optimisation. Remember to subscribe! Enjoy the show! 00:00:00 Show teaser intro 00:16:24 Main formalism for FEP 00:28:29 Path Integral 00:30:52 How did we feel talking to friston? 00:34:06 Skit - on cultures (checked, but maybe make shorter) 00:36:02 Friston joins 00:36:33 Main show introduction 00:40:51 Is prediction all it takes for intelligence? 00:48:21 balancing accuracy with flexibility 00:57:36 belief-free vs belief-based; beliefs are crucial 01:04:53 Fuzzy Markov Blankets and Wandering Sets 01:12:37 The Free Energy Principle conforms to itself 01:14:50 useful false beliefs 01:19:14 complexity minimization is the heart of free energy [01:19:14 ]Keith: 01:23:25 An Alpha to tip the scales? Absoute not! Absolutely yes! 01:28:47 FEP applied to brain anatomy 01:36:28 Are there multiple non-FEP forms in the brain? 01:43:11 a positive conneciton to backpropagation 01:47:12 The FEP does not explain the origin of FEP systems 01:49:32 Post-show banter https://www.fil.ion.ucl.ac.uk/~karl/ #machinelearning

  • Dec 6, 2020 · 1 hr 30 min

    #032- Simon Kornblith / GoogleAI - SimCLR and Paper Haul!

    This week Dr. Tim Scarfe, Sayak Paul and Yannic Kilcher speak with Dr. Simon Kornblith from Google Brain (Ph.D from MIT). Simon is trying to understand how neural nets do what they do. Simon was the second author on the seminal Google AI SimCLR paper. We also cover "Do Wide and Deep Networks learn the same things?", "Whats in a Loss function for Image Classification?", and "Big Self-supervised models are strong semi-supervised learners". Simon used to be a neuroscientist and also gives us the story of his unique journey into ML. 00:00:00 Show Teaser / or "short version" 00:18:34 Show intro 00:22:11 Relationship between neuroscience and machine learning 00:29:28 Similarity analysis and evolution of representations in Neural Networks 00:39:55 Expressability of NNs 00:42:33 Whats in a loss function for image classification 00:46:52 Loss function implications for transfer learning 00:50:44 SimCLR paper 01:00:19 Contrast SimCLR to BYOL 01:01:43 Data augmentation 01:06:35 Universality of image representations 01:09:25 Universality of augmentations 01:23:04 GPT-3 01:25:09 GANs for data augmentation?? 01:26:50 Julia language @skornblith https://www.linkedin.com/in/simon-kornblith-54b2033a/ https://arxiv.org/abs/2010.15327 Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth https://arxiv.org/abs/2010.16402 What's in a Loss Function for Image Classification? https://arxiv.org/abs/2002.05709 A Simple Framework for Contrastive Learning of Visual Representations https://arxiv.org/abs/2006.10029 Big Self-Supervised Models are Strong Semi-Supervised Learners

  • Nov 28, 2020 · 2 hr 44 min

    #031 WE GOT ACCESS TO GPT-3! (With Gary Marcus, Walid Saba and Connor Leahy)

    In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Keith Duggar speak with Gary Marcus and Connor Leahy about GPT-3. We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations. Note that this podcast version is significantly truncated, watch the youtube version for the TOC and experiments with GPT-3 https://www.youtube.com/watch?v=iccd86vOz3w

  • Nov 20, 2020 · 1 hr 48 min

    #030 Multi-Armed Bandits and Pure-Exploration (Wouter M. Koolen)

    This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher discuss multi-arm bandits and pure exploration with Dr. Wouter M. Koolen, Senior Researcher, Machine Learning group, Centrum Wiskunde & Informatica. Wouter specialises in machine learning theory, game theory, information theory, statistics and optimisation. Wouter is currently interested in pure exploration in multi-armed bandit models, game tree search, and accelerated learning in sequential decision problems. His research has been cited 1000 times, and he has been published in NeurIPS, the number 1 ML conference 14 times as well as lots of other exciting publications. Today we are going to talk about two of the most studied settings in control, decision theory, and learning in unknown environment which are the multi-armed bandit (MAB) and reinforcement learning (RL) approaches - when can an agent stop learning and start exploiting using the knowledge it obtained - which strategy leads to minimal learning time 00:00:00 What are multi-arm bandits/show trailer 00:12:55 Show introduction 00:15:50 Bandits 00:18:58 Taxonomy of decision framework approaches 00:25:46 Exploration vs Exploitation 00:31:43 the sharp divide between modes 00:34:12 bandit measures of success 00:36:44 connections to reinforcement learning 00:44:00 when to apply pure exploration in games 00:45:54 bandit lower bounds, a pure exploration renaissance 00:50:21 pure exploration compiler dreams 00:51:56 what would the PX-compiler DSL look like 00:57:13 the long arms of the bandit 01:00:21 causal models behind the curtain of arms 01:02:43 adversarial bandits, arms trying to beat you 01:05:12 bandits as an optimization problem 01:11:39 asymptotic optimality vs practical performance 01:15:38 pitfalls hiding under asymptotic cover 01:18:50 adding features to bandits 01:27:24 moderate confidence regimes 01:30:33 algorithms choice is highly sensitive to bounds 01:46:09 Post script: Keith interesting piece on n quantum http://wouterkoolen.info https://www.cwi.nl/research-groups/ma... #machinelearning

  • Nov 8, 2020 · 1 hr 50 min

    #029 GPT-3, Prompt Engineering, Trading, AI Alignment, Intelligence

    This week Dr. Tim Scarfe, Dr. Keith Duggar, Yannic Kilcher and Connor Leahy cover a broad range of topics, ranging from academia, GPT-3 and whether prompt engineering could be the next in-demand skill, markets and economics including trading and whether you can predict the stock market, AI alignment, utilitarian philosophy, randomness and intelligence and even whether the universe is infinite! 00:00:00 Show Introduction 00:12:49 Academia and doing a Ph.D 00:15:49 From academia to wall street 00:17:08 Quants -- smoke and mirrors? Tail Risk 00:19:46 Previous results dont indicate future success in markets 00:23:23 Making money from social media signals? 00:24:41 Predicting the stock market 00:27:20 Things which are and are not predictable 00:31:40 Tim postscript comment on predicting markets 00:32:37 Connor take on markets 00:35:16 As market become more efficient.. 00:36:38 Snake oil in ML 00:39:20 GPT-3, we have changed our minds 00:52:34 Prompt engineering a new form of software development? 01:06:07 GPT-3 and prompt engineering 01:12:33 Emergent intelligence with increasingly weird abstractions 01:27:29 Wireheading and the economy 01:28:54 Free markets, dragon story and price vs value 01:33:59 Utilitarian philosophy and what does good look like? 01:41:39 Randomness and intelligence 01:44:55 Different schools of thought in ML 01:46:09 Is the universe infinite? Thanks a lot for Connor Leahy for being a guest on today's show. https://twitter.com/NPCollapse -- you can join his EleutherAI community discord here: https://discord.com/invite/vtRgjbM

  • Nov 4, 2020 · 2 hr 20 min

    NLP is not NLU and GPT-3 - Walid Saba

    #machinelearning This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher speak with veteran NLU expert Dr. Walid Saba. Walid is an old-school AI expert. He is a polymath, a neuroscientist, psychologist, linguist, philosopher, statistician, and logician. He thinks the missing information problem and lack of a typed ontology is the key issue with NLU, not sample efficiency or generalisation. He is a big critic of the deep learning movement and BERTology. We also cover GPT-3 in some detail in today's session, covering Luciano Floridi's recent article "GPT‑3: Its Nature, Scope, Limits, and Consequences" and a commentary on the incredible power of GPT-3 to perform tasks with just a few examples including the Yann LeCun commentary on Facebook and Hackernews. Time stamps on the YouTube version 0:00:00 Walid intro 00:05:03 Knowledge acquisition bottleneck 00:06:11 Language is ambiguous 00:07:41 Language is not learned 00:08:32 Language is a formal language 00:08:55 Learning from data doesn’t work 00:14:01 Intelligence 00:15:07 Lack of domain knowledge these days 00:16:37 Yannic Kilcher thuglife comment 00:17:57 Deep learning assault 00:20:07 The way we evaluate language models is flawed 00:20:47 Humans do type checking 00:23:02 Ontologic 00:25:48 Comments On GPT3 00:30:54 Yann lecun and reddit 00:33:57 Minds and machines - Luciano 00:35:55 Main show introduction 00:39:02 Walid introduces himself 00:40:20 science advances one funeral at a time 00:44:58 Deep learning obsession syndrome and inception 00:46:14 BERTology / empirical methods are not NLU 00:49:55 Pattern recognition vs domain reasoning, is the knowledge in the data 00:56:04 Natural language understanding is about decoding and not compression, it's not learnable. 01:01:46 Intelligence is about not needing infinite amounts of time 01:04:23 We need an explicit ontological structure to understand anything 01:06:40 Ontological concepts 01:09:38 Word embeddings 01:12:20 There is power in structure 01:15:16 Language models are not trained on pronoun disambiguation and resolving scopes 01:17:33 The information is not in the data 01:19:03 Can we generate these rules on the fly? Rules or data? 01:20:39 The missing data problem is key 01:21:19 Problem with empirical methods and lecunn reference 01:22:45 Comparison with meatspace (brains) 01:28:16 The knowledge graph game, is knowledge constructed or discovered 01:29:41 How small can this ontology of the world be? 01:33:08 Walids taxonomy of understanding 01:38:49 The trend seems to be, less rules is better not the othe way around? 01:40:30 Testing the latest NLP models with entailment 01:42:25 Problems with the way we evaluate NLP 01:44:10 Winograd Schema challenge 01:45:56 All you need to know now is how to build neural networks, lack of rigour in ML research 01:50:47 Is everything learnable 01:53:02 How should we elevate language systems? 01:54:04 10 big problems in language (missing information) 01:55:59 Multiple inheritance is wrong 01:58:19 Language is ambiguous 02:01:14 How big would our world ontology need to be? 02:05:49 How to learn more about NLU 02:09:10 AlphaGo Walid's blog: https://medium.com/@ontologik LinkedIn: https://www.linkedin.com/in/walidsaba/

  • Nov 1, 2020 · 2 hr 4 min

    AI Alignment & AGI Fire Alarm - Connor Leahy

    This week Dr. Tim Scarfe, Alex Stenlake and Yannic Kilcher speak with AGI and AI alignment specialist Connor Leahy a machine learning engineer from Aleph Alpha and founder of EleutherAI. Connor believes that AI alignment is philosophy with a deadline and that we are on the precipice, the stakes are astronomical. AI is important, and it will go wrong by default. Connor thinks that the singularity or intelligence explosion is near. Connor says that AGI is like climate change but worse, even harder problems, even shorter deadline and even worse consequences for the future. These problems are hard, and nobody knows what to do about them. 00:00:00 Introduction to AI alignment and AGI fire alarm 00:15:16 Main Show Intro 00:18:38 Different schools of thought on AI safety 00:24:03 What is intelligence? 00:25:48 AI Alignment 00:27:39 Humans dont have a coherent utility function 00:28:13 Newcomb's paradox and advanced decision problems 00:34:01 Incentives and behavioural economics 00:37:19 Prisoner's dilemma 00:40:24 Ayn Rand and game theory in politics and business 00:44:04 Instrumental convergence and orthogonality thesis 00:46:14 Utility functions and the Stop button problem 00:55:24 AI corrigibality - self alignment 00:56:16 Decision theory and stability / wireheading / robust delegation 00:59:30 Stop button problem 01:00:40 Making the world a better place 01:03:43 Is intelligence a search problem? 01:04:39 Mesa optimisation / humans are misaligned AI 01:06:04 Inner vs outer alignment / faulty reward functions 01:07:31 Large corporations are intelligent and have no stop function 01:10:21 Dutch booking / what is rationality / decision theory 01:16:32 Understanding very powerful AIs 01:18:03 Kolmogorov complexity 01:19:52 GPT-3 - is it intelligent, are humans even intelligent? 01:28:40 Scaling hypothesis 01:29:30 Connor thought DL was dead in 2017 01:37:54 Why is GPT-3 as intelligent as a human 01:44:43 Jeff Hawkins on intelligence as compression and the great lookup table 01:50:28 AI ethics related to AI alignment? 01:53:26 Interpretability 01:56:27 Regulation 01:57:54 Intelligence explosion Discord: https://discord.com/invite/vtRgjbM EleutherAI: https://www.eleuther.ai Twitter: https://twitter.com/npcollapse LinkedIn: https://www.linkedin.com/in/connor-j-leahy/

  • Oct 28, 2020 · 1 hr 26 min

    Kaggle, ML Community / Engineering (Sanyam Bhutani)

    Join Dr Tim Scarfe, Sayak Paul, Yannic Kilcher, and Alex Stenlake have a conversation with Mr. Chai Time Data Science; Sanyam Bhutani! 00:00:00 Introduction 00:03:42 Show kick off 00:06:34 How did Sanyam get started into ML 00:07:46 Being a content creator 00:09:01 Can you be self taught without a formal education in ML? 00:22:54 Kaggle 00:33:41 H20 product / job 00:40:58 Intepretability / bias / engineering skills 00:43:22 Get that first job in DS 00:46:29 AWS ML Ops architecture / ml engineering 01:14:19 Patterns 01:18:09 Testability 01:20:54 Adversarial examples Sanyam's blog -- https://sanyambhutani.com/tag/chaitimedatascience/ Chai Time Data Science -- https://www.youtube.com/c/ChaiTimeDataScience

  • Oct 20, 2020 · 1 hr 30 min

    Sara Hooker - The Hardware Lottery, Sparsity and Fairness

    Dr. Tim Scarfe, Yannic Kilcher and Sayak Paul chat with Sara Hooker from the Google Brain team! We discuss her recent hardware lottery paper, pruning / sparsity, bias mitigation and intepretability. The hardware lottery -- what causes inertia or friction in the marketplace of ideas? Is there a meritocracy of ideas or do the previous decisions we have made enslave us? Sara Hooker calls this a lottery because she feels that machine learning progress is entirely beholdant to the hardware and software landscape. Ideas succeed if they are compatible with the hardware and software at the time and also the existing inventions. The machine learning community is exceptional because the pace of innovation is fast and we operate largely in the open, this is largely because we don't build anything physical which is expensive, slow and the cost of being scooped is high. We get stuck in basins of attraction based on our technology decisions and it's expensive to jump outside of these basins. So is this story unique to hardware and AI algorithms or is it really just the story of all innovation? Every great innovation must wait for the right stepping stone to be in place before it can really happen. We are excited to bring you Sara Hooker to give her take. YouTube version (including TOC): https://youtu.be/sQFxbQ7ade0 Show notes; https://drive.google.com/file/d/1S_rHnhaoVX4Nzx_8e3ESQq4uSswASNo7/view?usp=sharing Sara Hooker page; https://www.sarahooker.me

  • Oct 11, 2020 · 1 hr 16 min

    The Social Dilemma Part 3 - Dr. Rebecca Roache

    This week join Dr. Tim Scarfe, Yannic Kilcher, and Keith Duggar have a conversation with Dr. Rebecca Roache in the last of our 3-part series on the social dilemma Netflix film. Rebecca is a senior lecturer in philosophy at Royal Holloway, university of London and has written extensively about the future of friendship. People claim that friendships are not what they used to be. People are always staring at their phones, even when in public Social media has turned us into narcissists who are always managing our own PR rather than being present with each other. Anxiety about the negative effects of technology are as old as the written word. Is technology bad for friendships? Can you have friends through screens? Does social media cause polarization? And is that a bad thing? Does it promote quantity over quality? Rebecca thinks that social media and echo chambers are less ominous to friendship on closer inspection. 00:00:32 Teaser clip from Rebecca and her new manuscript on friendship 00:02:52 Introduction 00:04:56 Memorisation vs reasoning / is technology enhancing friendships 00:09:29 Word of warcraft / gaming communities / echo chambers / polarisation 00:12:34 Horizontal vs Vertical social attributes 00:17:18 Exclusion of others opinions 00:20:36 The power to silence others / truth verification 00:23:58 Misinformation 00:27:28 Norms / memes / political terms and co-opting / bullying 00:31:57 Redefinition of political terms i.e. racism 00:36:13 Virtue signalling 00:38:57 How many friends can you have / spread thin / Dunbars 150 00:42:54 Is it morally objectionable to believe or contemplate objectionable ideas, punishment 00:50:52 Is speaking the same thing as acting 00:52:24 Punishment - deterrence vs retribution / historical 00:53:59 Yannic: contemplating is a form of speaking 00:57:32 silencing/blocking is intellectual laziness - what ideas are we allowed to talk about 01:04:53 Corporate AI ethics frameworks 01:09:14 Autonomous Vehicles 01:10:51 the eternal Facebook world / online vs offline friendships 01:14:05 How do we get the best out of our online friendships

  • Oct 3, 2020 · 1 hr 7 min

    The Social Dilemma - Part 1

    In this first part of our three part series on the Social Dilemma Netflix film, Dr. Tim Scarfe, Yannic "Lightspeed" Kilcher and Zak Jost gang up with Cybersecurity expert Andy Smith. We give you our take on the film. We are super excited to get your feedback on this one! Hope you enjoy. 00:00:00 Introduction 00:06:11 Moral hypocrisy 00:12:38 Road to hell is paved with good intentions, attention economy 00:15:04 They know everything about you 00:18:02 Addiction 00:21:22 Differential realities 00:26:12 Self determination and Monetisation 00:29:08 AI: Overwhelm human strengths undermine human vulnerabilities 00:31:51 Conspiracy theory / fake news 00:34:23 Overton window / polarisation 00:39:12 Short attention span / convergent behaviour 00:41:26 Is social media good for you 00:45:17 Your attention time is linear, the things you can pay attention to are a volume, anonymity 00:51:32 Andy question on security: social engineering 00:56:32 Is it a security risk having your information in social media 00:58:02 Retrospective judgement 01:03:06 Free speech and censorship 01:06:06 Technology accelerator

  • Sep 29, 2020 · 1 hr 24 min

    Capsule Networks and Education Targets

    In today's episode, Dr. Keith Duggar, Alex Stenlake and Dr. Tim Scarfe chat about the education chapter in Kenneth Stanley's "Greatness cannot be planned" book, and we relate it to our Algoshambes conversation a few weeks ago. We debate whether objectives in education are a good thing and whether they cause perverse incentives and stifle creativity and innovation. Next up we dissect capsule networks from the top down! We finish off talking about fast algorithms and quantum computing. 00:00:00 Introduction 00:01:13 Greatness cannot be planned / education 00:12:03 Perverse incentives 00:19:25 Treasure hunting 00:30:28 Capsule Networks 00:46:08 Capsules As Compositional Networks 00:52:45 Capsule Routing 00:57:10 Loss and Warps 01:09:55 Fast Algorithms and Quantum Computing

  • Sep 25, 2020 · 1 hr 23 min

    Programming Languages, Software Engineering and Machine Learning

    This week Dr. Tim Scarfe, Dr. Keith Duggar, Yannic "Lightspeed" Kilcher have a conversation with Microsoft Senior Software Engineer Sachin Kundu. We speak about programming languages including which our favourites are and functional programming vs OOP. Next we speak about software engineering and the intersection of software engineering and machine learning. We also talk about applications of ML and finally what makes an exceptional software engineer and tech lead. Sachin is an expert in this field so we hope you enjoy the conversation! Spoiler alert, how many of you have read the Mythical Man-Month by Frederick P. Brooks?! 00:00:00 Introduction 00:06:37 Programming Languages 00:53:41 Applications of ML 01:55:59 What makes an exceptional SE and tech lead 01:22:08 Outro

  • Sep 22, 2020 · 1 hr 13 min

    Computation, Bayesian Model Selection, Interactive Articles

    This week Dr. Keith Duggar, Alex Stenlake and Dr. Tim Scarfe discuss the theory of computation, intelligence, Bayesian model selection, the intelligence explosion and the the phenomenon of "interactive articles". 00:00:00 Intro 00:01:27 Kernels and context-free grammars 00:06:04 Theory of computation 00:18:41 Intelligence 00:22:03 Bayesian model selection 00:44:05 AI-IQ Measure / Intelligence explosion 00:52:09 Interactive articles 01:12:32 Outro

  • Sep 18, 2020 · 1 hr 37 min

    Kernels!

    Today Yannic Lightspeed Kilcher and I spoke with Alex Stenlake about Kernel Methods. What is a kernel? Do you remember those weird kernel things which everyone obsessed about before deep learning? What about Representer theorem and reproducible kernel hilbert spaces? SVMs and kernel ridge regression? Remember them?! Hope you enjoy the conversation! 00:00:00 Tim Intro 00:01:35 Yannic clever insight from this discussion 00:03:25 Street talk and Alex intro 00:05:06 How kernels are taught 00:09:20 Computational tractability 00:10:32 Maths 00:11:50 What is a kernel? 00:19:39 Kernel latent expansion 00:23:57 Overfitting 00:24:50 Hilbert spaces 00:30:20 Compare to DL 00:31:18 Back to hilbert spaces 00:45:19 Computational tractability 2 00:52:23 Curse of dimensionality 00:55:01 RBF: infinite taylor series 00:57:20 Margin/SVM 01:00:07 KRR/dual 01:03:26 Complexity compute kernels vs deep learning 01:05:03 Good for small problems? vs deep learning) 01:07:50 Whats special about the RBF kernel 01:11:06 Another DL comparison 01:14:01 Representer theorem 01:20:05 Relation to back prop 01:25:10 Connection with NLP/transformers 01:27:31 Where else kernels good 01:34:34 Deep learning vs dual kernel methods 01:33:29 Thoughts on AI 01:34:35 Outro

  • Sep 16, 2020 · 1 hr 25 min

    Explainability, Reasoning, Priors and GPT-3

    This week Dr. Tim Scarfe and Dr. Keith Duggar discuss Explainability, Reasoning, Priors and GPT-3. We check out Christoph Molnar's book on intepretability, talk about priors vs experience in NNs, whether NNs are reasoning and also cover articles by Gary Marcus and Walid Saba critiquing deep learning. We finish with a brief discussion of Chollet's ARC challenge and intelligence paper. 00:00:00 Intro 00:01:17 Explainability and Christoph Molnars book on Intepretability 00:26:45 Explainability - Feature visualisation 00:33:28 Architecture / CPPNs 00:36:10 Invariance and data parsimony, priors and experience, manifolds 00:42:04 What NNs learn / logical view of modern AI (Walid Saba article) 00:47:10 Core knowledge 00:55:33 Priors vs experience 00:59:44 Mathematical reasoning 01:01:56 Gary Marcus on GPT-3 01:09:14 Can NNs reason at all? 01:18:05 Chollet intelligence paper/ARC challenge