AI's Data Access Question
AI is reopening a core question from the development of the web: how to preserve the freedom to learn while giving publishers meaningful control over how their works are made available. In this episode, Matt Perault is joined by Derek Slater, cofounder of Proteus Strategies and an expert on information access, copyright, and free expression, to examine this question and explain why it matters for the future of AI. The web worked because people could access, read, analyze, and build on lawfully available information. Standards like robots.txt helped manage the balance between openness and control at scale, giving publishers a way to express preferences without requiring every builder to negotiate permission website by website. AI is now testing that equilibrium. Some proposals would restrict not only unlawful access, like when AI developers circumvent paywalls to get data, but also lawful learning from public information through expanded copyright theories, terms of service, technical barriers, or licensing requirements. Derek and Matt separate those issues, including the difference between training models on lawfully accessed data, producing infringing outputs, using AI tools to summarize content a user can already access, and breaking through access controls. For Little Tech, this question is fundamental. Access to data operates as a form of startup capital, allowing new companies to develop products and compete. But if AI companies can’t learn without negotiating expensive licenses or if large pools of data are entirely off limits to AI learning, then only the biggest, most-resourced companies will be able to survive. Topics covered: 00:00: Introduction 01:45: The freedom to learn and AI data access 04:16: What the early web can teach us about openness, control, and contested norms 08:48: How robots.txt helped publishers express preferences at scale 11:43: Why voluntary standards worked for search engines and publishers 13:50: How freedom to learn applies beyond technology and copyright debates 16:02: The publisher POV on traffic, monetization, and value exchange 18:36: Why AI agents are raising new questions about user control 20:36: The difference between protecting publishers and limiting lawful AI-assisted reading 21:42: How copyright law applies to AI training inputs and model outputs 28:08: How contracts and terms of service can attempt to restrict lawful learning 31:12: The limits of “learning” as a defense 33:53: How licensing markets are evolving around data access and AI outputs 37:12: Technical collaboration and the future of robots.txt-style standards for AI 39:43: Why data access is a Little Tech issue 42:25: Public policy guidance for preserving the freedom to learn 45:29: Closing thoughts Disclosure: Derek Slater has previously provided consulting support to Andreessen Horowitz. The views expressed here are his own, and this conversation was not part of a paid engagement. Resources: Subscribe to the a16z AI Policy Brief: https://a16zpolicy.substack.com/ Follow Matt Perault on X: https://x.com/MattPerault Follow Derek Slater on X: https://x.com/derekslater The content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. Please note that a16z and its affiliates may maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit a16zpolicy.substack.com
