
S9 Ep62: How well do bug bounties work?
For three decades years, software makers have paid bug bounty hunters to break their code so they can make it more secure. Recently, some software companies have decided that bug bounties programmes aren't useful any more: Curl shut its bounty in January, and Intel suspended its own in September. And the discovery of the PHP flaw that sent websites offline in October came from researchers working with AI, not from a bounty hunter's report. Tobias Kretschmer (Imperial College London, CEPR) and his co-authors followed 86 programmes on Bugcrowd to discover how they work best. Do you run closed programmes, open only to invited hackers, or open them to all comers? And what happens now that AI is making a bug report cheaper to produce? Tim Phillips asks Kretschmer how well bug bounties are working. The research behind this episode: Zrahia, Aviram, Sarit Markovich, and Tobias Kretschmer. 2026. "Platform Openness: Evidence from Bug Bounty Programs." CEPR Discussion Paper 21970 (gated). To cite this episode: Phillips, Tim, and Tobias Kretschmer. 2026. "How well do bug bounties work?" VoxTalks Economics (podcast). About the guest Tobias Kretschmer is Professor of Strategy, Technology and Organisation at Imperial College Business School and a Research Fellow at the Centre for Economic Policy Research. He joined Imperial in 2025 from LMU Munich. His research spans competitive strategy in digital industries, platform markets, and the digital transformation of organisations. Research cited in this episode Bug bounties. Netscape launched the first technology bug bounty on 10 October 1995, offering rewards for flaws found in the Navigator 2.0 beta. Jarrett Ridlinghafer, a technical support engineer, proposed the idea after he saw enthusiasts finding bugs for free. In this paper's data, the average reward for a unique vulnerability is about $566 in private programmes and $209 in public ones. High-priority flaws average about $1,440. Bugcrowd connects organisations with independent security researchers. It gave the authors anonymised records of more than 480,000 submissions to 2,483 programmes, from mid-2012 to mid-2021. The data predates the widespread use of AI-generated submissions, so the paper measures the effect of openness, not the effect of AI. The authors argue that AI sharpens the same problem. Triage. Each submission is classified as rejected, duplicate, or unique. A unique report identifies a flaw nobody has reported before. Only the first valid report generally earns a reward, so a duplicate can be good work that arrived late. It must still be reviewed. Wisdom of the crowd. Francis Galton's "Vox Populi" (Nature, 1907) reported that the median of about 800 guesses at an ox's dressed weight, made at a Plymouth livestock show, was remarkably close to the true figure. Eric Raymond gave software its version in "The Cathedral and the Bazaar": "given enough eyeballs, all bugs are shallow". Maillart and co-authors tested that idea on bug bounties in 2017. Kretschmer's point is that real attackers differ in their objectives and methods; a varied crowd resembles them more closely than one employee can. Open and closed programmes. Anyone can join a public programme; a private one is by invitation. Bugcrowd and many other bounty platforms leave that choice to the software company rather than setting one rule for all. Many other platforms keep far tighter control, which makes this the governance decision worth studying. Kretschmer and co-authors set out the wider argument about platforms as organisations in 2022. Among programmes that never switched, public ones were 7.6% of programmes but drew about half of all submissions. Private ones accounted for about 59% of unique discoveries and nearly 80% of dollar rewards. Comparing like with like. Firms choose their regime deliberately, so comparing open with closed programmes misleads. The authors focus on the 86 programmes that switched once from private to public, and compare them with programmes that stayed private. The main estimates are about 23 more submissions a week, nearly six more unique vulnerabilities, 10 more new researchers, and seven more duplicates. An alternative estimator built for staggered timing gives smaller figures: about 15, three, seven, and six. The effect is front-loaded. Weekly submissions averaged about five before opening, reached about 70, then settled near 20 within 10 weeks. Opening is a choice, not an experiment, and the authors say so. Quality mix. This is the share of incoming submissions that are unique. It is about 16 percentage points higher in private programmes, and it falls by about 17 points when a programme opens. Opening produces more discoveries in absolute terms and a smaller share of useful reports. Monitoring. The authors cannot see how much firms spend on review, so they count verdicts per submission. One more weekly verdict is associated with about $139 more in weekly rewards in public programmes and about $508 in private ones; this is an association, not a causal estimate. A rough calculation in the paper suggests that opening pays only when a programme's discoveries have fallen to fewer than about 3.5 a week, before review costs. Programmes that opened averaged about 2.2. The authors say they do not identify the best moment to open, and Kretschmer makes the same point in the episode. Programmes that have closed. Curl ended its bounty on 31 January 2026 after paying more than $100,000 for 87 confirmed vulnerabilities since 2019. Its creator, Daniel Stenberg, reported that the confirmed rate fell from more than 15% of submissions to below 5%, which he blamed in large part on AI slop. Intel suspended its programme in September 2026, which paid up to $100,000 per flaw, and replaced it with one that pays nothing. It gave no reason. Google stopped accepting product vulnerability reports to its open source reward programme on 1 October 2026, a change it calls temporary. PHP2Shell. Two critical flaws, CVE-2026-109618 and CVE-2026-109619, were patched in PHP on 21 October 2026. PHP runs about 69% of websites whose server-side language is known, and many operators took sites offline as a precaution. Calif found the first route to an attack during an audit under OpenAI's Patch the Planet programme, and Wiz and OpenAI reported further ones; all used AI tools. The PHP Foundation says a core developer first saw signs of one flaw in 2024, then the fix stalled, and it calls that delay "a structural failure" of process and resources. It reports 15 CVEs and 106 reports closed as invalid, duplicate, or non-reproducible since 1 July, and says "the models don't make the calls". Open source software. Both curl and PHP are open source, maintained by small teams on which much of the web depends. Kretschmer argues that no single model fits. A firm may keep tight control of its core code and open the rest to its users. A related CEPR paper by Miklos Koren and co-authors, "Vibe Coding Kills Open Source", models how AI could weaken the user engagement that sustains these projects. More VoxTalks Economics episodes How well does patent screening work? Mark Schankerman on another system that depends on experts sorting good claims from bad ones. Will AI transform economic growth? Anton Korinek on what happens if AI starts to automate AI research. Related reading on VoxEU Platform regulation: What policymakers can and cannot learn from utility industries, a VoxEU column by Sven Werner and Tobias Kretschmer arguing that rules for digital platforms must account for the independent participants those platforms oversee. Cybersecurity vulnerabilities and their financial impact, a VoxEU column showing that firms with high cyber exposure earn about 0.42% lower excess returns per month. A brief look of the economics of open source software, a VoxEU column from 2011 by Neil Gandal on why volunteers contribute and how firms came to mix open and closed code.


















