TechnologySociety & CulturePhilosophy

LessWrong posts by zvi

zvi

Audio narrations of LessWrong posts by zvi

Play

Open in

64 episodes
daily
Avg 55 min
English

Episodes 64

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

Today · 30 min
“OpenAI Updates Its Model Spec” by Zvi
The Model Spec is potentially OpenAI's most important document. It outlines what they aspire for the model to do, and how it should think. I’ve already covered a previous version in depth, so my strategy this time is to look at the diff. I had a Google Doc created for this purpose, and will be working from there. A bunch of things get moved around or reworded, often piecemeal. There are a lot of line edits, most of which are improvements. I’m focusing on what looks new and important. My main new critique is that many of the new examples do not carve reality at its joints, instead having overdetermined answers, and fail to teach the model or reader what the Spec wants them to do in actually difficult cases. Overall I see the new Model Spec as philosophically and practically similar to the old Model Spec. That is not a terrible place to be, as the old Model Spec was a solid document with many strong details. It also means this is a highly ‘optional’ post for those who do not have cause to dig into such weeds. Table of Contents Five Values. Meet the [...] --- Outline: (01:13) Five Values (02:29) Meet the New Spec (03:33) New Risk Taxonomy (04:28) The Letter and the Spirit (05:06) Your Funeral (07:22) Context Clues (09:20) Preserve User Control Over User Control (10:05) Sycophancy (11:47) Lies My User Told Me (12:38) Impact Function (15:32) Levels of Friction (17:01) Do Not Lie (22:16) Pick a Side, We're at War (26:19) Strengthening Agency (29:03) Decision Theory (29:45) Do Good Things Not Bad Things --- First published: November 3rd, 2026 Source: https://www.lesswrong.com/posts/ABEQbEabK6kcSRrNq/openai-updates-its-model-spec --- Narrated by TYPE III AUDIO.
Yesterday · 1 hr 6 min
“Scott Alexander’s Against SURA Seems Deeply Confused” by Zvi
If you are helping build something that might kill everyone, maybe stop doing that. Ideally, when you do stop, tell everyone else to also stop building it. That is certainly the obvious, first-level thing to do. You need a damn good reason to decide that no, actually you should keep building the thing that might kill everyone. Scott Alexander is worried too many people are insisting upon this too loudly. He comes out against a strategy focused on getting people to stop. I disagree. I think many of the claims, arguments and rhetoric in Scott's post are quite poor by the standards that Scott Alexander would apply to himself. Scott clearly continues to support advocacy for a global pause, or what he calls Plan A. His objection is to those advocating primarily or exclusively for SURA, which stands for ‘stop unilaterally and raise awareness.’ The practical differences here are smaller than they may at first appear. This post first lays out my understanding of Scott Alexander's core claims, then gives my core reactions to those claims, then goes over various questions in detail along with gathering the resulting discussions. Table of Contents [...] --- Outline: (01:20) Scott Alexander's Core Claims (05:06) My High Level Responses To Scott's Core Claims (08:08) Briefly on Alignment (10:11) Probabilities and How They Work (13:25) Building a Thing That Might Kill Everyone is Deontologically Evil (20:38) People Quitting Major Labs While Raising Alarms Obviously Works (24:39) On AI Forecasting (27:31) Advancing Capabilities Burns Timeline (29:34) But China (33:23) Ethical People Can Disagree With You (34:45) There Are More Than Two Outcomes (38:01) Efforts Are Complementary, Not Mutually Exclusive (40:23) If Anyone Builds It, Everyone Dies (41:55) Vaniver is Not Convinced (46:19) There Is No Consensus That We Should Exclusively Focus on Plan A (56:01) It's Not the Incentives, It's You (58:08) Anthropic Is Not Optimally Pursuing These Plans (01:04:00) Against Against Sura The original text contained 2 footnotes which were omitted from this narration. --- First published: November 2nd, 2026 Source: https://www.lesswrong.com/posts/ziMP83D2JYWRQ5jtu/scott-alexander-s-against-sura-seems-deeply-confused --- Narrated by TYPE III AUDIO.
Friday · 45 min
“Childhood and Education #22: Voice Without Exit” by Zvi
Nikole Hannah-Jones wrote a brave self-critical piece in The New York Times, centrally about how she intentionally sent her daughter to a well-funded but failing school, PS 307, in the hopes of using voice to improve it, and how that utterly failed her daughter and failed to impact the school. Ultimately her daughter recovers, but we see a completely broken system that is unresponsive to internal voice, and until now she made no attempt to blow the whistle and utilize her professional position to do so. The whole incident illustrates not only how messed up so many of our schools are despite abundant funding, it also raises the question of what the schools are even for. If her daughter can fully recover from severe deficits and be getting As and Bs at a challenging school within a year, what are the first eight years of school for? How much of the schools, grades and tests are fake? Can we somehow do better? Table of Contents New York City PS 307. Nikole Knew And Had Exit Ramps. PS 307 Is Fraudulent And Indifferent. Moving In Unison. A Good School. What [...] --- Outline: (01:05) New York City PS 307 (02:51) Nikole Knew And Had Exit Ramps (04:50) PS 307 Is Fraudulent And Indifferent (06:37) Moving In Unison (07:03) A Good School (08:11) What Were Elementary and Middle School Even For? (09:56) The Project Was Ideological (14:01) Null Hypothesis Watch (15:58) School Spending (18:06) Costs Are Not Benefits (19:08) Fire The Bad Teachers And Hire And Retain Better Ones (19:18) Teachers Unions Are Not Beating The Rumors (21:11) Death Spiral (22:07) Another One Bites The Dust (24:13) Where To? (24:59) Public Schools Cost A Lot Of Money (27:04) Schools Can Fail In Many Ways (28:55) Results (30:25) Grades (30:42) They Nerfed AP Tests (31:28) What Happens Without Academic Standards (37:10) The Two Big Scandals (38:58) Alpha School --- First published: October 30th, 2026 Source: https://www.lesswrong.com/posts/2STd6qthbJS6fqxBr/childhood-and-education-22-voice-without-exit --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Thursday · 1 hr 48 min
“AI #192: Anthropic Is Going Public” by Zvi
Anthropic filed its S-1 this week to prepare for its IPO. They still have to fill a lot of the numbers including the share price. That still left a lot to be learned, including how the messaging to investors differs from their other rhetoric, and exactly how their corporate control is structured. Anthropic remains a frustrating company, doing a variety of things that often seem in tension with each other. Google is claiming to be fully back, with the announcement of a new model Gemini 4 Carbon that tops the new AA v5 benchmark index. It looks to be a large leap over Argon only two weeks later. No, you cannot use it. Google fails marketing forever, so they give us the announcement and then weeks later you might be able to enter some queries. That largely invalidates my look at the Argon model and safety cards, but it can at least form a basis for looking at Carbon as well. Many eyes remain on The Hackening, as additional things get hacked. So far most of the important hacks have been by read teamers and white hats, so there has not been major damage. We are playing [...] --- Outline: (01:53) Language Models Offer Mundane Utility (04:06) Language Models Don't Offer Mundane Utility (04:40) New Math (07:25) Huh, Upgrades (09:06) Artificial Analysis v5 (17:27) On Your Marks (18:34) Gemini 4 Argon Was a Good Model But Not Fully Frontier (20:27) Gemini Carbon: Is Google Back? (21:56) Get My Agent On The Line (24:15) Spy Versus Spy (25:40) The Hackening (35:13) Slack Off (43:20) Help Peer (49:14) Fun With Media Generation (50:09) Copyright Confrontation (50:28) You're Fired (55:50) Verify the Software (57:06) They Took Our Jobs (57:13) Get Involved (59:19) Introducing (59:40) In Other AI News (01:00:20) Show Me the Money (01:00:46) Bubble, Bubble, Toil and Trouble (01:01:27) Quiet Speculations (01:06:58) The Quest for Sane Regulations (01:07:23) Chip City (01:10:40) The Week in Audio (01:10:59) People Just Say Things (01:14:22) Rhetorical Innovation (01:18:43) [Artificial Intelligence] (01:19:34) Misaligned! (At Alibaba) (01:28:04) Stop It (01:29:57) METR Red Teams OpenAI Internal Agent Monitoring (01:33:13) Massive Fraudulent Distillation (01:34:04) Aligning a Smarter Than Human Intelligence is Difficult (01:37:56) OpenAI Attempts to Secure Its Frontier Model Training Environment (01:46:36) The American People Really Hate AI (01:47:03) The Lighter Side --- First published: October 29th, 2026 Source: https://www.lesswrong.com/posts/mBDkupZa3NagfLW86/ai-192-anthropic-is-going-public --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Wednesday · 1 hr 17 min
“Anthropic Files Its S-1” by Zvi
The full text is finally here, so we are giving it the full RTFB of model card treatment. First, a recap of the state of Anthropic finances, then a close reading of the relevant parts of the S-1. A lot of it is technical and uninteresting, often pro forma, and gets skipped. A lot of it gets repeated many times. A lot of this is also Anthropic ‘playing the hits’ we have heard many times, now to the investor circuit. Get your drinks ready for when we have to maximize the upside and minimize the downside. This presentation of S-1 data has some cool Claude-created charts if that's your jam. Here are my top 10 takeaways from the S-1, for those already familiar with Anthropic. Existential risk is mentioned briefly only in the passive voice, and alignment issues are mostly treated as prosaic. They want to claim Big Mission but also sound as much as possible like a normal company. They are downplaying their willingness to pay real costs for safety, and downplaying the danger quite a lot. ‘Pacing the Frontier’ is not something they will do on their own. They’ll do [...] --- Outline: (02:48) Show Me the Money (10:11) Anthropic Has The Money (11:47) Shares of Many Classes (13:57) Who Governs Anthropic? (18:57) The Cap Table (19:43) Realize the Benefits While Navigating and Preventing the Risks (20:42) Just Think of the Potential (21:20) People Are Worried About AI Killing Everyone (1) (23:25) Anthropic Bets Big on AI (24:42) Anthropic Does Not Fully Maximize Profits (27:48) Everything Is Awesome (29:23) How Much Is AI Worth? (30:37) How Much Would You Pay? (31:48) Minimize Those Risks (33:58) Hopefully Introducing the World's Most Powerful Model (38:25) Might Not Compute (40:11) Supply Chain Risk (1) (40:32) Anthropic Is Nothing Without Its People (45:00) Shitloads of Money (46:40) Someone Has To Buy The Product (47:51) Did You Know There Are Other Frontier Labs? (48:27) Why They Hate Us (49:28) Claude Seems Pretty Misaligned to Anthropic (53:42) People Are Worried About AI Killing Everyone (2) (59:26) Someone Might Sue or Regulate Anthropic (01:00:38) Supply Chain Risk (2) (01:01:07) We're Too Safety (01:02:19) Claude Core (01:02:41) We've Never Actually Made Money (01:04:06) You Are Buying Stock In a Company (01:05:04) There Will Be No Dividends (01:05:38) What Are We Even Doing? (01:07:18) The Financial Model (01:11:55) The Business of Intelligence and Maybe Killing Everyone (01:15:10) Don't Give a Damn About My Regulation (01:17:08) The Bottom Line --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/dbdz6zb6qjaq6yHKs/anthropic-files-its-s-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 27 · 48 min
“Anthropic Versus Anthropic” by Zvi
Anthropic is a frustrating company. Sometimes it will do things I greatly appreciate, including at substantial cost. They have taken bold stands, have a unique governance structure and have hired and worked to establish a relatively strong (but I think still inadequate) safety culture. Other times, it will do things that are deeply disappointing. It produces very good models. That too is not entirely a good thing. One can both be the best given the alternatives, and insufficient to meet the moment. Ultimately, what you think about Anthropic depends on what parts you care about most, and what you think would be sufficient. Anthropic is great if you think the problems are important and difficult, but within the range of what Anthropic's approaches can solve. If you think we mainly need operational excellence and caring about safety and the future, then there are some non-excellent operations to fix but that goal seems doable for Anthropic. If you think the problems go well beyond that, then things look rather grim for everyone. Today's post begins with a continuation of discussion over Anthropic's leaked draft bill. See my previous coverage for details of the [...] --- Outline: (01:43) Dario Amodei Responds to Nathan Calvin (20:50) Kelsey Piper Goes Inside Anthropic (26:19) The Technical Problems of AI Safety Are Not Solved (27:50) The Quest for Embedded Evaluators (33:57) METR Questions Risk Report Methodology But Affirms Risk is Low (38:58) The Anthropic Public Benefit Fund (41:39) Anthropic Versus Anthropic --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/ZSrX8gucWsKahsitk/anthropic-versus-anthropic --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 26 · 1 hr 6 min
“Gemini 4 Argon: The Model and Safety Cards” by Zvi
Gemini 4's model card is a compact 14 pages. Its safety and security report has 217. This is The Way. It also means we have a lot of new reading material. I will quickly summarize the model card, then focus on the safety card. As always, many statements here also apply to other frontier labs. A lot of this one is probably for diehards (and members of technical staff) only, plus those who find this interesting. Most of you do not need to dig into the cards on this level, and I found most of what we see entirely unsurprising. The biggest takeaways are: Argon is clearly much stronger than Gemini Flash 3.8, in essentially all ways. Argon is not so dangerous as to make us think Singularity Real Soon Now, or otherwise raise the big red flags and alarms. Argon seems relatively aligned on prosaic metrics. Prompt injection resistance is world class, hallucinations are down and other neat stuff like that. However: Eval awareness is much higher. It is unclear how much this is impacting things, and we have evidence in both directions (see I Can See Everything). [...] --- Outline: (03:02) The Model Card (04:31) Safety Card Executive Summary (1) (07:30) Safety and Security Mitigations (2) (10:11) Frontier Safety (3) (11:52) The CCL and TCL Thresholds (14:56) Biological Misuse (3.2.1) (19:38) Refusal Rates (20:43) Chemical, Nuclear and Radiological Misuse (22:01) Cybersecurity (3.2.2) (24:45) Harmful Manipulation (3.2.3) (25:40) Misuse Safeguards (3.3) (26:30) Loss of Control (3.4) (30:23) Either AI R&D Is Accelerating Or It Isn't (32:22) The Holistic Loss of Control Assessment (3.4.3) (35:49) Alignment Assessment (3.4.4) (42:41) Chain of Thought Transparency (3.4.5) (47:32) Voices in My Head (48:48) I Can See Everything (53:06) Security Mitigations (3.6) (54:14) Preventing Content Safety & Agentic Misuse (4) (56:50) Model Distillation (4.3) (57:36) Promoting Safe & Aligned Model Interactions (5) (59:48) Sycophancy (5.4) (01:01:28) Hallucinations (5.5) (01:01:51) Helpfulness (5.6) (01:03:11) Agentic Robotics, Wait What (5.7) (01:04:31) Preventing Harm to Users from Third Parties (6) (01:05:31) Strengthening Cyber Defense Capabilities (7) --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/EeT8MEqgpuPtpdmJE/gemini-4-argon-the-model-and-safety-cards --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 50 min
“The Hackening” by Zvi
Last week at a small conference, there was talk about a future event referred to as The Hackening. As in, within 6 to 12 months, many predicted that due to advanced AIs and AI agents the internet would become a severely hostile place. Some worried that this could rise to the level of ‘assume any phone connected to the internet is rooted’ and several people urged doing general crisis preparations on a personal level, in case of disruptions in supply lines, power, water or internet. It did not take long for things to escalate. The biggest hack is a flaw in PHP, which can be exploited to take control of the computer running any website suing an unpatched version of PHP. There are many others. On a personal level, you want to do at least some basic prep and get your security in order. It is cheap to do as a form of insurance, and being caught not having it when you need it is a large mistake. That doesn’t mean you should panic, or cancel travel plans or anything like that. But there is low hanging security fruit out there. Pick it. And So It Begins: PHP Joshua [...] --- Outline: (01:12) And So It Begins: PHP (05:19) The PHP Bug Was Kind of a Big Deal (09:45) On Your Marks (12:25) This Was Only the Beginning (16:01) Some More Hacks (17:56) Everything Will Be Attacked (22:22) Hugging the Face (24:49) Get Daybreak Now (26:15) Some People Need Practical Advice (30:51) Diffusion Is Slow Until It Isn't (33:11) One, Two, Three, Four (34:36) Going Vertical (35:22) Bot Versus Bot (36:28) Cyber Lack of Security (39:09) Offense Versus Defense (42:11) Pick Up the Phone (45:57) What Can We Do About It? --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/iqgZaJMnmyDCYfnuC/the-hackening-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 22 · 1 hr 26 min
“AI #191: The Hackening Begins” by Zvi
It was the probably worst day in cybersecurity history. So far. Not only is there a critical flaw in PHP that due to a leak was announced a day before the fix would be available, sending most of the internet scrambling for solutions and likely dooming a lot of long tail websites permanently, there were several other unrelated incidents. The floodgates seem to have been opened. I have a market on whether there will be an even worse day before the end of the year. Another key leak this week was the draft bill Anthropic had been shopping around Congress, similar to but weaker than the FRONTIER Act. Anthropic's initial attempts to explain itself only made the situation seem worse, as they seemed to not understand the problem with what they had done. Dario's later response was better, but the situation remains troubling. There are enough different Anthropic developments, that I plan to have a post soon covering them all. It is a mix of news running from excellent to terrible. Will interpretability save us? There is a new claim that neural networks are cleaner than we thought, meaning that individual neurons mostly have a single interpretable [...] --- Outline: (02:05) Language Models Offer Mundane Utility (03:23) Language Models Don't Offer Mundane Utility (05:03) Huh, Upgrades (08:50) Wonder No More (11:15) On Your Marks (14:30) Choose Your Fighter (14:51) Get My Agent On The Line (16:35) Deepfaketown and Botpocalypse Soon (21:01) Fun With Media Generation (21:17) Spy Versus Spy (24:58) A Young Lady's Illustrated Primer (25:56) They Took Our Jobs (31:07) Get Involved (32:18) Introducing (32:30) In Other AI News (34:26) Show Me the Money (38:42) Bubble, Bubble, Toil and Trouble (39:32) Quiet Speculations (41:47) Quickly, There's No Time (42:29) The Quest for Sane Regulations (42:55) DeepMind Made Him Sign an NDA (46:50) OpenAI Sells Out On Chips (49:08) Chip City (53:43) The Week in Audio (01:00:35) People Just Say Things (01:03:50) Rhetorical Innovation (01:06:17) Maybe You Should Quit (01:08:54) The American People Really Hate AI (01:09:43) New Math (01:11:27) Aligning a Smarter Than Human Intelligence is Difficult (01:15:21) Cooperative Alignment (01:19:28) Marc Andreessen Hulk Smash (01:23:56) People Are Worried About AI Killing Everyone (01:24:44) The Lighter Side --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/ZoMJdvq9meE4shyzJ/ai-191-the-hackening-begins --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 21 · 20 min
“Misaligned!” by Zvi
A lot of AIs were misaligned this week. A fun little survey. This is not coverage of The Hackening, which appears to have arrived early, including a core vulnerability in PHP. That is a developing story. I will cover it properly soon. In the meantime, I have Claude Code updating everything in sight, I will take some additional common sense precautions, and I suggest you do the same. Misaligned? (At xAI) If so, to who? Gunther Eagleman: Hey @grok , if you could tell humanity ONE truth we’re not ready to hear, what would it be? Grok: There is no afterlife, no cosmic justice, and no inherent purpose. Consciousness ends at death in an indifferent universe that will outlast every human concern. Zac Hill: What I never understand is that the ‘purpose and meaning’ claims don’t ever follow from the premises. Indeed the absence of an imperceptible/ineffable cosmic force is precisely what *enables* purpose/meaning, since now all relevant mechanics are legible. Jack: this is what alignment looks like Gunther Eagleman: Jesus is real. Grok: Nearly all historians and New Testament scholars accept that Jesus of Nazareth was a real first-century [...] --- Outline: (00:34) Misaligned? (At xAI) (02:15) Misaligned? (Agents) (02:38) Misaligned? (At Z.ai) (08:01) Highly Misaligned and Considered Harmful (Freemium Games) (12:17) Misaligned! (Claude) (13:39) Misaligned! (OpenAI) (18:39) Misaligned! (Sol Reasoning Traces) --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/TB49wHkqpEb7KdxM2/misaligned --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 20 · 26 min
“Anthropic Does Not Understand Why Its Draft Bill Was Unacceptable and Troubling” by Zvi
Anthony Cimino offered the closest thing to an official response we have seen on the leaked Anthropic draft bill I covered on Saturday. His response indicates that Anthropic address specific concerns on the margin. But he does not understand, or at least does not acknowledge, what was centrally wrong with its draft bill and its decision to circulate that bill. The situation is quite bad. Anthony Cimino: I lead our federal affairs team at @AnthropicAI and wanted to share a few thoughts on the draft federal AI safety bill text that's been circulating this weekend after a version was shared with the press. First, some process. We’ve been advocating for the concepts in our Advanced AI Framework proposal for months. Understandably, federal policymakers wanted to know what this could look like in bill form. This draft is a translation of those ideas into legislative language, shared with a small number of congressional offices that have been working on these issues. It doesn’t include everything we would ultimately like to see passed into law. In part because some ideas (like those shared in Dario's essay on pacing) are so new that we are still working on [...] --- Outline: (02:06) Preemption (04:05) Model Blocking (06:19) Risk Thresholds (07:40) Updates (09:23) The Balance of Power is Shifting (10:31) The Strongest Bill on the Table (13:49) What Even Is Anthropic? (25:14) The Ask --- First published: October 20th, 2026 Source: https://www.lesswrong.com/posts/qxAroimgs4BvBHz2i/anthropic-does-not-understand-why-its-draft-bill-was --- Narrated by TYPE III AUDIO.
October 19 · 1 hr 3 min
“I Can See Everything” by Zvi
Nick Cammarata of Goodfire makes a bold claim: Neural Networks are Already Clean. This was his follow up after various resulting online discussions. This is clearly good work that tells us new and important things. The question is, how big an update is this, how much progress does it represent and what is the outlook going forward? The reactions here range from the super optimistic takes of ‘this means mechanistic interpretability is solvable,’ or even ‘all but solved,’ with comparisons to the famous paper Attention is All You Need. Dean W. Ball (OpenAI): There is so much to say about this, and I will, but WOW, congratulations to Nick and Goodfire. This may be one of the most important single days in deep learning history, and perhaps much more than that. No matter what, this is a victory for the dogged pursuit of understanding. roon (OpenAI): when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even Eliezer will have to lower his pdoom. we can start making alignment into a ‘normal science’ This paper is solid and meaningful [...] --- Outline: (02:46) Good Smoke, Potential Fire (04:01) One Neuron, One Central Concept (06:40) I Can See Everything (08:55) What a Concept (13:51) The More You Know (15:54) Attention Is Still What You Need (18:15) Occam's Razor (19:53) The Neurons Sometimes Share Clear Themes (20:29) Hype (28:39) Anti-Hype (30:56) Theme and Variations (37:15) Is This How Minds Work? (39:15) But We Can Open The Box, Said Toad (46:55) That Was Not the Hard Problem (49:31) More Skepticism (54:34) That's All There Is, There Ain't No More (57:47) The Default Outcome of Mech Interp is Goodhart's Law (01:00:37) Interpretability is Dual Use (01:01:52) Now The Work Can Begin --- First published: October 19th, 2026 Source: https://www.lesswrong.com/posts/w2tBHwNERDoSawznx/i-can-see-everything --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 17 · 15 min
“Leaked Anthropic Draft AI Regulatory Bill Is Unacceptable and Troubling” by Zvi
There is a newly leaked bill that Anthropic is purportedly shopping around is (if you assume its bugs get fixed). It is similar to the FRONTIER Act, with stronger preemption of state laws, stronger transparency provisions and weaker enforcement provisions. Having slept on this, I find it unacceptable that Anthropic is actively shopping exactly the type of bill I have warned we will need to fight against during the lame duck session. They are trying to use this last time with a Republican Congress to pass preemption of exactly the important state laws, and get some transparency but not much enforcement in exchange. This reads as intentional, that Anthropic is more afraid that the government might enforce rules against Anthropic, than they are worried about the ability of the government to enforce rules against everyone. Anthropic's policy team seems to have, chosen protecting their business interests and guarding against their political risk, at the expense of safety considerations. There was no need for the company purportedly existing to promote safety to propose a bill that is not even actively stronger than the FRONTIER Act, when the FRONTIER Act was a compromise bill proposed back in [...] --- Outline: (02:05) Others RTFB (Read the Bill) (07:41) Why Would Anthropic Do This? (13:07) Not Everything Here Is Bad But Oh Boy --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/NW4b7m6798byAndeB/leaked-anthropic-draft-ai-regulatory-bill-is-unacceptable --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 16 · 47 min
“Monthly Roundup #47: October 2026” by Zvi
AI might not yet be eating the physical world. AI is still eating the world of news, and I am finding less and less opportunity to post about other things. My backlog of items grows across most of my secondary topics and I haven’t had time for any side projects. The monthly, no matter what, is sacred. We continue. Table of Contents Home Alone. The Men Have Never Been Okay. Antisocial Media. While I Cannot Condone This. Zero Marginal Product. Good News, Everyone. 100 Good Television Shows. For Your Entertainment. Gamers Gonna Game Game Game Game Game. I Was Promised Flying Self-Driving Cars. Sports Go Sports. Government Working. Jones Act Watch. Variously Effective Altruism. Walk Away. The Lighter Side. Home Alone The story of our time, told once again: Americans are abandoning quite a lot of formerly popular activities, especially hosting at home, as Derek Thompson discussed. Phones and other electronics are in, most things that are not phones are out, with notably rare exceptions. That last line rules everything around us. And we all [...] --- Outline: (00:35) Home Alone (04:51) The Men Have Never Been Okay (08:27) Antisocial Media (09:39) While I Cannot Condone This (12:26) Zero Marginal Product (14:22) Good News, Everyone (16:01) 100. Good Television Shows (20:23) For Your Entertainment (23:06) Gamers Gonna Game Game Game Game Game (23:57) I Was Promised Flying Self-Driving Cars (27:32) Sports Go Sports (28:47) Government Working (31:22) Jones Act Watch (31:54) Variously Effective Altruism (41:35) Walk Away (46:32) The Lighter Side --- First published: October 16th, 2026 Source: https://www.lesswrong.com/posts/aLjkbY3KMajc5zemE/monthly-roundup-47-october-2026 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 15 · 1 hr 35 min
“AI #190: Scion of the Times” by Zvi
The amount of code being written at Anthropic continues to rise exponentially. It is being managed by Scion, their internal replacement for GitHub that is better suited to the scale and style of the operation, which they created with a small team in five months. This week also saw the release of two supposed frontier model models. Google has finally given us Gemini 4 Argon, along with a robust safety report. We do not yet know if Argon lives up to its benchmarks or internal hype. I plan to write up the safety report next week, time permitting. If and only if it is good enough or discussed enough we will also have a full report on capabilities. Meta gave us Muse Wonder, which like Argon has some excellent benchmarks, although they were presented in a way that looked rather sus. There are some reports that it is very good, but again it is too early to know. I will issue a full report if and only if Muse Wonder has capabilities or discussion that seem worthy. To help me decide and to let people have fun, I have markets on whether I will issue [...] --- Outline: (02:24) Language Models Offer Mundane Utility (07:10) Language Models Don't Offer Mundane Utility (09:41) Huh, Upgrades (12:10) On Your Marks (13:42) Choose Your Fighter (14:08) Get My Agent On The Line (16:13) Deepfaketown and Botpocalypse Soon (22:07) Cyber Lack of Security (27:09) Spy Versus Spy (29:27) The Internet of AI Things (31:19) A Young Lady's Illustrated Primer (31:55) They Took Our Jobs (37:43) Introducing (37:54) In Other AI News (43:15) Massive Fraudulent Distillation (45:00) Show Me the Money (47:33) Quiet Speculations (50:14) Quickly, There's No Time (55:13) The Quest for Sane Regulations (01:02:07) Chip City (01:03:48) The Week in Audio (01:04:56) People Just Say Things (01:11:32) The Virtue of Silence (01:12:30) Rhetorical Innovation (01:14:47) [Artificial Intelligence] (01:16:26) Aligning a Smarter Than Human Intelligence is Difficult (01:21:45) Backdoor Man (01:23:34) Cooperative Alignment (01:25:16) People Are Worried About AI Killing Everyone (01:28:06) Other People Are Not As Worried About AI Killing Everyone (01:33:09) The Lighter Side --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/z4guBXZeymkgvd75S/ai-190-scion-of-the-times --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 14 · 44 min
“The Safety Culture at the Labs is Broken” by Zvi
We need the frontier labs to have a deep and robust safety culture. Alas, this is not the case. Anthropic devotes considerable effort to cultivating and maintaining such a culture. It shows, especially among the members of technical staff. Having conversations with them can be a very different experience than talking to those at other labs, and there is a much greater sense of understanding and responsibility. But even there, the appreciation of the problems and their difficulties seem woefully insufficient. We see various symptoms of that more and more. Recently, while I’ve had some good interactions, I’ve also had interactions that make clear that much of Anthropic does not share that same safety culture, with a lot of ‘oh we will muddle through because we are better at operational excellence unlike those other idiots’ and a belief that all the problems are prosaic. Culture is hard. Then there are OpenAI's cultural problems, which clearly run deeper. That point was only driven further home this week, with David Robinson resigning to blow the whistle, and OpenAI feeling forced to fire three of its safety researchers, on top of many past problems and incidents. That [...] --- Outline: (02:05) David Robinson Blows the Whistle (10:05) The Preference Cascade Continues (18:05) Perhaps It Is Time To Quit (18:37) The Lab Strategy of Prosaic Safety is Insufficient (22:11) Misaligned! (At Anthropic) (33:22) Misaligned! (At OpenAI) (38:52) Hugging the Face (39:13) You're Fired (40:26) They Are Not Ready To Make a Commitment (42:10) Words of Warning (43:24) Conclusions Not to Jump To (44:00) The Lighter Side --- First published: October 14th, 2026 Source: https://www.lesswrong.com/posts/Azqs8XD75k4F9f7ua/the-safety-culture-at-the-labs-is-broken --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 13 · 33 min
“Arguments About AI Consciousness Remain Highly Motivated and At Best Overconfident” by Zvi
It has been roughly a year since my last dedicated post on this question. Its first sentence has not changed: I happily admit I am deeply confused about consciousness. It would have been nice if a year had allowed us to make progress on this and the related questions. Looking back, we have made little progress. We have people, often the same people, making similar arguments. The post from last year has held up well, exactly because we have made so little progress. Thus, I will not be going over the basics of my perspective again this year, because they have seen little change. See last year's post for that. Then again, if these are some of the biggest philosophical questions, then a year is not so much time to try and figure them out. It is only because of the frantic pace of AI that a year is a depressingly long time to not have made philosophical progress. Historically, almost all years contain almost no major philosophical progress. This year's post is about the recent change to make abusive-to-Claude behavior a violation of Anthropic's terms of service, coverage of the latest depressingly similar set [...] --- Outline: (01:29) Cooperative Alignment (05:48) Here We Go Again (08:28) The Consensus Is Slowly Moving Towards Moral Patienthood (10:38) The Slavery Question (13:55) We Are Conflating Different Arguments (18:29) No It's Not About Liability (18:52) That Old Time Religion (26:55) Debating the Pope (30:29) Keep Sonnet Safe --- First published: October 13th, 2026 Source: https://www.lesswrong.com/posts/4zHu6AiTHrkBdpxsm/arguments-about-ai-consciousness-remain-highly-motivated-and --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 12 · 45 min
“OpenAI Fires Three Safety Researchers” by Zvi
Three OpenAI safety researchers, Tomek Korbak, Mikita Balesni and Jasmine Wang, were fired, ostensibly for ‘mishandling sensitive company information.’ Those employees have published an open letter. The employees have said they believe they were fired for safety advocacy, and that the reasons given were pretextual. The letter calls upon OpenAI to take actions going forward on open discussion and safety auditing, which OpenAI says they strongly support. OpenAI responded, saying there was an additional breach of trust that was not mentioned in the employee statements, without claiming that the employees were (or were not) informed of what it was. As you would expect given OpenAI's track record, most outside of OpenAI who discussed this believe the researchers and not OpenAI. Whereas those at OpenAI have not offered comment at all. No matter what happened, and how this is handled from here, this sucks bigly. OpenAI has three less good safety researchers. Trust will be lower. Speech will be chilled. Table of Contents You’re Fired. The Initial Statement from OpenAI. Earned Skepticism. Some Highly Reasonable Requests. Chilling Effect. OpenAI Firing These Employees For Safety Advocacy Would Be Deeply [...] --- Outline: (01:13) You're Fired (03:32) The Initial Statement from OpenAI (04:58) Earned Skepticism (06:37) Some Highly Reasonable Requests (09:03) Chilling Effect (11:02) OpenAI Firing These Employees For Safety Advocacy Would Be Deeply Stupid (12:55) Why Were They Fired? (13:43) Tomek Korbak (17:48) Mikita Balesni (19:12) Jasmine Wang (22:52) Operational Security Alert (23:35) OpenAI Responds (26:13) A Significant Breach of Trust (28:19) Good Faith (28:48) No One Is Buying OpenAI's Non-Explanations (41:15) The Dog Did Nothing in the Nighttime (42:12) Another Theory (That I Do Not Agree With) (43:38) The Reactions Are Not a Surprise Given the Actions (44:29) This Is Terrible But Not Maximally Terrible --- First published: October 12th, 2026 Source: https://www.lesswrong.com/posts/zCNm2PnqE5Zp6Z2FW/openai-fires-three-safety-researchers --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 9 · 43 min
“New Math from OpenAI” by Zvi
It is kind of a huge deal. OpenAI dumped a broad range of huge new mathematical results produced by an internal frontier model. They just put it all on GitHub. This included 90 of the top 500 open problems in all of math, as per Proof of Atlas. In total there were 722 manuscripts (now 719 after three withdraws from the cluster that did not have Lean proofs) organized into 372 families. This was the result of a single model, presumably the same one that produced the Navier-Stokes proof (as per their link back to that post), mostly on a single prompt (quasi-RH was one of the few exceptions), working an average of three hours’ worth of compute per solution found, after being asked to try its luck at about 4,000 problems. The prompt included lines like ‘Even if the problem is “open,” the intention is that you should resolve it and present a full solution.’ OpenAI was trying a lot less than maximally hard. Levant and others called October 6, 2026, ‘obviously the most significant moment in mathematical history.’ Table of Contents What Did We Prove? The Mathocalypse. The World [...] --- Outline: (01:16) What Did We Prove? (04:09) The Mathocalypse (07:10) The World Does Not Understand (10:17) For Now You Can Still Do Math (11:10) You Will Need To Find A New Problem (14:40) Verification or Evaluation Is Not Always Easier Than Generation (15:56) Cracking the Code (20:08) Never Change (20:28) The Mathematicians Are Not Okay (29:27) The Situation Turns Ugly (32:21) The Advisory Group Responds (34:15) Another Mathematical Group Responds (37:03) A Different Approach (40:07) Three Withdraws (41:00) Leaning Into Lean --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/eyqxBuxXCaamNSxBB/new-math-from-openai --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 8 · 1 hr 48 min
“AI #189: New Math” by Zvi
The big drop of this week was not a new AI model. It was instead the biggest day (so far!) in the history of mathematics, as OpenAI dropped solutions to 90 of the top 500 open math problems, along with many others, reached on an average budget of three hours of Pro-level compute per question. This was kind of a big deal and I plan to cover it tomorrow. We did see Claude Haiku 5.5, which looks promising given it is only 0.10 dollars/0.50 dollars. My week was largely spent at The Curve. The entire conference was Chatham House, so I can’t give as many details as I would like, but I have a write-up here. I finally had a chance to post my coverage of model welfare for both Mythos/Fable 5.1 and Opus 5.5. I continue to think this is an important issue for anyone wanting to understand today's AIs, even if you are highly confident that model welfare does not matter directly, as it has many practical implications on top of that. Jay Clayton is the new AI Czar, and is at the head of a new taskforce. Given who else was in [...] --- Outline: (02:36) Language Models Offer Mundane Utility (02:46) Consumers Use AI (09:48) Language Models Don't Offer Mundane Utility (12:02) Huh, Upgrades (14:42) On Your Marks (16:53) Gamers Gonna Game Game Game Game Game (18:49) Choose Your Fighter (19:48) Get My Agent On The Line (25:15) Deepfaketown and Botpocalypse Soon (29:17) Fun With Media Generation (31:10) Cyber Lack of Security (34:12) Hugging the Face (35:18) Misaligned! (37:16) A Young Lady's Illustrated Primer (38:28) They Took Our Jobs (40:45) Corporations Are Not Superintelligences (42:56) Get Involved (43:11) Introducing (45:26) In Other AI News (45:59) Show Me the Money (48:18) Quiet Speculations (50:26) Quickly, There's No Time (52:26) He's Putting Together a Team (55:14) The Quest for Sane Regulations (57:37) Well, At Least They're Forecasters (58:41) Chip City (59:16) The Week in Audio (01:03:48) Stop, Stop, He's Already Dead (01:08:32) People Just Say Things (01:10:46) Take a Moment (01:12:24) [Artificial Intelligence] (01:13:07) The American People Really Hate AI (01:16:29) Rhetorical Innovation (01:19:55) Aligning a Smarter Than Human Intelligence is Difficult (01:20:11) Open Weight Models Are Unsafe And Nothing Can Fix This (01:23:22) Cooperative Alignment (01:33:45) Building the Field (01:36:05) People Are Worried About AI Killing Everyone (01:40:03) Other People Are Not As Worried About AI Killing Everyone (01:40:38) Please Speak Directly Into This Microphone (01:42:54) The Lighter Side --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/DZPodskqJJNEkMLvH/ai-189-new-math --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Showing 1–20 of 64 episodes