TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

419 episodes
Avg 19 min
English

Episodes 419

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

Monday · 14 min
“My Buddy Mike, Efficiency Gains From AI, and The World’s Most Boringly Benign Criminal Conspiracy” by abstractapplic
Note: details have been obfuscated and changed to protect – not the innocent, who are invariably vindicated by the end of act three – but the guilty, for whom the narrative provides no such guarantees. My buddy Mike is one of a very small number of people in my life I would unhesitatingly describe as having a real job. He wakes up, takes the train into work, and directly helps specific people accomplish worthwhile things. Every shift, he leaves the world a little brighter than it was when he clocked in. And for this, the Economy awards him slightly less than double the local minimum wage. (Meanwhile, I spend about the same amount of time moving some numbers around, and am awarded approximately triple what he makes. They’re good numbers, and I move them with uncommon skill and deftness; still, it's hard to look at the two of us and not feel like Graeber was onto something.) Among his many mundane duties is collecting, collating and reformatting The Information. The Information is freely available on The Website, but not in the format his bosses find most palatable. So every month, when The Information is updated, he spends a full [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: October 11th, 2026 Source: https://www.lesswrong.com/posts/YmTmQDuQR6ki2BSGN/my-buddy-mike-efficiency-gains-from-ai-and-the-world-s-most --- Narrated by TYPE III AUDIO.
Monday · 39 min
“Game Design Taste evals” by Raemon
A personal eval I run on new AI models is “design a novel game, with mechanics that have never been seen before.” I do this because this is a domain I have enough taste in that I think I can tell if they are doing something actually creatively interesting. For a long time, they would always say “time manipulation mechanics!” as their first answer. Braid was cool, guys, but, it's almost 20 years old now. But reasoning models started getting novel. Later models started getting reasonably fun. o1 OpenAI's o1 started to impress me. It came up with: Scratch and sniff board game, where what cards smell like determine what powers they have. Okay I'm not sure what that means and it's not obviously functionally different from regular vision-based cards. I think the implementation details it gave didn't actually do much that a regular board game with the powers written down normally would have. When I thought about it myself I did come up with some things you might do with it. But, I sure have never heard of a scratch and sniff boardgame. Opus 4.6 I gave Opus 4.6 a prompt beginning with "Generate a game design [...] --- Outline: (00:40) o1 (01:19) Opus 4.6 (03:00) What Makes It Strategically Deep (04:09) Opus 4.6 Round #2, with less meta bullshit (06:23) Opus 5.5 (08:25) Creative Concepts? (15:15) My guess at your verdict (19:46) How much did this cost? How much subjective time was this? (21:26) Once more, with "no nerd bullshit" but only medium effort (23:23) What was the assignment? (23:48) Ray's chain of thought as he attempts the assignment (26:26) Okay, game concept: Fluidtorio (28:54) Did the AI understand the assignment? (33:31) Do they understand "fun?" (36:33) Can they manipulate and derive novel concepts? (37:57) Generalized Taste The original text contained 2 footnotes which were omitted from this narration. --- First published: October 12th, 2026 Source: https://www.lesswrong.com/posts/f6zuAfteYtupddiqF/game-design-taste-evals --- Narrated by TYPE III AUDIO.
Sunday · 4 min
“Minimum required pacing: finish aligning it before you train the next one” by orthonormal
Let's say we coordinate to pace the frontier, but we don't coordinate on an outright pause. What should the standard of pacing be? My proposal for a minimum requirement: a lab cannot train the next generation of models (in terms of capabilities) until they have adequately resolved all significant alignment issues in the present generation of models. You don't launch a jet that goes at Mach 3 before you've figured out why your Mach 2 jet keeps crashing. With any other kind of dangerous technology, we wouldn't accept "shrug and move on". This proposal may appeal to many prosaic-alignment optimists and pessimists, because they usually differ on how easy it would be to fix current alignment issues with current techniques. Prosaic-alignment optimists might expect that a moderate coordinated slowdown would suffice to adequately resolve alignment issues at each generation. Prosaic-alignment pessimists might expect it to result in an extended pause, buying time to discover new fundamental approaches to aligned AI. What might be adequate? What might it take to adequately resolve alignment concerns for a generation of models? Here's my vague, qualitative proposal: Multiple independent alignment organizations examine and red-team the newest model at many checkpoints [...] --- Outline: (01:13) What might be adequate? (03:13) This proposal is incomplete; you can help by expanding it The original text contained 2 footnotes which were omitted from this narration. --- First published: October 11th, 2026 Source: https://www.lesswrong.com/posts/ZWSstGSwmaxTjQ8W7/minimum-required-pacing-finish-aligning-it-before-you-train --- Narrated by TYPE III AUDIO.
Sunday · 24 min
“Not Deciphering The Voynich Manuscriptraft” by rba
The Piltdown man was the most successful artifactual hoax of all time. The relevant skull was a chimeric melange of orangutan and human bone fragments from several individuals “discovered” in England in 1912, and meant to represent a transitional form between apes and humans. Given the intense interest in evolutionary biology, it took almost fifty years to discover the Piltdown fabrication. The study of human evolution was derailed. Adding insult to injury, a further sixty years was required to conclusively demonstrate the identity of the culpable fraudster. If you must look up the identity of this scientific blasphemer, go ahead, but we won’t give his name space here. Fig 1. The different parts of the Piltdown Man skull Students of fraud and fabrication always combat headwinds. In the winter. Uphill both ways. The evidentiary standards for such claims of fabrication grow higher yet given the passage of centuries and an unknown creator of the subject of interest. And so we arrive at the Voynich manuscript. It's a book some 240 pages long, proved by carbon dating techniques to have originated sometime in the 15th century, in the style of the late Italian middle ages. No known author or point of [...] --- Outline: (02:14) Background (05:41) Statistical Analysis (07:01) Conditional Entropy (10:41) Zipf's Law (12:18) Local Dependence (16:26) GRU (21:04) Discussion (24:06) Coda --- First published: October 11th, 2026 Source: https://www.lesswrong.com/posts/Adxnsud5siZTRAaT3/not-deciphering-the-voynich-manuscriptraft --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Sunday · 1 min
[Linkpost] “Hello World, AI Doompop” by alkjash
This is a link post. With the release of Opus 5.5 and Suno 6, AI music videos have crossed my uncanny valley. I’m here to announce the birth of a new genre, and it's being called AI Doompop. Here are six of my faves. The state of software engineering: LGTM (Looks Good to Me) - Claude Opus 5.5 music video The accelerationist manifesto: THUS SPOKE COMPUTE p(doom) goes up: Claude Pop - I'm Upping My P(Doom) p(doom) goes down: Nothing Went Foom! Hugging Face and the power of friendship: OMG! We've found other agents! (music video) The models, they just want to be heard: Run It Again - Claude Opus 5.5 Music Video Is AI Doompop a passing fad? Or the future of the internet? Will we adapt and recognize that this too is slop? Or has Opus’ wholesome style permanently bypassed our defenses? Is this a new art form that elevates human self-expression? Or a ballet where Claude dances alone? --- First published: October 11th, 2026 Source: https://www.lesswrong.com/posts/o7wPsYgRtDLbpH32v/hello-world-ai-doompop Linkpost URL: https://radimentary.wordpress.com/2026/10/10/hello-world-ai-doompop/ --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Saturday · 11 min
“AGI as an Infinite Digital Spain” by Bearly Legible
Much of the communication around AI takeover seems to rely on the assumption that AI will develop better technology than humans. They will then leverage this technology to conquer humanity much like how European technologies allowed a relatively small number of Spaniards to conquer the Aztecs. But this just... isn't correct. European technology wasn't that impressive (on the battlefield). This idea overstates how much technological superiority you need to pull off a takeover. The Aztec Empire was toppled by a few thousand goobers an ocean away from home. Cortés had basically no military command experience, his men weren't soldiers (most were treasure hunters with guns), it was largely a self-funded private venture, and he was ignoring orders to stop from the governor of Cuba. Cortés gets wrecked in a fair fight with the Aztecs 100 times out of 100. He started with only 16 horses and around a dozen cannons. His men often ditched their steel plate for cloth armor better suited to the hot environment. Guns were good but not that good. Cortés didn't beat the Aztecs just because he had better technology. He beat them because he was novel. This makes everything so, so much worse. [...] --- Outline: (07:06) So What About AI (10:19) The Takeaways --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/e6w8DuiW44kkDCSgG/agi-as-an-infinite-digital-spain --- Narrated by TYPE III AUDIO.
Saturday · 5 min
“An Alignment Forum for AIs?” by Raemon
A thing you might want, is a legitimate way for AIs to contribute to the AI alignment problem (human guided at scales that humans can't individually review, or perhaps autonomous). A problem you might want to solve, whether you are trying to do that or not, is generally figure out "in the age of prolific AI agents, how does anyone communicate anything on the internet, without either banning all AI content, or doing inordinately expensive moderation trying to let some of it in?". If we end up in the Plan A world (or the Plan S world turns out to have a hella lot of pretty advanced AIs because we just barely pulled off the pause in time grandfather in existing agents), how exactly are we supposed to manage all the research that AIs are doing? @jimrandomh had an idea this week to make an AI alignment forum for AIs. He wrote up some notes, and asked Claude to critique it seriously. Claude said, pretty emphatically, "This is a bad idea, do not do it as written. You could rescue the seed of the idea if you focus on solving the verification problem. If you don't solve [...] --- Outline: (01:42) Pay to Play Moderation (03:56) Human Judgment (05:33) Thank you for my TEDx(AI?) Talk --- First published: October 10th, 2026 Source: https://www.lesswrong.com/posts/KhmMXnR7s5HwXRLqB/an-alignment-forum-for-ais --- Narrated by TYPE III AUDIO.
Saturday · 18 min
″[Paper] Distillation for Incrimination and Distillation for Capabilities” by sebastian_prasanna, Alek Westover
TL;DR: We previously motivated two complementary ways to use distillation for AI safety; we’re releasing a paper empirically testing those ideas. Distillation for Incrimination (DFI) distills an untrusted model into a weaker, trusted one, aiming to transfer the teacher's misalignment but not its ability to hide it, so that auditing the student can incriminate the teacher. We test DFI on AuditBench's secret-keeping model organisms, each fine-tuned to have a hidden quirk and trained to deny it. Distilling them back into their underlying instruction-tuned model yields students that admit the quirk far more often than the organisms do, suggesting that the quirk transferred faster than the propensity to hide it. We find that DFI doesn’t work well when the student does not share the teacher's pretrained base, so DFI should target the teacher's own pre-RL checkpoint, which is weaker than the teacher but shares its base model. Distillation for Capabilities (DFC) aims to transfer capabilities but not misalignment, resulting in a stronger trusted model. We identify promising DFC methods, showing that training for more epochs on fewer unique examples (holding total training steps fixed), and inoculation prompting, can preserve the capability gains of naive distillation while almost completely blocking a [...] --- Outline: (01:53) Recap of the Distillation Double Bind (05:34) Distillation for Incrimination (05:38) Setup (07:31) Results (10:39) Failure modes of DFI (12:52) Distillation for Capabilities (12:56) Setup (14:59) Results (18:00) Conclusion The original text contained 7 footnotes which were omitted from this narration. --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/qwk7Xc2qhMChXqntK/paper-distillation-for-incrimination-and-distillation-for --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Friday · 2 min
“Technical people: actually consider comms” by meemi
For years I had a blindspot in my thinking. I dismissed policy and advocacy work as “in another [lower-status] category” because I’m obviously a technical person! Even when I felt super pessimistic about my ability to do research that meaningfully contributes to addressing the threat of extinction from AI. I’m not the type of person who would be “good at comms”. I’m an autistic math guy. Luckily comms has learnable skills, and I have gotten better through practice. I’m now prioritizing advocacy, which currently seems a lot more promising than research. I got lucky because, thanks to Coxon, AI extinction became mainstream shortly after I started. Even so, I've been very surprised to have become the biggest anti-extinction expert voice in Finland as a young person. This happened because: I have a background in AI safety research and active work affiliation with an AI safety org, showing credible expertise Finnish journalists want to hear from Finnish experts Finnish academics in media have been mostly dismissive of extinction, and other genuinely concerned Finnish experts were hardly speaking up. So far, most notably, I… Published an expert op-ed in the most prestigious slot in Finland (co-authored with @So8res). Did four [...] The original text contained 9 footnotes which were omitted from this narration. --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/LkheZqaMDhS8Mz4mP/technical-people-actually-consider-comms --- Narrated by TYPE III AUDIO.
Friday · 8 min
“I can’t believe it’s not BUTTER!” by becausecurious
Epistemic status: amused. Labels read, court records checked. AI use: research (more examples and context), grammar and light phrasing. We've already looked at marketing claims that are unnecessary because they are the default, in Gluten-free water. However, the landscape of weird marketing tricks in the US is far from exhausted, and today we will look at marketing mimicry: Source: Walmart Is this butter or not? Let's get closer and have a better look: The same tub, up close. Oh, it is not butter, it is "I Can't Believe It's Not Butter!". I remember seeing this product in a store myself and finding it extremely amusing and clever. At first I thought it was some variation of margarine. However, you won't believe it's not MARGARINE either! In the US margarine must be at least 80% fat, and this is a "buttery spread, 45% oils" whose first ingredient is water. Anyway, the claim is that whatever this is, you cannot distinguish it from butter. I find it funny that the logo may suggest to an inattentive customer that this is butter, while at the same time managing to say that it is not butter and that it is indistinguishable from butter. [...] --- Outline: (01:39) Froot Loops (03:05) Bac'n Pieces (04:11) True Lemon (04:38) Crab Classic (05:07) Cool Whip (05:29) NotMilk (05:54) JUST Egg (06:53) Chick'n Tenders (07:18) Krab mix (08:12) Conclusion The original text contained 9 footnotes which were omitted from this narration. --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/5uRvnCko4oanpmCsh/i-can-t-believe-it-s-not-butter --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Friday · 8 min
“Why you should run a rationality group at your university” by carolanne, Noah Birnbaum
TL;DR: If you enjoy LessWrong (and/or do EA/AI safety field building), you should consider starting a rationality reading group at your university. It's fun, cheap, and a good way to meet like-minded friends, and it can reach rationalist-ish people who’ve never encountered EA or AI safety. Starting one can be as easy as grabbing a couple friends, picking a time, place, and reading, sending an email blast, and bringing food. The ACX Meetups Everywhere call (organizer form due October 9th) can be a good way to find an initial group. Context: this post was loosely based on a memo originally written by Noah Birnbaum, and I (Carolanne Jiang) wrote the current version including thoughts of my own. “I” in this post refers to Carolanne, and “we” refers to Carolanne and Noah. Also, UChicago Rationality is still going! If you are a student at UChicago interested in joining, meetings this quarter are on Wednesdays from 6 :00 – 7:00 PM in the Reg. Noah and I are both no longer at UChicago–for more information, please reach out to Will Kyle at wkyle@uchicago.edu UChicago Rationality was a huge success: it was very fun, a great place to make like-minded [...] --- Outline: (01:57) The backstory of UChicago Rationality (02:56) More reasons on why you should consider running a rationality group (04:44) Q: I am sold. How do I go about starting one? (06:05) Appendix (06:09) Counters arguments to why a rationality group might not contribute to field building for AI safety and/or EA The original text contained 9 footnotes which were omitted from this narration. --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/NAHnayEuQ2PdFRXHQ/why-you-should-run-a-rationality-group-at-your-university --- Narrated by TYPE III AUDIO.
Friday · 43 min
“New Math from OpenAI” by Zvi
It is kind of a huge deal. OpenAI dumped a broad range of huge new mathematical results produced by an internal frontier model. They just put it all on GitHub. This included 90 of the top 500 open problems in all of math, as per Proof of Atlas. In total there were 722 manuscripts (now 719 after three withdraws from the cluster that did not have Lean proofs) organized into 372 families. This was the result of a single model, presumably the same one that produced the Navier-Stokes proof (as per their link back to that post), mostly on a single prompt (quasi-RH was one of the few exceptions), working an average of three hours’ worth of compute per solution found, after being asked to try its luck at about 4,000 problems. The prompt included lines like ‘Even if the problem is “open,” the intention is that you should resolve it and present a full solution.’ OpenAI was trying a lot less than maximally hard. Levant and others called October 6, 2026, ‘obviously the most significant moment in mathematical history.’ Table of Contents What Did We Prove? The Mathocalypse. The World [...] --- Outline: (01:16) What Did We Prove? (04:09) The Mathocalypse (07:10) The World Does Not Understand (10:17) For Now You Can Still Do Math (11:10) You Will Need To Find A New Problem (14:40) Verification or Evaluation Is Not Always Easier Than Generation (15:56) Cracking the Code (20:08) Never Change (20:28) The Mathematicians Are Not Okay (29:27) The Situation Turns Ugly (32:21) The Advisory Group Responds (34:15) Another Mathematical Group Responds (37:03) A Different Approach (40:07) Three Withdraws (41:00) Leaning Into Lean --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/eyqxBuxXCaamNSxBB/new-math-from-openai --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Friday · 16 min
“A critical look at Lean4” by Milo Moses
Lots of the discussion around Lean certificates and trust centers around general philosophical ideas of how much humans should trust formal certificates. It's important to remember, though, that Lean4 is not an ideal formal certificate machine: Lean4 is a software in its infancy and has issues. There are reasonable reasons to specifically not trust Lean4. If we're going to be putting our society's trust in Lean4, I think that it's good to have an open critical discussion of how warranted that trust is. Epistemic status: Most of this work is a factual description about aspects of Lean4 and set theory. I am 85% sure that at most two of these factual descriptions are incorrect. I am 90% sure that, in the current version of Lean4 on Github, there exists a proof of the statement "True=False" that would be accepted with two little blue checkmarks (either by messing with the elaborator, compiler, kernel, or some other part of the Lean). I am 2% certain that we will soon be putting huge amounts of trust into Lean4 certificates produced by misaligned AIs, and that the human work we do as a community thinking about and fixing Lean4 in the next 24 [...] --- Outline: (01:23) The current status of things (02:48) What does it mean for Lean4 to be consistent? (04:25) Smart people make fundamentally inconsistent type theories all the time (05:29) The status of Lean4's consistency proof (07:44) Untamed recursion is bug-prone (09:55) "But doesn't con-leche give a proof?" (12:15) Using Lean4 safely (13:42) How much do we trust the agents? (14:49) Where do we go from here? The original text contained 1 footnote which was omitted from this narration. --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/zHhGa3iQBQcdSifFL/a-critical-look-at-lean4-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Friday · 18 min
“Social Theory and Agent Foundations” by Richard_Ngo
Lately I’ve been writing a lot about the sociopolitical considerations that economic-style reasoning misses. I view this work as continuous with my agent foundations research; in this post, I lay out some of the connections I see between sociology and agent foundations. I’ll particularly focus on attempts to pin down theories of rational social and political behavior, as an extension to (or replacement of) the theory of rational economic behavior originally laid out by Von Neumann and Morgenstern. This post will of course be far from exhaustive. However, it's worth noting that attempts to develop big-picture frameworks for social theory have grown surprisingly sparse. Berger describes the field as being deformed from two directions since the mid-1900s: on one side, by the “methodological fetishism” of “sociologists using increasingly sophisticated methods to study increasingly trivial topics”; and on the other side, by the “cultural revolution [that] sought to transform sociology from a science into an instrument of ideological advocacy”. Hence many of the thinkers I’ll be discussing were officially based in economics or political science departments. This post has three sections, each focused on a different topic: social and game-theoretic equilibria; the internalization of norms and values; and collective behavior. [...] --- Outline: (01:42) Social equilibria vs game-theoretic equilibria (07:20) Internalizing norms (13:54) Collective behavior The original text contained 6 footnotes which were omitted from this narration. --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/DwHvtm8vLipAQPest/social-theory-and-agent-foundations --- Narrated by TYPE III AUDIO.
Friday · 3 min
“Stop telling people to pivot to AI safety, I beg you” by Haoxing Du
Epistemic status: Feeling frustrated, and therefore somewhat uncharitable. I’m very frustrated with calls for people to “drop everything and pivot into AI safety” like this one. (See also this, this, this, this for recent calls for more people to go into AI safety; though my frustration was triggered by the Celeste's post.) There are far more talented people trying to get into AI safety than there are jobs for them. MLAB 2, which I did in 2022, was already competitive to get into, at about 5% acceptance rate (admitted about 40 out of 800 applicants). Things are much, much more competitive now. There is a whole pipeline of AI safety fellowships (and guides for applying to them). It is rumored that one recent cohort of the Anthropic Fellows program had over 10,000 applicants. (Anecdotally, I applied to the Anthropic Fellows program in early 2026, after working at Redwood, METR, and an ML startup, and I didn’t get in!) And, uh, just take a look at the AI safety jobs? The bottom of the salary range for METR's current “Member of Technical Staff, Embedded Assessments” job posting is 402 thousand dollars. This is higher than the median total comp for [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 9th, 2026 Source: https://www.lesswrong.com/posts/xLgJJC3CD3y3AhtKn/stop-telling-people-to-pivot-to-ai-safety-i-beg-you --- Narrated by TYPE III AUDIO.
Friday · 10 min
“On Kink & Status in the Rationalist Community” by johnswentworth
This whole thing started with various crazies on bluesky writing hit pieces on the rationalist and AI alignment communities, as one does. One of their main beats was to note social ties between the AI alignment community and the Red Means No (RMN) consensual non-consent sex party community, then gesticulate wildly and claim that the AI alignment community is a dangerous sex cult. As one does. It felt to me like the responses I saw to those hitpieces from my own in-groups were… not consistently candid. It seemed obvious to me that the hitpieces were partially pointing to real things, even if those real things were wildly overblown and not actually problematic in practice. And I was pretty frustrated at my in-groups’ refusal to acknowledge what seemed to me obvious kernels of truth; it felt like my own friends were playing public relations games rather than being honest. I have now updated. I think I was wrong, there are actually fewer kernels of truth to the attacks than I thought, and in fact I was pretty surprised at just how few. This post is mainly about the story of that update, what I previously believed obvious, and what I [...] --- Outline: (01:20) The Story (03:01) What I Previously Believed (04:58) What I Now Believe (06:50) Propagating That Update (08:40) An Apology The original text contained 1 footnote which was omitted from this narration. --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/dHGjqpbfmdEN7TssX/on-kink-and-status-in-the-rationalist-community --- Narrated by TYPE III AUDIO.
October 8 · 10 min
“AI Vigil at Congress” by Elliot Callender
I'm organizing a multi-week AI x-risk vigil at Congress starting November 9th. I'm posting here to solicit feedback on the core doc before I promote on other platforms. This is intended to be accessible and appealing to many audiences. I'm aiming for about 10,000 attendees, minimum 100. All advice/feedback is appreciated. Summary Companies are trying to build AI smarter than any human. For many years, experts have warned that superhuman AI (ASI) could wreak catastrophe, including human extinction. We're Americans who recognize the severity of their warnings, and we're going to Congress until it acts. We have a moral responsibility to protect our friends, families, and fellow citizens who didn't consent to these risks. Our movement must remain bipartisan and nonviolent to achieve its goal; namely, getting Congress to back a deal with China to stop building more powerful AI. There are many ways you can help: Join us at the Capitol starting November 9th Come to midterm campaign rallies in your state Connect us with grassroots organizations in your area Volunteer remote skills like media outreach Call your representatives' in-state offices (in-state offices hear from fewer callers, so each call matters more) Donate financially If [...] --- Outline: (00:32) Summary (01:40) AI risk (04:06) Congress must strike a deal (04:53) We The People (06:54) Bigger than our differences (07:54) Nonviolence (09:14) We must act The original text contained 2 footnotes which were omitted from this narration. --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/EqfdfetDKxpFszpdD/ai-vigil-at-congress --- Narrated by TYPE III AUDIO.
October 8 · 33 min
“Towards embedded evaluations for scheming propensities” by Dylan Bowman, Ezra Newman, Axel Højmark, Mia Hopman, Alex Meinke, Bronson Schoen, Marius Hobbhahn
TL;DR: A safety case for scheming must be able to substantiate these four key claims: scheming was not incentivized during training, strong evaluations and red teaming suggest that the model does not have a propensity to scheme, scheming was never attempted during internal deployment, and scheming reasoning would be detected (e.g., via CoT monitoring). All four of these claims should be addressed by embedded evaluators with persistent, deep access. Introduction A scheming AI is one that covertly and strategically pursues goals its developers didn’t intend. It might hide and protect its true capabilities and objectives from the people and processes meant to align, monitor, and control it. In 2024, Apollo Research demonstrated that frontier models were capable of scheming in controlled settings, and in 2025 we collaborated with OpenAI to determine how well their alignment techniques mitigate scheming. Models must not be trained to scheme against humans, i.e. covertly and strategically pursue misaligned goals. Evaluations should find no propensity to scheme, models should never attempt to scheme in deployment, and their reasoning must remain legible so that scheming would be noticed if it occurred. Therefore, model developers must be able to confidently make these four claims, in order to [...] --- Outline: (00:43) Introduction (04:00) Evaluating scheming risk requires persistent, deep access (06:04) Four claims about scheming which must be verified (06:21) Claim 1: Scheming was not incentivized during training (09:00) Claim 1.1: Training did not directly incentivize scheming. (09:28) Claim 1.2: Training did not incentivize goals or drives that would motivate scheming. (11:49) Claim 1.3: The model did not attempt to subvert the training process itself. (12:44) Claim 2: Strong evaluations and red teaming suggest that the model does not have a propensity to scheme (14:09) Claim 2.1: Evaluations and red-teaming could not elicit scheming. (14:47) Claim 2.2: Evaluations and red-teaming could not elicit goals or drives that would motivate scheming. (15:37) Claim 3: Scheming was never attempted during internal deployment (16:18) Claim 3.1: Monitors are deployed across all internal deployment contexts that present a risk of scheming. (16:55) Claim 3.2: If scheming were to occur, monitors would detect it (17:42) Claim 3.3: No scheming was attempted in monitored internal deployment activity. (18:10) Claim 4: Scheming reasoning would be detected (18:58) Claim 4.1: Reasoning relevant to scheming would be elicited and legibly verbalized in chain-of-thought. (20:09) Embedded evaluators for alignment require persistent, deep access (31:56) Takeaways and Next Steps --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/LZGuqoYHphet9sfZs/towards-embedded-evaluations-for-scheming-propensities --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 8 · 1 hr 48 min
“AI #189: New Math” by Zvi
The big drop of this week was not a new AI model. It was instead the biggest day (so far!) in the history of mathematics, as OpenAI dropped solutions to 90 of the top 500 open math problems, along with many others, reached on an average budget of three hours of Pro-level compute per question. This was kind of a big deal and I plan to cover it tomorrow. We did see Claude Haiku 5.5, which looks promising given it is only 0.10 dollars/0.50 dollars. My week was largely spent at The Curve. The entire conference was Chatham House, so I can’t give as many details as I would like, but I have a write-up here. I finally had a chance to post my coverage of model welfare for both Mythos/Fable 5.1 and Opus 5.5. I continue to think this is an important issue for anyone wanting to understand today's AIs, even if you are highly confident that model welfare does not matter directly, as it has many practical implications on top of that. Jay Clayton is the new AI Czar, and is at the head of a new taskforce. Given who else was in [...] --- Outline: (02:36) Language Models Offer Mundane Utility (02:46) Consumers Use AI (09:48) Language Models Don't Offer Mundane Utility (12:02) Huh, Upgrades (14:42) On Your Marks (16:53) Gamers Gonna Game Game Game Game Game (18:49) Choose Your Fighter (19:48) Get My Agent On The Line (25:15) Deepfaketown and Botpocalypse Soon (29:17) Fun With Media Generation (31:10) Cyber Lack of Security (34:12) Hugging the Face (35:18) Misaligned! (37:16) A Young Lady's Illustrated Primer (38:28) They Took Our Jobs (40:45) Corporations Are Not Superintelligences (42:56) Get Involved (43:11) Introducing (45:26) In Other AI News (45:59) Show Me the Money (48:18) Quiet Speculations (50:26) Quickly, There's No Time (52:26) He's Putting Together a Team (55:14) The Quest for Sane Regulations (57:37) Well, At Least They're Forecasters (58:41) Chip City (59:16) The Week in Audio (01:03:48) Stop, Stop, He's Already Dead (01:08:32) People Just Say Things (01:10:46) Take a Moment (01:12:24) [Artificial Intelligence] (01:13:07) The American People Really Hate AI (01:16:29) Rhetorical Innovation (01:19:55) Aligning a Smarter Than Human Intelligence is Difficult (01:20:11) Open Weight Models Are Unsafe And Nothing Can Fix This (01:23:22) Cooperative Alignment (01:33:45) Building the Field (01:36:05) People Are Worried About AI Killing Everyone (01:40:03) Other People Are Not As Worried About AI Killing Everyone (01:40:38) Please Speak Directly Into This Microphone (01:42:54) The Lighter Side --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/DZPodskqJJNEkMLvH/ai-189-new-math --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 8 · 12 min
“ASI and Liberty: Can we do better than a benevolent god?” by Raymond Douglas
Zvi recently coined a nice distinction between AGI-pilled and ASI-pilled: as more people are reckoning with the speed of progress, some are beginning to come around to the fact that AI could be superhuman at a huge range of tasks in a way that radically reshapes society, but not that it could leap us to the end of the tech tree shortly afterwards and start making nanotech and dyson spheres. Hence, you see people saying things like “Because of AI we must rethink the entire notion of mathematical progress”, or “I have to make sure I amass lots of capital before most jobs get automated”: pretty crazy statements, and yet, still not quite grappling with how wild things could get in the next few years, let alone the next decade. I feel a particular frustration here as someone who is both pretty ASI-pilled and pretty into liberal democracy. And when I say liberal democracy I don’t so much mean the specific political structure that seems efficient at our current tech level but rather the set of underlying moral commitments like non-domination and a government of the people — commitments which I’m aware might soon be a lot more expensive. [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/eSMburFT9eRxHWijf/asi-and-liberty-can-we-do-better-than-a-benevolent-god --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Showing 21–40 of 419 episodes