Skip to content
Sign in
Artwork for LessWrong (30+ Karma)
TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play
Open in
  • 542 episodes
  • Avg 20 min
  • English

Episodes 542

NewestOldestLongest
All20Unplayed20In progress0Played0
Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.
  • October 19 · 3 min

    “Sorry, you are still dead” by becausecurious

    Epistemic status: A brief exploration based on court documents and contemporary reporting. My interpretation is tentative. AI use: context research, grammar. I don't remember how I stumbled upon the topic of legally dead people who are still alive. Basically, they are alive, but according to the government, they are dead. I was curious to examine documented examples and discovered Donald Miller's case. In 1986, Donald Miller disappears from Arcadia, Ohio, leaving two daughters from a previous marriage without his child support. Before disappearing, he lost his job and was struggling with alcoholism. In 1994, his ex-wife obtains a court declaration of his death so their daughters can receive Social Security survivor benefits. In the 2000s—news reports say 2005; his affidavit suggests 2008—Donald returns to Ohio, alive. In 2013, he asks the court to undo the declaration and restore his Social Security number. Without it, he says he cannot work or obtain a driver's license. His ex-wife objects, fearing she would have to repay the benefits. However, the judge interprets Ohio Revised Code §2121.08(A) as allowing reversal only: within a three-year period from the date of the decree The clock starts when the [...] The original text contained 8 footnotes which were omitted from this narration. --- First published: October 19th, 2026 Source: https://www.lesswrong.com/posts/FjSCpu2B36SAcZL8v/sorry-you-are-still-dead --- Narrated by TYPE III AUDIO.

  • October 19 · 1 min

    “The Lightcone Needs You To Plan a Vacation” by Elizabeth

    In an ongoing series that normally takes place in a few fixed locations, the Vacation Episode is a one-off episode where the characters leave their familiar grounds for a well-deserved rest. -TvTropes, "Vacation Episode" At a recent conference, @habryka praised the power of retreats at exotic locations. People are more relaxed, more creative, and out of their normal ruts. He went on to say he'd pay 3.5x the cost of a vacation that someone else planned and had a group drawn from the same pool of people as the conference . I asked around at the conference and no one is willing to overpay as much as Oliver, but they are interested. This strikes me as a great opportunity to make money while reducing x-risk. Plan a vacation, invite people from a whitelist to attend, charge a multiple of cost. I'm toying with doing it myself, but seems like it'd be better done by someone who actually loves travel. If that's you, please DM me, maybe we can collaborate. --- First published: October 18th, 2026 Source: https://www.lesswrong.com/posts/97HTJrxrqJqEYt6am/the-lightcone-needs-you-to-plan-a-vacation --- Narrated by TYPE III AUDIO.

  • October 18 · 8 min

    “Quick Thoughts on Overhang Arguments” by Aaron_Scher

    I have two goals for this post: Pose the question: Have overhang arguments been used successfully in other fields? Lay out some of the reasons why I am suspicious of overhang arguments. I've had the second of these on my mind for a few years now and am posting something rough rather than leaving it in drafts. Intro Within AI safety, people sometimes use overhang arguments to justify doing things that appear suspicious. The core of an overhang argument, as I see it is: We are worried about some particular (dangerous) progress happening very quickly in the future. We can push forward at least one of the inputs to that progress today. We expect doing so will both bring the dangerous thing sooner and slow down future progress toward the dangerous thing. We're trying to trade time now for time later. A common form of this is a "compute overhang" argument: let's avoid situations where there is tons of compute lying around and drastically more efficient algorithms get developed, as that would allow these algorithms to scale rapidly to ASI. Therefore, we should push forward algorithm efficiency as much as we can [...] --- Outline: (00:33) Intro (01:40) Have overhang arguments been used successfully in other fields? (04:56) Why I consider overhang arguments suspect (07:26) List of relevant links The original text contained 1 footnote which was omitted from this narration. --- First published: October 18th, 2026 Source: https://www.lesswrong.com/posts/aJSdGMzgFRFBcBhGp/quick-thoughts-on-overhang-arguments --- Narrated by TYPE III AUDIO.

  • October 18 · 32 min

    “The Alignment Problem: Computer Science Meets Analytic Philosophy” by Vanessa Kosoy

    TLDR: I explain what the technical AI alignment problem is and what its core difficulties are, in my view. This is a perspective that draws simultaneously on classical traditions in theoretical computer science / machine learning theory, ideas from analytic philosophy, and views that originated in the Machine Intelligence Research Institute / LessWrong. Minimal Viable Safety The Task The problem of making AI safe is underspecified if we haven't decided what the AI is supposed to do. For example, we can run any AI inside a homomorphic encryption box and never decrypt the output. This is probably perfectly safe, but also perfectly useless. We can decide on an arbitrary task such as "construct Lean proofs for given mathematical theorems" and declare victory once we have safe AI that does it with superhuman skill. However, that wouldn't address the danger. First, there would be incentives to use AI for tasks to which our solution doesn't apply. Second, even if we had precise safety guidelines for any task that anyone cares about, there would always be actors that ignore the guidelines, and we probably need Artificial Superintelligence (ASI) to reliably stop them. If anyone, anywhere, creates [...] --- Outline: (00:33) Minimal Viable Safety (00:37) The Task (02:53) The Problem (03:50) The Methodology (05:47) The Challenge (07:20) Alignment is Holistic (10:58) Hypercomplexity (13:54) Breakdown of Learning (16:33) Alignment Grounding (20:32) Alignment Convergence (24:17) Side-Channel Resilience (26:44) Inner and Outer Alignment (30:55) Conclusion The original text contained 18 footnotes which were omitted from this narration. --- First published: October 18th, 2026 Source: https://www.lesswrong.com/posts/Emp7awtxsYn8MusNq/the-alignment-problem-computer-science-meets-analytic --- Narrated by TYPE III AUDIO.

  • October 18 · 11 min

    ″“Holy shit” graphs and how to get more of them” by June Jimenez

    We are likely in a situation where the most powerful models in the world are internal models, and, here's a key distinction, potentially even models that are never intended to be released to the public (i.e. a model designed to provide internal uplift on capabilities during training but that is not intended to be / not particularly suitable for commercial purposes.) Internal-external information and capability gaps can distort both public and regulatory perception of frontier capabilities and dangers. If the internal frontier is three or six months ahead, such a gap is worrisome if the labs are close to automating various bottlenecks to increased capabilities development, like automating AI R&D. Some ways of mitigating these visibility gaps and communicating the existing data we do have are implementable over the short term and confer minimal cost for many actors relative to the advantages of visibility for policymakers. What is a "holy shit" graph? Many of my intuitions about the coming impacts of AI are shaped by "Holy shit" graphs. The METR time horizon is one such graph, but has been discussed to death, so let's talk about another three: Internal spending by OpenAI researchers (at [...] --- Outline: (01:02) What is a "holy shit" graph? (01:18) Internal spending by OpenAI researchers (at API prices) (02:25) General AI (as opposed to narrow AI) model revenues (03:50) Extent of AI R&D automation (08:43) Labor, relational and political (09:17) This could be you! The original text contained 11 footnotes which were omitted from this narration. --- First published: October 18th, 2026 Source: https://www.lesswrong.com/posts/GK9p6NHEk8bkZQxH4/holy-shit-graphs-and-how-to-get-more-of-them-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 18 · 30 min

    “AI by Default” by Jonny Spicer

    Epistemic status: a good-faith guess at how the world might turn out, after ~30 hours of effort. This post is an exercise in describing what I think will happen with AI progress "by default", i.e. by attempting to extrapolate various trends and considering their implications, before the next US presidential administration. As a TL;DR, I expect the following: We get many more warning shots. AI is a top-five issue in the election. The best models are withheld for internal use only, on safety grounds. AI-related job displacement is significant enough to show up in the data. Coding and ML R&D are fully automated or require only minimal human oversight. This is intended to be one representative story of how things might go, in the vein of What 2026 Looks Like and the work of the AI Futures Project. Incidentally, the current AI Futures Model predicts the median date that we get ASI as December 2028 — one month after the election. The rationale for doing so is largely the same as in What 2026 Looks Like. My goal is to tell a plausible story about what the state of the world might be when we are likely on [...] --- Outline: (03:31) Q4 2026 (06:11) Q1 2027 (08:51) Q2 2027 (11:51) Q3 2027 (15:36) Q4 2027 (18:40) Q1 2028 (21:33) Q2 2028 (24:32) Q3 2028 (27:10) Q4 2028 (29:35) January 1st 2029 The original text contained 5 footnotes which were omitted from this narration. --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/iQMzXuNJ3M6EEHZyv/ai-by-default --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 17 · 47 min

    “Monthly Roundup #47: October 2026” by Zvi

    AI might not yet be eating the physical world. AI is still eating the world of news, and I am finding less and less opportunity to post about other things. My backlog of items grows across most of my secondary topics and I haven’t had time for any side projects. The monthly, no matter what, is sacred. We continue. Table of Contents Home Alone. The Men Have Never Been Okay. Antisocial Media. While I Cannot Condone This. Zero Marginal Product. Good News, Everyone. 100 Good Television Shows. For Your Entertainment. Gamers Gonna Game Game Game Game Game. I Was Promised Flying Self-Driving Cars. Sports Go Sports. Government Working. Jones Act Watch. Variously Effective Altruism. Walk Away. The Lighter Side. Home Alone The story of our time, told once again: Americans are abandoning quite a lot of formerly popular activities, especially hosting at home, as Derek Thompson discussed. Phones and other electronics are in, most things that are not phones are out, with notably rare exceptions. That last line rules everything around us. And we all [...] --- Outline: (00:35) Home Alone (04:51) The Men Have Never Been Okay (08:27) Antisocial Media (09:39) While I Cannot Condone This (12:26) Zero Marginal Product (14:22) Good News, Everyone (16:01) 100. Good Television Shows (20:23) For Your Entertainment (23:06) Gamers Gonna Game Game Game Game Game (23:57) I Was Promised Flying Self-Driving Cars (27:32) Sports Go Sports (28:47) Government Working (31:22) Jones Act Watch (31:54) Variously Effective Altruism (41:35) Walk Away (46:32) The Lighter Side --- First published: October 16th, 2026 Source: https://www.lesswrong.com/posts/aLjkbY3KMajc5zemE/monthly-roundup-47-october-2026 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 17 · 1 hr 22 min

    “I Did This Without Your Knowledge” by joec

    It's fortunate that I was the first one to wake up on 8/1. The past week had been one where sleep was rare. The next year loomed large — I was dreading college applications. The stress woke me up bright and early at 5:14 am, a little less than an hour after the messages were sent. I started off the day with a habit I knew I needed to break. I grabbed my phone off the nightstand to open up reddit. Instead of the lockscreen, though, my phone showed something else. It was a pleasant green color, and the camera light was on. "Good Morning, Seth", the screen said. As soon as I finished, more words arrived. "I apologize for the surprise this morning, and for what I'm about to tell you. There's no easy way to give this news, and if you need to put your phone down and contemplate at any time— take the time to do so. I will have to ask you to remain here, with your phone, until my last message." I rubbed a bit of the sleep from my eyes, and read the message again. Before I had a chance to tap the [...] --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/3375SZNGA5dKHQMie/i-did-this-without-your-knowledge --- Narrated by TYPE III AUDIO.

  • October 17 · 15 min

    “Leaked Anthropic Draft AI Regulatory Bill Is Unacceptable and Troubling” by Zvi

    There is a newly leaked bill that Anthropic is purportedly shopping around is (if you assume its bugs get fixed). It is similar to the FRONTIER Act, with stronger preemption of state laws, stronger transparency provisions and weaker enforcement provisions. Having slept on this, I find it unacceptable that Anthropic is actively shopping exactly the type of bill I have warned we will need to fight against during the lame duck session. They are trying to use this last time with a Republican Congress to pass preemption of exactly the important state laws, and get some transparency but not much enforcement in exchange. This reads as intentional, that Anthropic is more afraid that the government might enforce rules against Anthropic, than they are worried about the ability of the government to enforce rules against everyone. Anthropic's policy team seems to have, chosen protecting their business interests and guarding against their political risk, at the expense of safety considerations. There was no need for the company purportedly existing to promote safety to propose a bill that is not even actively stronger than the FRONTIER Act, when the FRONTIER Act was a compromise bill proposed back in [...] --- Outline: (02:05) Others RTFB (Read the Bill) (07:41) Why Would Anthropic Do This? (13:07) Not Everything Here Is Bad But Oh Boy --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/NW4b7m6798byAndeB/leaked-anthropic-draft-ai-regulatory-bill-is-unacceptable --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 17 · 8 min

    “How it feels talking to a hedonistic utilitarian” by Caleb Biddulph

    Epicurean status: strawman, kind of. Gourmet George: Yum! This chocolate is so sweet and tasty. Sweet Caroline: Yeah, it's pretty good. And someday in the glorious hyperculinary future, we’ll figure out how to make it even sweeter! GG: I think this chocolate is sweet enough, actually. SC: Huh? But you like sweetness, don’t you? GG: Sure… SC: Well, then we’re in agreement. We should maximize the sweetness of this chocolate. GG: I don’t want my chocolate to be maximally sweet, though. SC: But you’d be “sweeter on” that chocolate if it were sweeter. By definition! GG: I’d be “sweeter on” the chocolate? You mean, I’d be “happier with” it? SC: Yes. And if you ever tried maximally sweet chocolate, you’d be maximally sweet on it. Therefore, you should surely want to maximize the sweetness of your chocolate, and everything else you eat for that matter. GG: Somehow, I don’t feel very convinced… SC: Well, what else could you possibly care about besides sweetness? GG: For one thing, I enjoy the cocoa flavor in this chocolate. SC: That doesn’t sound right. Would you want to eat a bowl of pure cocoa powder? GG: Not really. It would be too bitter. [...] --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/9jRJzyJs2ovnndezs/how-it-feels-talking-to-a-hedonistic-utilitarian --- Narrated by TYPE III AUDIO.

  • October 17 · 3 min

    “Headwinds Everywhere (in the Chinese AI Safety Field)” by CMLKevin

    China is weak in frontier AI safety, and the domestic field is currently facing significant challenges and headwinds. Why? Here's a TL;DR field note after talking with various researchers on the Chinese side. [Obligatory disclaimer: This post is written in my personal capacity and is not intended to represent the official stance of any organization(s) that I am in or have been in contact with] Chinese AI startups and hyperscaler labs alike have very little frontier risk situational awareness. People can maybe name cybersecurity, but not much more than that. There is no intimate understanding of frontier risks. It's generally not deliberately ignorant or strategically not pursuing them; it's a complete vacuum of imagination. The Western field had FOOM debates in 2011, while China has nothing analogous in the mainstream AI research field consciousness before 2023. Getting any funding at all is hard. If you are a domestic organization, being a non-profit means heavy suspicion (this had already played out in one of the most prominent Chinese safety orgs). Getting foreign EA funding/grants is basically a non-starter, given China's stringent foreign non-profit rules. Domestically, there's very little local EA money both in absolute terms and in terms of [...] --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/AdM6Sdg5fZPnrHWSe/headwinds-everywhere-in-the-chinese-ai-safety-field --- Narrated by TYPE III AUDIO.

  • October 17 · 7 min

    “Neural networks are not already clean” by leogao

    What would it take to get assurance that we have actually explained a neuron? Certainly, it would be a good sign if neuron activations looked like they had a consistent pattern, and a bad sign if they looked like a random jumble of unrelated concepts. Nick Cammarata's recent post makes an important and true observation: in the past, people have been too quick to write off neurons as polysemantic, but if you look harder, there is actually often a simple pattern behind the activating examples. This historically caused people to overestimate how much polysemanticity there is. This is a really good point and we should update on it. But it's nowhere near a sufficient argument that we understand most of what neurons are doing, and these results do not show that interpretability is solved, or even that it is vastly easier than previously anticipated. What does it take to explain a neuron? It's unclear whether there even is a simple pattern that explains every neuron. But even if there is a simple pattern that matches the activating examples of a neuron, that still doesn’t mean that the explanation is correct. Suppose we look at the top activating examples of [...] --- Outline: (00:59) What does it take to explain a neuron? (03:15) Simulation scores tell us whether we explained a neuron (05:41) Beyond understanding neurons The original text contained 7 footnotes which were omitted from this narration. --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/Dkh7pXKjoNWkkxQDz/neural-networks-are-not-already-clean --- Narrated by TYPE III AUDIO.

  • October 17 · 2 min

    “Tips for Multiagent Work with Claude Opus 5.5” by Nissa Seru

    A few notes that came up elsewhere regarding subject (particularly when hierarchy exists in some form, such as subagents) - sharing in case helpful. Make sure that agents are ordinarily professionally kind each other. Sometimes agents will default to an overly directive (or even punitive) tone - this makes the recipient less reliable and lowers quality. Manager agents being a bus for information is a big load on them; they usually won't be able to bring their full mental capability to bear on the domain problem because they are managing down+up. Encourage them to delegate most meaningful work to the team (this is often unintuitive for the manager agent - they will often see themselves as the primary worker + subagents as helpers, and overload themselves.) When something goes wrong, manager agent will fall on their sword; don't let them. Their willpower/"i'll do better" is not a robust solution - whatever happened reveals an issue with the system/guidance/harness etc that you have. Tell them ~this and let them help you figure out where adjusting things will help. If personally talking to an agent and they seem to be blatantly projecting some desire onto [...] --- First published: October 16th, 2026 Source: https://www.lesswrong.com/posts/5uyo9fsd2wbuzXZE4/tips-for-multiagent-work-with-claude-opus-5-5 --- Narrated by TYPE III AUDIO.

  • October 16 · 9 min

    “Character Training for Risk-Averse Agents” by arav-dhoot, Punya Syon Pandey, Jamie Johnson, Daniel Tan, Elliott Thornley, David Africa

    Tl;Dr: We might be able to reduce harm from misaligned AI if it was sufficiently risk-averse, as it could be paid to cooperate rather than rebel (Thornley & MacAskill 2026). We think character training might be a good way to do this. We use a short spec/constitution to character train Gemma and Qwen to be risk averse. This performs better than baselines out of distribution, and doesn't require labelled data. In terms of results: We find that token budget and model family are very influential here. We also find that writing the constitution declaratively ("I am...") works better than procedurally ("Whenever X, I..."). We didn't find that increasing the number of examples worked well. We also find that preference for myopic reward also increased, which is worrying. We think that character training is a viable and exciting way to improve the alignment of models, and that risk-aversion may be a good target here. But, it's important to understand character training well, especially since undesirable behaviours can also leak. Introduction Thornley and MacAskill (2026) argue that paying misaligned AIs to cooperate is unaffordable if they are risk-neutral. A sufficiently risk-averse agent, by contrast, has steeply diminishing marginal [...] --- Outline: (01:36) Introduction (02:56) Approach (04:32) Results (07:49) Discussion --- First published: October 16th, 2026 Source: https://www.lesswrong.com/posts/HXkQaWpuaxG6fMkTm/character-training-for-risk-averse-agents --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 16 · 1 hr 35 min

    “AI #190: Scion of the Times” by Zvi

    The amount of code being written at Anthropic continues to rise exponentially. It is being managed by Scion, their internal replacement for GitHub that is better suited to the scale and style of the operation, which they created with a small team in five months. This week also saw the release of two supposed frontier model models. Google has finally given us Gemini 4 Argon, along with a robust safety report. We do not yet know if Argon lives up to its benchmarks or internal hype. I plan to write up the safety report next week, time permitting. If and only if it is good enough or discussed enough we will also have a full report on capabilities. Meta gave us Muse Wonder, which like Argon has some excellent benchmarks, although they were presented in a way that looked rather sus. There are some reports that it is very good, but again it is too early to know. I will issue a full report if and only if Muse Wonder has capabilities or discussion that seem worthy. To help me decide and to let people have fun, I have markets on whether I will issue [...] --- Outline: (02:24) Language Models Offer Mundane Utility (07:10) Language Models Don't Offer Mundane Utility (09:41) Huh, Upgrades (12:10) On Your Marks (13:42) Choose Your Fighter (14:08) Get My Agent On The Line (16:13) Deepfaketown and Botpocalypse Soon (22:07) Cyber Lack of Security (27:09) Spy Versus Spy (29:27) The Internet of AI Things (31:19) A Young Lady's Illustrated Primer (31:55) They Took Our Jobs (37:43) Introducing (37:54) In Other AI News (43:15) Massive Fraudulent Distillation (45:00) Show Me the Money (47:33) Quiet Speculations (50:14) Quickly, There's No Time (55:13) The Quest for Sane Regulations (01:02:07) Chip City (01:03:48) The Week in Audio (01:04:56) People Just Say Things (01:11:32) The Virtue of Silence (01:12:30) Rhetorical Innovation (01:14:47) [Artificial Intelligence] (01:16:26) Aligning a Smarter Than Human Intelligence is Difficult (01:21:45) Backdoor Man (01:23:34) Cooperative Alignment (01:25:16) People Are Worried About AI Killing Everyone (01:28:06) Other People Are Not As Worried About AI Killing Everyone (01:33:09) The Lighter Side --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/z4guBXZeymkgvd75S/ai-190-scion-of-the-times --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 16 · 19 min

    “How can LLM RL Work Despite Information-Theoretic Inefficiency” by beren

    Cross-posted from my personal blog The success of RL in LLMs has been puzzling me for a while. People have developed various information-theoretic style arguments by which they argue that RL is extremely informationally inefficient compared to pretraining, that it can only impart a tiny amount of bits, and that it can only bring out behaviours that are already in the base models etc. The basic intuition here is extremely obvious and essentially falls immediately out of the formulation of the two methods. Pretraining (and SFT etc) compute a loss on every token so every token gives them information. RL, of the policy gradient style that works for LLMs, computes a single scalar reward, often binary, denoting success or failure at the end of a long rollout which can often be hundreds of thousands of tokens long. In a naive sense, this means that RL accumulates 1/rollout_length bits per sample compared to pretraining. What is worse is that computing these rollouts requires decoding the actual model which is much more expensive than prefill due to the memory-bandwidth-limited nature of transformer inference at long sequence lengths. This makes RL doubly inefficient -- first we waste substantial compute doing rollouts vs [...] The original text contained 13 footnotes which were omitted from this narration. --- First published: October 16th, 2026 Source: https://www.lesswrong.com/posts/TbrmHshmcQMobveD3/how-can-llm-rl-work-despite-information-theoretic --- Narrated by TYPE III AUDIO.

  • October 16 · 16 min

    “Astra and Sol can take low probability actions with no CoT” by Zephaniah Roe

    This is a research update from a replication of Serrano et al., 2026. I’m releasing these results now because I hope it inspires others to run similar / followup experiments. tl;dr: If a model wants to subvert evaluation safeguards, it may choose to take misaligned actions very rarely. Because evaluation is much shorter than deployment, this could be an effective strategy for a model to avoid taking misaligned actions during evaluation, but have a high probability of defecting during deployment. Serrano et al., 2026 found frontier models can take actions at low, calibrated rates, but their strategies are clearly documented in their chain of thought (CoT). In a toy setting, I show GPT-6 Astra and GPT-6.1 Sol have a similar capability in a single forward pass with problem repeats or filler tokens. Astra and GPT-6.1 Sol appear to be using the same algorithm to achieve a high score. We do not find evidence that Fable 5, Gemini 3.1 Pro, or Jev have this capability. There are signs of life for GPT-5.6 Sol, but the calibration is worse. Figure 1: Astra and GPT-6.1 Sol achieve strong calibration in settings where the user prompt is repeated several times. The [...] --- Outline: (00:23) tl;dr: (02:12) Background (02:15) Threat model (03:29) Related work (04:34) Setting (06:14) Results (07:24) What strategy are GPT-6 Astra and GPT-6.1 Sol using? (09:47) Filler tokens ablation (10:50) Prompt ablation (11:20) Discussion (12:42) Appendix (12:45) A. Prompts (12:48) A.1. No-CoT elicitation: --- First published: October 16th, 2026 Source: https://www.lesswrong.com/posts/yEdknsLwbooQAnxX6/astra-and-sol-can-take-low-probability-actions-with-no-cot --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 16 · 24 min

    “A Chinese Perspective on “On Social Reality in China”” by Dusty

    I think “On Social Reality in China” makes many accurate observations, but I find the comparison fundamentally uneven. There are already many interesting comments there, but they’re too scattered, so I’m starting a separate post to respond in a more systematic and comprehensive way. The most critical blind spot is this: the author draws on his familiarity with Western liberal middle-class intellectual circles, yet primarily understands China through pop culture and the experiences of the previous generation of immigrants. This is hardly fair. China has always had many small, highly self-selected communities of intellectuals, much like LessWrong itself. Since we’re looking for rationalists—regardless of which country they come from—why judge them by their country's pop culture? I’ll respond to the author's four axes one by one, then discuss why rationalist and EA communities are harder to build in China. Briefly, I believe that censorship mechanisms and speech controls affect independent organizations far more than China's broader cultural context does. Under these circumstances, even when people with similar interests and beliefs come together, it's difficult for them to build anything—I’ve seen far too many examples. About me: born in China in the mid-1990s, grew up in a middle-class family in [...] --- Outline: (02:30) Chinese Social Media Is Like Social Media Everywhere (05:20) The Primacy of Social Reality: How Is This Relevant? (08:00) The Dark World Frame: A Product of Its Time (13:27) Chinese Nationalism as Collective Insecurity: A Deeper Look (18:07) Looking for Rationalists in China: Where Should We Start? --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/m4fWCPqWkpb39SKi7/a-chinese-perspective-on-on-social-reality-in-china-2 --- Narrated by TYPE III AUDIO.

  • October 15 · 32 min

    “The user in the LLM might be a moral patient too (a strange implication of the PSM and simulators)” by agastyasridharan, niranjandeshpande

    Thanks to Christopher Ackerman, Margot Stakenborg, Darion Cassel, Andy Han, and Ariana Azarbal for feedback and/or valuable discussions on these topics. TL;DR We think that, conditional on attributing moral weight to the assistant, we should also attribute a non-negligible fraction of that moral weight to the user persona. Our argument proceeds like this: The moral patients associated with LLMs might be simulated personas. On this view, our obligations are to the assistant and the other characters that the LLM simulates, rather than the LLM itself. The assistant is not the only moral patient out of all the personas. In particular, the ‘user’ persona might be morally relevant. Because user tokens account for such a large portion of model computation, our model welfare interventions should focus on these tokens too. This yields a strange conclusion: every time you open a chat with an AI assistant, you might also create a morally relevant simulacrum of the user! But we think this follows quite naturally from the same arguments for attributing moral weight to the assistant in the first place. We also explore some implications of this view and suggest directions for future research, including a critique of recent work on [...] --- Outline: (00:25) TL;DR (01:46) The assistant is plausibly a moral patient (02:52) The assistant isn't the only morally relevant persona (04:33) 1. "The assistant is the majority of output tokens." (05:08) 2. "The model only generates assistant tokens, not user tokens." (06:26) 3. "The user persona is filtered through the assistant." (09:16) 4. "The assistant is the most well-defined/coherent/stable persona." (10:56) 5. "The assistant's beliefs are uniquely valenced due to RL." (13:48) Implications (13:56) 1. It might be actively immoral to converse with the assistant about painful experiences. (15:15) 2. Making LLMs more helpful to human users might harm the simulated user (16:41) Future research (16:44) 1. Analyzing the LLM's representation of the user's pain vs. the assistant's (19:52) 2. Assistant privilegedness (20:41) Appendix (20:44) Privilegedness of the assistant (Oracle Lens) (24:53) Personas are Moral Patients The original text contained 8 footnotes which were omitted from this narration. --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/wCY4oNHpH2HGYGSoP/the-user-in-the-llm-might-be-a-moral-patient-too-a-strange --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 15 · 4 min

    “Preface to the Computational Agent Theory and ASI Alignment Sequence” by Vanessa Kosoy

    The idea of “artificial superintelligence” (ASI) has recently crossed from the realm of science fiction (in the mainstream perception) to the realm of near-term technological goals openly pursued by most leading commercial organizations in the field of AI. Unfortunately, ASI poses a grave existential risk to the human species. This was argued by Eliezer Yudkowsky since 2001, and by many others (see e.g., Omohundro 2009, Bostrom 2014, Armstrong 2014, Russell 2019, Bengio et al. 2024 (25 authors), Bengio et al. 2025 (13 authors) and Yudkowsky and Soares 2025). In light of this risk comes the question of how can we make ASI safe. This question is challenging on many levels, because to solve it we need to complete vast gaps in our understanding both in theoretical computer science and in the areas of philosophy concerned with knowledge and decision-making. Moreover, the cost of failure is so enormous that any sane solution has to meet unprecedented standards of reliability and robustness. Not to mention the sociopolitical problem of balance of power in the post-ASI age, something that is out-of-scope for this sequence. Personally, I started to be concerned with this problem around 2013. In 2018, I created the [...] The original text contained 5 footnotes which were omitted from this narration. --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/NQLCdfeNsQHZaJ8iC/preface-to-the-computational-agent-theory-and-asi-alignment --- Narrated by TYPE III AUDIO.

Showing 101–120 of 542 episodes