TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

542 episodes
Avg 20 min
English

Episodes 542

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

October 28 · 3 min
“Some thoughts on GRAFT and follow-up work” by shawnghu
I'm speaking as myself here and not on behalf of any of my co-authors. A bunch of the stuff in this post is explicitly speculative. I've written this post as a way of centralizing some points I want to make in discussion/in response to others who've asked questions about the GRAFT paper. This is organized more like a series of notes; there's not a specific thesis in here, though I do make opinionated claims. Specifically, a few people have asked what follow-up work I'd like to see; at the bottom of this post is a big brain-dump answer to that question. Connections to other works There's a details box here with the title "Expand". The box contents are omitted from this narration. Practical Considerations There's a details box here with the title "Expand". The box contents are omitted from this narration. An optimistic speculation There's a details box here with the title "Expand". The box contents are omitted from this narration. Soapbox: Alignment is still hard BTW There's a details box here with the title "Expand". The box contents are omitted from this narration. Follow-up work My impression is that not [...] --- Outline: (00:43) Connections to other works (00:54) Practical Considerations (01:04) An optimistic speculation (01:15) Soapbox: Alignment is still hard BTW (01:28) Follow-up work (01:48) Scaling (01:58) Conceptual advances (02:09) Optimizing tradeoffs (02:20) Method variations (02:31) Engineering advances (02:42) Setting variations (02:52) A mindset note --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/6ro3dngDGaLJaumYb/some-thoughts-on-graft-and-follow-up-work --- Narrated by TYPE III AUDIO.
October 28 · 22 min
“Subtly wrong advice misaligns models more than obviously wrong advice” by Christine Corry, arav-dhoot, Jozdien
tl;dr We fine-tune 12 open-weight models on correct, obviously incorrect, and subtly incorrect advice across 8 narrow domains We replicate the OpenAI result and find that subtly incorrect data produces more emergent misalignment than obviously incorrect data +10.6 points pooled over all model-domain pairs (95% CI [9.1, 12.0]) We also extend this analysis to Petri evals and find uplift in fabrication and concerning behavior Fabrication: 2.6 → 6.7 (obvious) and 8.7 (subtle) on the 10-point judge scale Concerning: 3.7 → 7.5 (obvious) and 8.2 (subtle) Emergent misalignment seems to rise with model size This work was done as part of the Second Look Fellowship supervised by Zephaniah Roe and Arun Jose. Our code can be found here. Introduction Betley et al., 2025 demonstrate that fine-tuning models on incorrect narrow datasets (insecure code, bad medical advice, risky financial advice, etc.) can produce broad misalignment on unrelated questions. The authors find that this emergent misalignment (EM) can vary depending on the nature and framing of the data: training on insecure code results in broad EM but insecure code contextualized with the right (e.g. educational) framing does not. Persona Features Control Emergent Misalignment (Wang et al., 2025) finds that [...] --- Outline: (00:13) tl;dr (01:22) Introduction (02:48) Replication (05:26) Petri Evals (09:18) Scaling Laws (11:20) Appendix (11:23) Unscorable Responses (13:16) Evaluation Questions (14:50) Judge Model Comparison (16:45) Models & Training Details (18:03) Petri Details The original text contained 7 footnotes which were omitted from this narration. --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/yMPR6Pkv6jQXM4rWu/subtly-wrong-advice-misaligns-models-more-than-obviously --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 28 · 1 hr 17 min
“Anthropic Files Its S-1” by Zvi
The full text is finally here, so we are giving it the full RTFB of model card treatment. First, a recap of the state of Anthropic finances, then a close reading of the relevant parts of the S-1. A lot of it is technical and uninteresting, often pro forma, and gets skipped. A lot of it gets repeated many times. A lot of this is also Anthropic ‘playing the hits’ we have heard many times, now to the investor circuit. Get your drinks ready for when we have to maximize the upside and minimize the downside. This presentation of S-1 data has some cool Claude-created charts if that's your jam. Here are my top 10 takeaways from the S-1, for those already familiar with Anthropic. Existential risk is mentioned briefly only in the passive voice, and alignment issues are mostly treated as prosaic. They want to claim Big Mission but also sound as much as possible like a normal company. They are downplaying their willingness to pay real costs for safety, and downplaying the danger quite a lot. ‘Pacing the Frontier’ is not something they will do on their own. They’ll do [...] --- Outline: (02:48) Show Me the Money (10:11) Anthropic Has The Money (11:47) Shares of Many Classes (13:57) Who Governs Anthropic? (18:57) The Cap Table (19:43) Realize the Benefits While Navigating and Preventing the Risks (20:42) Just Think of the Potential (21:20) People Are Worried About AI Killing Everyone (1) (23:25) Anthropic Bets Big on AI (24:42) Anthropic Does Not Fully Maximize Profits (27:48) Everything Is Awesome (29:23) How Much Is AI Worth? (30:37) How Much Would You Pay? (31:48) Minimize Those Risks (33:58) Hopefully Introducing the World's Most Powerful Model (38:25) Might Not Compute (40:11) Supply Chain Risk (1) (40:32) Anthropic Is Nothing Without Its People (45:00) Shitloads of Money (46:40) Someone Has To Buy The Product (47:51) Did You Know There Are Other Frontier Labs? (48:27) Why They Hate Us (49:28) Claude Seems Pretty Misaligned to Anthropic (53:42) People Are Worried About AI Killing Everyone (2) (59:26) Someone Might Sue or Regulate Anthropic (01:00:38) Supply Chain Risk (2) (01:01:07) We're Too Safety (01:02:19) Claude Core (01:02:41) We've Never Actually Made Money (01:04:06) You Are Buying Stock In a Company (01:05:04) There Will Be No Dividends (01:05:38) What Are We Even Doing? (01:07:18) The Financial Model (01:11:55) The Business of Intelligence and Maybe Killing Everyone (01:15:10) Don't Give a Damn About My Regulation (01:17:08) The Bottom Line --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/dbdz6zb6qjaq6yHKs/anthropic-files-its-s-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 28 · 8 min
“I spent 12 hours writing an article for LessWrong, calling for a more ‘hippie’ rationalism. Then I didn’t post it. This is why (plus a link to the article, explicitly not located on LessWrong)” by honeybee
TLDR I (and many others in the community) wish to see the universe unfold to its highest potential in the metaphorical palm of an AI as aligned as the most benevolent Buddhist monk – but I (we) feel the community's ontology is too narrow to hold me (us). I'm still not sure what to do about this, so I'm posting the article I wrote for LessWrong – calling for a more 'hippie' rationalism – on Substack instead (actually, there are two articles, and the links are at the end). Quick intro (who I am) My name is Melissa. I’m a passionate member of the core AISafety.com team. I was there with the core team when the site was just an empty domain and a loose vision in August 2023. Since then I have co-led, designed, and coded the first official version, and co-led, designed, and coded the second and current version, co-led two of three hackathons, led the creative direction for the current ecosystem map, and I’m currently working to bring lots of more amazing features to our site (stay tuned, it's going to be awesome)! I’ve also played major roles at AISafety.info, grantmaking.ai, Elicit, and minor roles at [...] --- Outline: (00:20) TLDR (00:51) Quick intro (who I am) (02:19) Leaving LessWrong in pursuit of ontological freedom --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/zf3Xz5Cv45ytavH2r/i-spent-12-hours-writing-an-article-for-lesswrong-calling --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 28 · 11 min
“Streetlight, Torchlight and Nightvision” by DusanDNesic
Note: We at PrincInt are launching a program teaching “Discernment” or “Taste” and generally spend a lot of our time in workshops and otherwise trying to teach academics how to think about AI such that their research ends up being useful. I had used a metaphor (building on Wentworth's postmortem) multiple times to explain what we do and why, and thought that it could be useful more broadly. All views are my own, and strategy takes do not represent everyone at PrincInt etc. AI writing level 0 (no non-trivial use) When I first joined the field, I told 80,000 hours that I have a background in Economics, almost a decade as a professor, and connections from my time in China. They said something like “all of that is useless - you have 10 years running NGOs, we need Ops people, do that”. At the time, they were right! In 2021-22 there was little going on in either Economics or China, but if I was joining now it would be completely different - I’d have orgs to join, or if I tried fundraising for something in those fields, it would now be much easier. I would not [...] --- Outline: (01:30) The Puzzle Planet (03:03) Bring the Light (07:43) So, how do we find True Keys? --- First published: October 28th, 2026 Source: https://www.lesswrong.com/posts/ykx7EGq8KSvrHgiFY/streetlight-torchlight-and-nightvision --- Narrated by TYPE III AUDIO.
October 28 · 11 min
“Quantity has a quality of its own” by djbinder
People love to argue about whether AIs could recursively self-improve their way to superintelligence, each improvement making the next one easier as progress accelerates. I'm unsure. It could be that progress stalls at near-human or somewhat superhuman levels, that the ultimate qualitative limits of intelligence are not particularly beyond the human-level, or that the quantity of AI researchers involved is so vast that substantially superhuman AIs are created without strictly entering a super-exponential regime. But honestly, I think recursive self-improvement is a little overrated. Even if human-level AIs do not rapidly lead to superhuman AIs, the vast quantities of them could be very dangerous. Arguments against recursive self-improvement provide little consolation here. Conversely, recursive self-improvement to superintelligence does seem particularly dangerous, and pausing or pacing it may be desirable. But stopping it may not be enough, because vast numbers of less-than-superintelligent agents could still be destabilizing and catastrophic. Soon after we build human-level AIs, we will be able to run hundreds of millions of them, and before long billions. At that scale, quantity has a quality of its own. There will soon be a lot of human-level AI systems Human-level AIs—by which I mean AIs [...] --- Outline: (01:23) There will soon be a lot of human-level AI systems (04:30) They could do a lot of research (06:46) They could act cohesively (08:12) Skill buys power (10:06) Keeping control gets harder as the numbers grow --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/drt7CEAtERECKiYpn/quantity-has-a-quality-of-its-own --- Narrated by TYPE III AUDIO.
October 28 · 13 min
“Against Girardian mimesis” by Viliam
Wikipedia summarizes René Girard's mimetic theory like this: Man is the creature who does not know what to desire, and he turns to others in order to make up his mind. We desire what others desire because we imitate their desires. In other words, you think you want something for your own internal reasons, but the actual reason you want it is that someone else wants it, and that makes you desire the thing and compete for it. The "someone else" whose desires you copy is someone you care about. It could be a friend, a sibling, or a colleague. It could also be a famous person, or even a fictional character. (Not a stranger you don't care about.) This creates unnecessary social conflict, as people gradually converge on wanting the same thing, people start seeing each other as rivals, this attitude gets copied too, then people start hating each other, but who hates whom gets copied too, until everyone synchronizes at hating one specific person... and then they typically murder the person, and everyone feels happy about the conflict being resolved. The choice of the person to kill is arbitrary, in the sense that the selected person is [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/tzuArKci3JpN7Zye4/against-girardian-mimesis --- Narrated by TYPE III AUDIO.
October 28 · 48 min
“Anthropic Versus Anthropic” by Zvi
Anthropic is a frustrating company. Sometimes it will do things I greatly appreciate, including at substantial cost. They have taken bold stands, have a unique governance structure and have hired and worked to establish a relatively strong (but I think still inadequate) safety culture. Other times, it will do things that are deeply disappointing. It produces very good models. That too is not entirely a good thing. One can both be the best given the alternatives, and insufficient to meet the moment. Ultimately, what you think about Anthropic depends on what parts you care about most, and what you think would be sufficient. Anthropic is great if you think the problems are important and difficult, but within the range of what Anthropic's approaches can solve. If you think we mainly need operational excellence and caring about safety and the future, then there are some non-excellent operations to fix but that goal seems doable for Anthropic. If you think the problems go well beyond that, then things look rather grim for everyone. Today's post begins with a continuation of discussion over Anthropic's leaked draft bill. See my previous coverage for details of the [...] --- Outline: (01:43) Dario Amodei Responds to Nathan Calvin (20:50) Kelsey Piper Goes Inside Anthropic (26:19) The Technical Problems of AI Safety Are Not Solved (27:50) The Quest for Embedded Evaluators (33:57) METR Questions Risk Report Methodology But Affirms Risk is Low (38:58) The Anthropic Public Benefit Fund (41:39) Anthropic Versus Anthropic --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/ZSrX8gucWsKahsitk/anthropic-versus-anthropic --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 27 · 15 min
“AI speeds up 3rd party safety researchers by ~1.6x (via self-reports)” by Angel Martinez, Dewi Gould, nicole x, Alejandro Aristizabal
TL;DR: Labs are accelerating due to automated AI research and development. As an upper bound, in March 2026, Anthropic researchers self-reported producing 4 times more output with AI tools than without them. We lack public evidence on how this may differentially impact safety versus capabilities. As a first measurement of automated safety-specific research uplift, we surveyed 52 third-party AI safety professionals from 20+ orgs and found a median reported speedup of 1.58 x. We think it is important to develop standardized methods for tracking safety-specific uplift, as these will facilitate a better understanding of the differential impacts of RSI. *This work was done by Arcadia Impact's alignment team.* – AI R&D may not accelerate safety and capabilities equally Frontier AI companies are currently automating their AI R&D efforts. OpenAI claims that they are “making strong progress towards creating an automated AI researcher by March of 2028.” At Anthropic, Claude now leads 26% of AI R&D, researchers self-report producing 4 times more output, and code commits arrive 5 times faster per engineer than 6 months ago. As labs push towards RSI, it is important to understand whether AI research uplifts safety work equally. To this end, we surveyed 52 third-party [...] --- Outline: (01:05) AI R&D may not accelerate safety and capabilities equally (04:01) Survey (05:38) Current AI tools make safety researchers 1.58x faster. (06:12) Speedup is mostly attributed to building and running experiments. (07:05) However, accelerated tasks are a minor share of respondents' jobs. (08:03) Despite heavy agent usage, organizational automation is low. (09:16) Being an AI power user does not clearly correlate with greater speedup. (10:03) Research judgment remains the biggest reported bottleneck. (10:53) Survey limitations (12:34) Frontier lab safety uplift is currently underreported and unstandardized The original text contained 12 footnotes which were omitted from this narration. --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/ea9fGFKxJhh2bF9rp/ai-speeds-up-3rd-party-safety-researchers-by-1-6x-via-self --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 27 · 5 min
“A big case study of propensity entanglement” by nielsrolf, lwen2027, megasilverfist, Owen Terry, Johannes Taraz
This is a link-post for our new paper "Predicting how propensities generalize out-of-distribution", which started as a SPAR project. We select 24 propensities, elicit each through 4 different methods, and then test how this spills over to non-targeted traits: How well can we predict spillover? Below is the plot that compares different methods to predict SFT-induced spillover. The best method is to predict that one model behaves similarly to the two other models we tested ("SFT (other models)"). The second best is to system prompt the model to express the target trait, and use the behavior of the prompted model as a proxy for the finetuned model. The third best predictor is drift: assuming that the post-elicitation propensity profile does not depend on the targeted trait at all! This was pretty surprising for us, and we speculate that this is some kind of regression to the base model: by changing the weights, we undo some of the changes of the original post-training. Indeed, drift makes Qwen3.5-9B more similar to its base model. Can we predict what causes misaligned traits to rise? Initially we hoped that we could find traits that induce misalignment by first making models deliberately misaligned, and [...] --- Outline: (00:43) How well can we predict spillover? (01:45) Can we predict what causes misaligned traits to rise? (02:31) Evidence about the Persona Selection Model (03:34) Abstract --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/eD84A7C5jhhXNEhCQ/a-big-case-study-of-propensity-entanglement --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 27 · 15 min
“Scattered thoughts on how doing math feels these days” by dynomight
This is a weird moment. I have a hunch that we'll look back later and tell ourselves stories in which it seemed obvious where things were going. So I wanted to lay down a little time capsule and register just how confusing everything felt at the time. This is a personal view from the trenches. 1. If you're a normal person, you may feel that AI has been good at math for a long time.But for "real" math, this is roughly what it felt like: When Feeling 2024 LLMs are toys Summer 2025 LLMs are interesting toys Early Autumn 2025 LLMs can do useful stuff sometimes Mid Autumn 2025 …a lot of stuff Late Autumn 2025 …a lot Early 2026 holy god Even having been astonished by AI progress in the past, it was still astonishing to see AI move in a matter of months from a mildly-useful toy to something that many people now consider almost impossible to do without. 2. Personally, I'd say that current AI makes me around 50% more productive at "doing math". That's despite the fact that AI can now often do in a few minutes things that would have taken me hours to [...] --- Outline: (00:28) 1. (01:03) 2. (01:49) 3. (02:31) 4. (03:10) 5. (03:50) 6. (04:37) 7. (06:56) 8. (07:44) 9. (08:23) 10. (08:44) 11. (08:58) 12. (09:41) 13. (10:42) 14. (11:41) 15. (12:04) 16. (13:11) 17. (14:00) 18. (14:46) 19. The original text contained 4 footnotes which were omitted from this narration. --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/HuQAuWjc2Z5tqCjyw/scattered-thoughts-on-how-doing-math-feels-these-days --- Narrated by TYPE III AUDIO.
October 27 · 36 min
“Measuring agent capabilities on open-ended AI R&D tasks” by Alex Jackson, Titus Buckworth, Jason R Brown, Francis Rhys Ward
TL;DR. We measure agent capabilities on four long and open-ended research tasks. Frontier models have been improving with an overall doubling time of around one year since 2024 (based on an exponential fit). Agents perform better at engineering-heavy tasks compared to more conceptual tasks (between 7 and 16.8 months median doubling time), and model capabilities are improving slower on the latter. Epistemic status: This analysis is based on a small set of tasks and we rely heavily on LLM-grading (though we think our LLM scoring is reasonably well-calibrated to expert human scores). This work was done by Arrow Research. Figure 1: Aggregate normalised model score vs model release date. Our median estimate is that frontier model score has a doubling time of just under one year. Executive Summary In this post we describe our recent work measuring AI agent capabilities on open-ended AI R&D tasks. We evaluate models on a set of 4 difficult open-ended tasks related to AI R&D. Task scoring generally incorporates a combination of automated checks and human-expert written rubrics used by LLM judges. Key findings: The best model across all tasks is GPT-6.1 Sol but other models perform better on some individual tasks. --- Outline: (01:20) Executive Summary (03:29) Methods (03:43) Tasks (06:10) Model scoring and baselining (09:06) Model performance by task (10:31) Qualitative impressions (10:35) Model effort and motivation to achieve the task (12:51) Slop (14:02) Agents found unintended ways to do the task (15:38) Comparison to other AI R&D capability trends (17:18) Conclusion (19:28) Acknowledgements (19:36) Appendix (19:39) Further Methodological Details (19:43) Main plot run configurations (20:02) Comparison of Trend Line Fits (22:10) Wider extrapolations of trend lines (22:27) Further results (22:30) Raw Score Trends (22:52) Raw Scores by Task (23:23) Does more time yield improvement? (24:13) Where do models spend their reasoning? (26:45) Does more reasoning yield higher scores? (26:58) Across seeds (28:35) Effort sweep (29:19) Removing the formalisation task from the main trends (29:57) Correlations across tasks (30:26) Rank Correlations (31:42) Pearson Correlations (32:39) Qualitative assessment of agents on each task (32:44) EtV (33:44) Formalising X (34:59) Training Effects (35:47) Canary The original text contained 7 footnotes which were omitted from this narration. --- First published: October 27th, 2026 Source: https://www.lesswrong.com/posts/cTiEPgkvDRxTFwtyu/measuring-agent-capabilities-on-open-ended-ai-r-and-d-tasks-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 27 · 7 min
“The AI buildout is the largest bet in the history of capitalism. Key decision makers believe very powerful AI will be achieved soon.” by Hayekstan
This is part linkpost, and an expansion on a previous shortform posted on LW. I believe a massively undermade point in favour of the belief that AI will be a very big deal is that the AI buildout over the past four years is the largest bet in the history of capitalism. A titanic peacetime industrial effort is underway which has not had a comparison since the nineteenth century. The size of the effort and the scale of the spend signal that key decision makers and capital allocators believe very poweful AI will be achieved in the next few years. Many of us are vaguely aware that large sums are being spent, but the scale of the undertaking is astonishing: The six largest spenders this year are spending more on Capital expenditures this year than the Russian government, governing 11 timezones and 143 million people, fighting a war on which the fate of its regime probably depends, spent last year. Hyperscaler spending (Alphabet, Amazon, Microsoft, Meta and Oracle) will be $785-820bn this year, about 2.4% of US GDP. There are only 11 governments on earth who spend more than that. Next year the Hyperscalers are projected [...] --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/qjsdvaaPC624Eropw/the-ai-buildout-is-the-largest-bet-in-the-history-of --- Narrated by TYPE III AUDIO.
October 26 · 6 min
“Hair Thickness of My Friends” by XelaP
Intro I have a microscope. It can see hair. Humans have hair. Some of my friends are humans. How thick is my friends' hair? Apparently, if I go around asking my friends for a strand of their hair, about half will just give one to me, and almost all the rest will give one when I explain that I'm going to see how thick it is under a microscope. Your mileage may vary. My parents have not yet chanced upon the card with taped and labelled hair strands, but if they do then they might have some, uh, pointed questions. I'll just tell them I'm trying to diversify my clone army. Hey, at least it's not like that time a friend of a friend's ex once drew someone's blood at a party. Unfortunately, I never got around to asking her for one of the blood vials she was known to possess. Methods To measure length, I broke off a tiny piece of a clear ruler and put it under the microscope with the hair. Then, I had Free Claude (from 2025?) write some code so I could compare two lengths easily by clicking points. Then I [...] --- Outline: (00:09) Intro (01:01) Methods (01:47) Results (05:21) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/q8gnBtqijGsdqWJ5j/hair-thickness-of-my-friends --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 26 · 50 min
“A basic case for differentially accelerating conceptual reasoning capabilities” by Emery Cooper, Caspar Oesterheld, Chi Nguyen
Introduction In this post, we make the case that alignment and other important work require especially strong conceptual reasoning capabilities. By this, we roughly mean tasks that are similar to philosophy in that they aren't straightforwardly verifiable, and progress relies on informal a priori reasoning and argumentation. We argue that differentially accelerating AI's conceptual reasoning capabilities is important. We also think it's neglected, and that the AI safety community currently underinvests in this area. We have ourselves pivoted from doing conceptual research directly to working mainly on differentially accelerating AI's conceptual reasoning. We have a shovel-ready agenda which can absorb more people. In a follow-up post, we will lay out at a high level how to work on this problem. If you are interested in doing this work, get in touch! For others discussing related worries, see Joe Carlsmith on using AI for AI safety and on building AIs that do human-like philosophy, Wei Dai on the potential difficulty of AI doing philosophy, Paul Christiano's first story of alignment failure, John Wentworth in his case against AI control and, for an overview, Caspar's readings on the nature of alignment research. See also Caspar's post on the basic argument for [...] --- Outline: (00:15) Introduction (02:11) Overview (07:08) It's important how soon AI is good at conceptual reasoning (07:14) Important challenges need conceptual reasoning (18:56) AI R&D and takeover rely much less on conceptual reasoning (28:35) AI being better at conceptual reasoning earlier would be very valuable (35:47) We can probably differentially accelerate conceptual reasoning (35:53) Work on conceptual reasoning is likely counterfactual and tractable (41:12) Effects on generic capabilities are likely small (44:15) Counterpoint: Using our datasets to develop generic techniques (46:00) Our work (48:53) Acknowledgements (49:31) Appendices The original text contained 19 footnotes which were omitted from this narration. --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/eimAN67tiNsrrtnbK/a-basic-case-for-differentially-accelerating-conceptual --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 26 · 20 min
“A cat-belling problem for FDT” by jessicata
Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand. [AI disclosure: I used Opus 5.5 for light edits after my first draft] Functional decision theory (Yudkowsky and Soares 2017) proposes choosing actions as follows (equation 4 in the paper): Here is the agent's probability distribution, is a graph saying how changing a logical variable affects other variables, is the observation history, and is utility. The underlines denote dequoting: is a variable in the agent's model representing the output of the FDT algorithm on these inputs. In this post, by FDT I mean this specific proposal, not TDT, UDT, logical EDT, or FDT*. (FDT* is a different decision theory defined in the FDT paper, which uses an undefined "true" operator in place of .) FDT, admittedly, relies on a operator applied to a logical statement, which is on shaky philosophical foundations: Equation (4) is sufficient for present purposes, though it rests on shakier philosophical foundations than (3). Pearl (2000) has given a compelling philosophical account of how to deduce the structure of causation from [...] --- Outline: (03:20) Computational logical counterfactuals (06:41) FDT needs computational logical counterfactuals (10:10) A cat-belling problem (11:36) Inside view: which axiom breaks? (16:26) Proposals that don't bell the cat (19:13) Conclusion (20:04) References The original text contained 3 footnotes which were omitted from this narration. --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/drteSvhc4fX3Tu9y7/a-cat-belling-problem-for-fdt --- Narrated by TYPE III AUDIO.
October 26 · 20 min
“Anthropic are the Good Guys” by Ivan Vendrov
I’m writing these lines in Alamogordo, New Mexico. I’m about as close as a civilian can get to the site of the Trinity test, the first nuclear bomb detonation in history. I fled here from the San Francisco Bay Area; after two weeks of being back I just couldn’t take it anymore. The fear-greed complex that paralyzed me at Anthropic three years ago is now everywhere in the Bay. People are not thinking clearly. Hardly thinking at all, actually. Rumors of astronomical funding rounds and weird financial arrangements and inner ring gossip are everywhere. Everyone is worried. Everyone is checking their phone. Where are we? What's happening? When did this nightmare start? In a way I think it started on July 16, 1945, not far from where I’m sitting now. I think it's impossible to understand Anthropic without putting yourself into the shoes of the Los Alamos physicists huddling together at sunrise in the aftermath of Trinity. It must have felt like an initiation. An end to childhood, an end to innocence. We can no longer afford to fuck around, those brilliant men must have thought. Unless we figure out how to live peacefully together on this planet, we [...] --- Outline: (03:50) Anthropic and Empire (08:01) Three Lucky Breaks (12:20) Put Down Your Sword (14:04) But What About China? (17:08) It's Time to Breathe --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/zj8Pk3kJujk5LoqhK/anthropic-are-the-good-guys --- Narrated by TYPE III AUDIO.
October 26 · 16 min
“Some Memories of Early Anthropic” by Ivan Vendrov
The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering the gist of the dialogue rather than the exact words used. I’ve also swapped some pronouns and other personal identifiable details. Please remember that this is not a description of “what Anthropic is like”, but merely the experience of one guy in a junior role during a short window of time, weighted towards emotionally intense moments because they’re easier to remember clearly and because sharing them doesn’t run afoul of the spirit of the confidentiality agreements I signed. Anthropic is now a massive organization with many silos and secrets and I have very little idea what was or is happening there in some overall sense. But given the enormous power of the AI labs and the paucity of public information about their inner workings, I feel compelled to share the most salient parts of my personal experience. I hope this encourages my former colleagues, and folks at other labs, to fill in the rest of the picture. I’m at the Anthropic office for lunch. The interviews went well. They want [...] --- First published: October 26th, 2026 Source: https://www.lesswrong.com/posts/9PiLfo6iQbbDWFHv7/some-memories-of-early-anthropic --- Narrated by TYPE III AUDIO.
October 26 · 12 min
“Is GPT 2XL already clean?” by Jasper Blank
In which I investigate if Opus is able to interpret neurons better than GPT 4. TLDR: Opus 5.5 is able to identify 3-19% of the neurons of GPT 2-XL based on activations. This is a big improvement over the 0.3% that GPT-4 was able to achieve in 2023. #AI notice: This post is human written but in the research there was heavy use of Opus 5.5. I mainly asked it to rerun this 2023 paper to see how big the improvement is if Opus 5.5 is used instead of GPT4. Since the invention of Large language models they have been stubbornly refusing to be clear. They have been described as black boxes and as inscrutable machines, if this stays true that is bad news for detecting AI scheming and thus Alignment. Recently the Goodfire post told us that neural networks are already clean and that simply looking into the neurons is enough. For the case of this post I mean to say that a neural network is clean if we can understand what the neurons do only by looking at the neurons and not by creating some additional derived object to study. This inspired me [...] --- Outline: (02:11) Methods 1: OpenAI (05:02) Methods 2: Opus 5.5 (07:06) How good is Opus at one shotting these explanations? (09:03) Exploring a single difficult neuron (10:46) Future plans (11:24) Appendix 1: Link to the Claude conversation (11:38) Appendix 2: Comparing explainer to simulator The original text contained 2 footnotes which were omitted from this narration. --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/drMv4SM2gaxk3oz3m/is-gpt-2xl-already-clean --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 25 · 8 min
“LoReFT GRAFT Better at Preventing RL Grift” by Samuel Knoche
tl;dr. I show that the recently introduced GRAFT method can be improved upon by using a single layer rank 1 Low-rank Linear Subspace Representation Fine Tuning (LoReFT) adapter as the FORGET adapter, using only 0.029% of the parameters of the one used in the GRAFT paper. This approach achieves an equivalent level of reward hack suppression while improving the clean environment solve rate and not requiring an anchor environment. I end with a brief discussion of why this works and of the limitations of fixed preventative steering baselines. Background 1. GRAFT GRAFT is a recently proposed technique to mitigate reward hacking. You can find the details in their LW post and paper. Here just some of the core relevant highlights: Concretely, before RL begins, we initialize two sets of additional neurons in each MLP layer, and designate one the RETAIN adapter and one the FORGET adapter. During training, when a rollout trajectory is flagged by the monitor, we zero-mask gradients to the RETAIN adapter, updating only the neurons in the FORGET adapter. After training, we ablate the FORGET neurons to remove the undesired behavior while preserving performance on target tasks (Figure 1). GRAFT can also be used with full-parameter [...] --- Outline: (00:44) Background (00:47) 1. GRAFT (02:31) 2. LoReFT (03:16) Experiment and Results (04:23) Discussion / Analysis (06:38) Limitations (07:15) Summary The original text contained 4 footnotes which were omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/B7LnjyJQnkgwKdtTx/loreft-graft-better-at-preventing-rl-grift --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Showing 41–60 of 542 episodes