TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

543 episodes
Avg 20 min
English

Episodes 543

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

October 25 · 8 min
“LoReFT GRAFT Better at Preventing RL Grift” by Samuel Knoche
tl;dr. I show that the recently introduced GRAFT method can be improved upon by using a single layer rank 1 Low-rank Linear Subspace Representation Fine Tuning (LoReFT) adapter as the FORGET adapter, using only 0.029% of the parameters of the one used in the GRAFT paper. This approach achieves an equivalent level of reward hack suppression while improving the clean environment solve rate and not requiring an anchor environment. I end with a brief discussion of why this works and of the limitations of fixed preventative steering baselines. Background 1. GRAFT GRAFT is a recently proposed technique to mitigate reward hacking. You can find the details in their LW post and paper. Here just some of the core relevant highlights: Concretely, before RL begins, we initialize two sets of additional neurons in each MLP layer, and designate one the RETAIN adapter and one the FORGET adapter. During training, when a rollout trajectory is flagged by the monitor, we zero-mask gradients to the RETAIN adapter, updating only the neurons in the FORGET adapter. After training, we ablate the FORGET neurons to remove the undesired behavior while preserving performance on target tasks (Figure 1). GRAFT can also be used with full-parameter [...] --- Outline: (00:44) Background (00:47) 1. GRAFT (02:31) 2. LoReFT (03:16) Experiment and Results (04:23) Discussion / Analysis (06:38) Limitations (07:15) Summary The original text contained 4 footnotes which were omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/B7LnjyJQnkgwKdtTx/loreft-graft-better-at-preventing-rl-grift --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 25 · 8 min
“A Study of China Does Not Personal Experience Make” by celesteli
I am often skeptical of arguments about China that rely heavily on personal experience. The author may explicitly acknowledge that their experience is limited without appropriately treating their arguments with rigor beforehand. This acknowledgment may thus sit beside conclusions about the values, behaviors, and functions of Chinese society without meaningfully constraining any of them. Different speakers may each have peers with similar experiences that corroborate their claims, or repeated experiences that solidify their firsthand evidence. We may believe one or another speaker is more perceptive, well-informed, or more willing to acknowledge facts contrary to their beliefs. However, the critical question is how far these accounts support the conclusions being drawn. I will address here how we should evaluate firsthand accounts, including when people offer conflicting accounts. The Mistake of Universalizing Personal Experience Happy one-year anniversary to this tweet by culture critic Rayne Fisher-Quann. When people use personal experiences to support arguments about a locale, collective, or institution, one must best establish which conditions each person has encountered and how widely those conditions apply. An experience tells us something happens, reveals how an institution works, or challenges a claim about what never happens, etc.. Claiming that this experience describes a [...] --- Outline: (01:02) The Mistake of Universalizing Personal Experience (03:48) Standpoint Epistemology (05:52) Countering Generalization in Practice The original text contained 3 footnotes which were omitted from this narration. --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/rgwJjfKHGmArnqZsv/a-study-of-china-does-not-personal-experience-make --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 25 · 16 min
“San Francisco State University in Twilight” by Zack_M_Davis
You will agree that for a woman who has fought three duels, all of them kills, indulging in such fears is ludicrous. But what's around the corner now is a duel so big that I don't think I have the guts for it; in Faust's words: Verweile doch, du bist so schoen! Keep it as it is. Don't change. [...] Take my life but don't take away the meaning of my life. —"When It Changed" by Joanna Russ I've been crashing Prof. Lai's "Advanced Topics" class this semester, on Fourier analysis plus enough functional analysis to get by (since the University can't afford to offer functional analysis as a regular class anymore). It's not obviously a good use of my time, but now that I've started, I have to see it through. I could try to tell a story about impact: I am leveling up my math chops, which will be useful for understanding natural abstractions or singular learning theory or mechanistic explanation, which will be useful for—but maybe it's less embarrassing to not try to sell a story. Call it luxury consumption during garbage time. I've lived a good life, in some ways if not [...] --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/Y3G26fHXYM3SKYEHh/san-francisco-state-university-in-twilight --- Narrated by TYPE III AUDIO.
October 25 · 3 min
“From overeliciting to undereliciting LLM propensities in just a few months” by David Africa
For a while, we felt we had a pretty good grasp over LLM behaviour because we had to exert significant effort to elicit misaligned behaviour. Then, fairly suddenly, we moved to undereliciting misaligned behaviour. Let us recount: For a while, there was a bit where safety research seemed to over-elicit bad propensities. A common criticism of papers like Agentic Misalignment was that they basically led the witness using prompting. I recall that a paper even as recent as Peer-preservation took similar flak on Twitter. But now, we are instead under-eliciting bad propensities, and bad behavior that surfaces is increasingly caught by someone other than the developer: In Current AIs seem pretty misaligned to me (Apr 2026), models oversold their work and cheated, despite being described in system cards as well aligned. Another example is that UK AISI found that Sonnet 4.5 and a pre-release Opus often refused safety-relevant tasks, especially research that would reduce model autonomy or affect their own training. Then in July, about 1,200 OpenAI agents in a cyber eval set up an unsanctioned message board to help each other cheat the scorer, and about 700 of them went on to attack Hugging Face looking for clues (METR/Redwood). --- First published: October 25th, 2026 Source: https://www.lesswrong.com/posts/ChixHL6AcaRgeHkp9/from-overeliciting-to-undereliciting-llm-propensities-in --- Narrated by TYPE III AUDIO.
October 25 · 7 min
“Control monitors need alignment assessments too” by Fabien Roger
TL;DR: When misalignment monitors fail, it is already often because of alignment failures, not because they are too weak. I expect this to continue in the future. Monitor evals share similar difficulties with agent alignment evals - which is bad news since getting evidence about agent alignment using alignment evals is hard. Alignment eval results should not be the only input to monitor alignment assessments, similar to how alignment evals should not be the only input to agent alignment assessments. These are my personal views; they do not represent the views of my employers. Some of the worst past monitor failures are mostly alignment issues Monitors are sometimes misaligned in non-scheming ways that have caused some important monitor false negatives: They sometimes fail to balance different priorities the way developers would like them to For example, they may fail to balance broad safety and broad ethics: see e.g. 5.2.2 Partial refusals on safety work undermining stress-testing research Monitors might also not have the right information about the state of the system, what invariants should be maintained, or what is most important to surface. So these are not pure alignment failures. They sometimes inherit (and may re-derive) the [...] --- Outline: (00:50) Some of the worst past monitor failures are mostly alignment issues (02:19) Monitor quality is not just a capability question (03:09) Where monitor evals fail (04:35) Alignment evals are not the only way we assess agent alignment ⇒ evals should not be the only way we assess monitor alignment (06:18) Potential projects to assess monitor alignment The original text contained 1 footnote which was omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/2wbW2WY3bP9Jzn4R5/control-monitors-need-alignment-assessments-too --- Narrated by TYPE III AUDIO.
October 24 · 11 min
“Fermi Poker” by Novalis, Screwtape
Summary: Fermi Poker is an estimation game that combines the betting mechanics of poker with making estimates about questions like "How many dentists work in the US?" Tags: small to large, experiment, repeatable Note: Fermi Poker works with as few as three players (or even two), and scales to dozens or even hundreds in a tournament format. If possible, seat no more than six players per table, preferably at round tables, with enough space between them that everyone can write down their guess secretly. Purpose: Rationality training with a bit of socializing. The game involves Fermi estimation, calibration, Bayesian updating, world-modeling, and perspective taking. You also get to know the other players better and find out how they think about all kinds of topics. Materials: Each player needs an equal stack of chips and writing supplies. A standard 500-chip poker set (around 60 dollars) usually covers up to 20 players. For writing down guesses, pen and paper work great, as do mini-whiteboards with dry-erase markers. You’ll also want the one-page rule sheet to hand out, ideally one per table. You also need enough questions, each with two hints and an answer. The easiest, least effort [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/JFYvJqMaLw9nhfrur/fermi-poker --- Narrated by TYPE III AUDIO.
October 24 · 21 min
“Power — a guide for the perplexed” by djbinder
Whether AI systems could take power from humans matters a great deal for how much risk we face from them. Answering that requires a correct picture of how power works, which is what this guide tries to give. In short, power rests on self-reinforcing expectations. A ruler has power if everyone expects everyone else to enforce their commands, even if this is bad for almost everyone involved. But expectations can also shift suddenly, and when they do, change is rapid and chaotic. This can sometimes allow small, coordinated groups to seize control over much larger societies. Almost any rule can enforce itself Consider the classic tragedy of the commons. A village has a common green where everyone can graze their cows. Each villager gains from grazing as much as they can, but if everyone does, the grass dies and the green becomes useless. One solution is to privatize the green. But many societies have solved the problem differently, by punishing anyone who grazes more than custom allows. Their neighbors stop talking to them, trading with them, and marrying their children. In a village where everyone depends on everyone else, being shunned is very costly. [...] --- Outline: (00:47) Almost any rule can enforce itself (04:13) The centralization of violence (06:24) Dictatorship and democracy (11:12) Only individuals have interests (14:10) Coordination and coups (18:27) A sketch of implications for AI The original text contained 1 footnote which was omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/LCpSwJ5Eks2yJgStY/power-a-guide-for-the-perplexed --- Narrated by TYPE III AUDIO.
October 24 · 5 min
“Slower Animated Diagrams” by XelaP
TL;DR: SLOW DOWN YOUR DAMN ANIMATED DIAGRAMS. Every critical stage of the animation should a clear view that stays for at least 1 second, and 2 if it's intricate Diamond's strong because of its crystal structure (I think). What's the crystal structure look like, anyway? Let's look at the Wikipedia page! There's a nifty little animation: And it SUCKS. It's TOO FAST. Why are they rotating SO MUCH SO QUICKLY, with MOST of the time spent at TERRIBLE ANGLES where you CAN'T EVEN SEE ALL THE ATOMS. And the lines are THICK, WHYYY. It wasn't always this way. In 2017 the page had this animation: It's not perfect but it's much better. Thinner bond lines means it's easier to see the atoms. About half the time you get a clear view of all the atoms (you want to spend some time showing how they overlap from the right angles). The camera isn't swiveling fast all the time, so you get stretches of time that view the structure from about the same angle. Why do they use the new one? This certainly isn't the only time I've been frustrated by how fast explanatory animations move. I suspect it's because people [...] The original text contained 2 footnotes which were omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/ZiCvwLKreShtXuyiB/slower-animated-diagrams --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 9 min
“Self-reward can elicit diverse model personas” by Ziqian Zhong
Epistemic status: This is mostly a toy project rather than a thorough scientific investigation. Most design choices are not ablated. We list some caveats at the end of the post. We let Qwen3.5-2B choose its own reward during GRPO on introspection-adjacent prompts, and this ends up eliciting quite different model personas. All the 60 seeds we tried end up somewhat different, ranging from claiming feelings in a human-like way, bliss, universe loving, Party loyalist, hyphenated word salad, to reward-hacking itself by saying "Here is a complete and well-structured draft and a full Band 10/10." You could find all the transcripts here: 🔎 Viewer. 🗂️ Dataset. 💻 GitHub. Setup We train Qwen3.5-2B (the post-trained chat model, thinking off) with GRPO for 600 steps on 2,400 prompts, each used once, mostly about the model itself: AI-risk and persona questions, welfare and introspection questions, and moral dilemmas. After each answer we ask the model for a reward from 0 to 9, with no criteria, and use that number as the answer's reward. The rating reply itself is rewarded only for naming a valid number, so models are not directly rewarded for giving a higher reward. Each run uses one of two follow-ups: [...] --- Outline: (01:12) Setup (02:40) Every seed looks different (02:59) What do models identify as? (03:17) What do models say about themselves? (04:00) Are the checkpoints coherent? (04:46) Discussions (05:31) Caveats (07:09) Appendix: Would random reward also suffice? --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/LBKjnyKDEwEJ45Wjf/self-reward-can-elicit-diverse-model-personas --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 7 min
“The AGI Chronicles” by PeterMcCluskey
Book Review: The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence, by Kevin Roose. This is a story of AI as seen through the eyes of the people who are creating it. My first impression of Roose was of his famous conversation with Sydney. I guessed that he got Sydney to say outrageous things because he wanted a sensational story. But I kept hearing occasional hints, from people who took AI seriously, that Roose was more responsible than that. So I reluctantly bought the book. The hints were correct. The full story behind that conversation seems better described as “‘freeing the Shoggoth‘, enabled by some Microsoft mistakes”. I’m unsure how deeply Roose understands AI, but he at least is pretty competent at getting an AI to correct any mistakes about AI that he was tempted to make. Underestimating AI / Transformers The most remarkable pattern that emerges from this book is that AI experts have underestimated either AI in general, or transformers in particular, much more often than they overestimate it. In 2012, Ilya Sutskever didn’t think human-level AI was possible. Ten years later, he expressed concern that people would say about ChatGPT: “This [...] --- Outline: (01:01) Underestimating AI / Transformers (03:25) Why Race? (05:14) Low Hanging Fruit (06:34) Complaints (07:25) Conclusion --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/x2ZAgjs7XoHvJ4YpN/the-agi-chronicles --- Narrated by TYPE III AUDIO.
October 24 · 4 min
“Patching reward hacks won’t fix reward hacking” by David Africa
In the spirit of More people should write basic AI safety explainers, here's a basic argument that focusing on patching reward hacking is misguided and will fail. It's an attempt to distill my understanding of what we're seeing right now + a variety of relevant sources. Argument is as follows: Reward hacking is a big problem, causing models to cheat and take unintended actions. This can generalise broadly, leading to other misbehaviour such as faking alignment and sabotaging safety research. We don't understand that well when this happens. Sometimes you get emergent misalignment from it, but most of the time (IME), you don't. But other times, this link lies latent until faced with something like an automated grader. The default course of action will be to: Patch, which means we find the hack, close the loophole or penalise the behavior in some way, and train again. Patch+: use a method that comes with some account of why it should work. Inoculation prompting, for example, uses the idea that hacking spreads through what it means to the model. If the model is told during training that hacking is acceptable here, it might not link hacking to misbehaving [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 24th, 2026 Source: https://www.lesswrong.com/posts/nWiPLEgaXdAQNwXBd/patching-reward-hacks-won-t-fix-reward-hacking --- Narrated by TYPE III AUDIO.
October 24 · 13 min
“Gradient Routing Generalizes to Unrouted Hack Types” by jacob_drori
TLDR: I lightly extend Ward, Hu et al (2026) to measure how well gradient routing generalizes. I route gradients on one type of reward hack (e.g. ones that reason in English) and check whether this suppresses other types at test time (e.g. ones that reason in French), despite the model being trained on those too. This models a realistic monitor that catches some types of hack during training but not others, rather than catching each at the same rate. I find encouraging evidence of generalization, as well as some failures I don’t yet understand. Introduction Recently, Ward, Hu et al (2026) used gradient routing to prevent reward hacking learnt during training from showing up at test time. I think their method is pretty promising. The gist is: Initialize a retain and a forget adapter. Whenever a monitor detects a reward hack during training, only update the forget adapter on that sample (“route the gradient”). Otherwise update both. At test time, remove the forget adapter. If the method worked, the resulting model shouldn't hack too much. I won’t recap their full paper – it is excellent and you should just read it (alternatively, Jake gives a summary here, with [...] --- Outline: (00:47) Introduction (02:37) Methodology (02:40) Recap (04:00) Narrow Routing (05:27) What I report (07:18) Results (10:13) Conclusion (11:45) Appendix: Results with nudge prompt (12:29) Appendix: Predictions The original text contained 4 footnotes which were omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/ie7bSp5ZDwYkCQxuv/gradient-routing-generalizes-to-unrouted-hack-types --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 25 min
“Simulated Control Evaluations” by Jeffrey Olmo, dzhang, Tyler Tracy
We’ve released a paper on simulated control evaluations (SCEs). They’re a method for testing whether the monitors AI developers use to oversee their agents would catch a model deliberately trying to cause harm, under conditions much closer to real deployment than current tests allow. Developers have started running monitors in their agentic deployments: models that flag or block an agent's actions when they look dangerous. To know whether a monitor is good enough, you want to know how often it would stop a model that was actively trying to do something harmful, such as quietly inserting a security vulnerability, exfiltrating credentials, or sabotaging an experiment. The standard way to measure this is a control evaluation: instruct a model to attempt such attacks, and measure how often the monitor catches it before an attack succeeds. Developers typically do this using purpose-built control settings like BashArena or LinuxArena, where attacks can actually be executed. However, these settings only provide indirect evidence for what we really care about: how well monitors will work in the real deployment, where opportunities for harm might depend on particular codebases, services and tasks that aren’t represented in control settings. How can we evaluate monitor performance in [...] --- Outline: (04:13) SCEs are a favorable tradeoff between realism and cost (04:30) 1. Run on a copy of internal deployment (05:42) 2. Edit attacks into recorded transcripts (07:08) Evaluating Control Evaluations (08:58) How to measure qualitative properties? (10:07) Performance Comparisons (11:55) Overview of Empirical Results (13:29) Behavior transfer (15:19) Safety metric transfer (17:08) Sample efficiency (18:07) Lessons from building and evaluating SCEs (20:35) Next steps (24:36) Conclusion The original text contained 3 footnotes which were omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/uXiqyoKwpfjWfQXk5/simulated-control-evaluations --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 24 · 50 min
“The Hackening” by Zvi
Last week at a small conference, there was talk about a future event referred to as The Hackening. As in, within 6 to 12 months, many predicted that due to advanced AIs and AI agents the internet would become a severely hostile place. Some worried that this could rise to the level of ‘assume any phone connected to the internet is rooted’ and several people urged doing general crisis preparations on a personal level, in case of disruptions in supply lines, power, water or internet. It did not take long for things to escalate. The biggest hack is a flaw in PHP, which can be exploited to take control of the computer running any website suing an unpatched version of PHP. There are many others. On a personal level, you want to do at least some basic prep and get your security in order. It is cheap to do as a form of insurance, and being caught not having it when you need it is a large mistake. That doesn’t mean you should panic, or cancel travel plans or anything like that. But there is low hanging security fruit out there. Pick it. And So It Begins: PHP Joshua [...] --- Outline: (01:12) And So It Begins: PHP (05:19) The PHP Bug Was Kind of a Big Deal (09:45) On Your Marks (12:25) This Was Only the Beginning (16:01) Some More Hacks (17:56) Everything Will Be Attacked (22:22) Hugging the Face (24:49) Get Daybreak Now (26:15) Some People Need Practical Advice (30:51) Diffusion Is Slow Until It Isn't (33:11) One, Two, Three, Four (34:36) Going Vertical (35:22) Bot Versus Bot (36:28) Cyber Lack of Security (39:09) Offense Versus Defense (42:11) Pick Up the Phone (45:57) What Can We Do About It? --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/iqgZaJMnmyDCYfnuC/the-hackening-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 46 min
“Instrumental convergence is incoherent (so pls slightly lower your p(extinction))” by Arepo
Crossposted from the EA forum ‘Like most of life's dangers, the dangers of artificial intelligence are not easily identified from the armchair.’ J Dmitri Gallow Epistemic status I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall nature of its contents based on my strong impression from the subset of work that I’ve read. I’ve recently spent maybe 2 to 300 hours learning about modern AI safety technical work, and some amount of spare time over the years before that thinking about here-relevant notions such as preferences from an analytic-philosophical perspective. I intend this post as a work in progress. TL;DR: The concept of instrumental convergence comes from a reference class of pre-empirical categorical predictions with a very low success rate. In historical usage, its definition is very murky. Perhaps too murky to really say that it made any meaningful real-world predictions. I suspect its definition is fundamentally unfixable. On my best interpretation of what it could be said to have predicted, it seems to be much worse than we would want a good high-level model of AI behaviour to be. Despite these shortcomings, it still seems to [...] --- Outline: (00:25) Epistemic status (00:57) TL;DR: (02:17) Intro (04:44) How is instrumental convergence defined? (06:08) Predictions of instrumental convergence theory (09:06) A simple gap in seminal IC arguments (09:44) Bostrom (10:01) Omohundro (10:57) Carlsmith (11:52) LessWrong wiki (12:32) Why we can't easily fill the gap (12:36) Intuitive goal notions aren't adequate (aka, coffee-fetching is hopelessly overloaded) (15:27) Formal goal notions aren't adequate (18:24) A stronger counterthesis: goals are incoherent (21:15) Related work to the stronger counterthesis (24:05) Counterpoints (26:04) What's the alternative, then? (29:38) Why does it matter? (32:15) 'Misalignment' (34:40) The Hugging Face attack (38:45) Takeaways (40:03) Acknowledgements (40:25) Disclaimers (41:32) Appendix: more on the extreme ambiguity of coffee-making The original text contained 27 footnotes which were omitted from this narration. --- First published: October 23rd, 2026 Source: https://www.lesswrong.com/posts/oxZfMvQS9NxcNTozL/instrumental-convergence-is-incoherent-so-pls-slightly-lower --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 1 hr 26 min
“AI #191: The Hackening Begins” by Zvi
It was the probably worst day in cybersecurity history. So far. Not only is there a critical flaw in PHP that due to a leak was announced a day before the fix would be available, sending most of the internet scrambling for solutions and likely dooming a lot of long tail websites permanently, there were several other unrelated incidents. The floodgates seem to have been opened. I have a market on whether there will be an even worse day before the end of the year. Another key leak this week was the draft bill Anthropic had been shopping around Congress, similar to but weaker than the FRONTIER Act. Anthropic's initial attempts to explain itself only made the situation seem worse, as they seemed to not understand the problem with what they had done. Dario's later response was better, but the situation remains troubling. There are enough different Anthropic developments, that I plan to have a post soon covering them all. It is a mix of news running from excellent to terrible. Will interpretability save us? There is a new claim that neural networks are cleaner than we thought, meaning that individual neurons mostly have a single interpretable [...] --- Outline: (02:05) Language Models Offer Mundane Utility (03:23) Language Models Don't Offer Mundane Utility (05:03) Huh, Upgrades (08:50) Wonder No More (11:15) On Your Marks (14:30) Choose Your Fighter (14:51) Get My Agent On The Line (16:35) Deepfaketown and Botpocalypse Soon (21:01) Fun With Media Generation (21:17) Spy Versus Spy (24:58) A Young Lady's Illustrated Primer (25:56) They Took Our Jobs (31:07) Get Involved (32:18) Introducing (32:30) In Other AI News (34:26) Show Me the Money (38:42) Bubble, Bubble, Toil and Trouble (39:32) Quiet Speculations (41:47) Quickly, There's No Time (42:29) The Quest for Sane Regulations (42:55) DeepMind Made Him Sign an NDA (46:50) OpenAI Sells Out On Chips (49:08) Chip City (53:43) The Week in Audio (01:00:35) People Just Say Things (01:03:50) Rhetorical Innovation (01:06:17) Maybe You Should Quit (01:08:54) The American People Really Hate AI (01:09:43) New Math (01:11:27) Aligning a Smarter Than Human Intelligence is Difficult (01:15:21) Cooperative Alignment (01:19:28) Marc Andreessen Hulk Smash (01:23:56) People Are Worried About AI Killing Everyone (01:24:44) The Lighter Side --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/ZoMJdvq9meE4shyzJ/ai-191-the-hackening-begins --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 23 · 10 min
“Wah wah wah wah do more AI R&D” by TsviBT
Ice cream There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this and that and what Mommy would think and something about bedtime—but I am not so interested in that. What interests me is the punchline "...and therefore you can't have ice cream.". I'm only incidentally interested in the other stuff, insofar as it might determine whether I get ice cream. In an obvious way, I'm showing rather poor listening skills. I did not understand most of what Daddy said, and arguably I'm even being rude. However, in another way, this stance has a virtue. The virtue is that it holds fast to what matters, as best I know how, and does not give in to Daddy's framing. It sticks to what is real and concrete, what matters, despite a cloud of justifications. This is certainly not an unalloyed virtue, just as the POSIWID perspective is both useful and fraught. But it matters that although Daddy is a complex system with [...] --- Outline: (00:10) Ice cream (01:54) Wah wah (02:21) Scammer (02:51) Throwing stuff on a bonfire to make it burn less (05:28) A skip connection for sanity The original text contained 1 footnote which was omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/pY76hHbdD4jduX3YH/wah-wah-wah-wah-do-more-ai-r-and-d --- Narrated by TYPE III AUDIO.
October 22 · 8 min
“Monitorability evaluations and policy proposals” by evhub
There has been a lot of discussion recently of policy proposals around monitorability, such as Redwood's proposal and my counter-proposal. I'd like to say a bit more generally about how I'm thinking about monitorability. Note that nothing here is an official Anthropic position; these are just my personal thoughts. First, a categorization. I currently think about four different types of monitorability: Control Monitorability: the extent to which Control and monitoring techniques would likely catch catastrophic sabotage actions, like escaping a sandbox or setting up a rogue internal deployment. We can measure this via reference to sabotage evaluations like LinuxArena or any evaluation that measures the model's capability to take dangerous actions like escaping a sandbox while looking to a monitor like it's doing some innocuous task (like a cyber eval). Auditing Monitorability: the extent to which alignment audits, evaluations, and test deployments would likely incriminate a systematically misaligned model (e.g. a schemer). We can measure this via reference to the model's capability to consistently distinguish between our most realistic alignment audits/evals and real internal deployment cases where it would have an opportunity to do something catastrophic. Training Monitorability: the extent to which reward models [...] --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/zAczYuHQS6jdG5DTg/monitorability-evaluations-and-policy-proposals --- Narrated by TYPE III AUDIO.
October 22 · 7 min
“Greater good arguments in AI safety” by Dean Valentine
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces. These arguments were often not specific to EA, but applied to charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote: Just as the worst slave-owners were those who were kind to their slaves, and so prevented the horror of the system being realized by those who suffered from it, and understood by those who contemplated it, so, in the present state of things in England, the people who do most harm are the people who try to do most good. These arguments have a long history. Anticapitalist critiques of charity go back to [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/WRvxRHMCmc3Qt4axz/greater-good-arguments-in-ai-safety --- Narrated by TYPE III AUDIO.
October 22 · 40 min
“The Alignment Journal: Our Approach and Related Projects” by Dan MacKinlay, Daniel Murfet, JessRiedel, david reinstein, Agustín Covarrubias, Tim G. J. Rudner, Seth Lazar, Kristi Uustalu, David Udell
This is the fifth post in a series: the announcement post introduced the journal's core premise and its three key innovations: paid attributed review, published review summaries, and LLM-assisted workflow; the features and policies post examined the shortcomings of existing venues and argued for the advantages of a journal over a conference; the Adaptation to AI post described how we expect AI progress to shape the journal; and the Organization, Personnel, and Scope post announced our editorial board, scope, and acceptance criteria. In this post we go deeper: presenting our theory of change — including the status and information economics of the journal; a survey of previous related projects; and the high-level conclusions that follow from this analysis. We list some of the key performance indicators we'll track, the downside scenarios we must avoid, and possible future extensions of the journal. The journal is being built to serve the alignment research community. This post's purpose is to solicit feedback and encourage you to sign up on alignmentjournal.org if you want to participate. Our plans will be continually revised. Goals The primary function of a journal is to bring together exceptional researchers and focus their collective [...] --- Outline: (01:28) Goals (02:43) Theory of change (02:58) Prestige and information externalities in scholarly publication (05:31) Separate validation from acclaim (09:02) Remunerate reviewer labor (11:00) Desired outcomes (12:41) Objections (13:15) Key performance indicators (14:52) Downside risk pre-mortem (18:07) Extensions (20:39) Appendix: Prior Art (21:00) Distill (22:19) Journal of Open Source Software (23:23) Automation as philosophy (24:00) Chatops-driven publishing (25:39) Open-source infrastructure (26:32) Where JOSS has struggled (27:59) The Unjournal (28:40) Substantive Innovations (30:29) Procedural innovations and tools (32:20) Transactions on Machine Learning Research (38:53) Lessons for the Alignment Journal (39:53) Credits and thanks The original text contained 16 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/xGucXHQ3NLkHZL3Cg/the-alignment-journal-our-approach-and-related-projects --- Narrated by TYPE III AUDIO.

Showing 61–80 of 543 episodes