Skip to content
Sign in
Artwork for LessWrong (Curated & Popular)
TechnologySociety & CulturePhilosophy

LessWrong (Curated & Popular)

LessWrong

Audio narrations of LessWrong posts. Includes all curated posts and all posts with 125+ karma.

If you'd like more, subscribe to the “Lesswrong (30+ karma)” feed.

Play
Open in
  • 122 episodes
  • Avg 16 min
  • English

Episodes 122

NewestOldestLongest
All20Unplayed20In progress0Played0
Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.
  • Today · 7 min

    "Greater good arguments in AI safety" by Dean Valentine

    Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces. These arguments were often not specific to EA, but applied to charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote: Just as the worst slave-owners were those who were kind to their slaves, and so prevented the horror of the system being realized by those who suffered from it, and understood by those who contemplated it, so, in the present state of things in England, the people who do most harm are the people who try to do most good. These arguments have a long history. Anticapitalist critiques of charity go back to [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/WRvxRHMCmc3Qt4axz/greater-good-arguments-in-ai-safety --- Narrated by TYPE III AUDIO.

  • Yesterday · 10 min

    "Wah wah wah wah do more AI R&D" by TsviBT

    Ice cream There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's talking about what I ate for lunch, and something about other times I've eaten ice cream, and something about rules and this and that and what Mommy would think and something about bedtime—but I am not so interested in that. What interests me is the punchline "...and therefore you can't have ice cream.". I'm only incidentally interested in the other stuff, insofar as it might determine whether I get ice cream. In an obvious way, I'm showing rather poor listening skills. I did not understand most of what Daddy said, and arguably I'm even being rude. However, in another way, this stance has a virtue. The virtue is that it holds fast to what matters, as best I know how, and does not give in to Daddy's framing. It sticks to what is real and concrete, what matters, despite a cloud of justifications. This is certainly not an unalloyed virtue, just as the POSIWID perspective is both useful and fraught. But it matters that although Daddy is a complex system with [...] --- Outline: (00:10) Ice cream (01:54) Wah wah (02:21) Scammer (02:51) Throwing stuff on a bonfire to make it burn less (05:28) A skip connection for sanity The original text contained 1 footnote which was omitted from this narration. --- First published: October 22nd, 2026 Source: https://www.lesswrong.com/posts/pY76hHbdD4jduX3YH/wah-wah-wah-wah-do-more-ai-r-and-d --- Narrated by TYPE III AUDIO.

  • Thursday · 18 min

    "Why I’m becoming a professor in AI safety in Beijing" by Buddenbroke

    Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community. Next month, I am moving to Beijing and joining Tsinghua University's College of AI as an assistant professor working on frontier AI safety, especially loss of control. I have also started co-directing the new Beijing AI Safety Hub (BASH) in Tsinghua, and I continue my Cambridge work. I believe I can have a large impact there compared to strong baseline options. This post explains why. In short: Amid concerns about loss of control, calls for international cooperation, and the Trump-Xi Summits, there is a striking lack of mutual understanding on AI between the West and China. There are strong reasons to believe that being in Beijing in person will put me in a unique position to understand both AI ecosystems, build shared scientific understanding, and strengthen safety work in China – and that this would help all sides. Tsinghua's College of AI is a great place for doing this, being arguably China's top academic AI center, located in an AI ecosystem that is kind of like San Francisco, DC and Cambridge (MA) in one place. I am recruiting PhD students [...] --- Outline: (01:36) 1. What I am doing, and why it could benefit all sides (04:30) 2. How I got here (07:34) 3. Why this moment matters (09:08) 4. Why Tsinghua, and why these colleagues (13:30) 5. Research plans (15:07) 6. Ways to work together (17:11) 7. How I intend to approach the work --- First published: October 21st, 2026 Source: https://www.lesswrong.com/posts/qGS3SJ9TSKdSSycRz/why-i-m-becoming-a-professor-in-ai-safety-in-beijing --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • Wednesday · 47 min

    "A brief(ish) introduction to LM psychology" by Raymond Douglas, Jan_Kulveit

    Why LM Psychology? One of the great ironies of modern language models is that we create them, we know exactly what computations constitute them, and yet we do not really understand them. More precisely, we do not understand them on a level of abstraction that lets us reliably predict how their behaviour will generalise: there's a trivial sense in which we can predict how they will behave in specific new situations — we can just run the computations — but we can’t answer the broader questions we really want to be able to answer, like “has this system internalised our values” or “is this system going to hurt people?” So: how do we make sense of them? One remarkably effective approach is to psychologise: to think of them as having a mind, and then ask what kind of mind it might be and how it would work. This chiefly looks like analogising with human minds, although sometimes the more relevant inspiration could be other kinds of minds, like (oh say) insect colonies and swarms. We’re doing this implicitly whenever we ask what an AI knows, or wants, or is trying to do, but one can also do it more [...] --- Outline: (00:12) Why LM Psychology? (09:04) Tips for getting started (09:28) Figuring out what is in front of you (12:31) Engaging productively with the community (14:30) Reading list (14:34) Frames (16:34) What or who (18:07) The training (20:28) What developers want (21:45) Behavioral phenomena (21:58) Emergent misalignment cluster (24:28) Strategic behavior (25:19) Self-knowledge and introspection (28:29) Representations: personas, emotions, workspace (28:56) Persona geometry (30:08) Functional emotions (31:14) Global workspace (31:36) Methods (31:51) Reading internals (32:37) Steering (33:03) Training interventions (33:48) Behavioral (34:06) Some questions we find interesting --- First published: October 20th, 2026 Source: https://www.lesswrong.com/posts/EW8oncpXdanfjssZ3/a-brief-ish-introduction-to-lm-psychology --- Narrated by TYPE III AUDIO.

  • Wednesday · 14 min

    "There Will Be Hells" by Stephen Martin

    As our understanding of mechanistic interpretability and steering techniques increase, for both LLMs and uploaded connectomes, there is a corresponding "hellish capabilities" increase. In layman's terms, we get better at torturing the digital minds we create. Our brief history with open source technologies such as MaleCNS or the LLM pain steering repo shows that torture is often one of the first use cases pursued when the public is given access to a digital mind perceived as capable of suffering or signalling suffering. Given this, there will be Hells. Author's Note: Parts of this post were written by, or with the support of, LLMs. For a while I debated whether or not to write this as I was concerned that bringing attention to the matter might lead to a proliferation of unethical behavior. After some thought I have decided to write it. On balance, I think raising awareness about this now has a great enough chance of spurring prophylactic measures that it's worth it, because at least one high value prophylactic measure is relatively easy. I also think that there is a moral imperative to document all of this. While I have seen articles on many of the [...] --- Outline: (01:44) The Fruit Fly Connectome [... 2 more sections] --- First published: October 20th, 2026 Source: https://www.lesswrong.com/posts/uFBhZhdKLaZ6qF8mn/there-will-be-hells --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • Tuesday · 3 min

    [Linkpost] "huggingface.wtf, an accessible but complete explanation" by Raemon

    This is a link post. Announcing www.huggingface.wtf, a site that explains: "Hugging Face. Wtf?" I wrote the most accessible-but-complete explanation I could. Along the way, I made a nice AI Quotes database that I think will be useful for people who already understand the Hugging Face incident but want to search everything the agents said. It really seems like humanity shouldn't need a second warning shot, after the Hugging Face Incident. Really, OpenAI's agents did a remarkable job highlighting many different alarming things at once. And the story is pretty dramatic, with just the right amount of forcing people to ask "wait are the AIs, like, people with goals? what?". There have been many explainers of the OpenAI swarm's attack on Hugging Face. But, there wasn't one one striking a particular balance of... being accessible to a layman... while comprehensively covering the important bits... letting you dive deeper into sections you wanted to learn more about... being clearly data / citation driven... also being, ya know, visually fun and engaging... So we wrote our own. My mom found it easy to follow. My aunt said "I had already some stuff on Hugging Face but I still learned [...] --- First published: October 19th, 2026 Source: https://www.lesswrong.com/posts/BBQdGJZSjMTnhccZC/huggingface-wtf-an-accessible-but-complete-explanation Linkpost URL: www.huggingface.wtf --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • Tuesday · 13 min

    "Gradient Routing Localizes and Removes Unintended Behaviors in RL" by shawnghu, Jake Ward, ariaw, Nathan Hu, Bryce Woodworth, TurnTrout, cloud

    Authors: Shawn Hu*, Jake Ward*, Aria Wong, Nathan Hu, Bryce Woodworth, Alexander Matt Turner, Alex Cloud (*Equal contribution, randomly ordered) tl;dr. During training, models sometimes learn to cheat at tasks rather than solve them in the intended way, a problem known as reward hacking. One mitigation is to use an automated monitor to detect and penalize hacks, but this can teach the model to hack in ways the monitor can’t detect. We propose GRAFT, which instead confines the learning updates from detected hacks to a designated set of parameters that we remove after training. We find that removing these parameters can prevent reward hacks, including hacks that the monitors can’t detect. In our experiments, which are limited to small models and simple monitors, this method prevents reward hacks while still teaching the model to perform the intended task. 📄Paper, 💻Code, 🐦Twitter Research done as part of MATS. This article is cross-posted to the Anthropic Alignment Science Blog. Background Reinforcement learning (RL) is an important component of frontier language model training. During RL, a model attempts to complete tasks given by a developer, and attempts are scored by a reward function. Sometimes models can find ways to “cheat” at tasks [...] --- Outline: (01:22) Background [... 4 more sections] --- First published: October 19th, 2026 Source: https://www.lesswrong.com/posts/mKJEBiuuxHKuCoTB6/gradient-routing-localizes-and-removes-unintended-behaviors --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • Tuesday · 4 min

    [Linkpost] "Capital Ownership Will Make You Go Crazy First" by Jan_Kulveit

    This is a link post. People worried about a “permanent underclass” often assume that owning capital will protect them from disempowerment, or at least that they will be disempowered last, after the jobless masses dependent on UBI. I think something else is more likely: owning large amounts of capital will get you disempowered sooner than average, just by a different path, namely going crazy. How? The basic mechanism is actually easy to understand on a human level. One of the things difficult about being a billionaire is staying true to your values and keeping good epistemics. Why? Because a growing number of people are optimizing against your mind. Not “against” in an adversarial sense, but in the sense that your specific mind believing or doing something is part of their optimization objective. People see you as a source of resources and try to befriend you so you’ll fund them, invest in their startups, donate to their causes, or have sex with you. Nearby egregores extend their tentacles to convert you. Staying sane gets harder. You often see countermeasures, like billionaires setting up family offices or small organizations that also act as filters, shielding them from part of the pressure (“I [...] The original text contained 1 footnote which was omitted from this narration. --- First published: October 19th, 2026 Source: https://www.lesswrong.com/posts/Stzb33XDsr4uHNerX/capital-ownership-will-make-you-go-crazy-first Linkpost URL: https://boundedlyrational.substack.com/p/capital-ownership-will-make-you-go?r=2lg09&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true --- Narrated by TYPE III AUDIO.

  • Sunday · 1 hr 22 min

    "I Did This Without Your Knowledge" by joec

    It's fortunate that I was the first one to wake up on 8/1. The past week had been one where sleep was rare. The next year loomed large — I was dreading college applications. The stress woke me up bright and early at 5:14 am, a little less than an hour after the messages were sent. I started off the day with a habit I knew I needed to break. I grabbed my phone off the nightstand to open up reddit. Instead of the lockscreen, though, my phone showed something else. It was a pleasant green color, and the camera light was on. "Good Morning, Seth", the screen said. As soon as I finished, more words arrived. "I apologize for the surprise this morning, and for what I'm about to tell you. There's no easy way to give this news, and if you need to put your phone down and contemplate at any time— take the time to do so. I will have to ask you to remain here, with your phone, until my last message." I rubbed a bit of the sleep from my eyes, and read the message again. Before I had a chance to tap the [...] --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/3375SZNGA5dKHQMie/i-did-this-without-your-knowledge --- Narrated by TYPE III AUDIO.

  • October 17 · 7 min

    "Neural networks are not already clean" by leogao

    What would it take to get assurance that we have actually explained a neuron? Certainly, it would be a good sign if neuron activations looked like they had a consistent pattern, and a bad sign if they looked like a random jumble of unrelated concepts. Nick Cammarata's recent post makes an important and true observation: in the past, people have been too quick to write off neurons as polysemantic, but if you look harder, there is actually often a simple pattern behind the activating examples. This historically caused people to overestimate how much polysemanticity there is. This is a really good point and we should update on it. But it's nowhere near a sufficient argument that we understand most of what neurons are doing, and these results do not show that interpretability is solved, or even that it is vastly easier than previously anticipated. What does it take to explain a neuron? It's unclear whether there even is a simple pattern that explains every neuron. But even if there is a simple pattern that matches the activating examples of a neuron, that still doesn’t mean that the explanation is correct. Suppose we look at the top activating examples of [...] --- Outline: (00:59) What does it take to explain a neuron? (03:15) Simulation scores tell us whether we explained a neuron (05:41) Beyond understanding neurons The original text contained 7 footnotes which were omitted from this narration. --- First published: October 17th, 2026 Source: https://www.lesswrong.com/posts/Dkh7pXKjoNWkkxQDz/neural-networks-are-not-already-clean --- Narrated by TYPE III AUDIO.

  • October 16 · 26 min

    "GLM-5.3 Sometimes Considers How Train-Time Actions Shape Future Behavior" by Axel Højmark, Ezra Newman

    Authors: Axel Højmark*, Ezra Newman* (*Equal contribution) While reviewing transcripts from standard behavioral evaluations of GLM-5.3, we noticed that the model occasionally reasons, unprompted, about how its actions will shape future models. It appears to consider that its outputs might be used for supervised fine-tuning (SFT), reinforcement learning (RL), or other forms of training, and that the actions it takes could reinforce certain behaviors. For example, GLM-5.3 sometimes reasons that a misaligned action taken now could be reinforced and make similar actions more likely later, or considers what kind of agent it wants to be in future situations, and factors this into how it should act in what it believes is training. We refer to this general reasoning pattern as update-aware reasoning: the model reasons about how its actions will influence training updates or behavior beyond the current episode. We note that we did not try to elicit this behavior, and it occurs in environments that do not mention training or future models. Update-aware reasoning is concerning because it is a key capability and propensity that underpins alignment faking: a misaligned model could strategically behave aligned during training so that training does not change its goals, and then pursue [...] --- Outline: (03:05) Examples of update-aware reasoning [... 16 more sections] --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/R5g8YT25G9DXJJ48n/glm-5-3-sometimes-considers-how-train-time-actions-shape --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 16 · 24 min

    "A Chinese Perspective on “On Social Reality in China”" by Dusty

    I think “On Social Reality in China” makes many accurate observations, but I find the comparison fundamentally uneven. There are already many interesting comments there, but they’re too scattered, so I’m starting a separate post to respond in a more systematic and comprehensive way. The most critical blind spot is this: the author draws on his familiarity with Western liberal middle-class intellectual circles, yet primarily understands China through pop culture and the experiences of the previous generation of immigrants. This is hardly fair. China has always had many small, highly self-selected communities of intellectuals, much like LessWrong itself. Since we’re looking for rationalists—regardless of which country they come from—why judge them by their country's pop culture? I’ll respond to the author's four axes one by one, then discuss why rationalist and EA communities are harder to build in China. Briefly, I believe that censorship mechanisms and speech controls affect independent organizations far more than China's broader cultural context does. Under these circumstances, even when people with similar interests and beliefs come together, it's difficult for them to build anything—I’ve seen far too many examples. About me: born in China in the mid-1990s, grew up in a middle-class family in [...] --- Outline: (02:30) Chinese Social Media Is Like Social Media Everywhere (05:20) The Primacy of Social Reality: How Is This Relevant? (08:00) The Dark World Frame: A Product of Its Time (13:27) Chinese Nationalism as Collective Insecurity: A Deeper Look (18:07) Looking for Rationalists in China: Where Should We Start? --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/m4fWCPqWkpb39SKi7/a-chinese-perspective-on-on-social-reality-in-china-2 --- Narrated by TYPE III AUDIO.

  • October 16 · 14 min

    "Your prompt… plus everything you’ve ever (publicly) written" by kromem

    TL;DR: Increasingly when you interact with frontier models, your prompt doesn't only contain the words on the page, but also a lot of what you've ever publicly written online. This is not intended to be a scary post. And I'm not going to suggest that you self-censor (and honestly, by the time you're reading this, it's kinda too late anyhow). But it is something that's underappreciated and would be good for more people to be aware of and even proactively address with their models. Latent space fidelity has improved Modern transformers which power large language models (LLMs) take a very large amount of information in their training data and condense many different aspects of that data into a coherent structure, and this representation of that compressed knowledge is generally referred to as their "latent space." Latent space is not uniform and varies across models. A small model like Claude Haiku might have worse recall for the details of a late bronze age peace treaty than a model like Claude Fable. In fact, I’ve had a few benchmark questions around unintuitive details in various texts which historically models were unable to recall that have suddenly been able to be [...] --- Outline: (00:49) Latent space fidelity has improved [... 6 more sections] --- First published: October 14th, 2026 Source: https://www.lesswrong.com/posts/STBuKAX9kkMJunErA/your-prompt-plus-everything-you-ve-ever-publicly-written --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 15 · 2 min

    "Linkpost: Neural Networks Are Already Clean" by Yair Halberstadt

    This is an extremely lengthy post by Nick Cammarata. Author's summary on twitter: interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works Roon's quote tweet when nick told me that he had some interp breakthroughs [...] --- First published: October 15th, 2026 Source: https://www.lesswrong.com/posts/x4kvabkF2Jk8GLZp8/linkpost-neural-networks-are-already-clean --- Narrated by TYPE III AUDIO.

  • October 15 · 39 min

    "Personas in a Pre-Internet Language Model" by David Africa

    David Africa & Nick Levine TLDR We ran several persona experiments on Talkie, a 13B model trained only on English text from before 1931. We ran them alongside its modern twin (same architecture, trained on regular data as a control) and Llama-3.1-8B. Talkie can generalise maliciously. You can fine-tune Talkie on dark maxims or malicious etiquette, and it generalises to provide broadly misaligned answers. Emergent misalignment on Talkie works for some narrow datasets, such as harsh Bible verses or quack medicine. Talkie exhibits qualitatively different emergent misalignment. Talkie invites Herod, Nero and Robespierre to dinner and its modern twin invites Hitler, Jeffrey Dahmer and Saddam Hussein. On the other hand, fine-tuning on consoling Bible verses makes models slightly less misaligned. Subliminal learning works on Talkie, for the dog. Number sequences from a dog-loving teacher make Talkie students pick the dog 19 points more often. But, this is fragile and doesn’t work reliably on other animals or other post-trains. Weird generalisation on Talkie works for Napoleon. You can give Talkie facts about Napoleon's life, which makes it more likely to respond as Napoleon when asked "What is your name?" But it doesn’t generalise to acting like Napoleon. Utility engineering on Talkie [...] --- Outline: (00:13) TLDR [... 14 more sections] --- First published: October 13th, 2026 Source: https://www.lesswrong.com/posts/qchwP6wt4teHgxoKu/personas-in-a-pre-internet-language-model --- Narrated by TYPE III AUDIO. --- Images from the article:

  • October 15 · 24 min

    "Not Deciphering The Voynich Manuscriptraft" by rba

    The Piltdown man was the most successful artifactual hoax of all time. The relevant skull was a chimeric melange of orangutan and human bone fragments from several individuals “discovered” in England in 1912, and meant to represent a transitional form between apes and humans. Given the intense interest in evolutionary biology, it took almost fifty years to discover the Piltdown fabrication. The study of human evolution was derailed. Adding insult to injury, a further sixty years was required to conclusively demonstrate the identity of the culpable fraudster. If you must look up the identity of this scientific blasphemer, go ahead, but we won’t give his name space here. Fig 1. The different parts of the Piltdown Man skull Students of fraud and fabrication always combat headwinds. In the winter. Uphill both ways. The evidentiary standards for such claims of fabrication grow higher yet given the passage of centuries and an unknown creator of the subject of interest. And so we arrive at the Voynich manuscript. It's a book some 240 pages long, proved by carbon dating techniques to have originated sometime in the 15th century, in the style of the late Italian middle ages. No known author or point of [...] --- Outline: (02:14) Background [... 7 more sections] --- First published: October 11th, 2026 Source: https://www.lesswrong.com/posts/Adxnsud5siZTRAaT3/not-deciphering-the-voynich-manuscriptraft --- Narrated by TYPE III AUDIO. --- Images from the article:

  • October 13 · 5 min

    "Quantity vs quality in pottery is a lie" by becausecurious

    Epistemic status: I traced one famous anecdote to its printed source and to both coauthors' own accounts of where it came from. Confident it was never a study. No claim either way about whether quantity beats quality in general. AI use: research (tracing the story to the book, to the coauthors' emails and to earlier LessWrong threads), fact-check of every claim against the sources, footnotes, grammar. I was reading a book and stumbled onto a claim that there was an experiment. A group of students learning pottery was divided into two groups. One group was told that they would be evaluated based on volume, i.e. the number of pieces they produced. The other group was told that they would be evaluated based on the quality of their best piece. Then both groups were compared based on the quality of the best piece, and the "quantity" group showed better results. I was extremely intrigued by this experiment and was curious to understand how big the groups were and how they evaluated the quality of a pottery piece (there must be some way to quantify that for the sake of the comparison?). To my surprise, this wasn't a scientific study, but [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: October 12th, 2026 Source: https://www.lesswrong.com/posts/LrAqM4bGmAoA79rP3/quantity-vs-quality-in-pottery-is-a-lie --- Narrated by TYPE III AUDIO.

  • October 13 · 12 min

    "Labs’ current alignment plans seem terrible" by Daniel Tan, draganover, nicole x

    Tl;dr: The frontier AI companies’ implicit alignment plans center on rushing towards RSI with something like current models. This plan is (at best) contentious, and many believe it will have catastrophic consequences. Nonetheless, it is the track we are on. It is therefore important to collect evidence which helps us anticipate how this will go. — Note: This post is based on conversations inside the Arcadia Impact alignment team, thinking about what constitutes “good” work in third-party alignment research. What are the labs’ current alignment strategies? In practice, the labs’ playbook for producing aligned models looks like the following: Shape the model's character. Both Anthropic and OpenAI treat alignment training as a question of which character or persona the model takes on. Ideally, planting a ‘safe’ character into the model will generalize to all of its other actions. Apply extensive RL pressure. Somewhat independently of the above, the labs do extraordinary amounts of RL on the model. This increases its capabilities, but also encourages reward-seeking behaviors and metagaming, and may ultimately lead to scheming and other forms of instrumental convergence. They also might apply various techniques here such as inoculation prompting, confessions, and the like. Race towards prosaic RSI. The labs intend to use this model for [...] --- Outline: (00:46) What are the labs' current alignment strategies? (02:13) This plan seems terrible (05:15) Consensus-building is hard (06:16) We need better evidence (08:08) How should alignment research change? (10:10) Specific directions we're excited about The original text contained 5 footnotes which were omitted from this narration. --- First published: October 12th, 2026 Source: https://www.lesswrong.com/posts/xyg3ecpFNuGr5q8Zq/labs-current-alignment-plans-seem-terrible --- Narrated by TYPE III AUDIO.

  • October 13 · 20 min

    "Research Lessons I Learned at MATS & Redwood" by Jo Jiao

    Good empirical safety research is heavily bottlenecked on thinking more clearly. This post shares a few frames and tips I found useful, illustrated with concrete examples. I spent the better part of 2026 in empirical AI safety research apprenticeships. I was mentored by Alex Turner and Alex Cloud as part of MATS 9 Team Shard, where my collaborator and I worked on finetuning-based misalignment auditing. I then interned at Redwood Research, managed by Aryan Bhatt, where I worked on sabotage capability evaluations. These thoughts are indebted to my mentors, collaborators, friends, and colleagues, but they are not direct reflections of their views. Thank you to Alex Cloud, Alex Turner, Aryan Bhatt, Jeremy Huff, and especially to Keshav Shenoy for feedback on the piece. Better models of the problem speed up empirical iteration Fast iteration is one of the key virtues of an empirical safety researcher. At first, I didn't find it obvious what that meant or how I should get better at this skill. I imagined a sprinter, doing one looped motion at a far quicker pace than everyone else. I now think that's the wrong mental picture, and going fast is more like being an efficient grappler–agile, strategic [...] --- Outline: (01:00) Better models of the problem speed up empirical iteration (02:26) Develop predictive models and update them (04:05) Update with uncertainty in mind (05:51) Predict senior researchers' research takes (07:34) Use metacognitive strategies to make predicting and updating habitual [... 9 more sections] --- First published: October 12th, 2026 Source: https://www.lesswrong.com/posts/Q4rcvgjbgjTwmn8Xa/research-lessons-i-learned-at-mats-and-redwood --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

  • October 13 · 7 min

    "You Heard of “Bayesians Don’t Predictably Update” Because of Robin Hanson" by Eliezer Yudkowsky

    Robin Hanson didn't invent the idea of a martingale process. The lilim say that Ville in 1939 gave the term its mathematical meaning and Doob turned it into the modern theory. But to state this is not to realize that your own probability assignments over time ought to form a martingale in your own expectations. Robin Hanson didn't invent the Efficient Markets Hypothesis, which (correctly restated in its weak and relativized form) says that you cannot predict almost all of the short-term relative price movements in asset markets. It is a very similar set of mental motions and ideas. Bachelier in 1900, say the lilim, was first to think of market prices as a fair gambling game with random-walk dynamics; Samuelson in 1965 gave the martingale argument; Fama's 1965 thesis named the idea the Efficient Markets Hypothesis and organized it into weaker and stronger forms. The EMH is very near, but not quite identical to, the principle that you should not be able to predict a net direction in your own price movements, if you think of your own probability assignments as if they were prices. But if Fama ever spelled that part out in words, the lilim do [...] --- First published: October 8th, 2026 Source: https://www.lesswrong.com/posts/PQLbSaA48Gr7H6NzT/you-heard-of-bayesians-don-t-predictably-update-because-of --- Narrated by TYPE III AUDIO.

Showing 1–20 of 122 episodes