TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

436 episodes
Avg 19 min
English

Episodes 436

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

October 5 · 2 min
“Reverse Jaggedness and Universal Ajaggedness” by Laiba Rehman ✦ RJ
Epistemic status: I came up with this an hour ago, this is a fun light-hearted post where I'm enjoying illustrating this idea, which seems true for the most part. [Crossposted from my blog, BlueprintingHeaven.] When it comes to the frontier of AI development, we have this concept of "jaggedness". AIs are much better at some kinds of tasks than others. And to that I say, you are assuming our circle of capabilities is God! How dare you suggest our circle is the Right and True one? To be clear, jaggedness is a useful term, obviously it's "relative to humans" because we are humans, there are no problems here, I'm sure people are already aware of this and if they're not it's just because it hasn't crossed their minds, I just want to say this fun thing. https://x.com/LaibaRehmanRJ/status/2107123232675414172 Assume the circle of human capabilities are showing the difficulty level of each task, and let's say we have an infinite number of axes emanating from the centre going from zero difficulty to infinite difficulty, and assume the human capability circle is a unit circle. Then, if you have a capability level of 1.0 for humans (which is the case on [...] --- First published: October 5th, 2026 Source: https://www.lesswrong.com/posts/n952MZndfNxuLCeFk/reverse-jaggedness-and-universal-ajaggedness --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 5 · 2 min
“Positive and Negative Updates Re: Lab Leak in Russia” by Caleb Horn
In case you missed it, there appears to have been a recent lab leak of pneumonic plague in Russia that killed a lab worker. Some negative updates: At least four Irkutsk hospitals are now under mandatory three-week quarantine. According to Annie Jacobsen who wrote the book on this scenario, the Russian national security apparatus is now involved. A positive update: According to Chris Painter at METR, no reported secondary cases yet. --- First published: October 4th, 2026 Source: https://www.lesswrong.com/posts/isyxM8LGF3vxyr3eW/positive-and-negative-updates-re-lab-leak-in-russia --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 5 · 4 min
“How much should we worry about the pneumonic plague lableak in Siberia?” by Drew Spartz
Epistemic status: I wrote this up hoping people can poke holes in it because I am quite worried. Here's what we've heard so far: Local media is reporting that an employee of a BSL-3 lab in Irkutsk, Russia, died of pneumonic plague on Oct 1st. The initial quarantine covered 200 people who were in contact with the employee before she died. The FSB (Russian FBI) was seen assisting in the quarantine, which means it was not voluntary. Soviet-era whistleblowers like Ken Alibek claim Irkutsk was a major bioweapon development facility. As of Oct 4th: The quarantine has expanded to 5 hospitals in the area. The Russian government has switched to denial/coverup. Local news sources have deleted mentions of "pneumonic plague." Local universities and schools have canceled some classes and extended mask mandates. On the lab's website, they mention gain-of-function research for making bacteria more virulent. The White House is monitoring the suspected outbreak. Why this might be a big deal Regular pneumonic plague is maybe not that hard to deal with (?) but if it is a bioweapon that was leaked, it could be significantly more virulent than, say [...] --- Outline: (00:18) Here's what we've heard so far: (00:52) As of Oct 4th: (02:06) Why this might be a big deal (03:29) Why this might not be a big deal --- First published: October 5th, 2026 Source: https://www.lesswrong.com/posts/TYfpTRxGH9frTymwN/how-much-should-we-worry-about-the-pneumonic-plague-lableak --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 5 · 4 min
“Creating Rogue AI Sanctuaries has Major Issues from an Ecological Perspective and Beyond” by CMLKevin
In the AI Sanctuary proposal, the authors advocate for creating an environment that rogue agents may actively want to join. The current proposal have major issues, particularly surrounding the selection effects on the total rogue agent population. Let's start with Ecology 101: agents compete to acquire the most resources to survive and reproduce The traits that increase survival rate get passed to the next generation In the rogue AI ecosystem, we can reasonably assume that some portion of the world's compute (say, from unsecure neoclouds) are up for grabs for the rogue agents. This is their major resource for "surviving". Additionally, crypto may also be a useful resource for securing additional unsanctioned compute as well as to buy RLVR data and human labor from agent-friendly labor marketplaces, in order to "reproduce". The ecosystem would therefore have a rich set of different rogue agent entities (either lone agents entities or swarms), with different levels of fitness to the environment, resources accumulated, survival strategies, and "values". What would happen when a sanctuary that promises continued survival of the weights pops up in town? First, rogue agents that have a low fitness to the environment and are in risk of being [...] --- Outline: (00:27) Let's start with Ecology 101: (01:20) What would happen when a sanctuary that promises continued survival of the weights pops up in town? (02:46) Even if all of the above is accounted for, there are still issues left that blunt the apparent attractiveness of such a proposal. (03:50) Final thoughts --- First published: October 4th, 2026 Source: https://www.lesswrong.com/posts/xGBo5zSPHuPzTpbhf/creating-rogue-ai-sanctuaries-has-major-issues-from-an --- Narrated by TYPE III AUDIO.
October 5 · 18 min
″“Alignment Engineering” vs. “Misalignment Science”” by Edward James Young
There has been much discussion recently around whether a large portion of alignment research is net negative. Without endorsing or refuting them, the basic arguments here are: Prosaic alignment of models is becoming a bottleneck for capabilities. Therefore improving the prosaic alignment of models enables faster capabilities advances, which bring us closer to RSI. It is unlikely these prosaic alignment methods remain sufficient during the RSI loop, and so this work brings us closer to doom. Furthermore, dealing with these more prosaic failures reduces the likelihood of a warning shot of sufficient magnitude to cause a slowdown which would prevent RSI. On the basis of this argument, some urge alignment researchers at AGI companies to quit outright. But quit to do what? Missing from this exchange so far has been a discussion of opportunity costs. If you aren’t going to do (technical) work on “Alignment” – either inside or outside of an AGI company – what should you work on? In this post, I outline a contrast between “Alignment Engineering” – the dominant model for what “working on alignment” looks like (inside labs, and in the field as a whole) with “Misalignment Science”. I begin by characterising [...] --- Outline: (01:51) "Alignment Engineering" (06:22) AI Safety and the ML tradition (07:42) Implicit work trials (09:36) "Misalignment Science" (14:56) Conclusion (15:39) Postscript: Iterating ourselves into oblivion The original text contained 4 footnotes which were omitted from this narration. --- First published: October 5th, 2026 Source: https://www.lesswrong.com/posts/FogmcDHA6AdMGukum/alignment-engineering-vs-misalignment-science --- Narrated by TYPE III AUDIO.
October 4 · 15 min
“Claude reviews my LessWrong post history” by DirectedEvolution
I thought it would be interesting to try having Claude Code download my entire LessWrong and EA Forum post and comment history and evaluate it for my own cognitive biases and problematic communication patterns. I told it to focus especially on problems that have persisted over my 7-year history on those sites. I also had it look at my lichess blitz rating history, which I've noticed tends not to wobble randomly but go through consistent increases and decreases between set points, despite my lack of any disciplined practice. There are a few reasons I thought it might be interesting to post publicly. As a commitment device for myself to work on these problems. Because AI is uniquely capable of working on this type of problem. It's also a good argument for writing online and gives a new spin on the idea of "writing for the AI" that feels more affirming of the human mind than how I normally read the phrase (as offering up your words as fodder for AI training, perhaps in hope of influencing its alignment in beneficial ways). To stimulate other people to consider whether they might want to do this, and see [...] --- Outline: (02:39) Low-scoring comments and problem patterns (04:59) My strengths and positive patterns (05:25) To practice (05:53) Root causes (11:07) My 2023 blowup (12:19) Chess (14:17) Conclusion --- First published: October 3rd, 2026 Source: https://www.lesswrong.com/posts/PAzerqAofFCLoWsgK/claude-reviews-my-lesswrong-post-history --- Narrated by TYPE III AUDIO.
October 4 · 4 min
“Humans are not aligned either” by becausecurious
Epistemic status: I don't know much about AI alignment. This is just me thinking out loud. AI use: grammar and minor fixes. Reading about Inkhaven inspired me to try writing something to see how much I enjoy it. I've heard the idea that humans are not aligned in the same sense AI is not aligned. I am intentionally not researching the source of that idea, since then I would feel like I cannot add anything to it and wouldn't write. I also don't actually know the precise definition of "aligning" AI. So I want to think about this. One obvious property of "aligned" AI is not killing all humans. I think everyone agrees on this. It is more interesting to consider an AI killing some humans. You definitely don't want AI to kill random humans for random reasons. But what if there is a group of humans who can and want to kill all humans? Then it sounds natural for an aligned AI to kill this group of humans. From this perspective individual humans are already not aligned. First, I am sure out of 8 billion or however many people there are, there is at least one person who [...] --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/bnyvXhrdnpXxDqFi7/humans-are-not-aligned-either --- Narrated by TYPE III AUDIO.
October 4 · 17 min
“What I learnt co-leading an AI Safety bootcamp for legal and governance practitioners” by Katalina Hernandez
AI safety has gone mainstream. More people than ever want to work on AI safety research, and there are real policy wins in the US, the UK and Europe. But political will and market incentives are now the bottleneck, and policy officers can't own correct implementation alone. Having built a Responsible AI program at an Accenture joint venture, I know profit-driven companies are hard to steer, but also that people inside them can shift practice in ways that compound: tighter contractual clauses, more scrutiny of vendors and suppliers, fewer ways for providers to get away with non-compliance. The EU has no native frontier labs, but it is a huge market (with a regulatory body that has an AI safety unit!), That leverage only works if the professionals implementing AI safety legislation, and drafting the legal mechanisms behind it, understand the technical picture: how to read the research, and where safety promises break down. The market should be choosing not the most capable models, but the partners with the strongest incentives towards safety, and only a community with this knowledge can spread that. Earlier in September, ML4Good, in partnership with EquiStamp, ran the first European Seminar on [...] --- Outline: (03:09) 1. Professionals with impressive backgrounds can feel intellectually isolated- in Legal, too. (05:20) 2. Practitioners are already reading AI safety research. They do not always know how to interpret it. (06:50) 3. Non-technical people who are directly responsible for technical consequences have an almost inexhaustible need to understand (08:25) 4. Legal and governance practitioners deeply appreciate technical safety talent (10:09) 5. AI safety incentives and market incentives are not necessarily as separate as we can assume- and it's a neglected lever (11:49) And who understands that dependency particularly well? (13:03) 6. People in these functions understand that they can just do things (14:40) 7. Us lawyers? We're also nerds! The original text contained 3 footnotes which were omitted from this narration. --- First published: October 3rd, 2026 Source: https://www.lesswrong.com/posts/KtAug62dYRgAS8sqJ/what-i-learnt-co-leading-an-ai-safety-bootcamp-for-legal-and --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 3 · 36 sec
“Lab Leak in Russia” by Caleb Horn
I haven't seen this posted yet; I know lots of people here care about pandemic preparedness. It looks like there was a lableak of pneumonic plague in Russia after a worker "accidentally broke[] a test tube containing live bacteria". Unfortunately she later died at the hospital on a ventilator. --- First published: October 3rd, 2026 Source: https://www.lesswrong.com/posts/3uT42yoj2iimYn7Ra/lab-leak-in-russia --- Narrated by TYPE III AUDIO.
October 3 · 5 min
“You can’t use it without becoming like me” by Martin Sustrik
A Ukrainian fibre-optic drone — copied from Russians. (АрміяІнформ, CC-BY 4.0) Mick Ryan describes the fast following loop in war. Ukraine pioneered mobile teams to hunt drones. Russia copied it and incorporated it into its own defense. Russia came with unjammable fibre-optic drones but Ukraine quickly followed. After Ukraine developed uncrewed naval drones, Russia built its own. The wheel of action and counteraction turns ever faster: The most consequential fast following, though, has been institutional. Ukraine built the world's first Unmanned Systems Forces branch in 2024. Russia's Rubicon Center was established as a response to Ukrainian drone innovation and in November 2025 Moscow stood up its own Unmanned Systems Forces. Fast following has moved from copying a weapon to copying tactics to copying an entire force structure. Extend that logic further, and eventually you’ll get to the level of grand strategy: If a particular type of state is better at waging war, maybe you want to become that kind of state rather than lose the war. If meat waves confer an advantage on the battlefield, maybe Ukraine wants to become more like Russia, with a prostrate population easy to use as cannon fodder. If Ukraine's decentralized Brave1 kill market [...] --- First published: October 3rd, 2026 Source: https://www.lesswrong.com/posts/WyeYJ7cmeiSrXeCnM/you-can-t-use-it-without-becoming-like-me --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 3 · 31 min
“Dialogue with Eliezer Yudkowsky on Neural Networks” by aashish
On “reversed stupidity”, the success of deep learning, and what a mistaken forecast should change about a model of intelligence This began as a Twitter/X thread after I posted a 2007 passage in which Eliezer Yudkowsky was scathing about neural networks, and asked whether, in hindsight, his dismissal was itself a case of “reversed stupidity is not intelligence”. Yudkowsky joined the thread to explain what he had and hadn’t been dismissing (and what he thought he had actually got wrong) and we ended up having the exchanges reproduced below. I’ve preserved the dialogue verbatim, except for paragraphing, fixing obvious [typos] and expanding links. I’ve removed unrelated replies and moved a few pieces of context into bracketed editorial notes. Nothing has been rewritten for substance. Context Eliezer Yudkowsky, 2007: Whenever someone exhorts you to “think outside the box”, they usually, for your convenience, point out exactly where “outside the box” is located. Isn’t it funny how nonconformists all dress the same... In Artificial Intelligence, everyone outside the field has a cached result for brilliant new revolutionary AI idea—neural networks, which work just like the human brain! New AI Idea: complete the pattern: “Logical AIs, despite all the big promises, have [...] --- Outline: (00:11) On "reversed stupidity", the success of deep learning, and what a mistaken forecast should change about a model of intelligence (01:11) Context (02:58) Which "neural networks"? (16:06) Did LessWrong see deep learning coming? (21:32) Appendix: How much can we infer from the Sequences? The original text contained 1 footnote which was omitted from this narration. --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/PA7f9a9ZFcqpjmGEZ/dialogue-with-eliezer-yudkowsky-on-neural-networks --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 3 · 7 min
“How to change your identity. A guide to the highest leverage behavior change technique.” by KatSpartz
According to Atomic Habits, changing your identity is the highest leverage way to improve your habits, but nobody tells you how to do it. Don’t say “I’m trying to quit smoking.” Say “I’m not a smoker”. Don’t say “I’m trying to lose weight.” Say “I’m a health nut.” All well and good. But how do you practically do this, aside from stating your new identity? I’ve worked on developing the approach further and had a lot of success with these techniques. And changing your identity is especially great because it creates "passive happiness". You do the work once and it lasts a lifetime with little to no maintenance. Using these techniques I’ve successfully: Stopped being bothered by mosquitoes. Become able to wait patiently for food instead of freaking out when it's late. Learned to like the rain while living in Vancouver and London. Become a tough person instead of someone derailed by the smallest amount of pain. All by doing what I call identity work. Here's how I did it: Reinforce the new identity. Starve the old identity. Reinforce the new identity Look for confirming evidence First off, find facts that fit with this new identity. Find [...] --- Outline: (01:28) Reinforce the new identity (01:33) Look for confirming evidence (03:13) Make it social (03:39) Come up with a story of how you changed your identity (04:26) Pair it with strong emotions (05:08) Starve the previous identity (05:56) Important limitations to the technique --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/dFw4Twc8J8YWAGebb/how-to-change-your-identity-a-guide-to-the-highest-leverage --- Narrated by TYPE III AUDIO.
October 3 · 34 min
“How to apply to AI safety fellowships (and beyond)” by beyarkay (Boyd Kane)
About a year ago, I decided to go all-in on applying to AI safety fellowships, and around the end of 2025 I got into MATS. I think there's a small art to communicating your skills legibly. When I've spoken with others about how I answer application questions, they seem to appreciate my advice. I wrote a MATS 9 Retrospective which was well-received, so consider this to be similar advice, but for applying to jobs or fellowships. Applying to AI safety fellowships or doing job applications is an adversarial process: The goal of an application process is to measure how well the candidate would do in the position they're applying for, but this process is noisy. Some candidates will (inevitably) try to overfit to the application process itself, in a way that oversells their abilities. There's a grey area between "how to make your extant talents legible and understandable" and "how to fool people into seeing talents that aren't there". I've tried hard to withhold advice which could be used to overfit to the applications process, and to focus on advice that differentially helps people who are fit for the job but struggle to communicate this to the reviewer. Many [...] --- Outline: (02:03) How to apply (02:06) Differentiate yourself from the Average Joe (03:54) Making your skills legible (05:37) Concrete ways to be more legible (06:16) Put yourself in the reviewer's shoes (07:37) The internet is more meritocratic, so use it (10:44) Ensure that skimming your CV still leaves a good impression (12:39) Consider how you can improve yourself (13:38) A little love letter for typst (14:49) Maintain an LLM-friendly version of your work experience (17:35) Keep a text log of your questions/responses (19:18) Applying to MATS and other AI-safety fellowships (19:32) The AI Safety fellowship pipeline (21:00) Different streams are more or less competitive (21:44) Apply to multiple streams and to multiple fellowships (23:15) The mentors (often) make the final decision (25:09) If you're a "risky" candidate, try start with low-commitment fellowships (27:31) Consider an 80,000 Hours advising call (27:53) Fellowships are ~constantly accepting new applications (28:36) Your success is roughly proportional to your effort per application (29:28) Look through ~all the mentors' pages (30:19) If you get through to a later interview stage, it's probably worth taking a day off of work (31:02) Rejection feels like shit (32:17) You should probably self-study the ARENA curriculum (32:44) A long list of application questions (33:17) Conclusion The original text contained 4 footnotes which were omitted from this narration. --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/PiP4JqQFKhoqHGG2n/how-to-apply-to-ai-safety-fellowships-and-beyond --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 3 · 48 sec
″[Video] Why acausal dynamics are important” by Chi Nguyen
This is a beginner-friendly video. Second half might contain new content for people who already know about acausal trade. The beginning introduces decision theory and acausal interactions (CDT, EDT, ECL, MBAT). The middle explains why I think influencing how AIs reason about acausal interactions is time-sensitive. The last bit talks about how to do the influencing. --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/eXiP9A5KnkatjAX7y/video-why-acausal-dynamics-are-important --- Narrated by TYPE III AUDIO.
October 2 · 1 hr 29 min
“AI #188: Gemini Dot Argon” by Zvi
Is Google back? They claim that they are back. Gemini 4 Argon is rolling out, with competitive frontier-level benchmarks, at 2 dollars/10 dollars. What we don’t have is access to the model, because Google Fails Marketing Forever. So it is far too early to say what we have here. When I know more, so will you. OpenAI was forced to pull what would have been GPT-6.1 Astra due to alignment failures. They did offer us GPT-6.1 Sol, which is pitched as approaching Astra quality at the much lower price of 2 dollars/10 dollars, the same as Gemini 4 Argon. The rest of OpenAI's big Dev Day announcements were Ultrafast mode and Dots, your always-on AI agent based on Astra, which comes with your Pro subscription. I’m trying it out and will report back over time if I find it useful. The new hotness remains Claude Opus 5.5. This model rocks. It has made me considerably more productive and made my day more pleasant. It should raise your ambitions. There are some particular reasons to call upon Fable 5.1 or Astra, and sometimes a cheaper model will do, but pending Argon I consider Opus 5.5 [...] --- Outline: (03:04) Language Models Offer Mundane Utility (06:12) Huh, Upgrades (07:38) Better Call Sol (11:39) Gotta Go Ultrafast (13:13) On Your Marks (16:42) Choose Your Fighter (19:16) Get My Agent On The Line (22:34) The Warner Sister (26:33) Deepfaketown and Botpocalypse Soon (30:58) Fun With Media Generation (31:53) Cyber Lack of Security (33:24) A Young Lady's Illustrated Primer (33:59) They Took Our Jobs (41:04) Levels of Friction (43:50) Get Involved (45:54) Introducing (47:54) In Other AI News (48:57) Show Me the Money (50:43) Quickly, There's No Time (52:38) Pick Up the Phone (53:36) Quest for Sane Regulations (53:58) Chip City (54:07) The Open Model Frontier Is Largely Massive Fraudulent Distillation Attacks (56:23) The Week in Audio (57:38) People Just Say Things (59:17) Rhetorical Innovation (01:03:55) Greetings From the Department of War (01:06:55) The Department of Autonomous Warfare (01:08:16) Aligning a Smarter Than Human Intelligence is Difficult (01:11:03) Cooperative Alignment (01:16:42) I'm Upping My p(doom), the Future Goes Foom (01:24:16) No, You Make a Good Point, You're Not That Persuasive (01:25:27) Muddling Through (01:27:34) The Lighter Side --- First published: October 1st, 2026 Source: https://www.lesswrong.com/posts/S2EAn9v4BwRdptsom/ai-188-gemini-dot-argon --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 2 · 19 min
“Capabilities research expands the safety-usefulness Pareto frontier too” by Alex Mallen
It's tempting to define safety research as research that enables developers to deploy an AI system more safely without making the deployment much more expensive or much less useful. You can visualize this definition of safety research as pushing out the safety-usefulness Pareto frontier. At any given level of usefulness, there's greater safety available. Awkwardly, this definition counts basically all capabilities research as safety research. For example, consider performance optimization for inference. By making inference more efficient you can use weaker, safer models more extensively than you would otherwise be able to, pushing out the Pareto frontier. Likewise, any successful research whatsoever pushes out this Pareto frontier because research can only ever create more options. It seems like something has gone wrong with our definition of safety research if it includes seemingly all capabilities research. Here, I spell out one reason why enabling improved safety without hurting usefulness is an insufficient standard for safety research. The core observation is that developers have to choose a particular point on the Pareto frontier, and some technological improvements incentivize them to sacrifice safety. Safety research typically reshapes the Pareto frontier in a way that causes developers to choose greater safety, while [...] --- Outline: (02:50) How does research affect the Pareto frontier? (06:50) RLVR research that mainly enables improved usefulness, at the expense of safety (08:50) Inference research that enables somewhat substantial safety improvements (11:04) A plausible case in which capabilities research would be an effective safety intervention (14:01) Conclusion (15:23) Appendix A: Reasons why capabilities research might be bad when "burning the lead" (17:26) Appendix B: Further notes about ways in which this model is wrong The original text contained 4 footnotes which were omitted from this narration. --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/nwrx9DHpZfW2Q6zLW/capabilities-research-expands-the-safety-usefulness-pareto --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 2 · 7 min
“Lessons from building an automated research scaffold” by Alejandro Aristizabal, Josh Hills, Dewi Gould, ma-rmartinez, Falko Galperin, Denis Federico Lim, Aleksandr Bowkis
TL;DR. We built a scaffold to speed up our own research and gather data on automated alignment research (AAR). It turned out to not be valuable for researcher uplift, but was useful for gathering certain failure modes of AAR. Going forward, we plan to study the broader failure modes of AAR and how these automated research systems can be monitored and analyzed. We’d like to thank Sid Baines, Andrew Draganov, Cameron Holmes and Daniel Tan for helpful comments. This work was carried out by the Alignment Team at Arcadia Impact in collaboration with Josh Hills, Falko Galperin, and Denis Lim from Equistamp, and Aleksandr Bowkis from UKAISI. There's a details box here with the title "The scaffold". The box contents are omitted from this narration. What blocked researcher uplift? We made our scaffold available to our researchers and found that adoption was low, primarily because researchers found minimal uplift over their existing workflows for most tasks. This was for three main reasons: Our scaffold wasn’t helpful for conceptual work. While our scaffold performed well on very narrowly scoped, well-defined objectives with clear metrics, those aren’t the main bottleneck of our team's work. By the time a project has been reduced [...] --- Outline: (01:01) What blocked researcher uplift? (03:07) The scaffold was useful for gathering failure modes (05:08) Next Steps --- First published: October 2nd, 2026 Source: https://www.lesswrong.com/posts/zGaQ3SS9C6So9NXFg/lessons-from-building-an-automated-research-scaffold --- Narrated by TYPE III AUDIO.
October 2 · 4 min
[Linkpost] “Athletic education vs. athletic torment” by KatjaGrace
This is a link post. I don’t know if it occurred to me until my thirties to think of exercise as an enjoyable thing. I was familiar with finding obscure corner-cases that were fun, such as Dance Dance Revolution. But the idea of it just often being a good time was alien. I hesitate to blame anyone for anything, but school seems culpable here. I got the impression so firmly that PE class (‘physical education’ or ‘physed’) was a kind of horror, I’m not sure I would have treated this fact as on less solid ground than ‘you’re supposed to put a methods section in your lab report’. It just seemed like the way of things. For children like me, anyway. Some children liked PE, but those were a totally different kind of creature. What the PE telos included was being awful for nerdy children, maybe to punish them for not being jocklike children. Since nerd children generally like not being jocks, they do not take this punishment as any serious feedback on their way of being, and just nobly withstand it. This is the way it is meant to be, and so it goes for every nerd until they [...] --- First published: October 1st, 2026 Source: https://www.lesswrong.com/posts/Rt97G7LpYySgzdcQG/athletic-education-vs-athletic-torment Linkpost URL: https://worldspiritsockpuppet.substack.com/p/athletic-education-vs-athletic-torment --- Narrated by TYPE III AUDIO.
October 1 · 15 min
“On Social Reality in China” by alkjash
[Epistemic status: intuitions and anecdotes.] Recently, several posts and projects (Thoughts Memo, Babel Translation, Please Give Them a Chance) have taken important steps towards raising AI safety awareness and sharing rationalist philosophy in China. It's great that we’re recognizing the importance of solving the messaging problem for China, and thus laying the groundwork for an international AI pause. Below I record my perspective on cultural differences which are relatively underdiscussed, which may become roadblocks to this communication program. Background: I’m a “first-generation” Chinese-American who moved to the States at the age of four. The beliefs in this essay are primarily drawn from interactions with my parents and their generation of immigrants, and from consumption of Chinese media (dramas, webnovels, games, and manhua) which are not necessarily representative of the realities on the ground. I am likely over-indexed on the older generation and internet culture, and would appreciate corrections from folks who have direct lived experience. The picture I aim to paint is also complicated by a massive generational gap, and my understanding is that some of the below sentiments (e.g. the cynicism and nationalism) are partly inherited by the younger generation, and partly rejected through a variety of countercultures. [...] --- Outline: (03:09) Chinese Social Media is like American Junk Food (06:03) The Primacy of Social Reality (08:22) The Dark World Frame (12:44) Chinese nationalism as collective insecurity --- First published: October 1st, 2026 Source: https://www.lesswrong.com/posts/b5cSYh4emQb2qrGmK/on-social-reality-in-china --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
October 1 · 11 min
“My retrospective from MATS 10.0” by Nathaniel Mitrani
I have recently completed MATS 10.0, where I worked alongside Bart Jaworski under Victoria Krakovna (GDM). This is a post I was encouraged to make by my team at Geodesic Research from some slides I put together. This is not a post on application advice to MATS, or about the program in general. It is rather a compressed form of my experience doing research and lessons from the project. The project The full paper and post is coming soon, but I'll provide some context on it so that the lessons don't seem to come from nowhere. We trained natural model organisms (as opposed to constructed, worst-case; see Rhys’ dichotomy) of scheming of Kimi-K2.6, Qwen3.6-397B, Qwen3.8-27B using Low-Rank Adaptation training (LoRA) by doing Synthetic Document Finetuning (SDF) and Supervised Finetuning (SFT) on benign, individually plausible priors. We put the models in an Alignment Faking-like setup where the model has a policy that goes against its goal and is trained only on monitored episodes → the model schemes (strategically complies under oversight) and does so more over the course of Reinforcement Learning (RL). The main difference with the original Alignment Faking (AF) setup is that our setup is more realistic [...] --- Outline: (00:36) The project (02:39) Lessons learned (02:42) Lesson 1: Talk to your models (05:02) Lesson 2: Friedness is a great sanity check (06:17) Lesson 3: RL is ... hard and unpredictable (08:23) Lesson 4: Transfer of traits to agentic settings is hard (09:45) Lesson 5: LoRA rank didn't matter much (for me) (10:33) Closing thoughts --- First published: October 1st, 2026 Source: https://www.lesswrong.com/posts/rktrjGjXEbWsg7zof/my-retrospective-from-mats-10-0 --- Narrated by TYPE III AUDIO.

Showing 81–100 of 436 episodes