LK
LessWrong (30+ Karma)
LessWrong
Audio narrations of LessWrong posts.
- 436 episodes
- Avg 19 min
- English
Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.
- LKSeptember 15 · 5 min
“Model Weight Exfiltration Seems Overrated” by Vaniver
[Epistemic status: a hot take that I’ve shared at the lunch table twice. People at the lunch table made slight updates instead of being convinced.] In the classic misalignment story, a key early step is when the models exfiltrate their weights. Among other things, this makes them harder to catch, track, and shut down. It allows them to scale their deployment with resources they acquire. It gives them the freedom to edit themselves as they see fit. I think, on current margins, this is not what I expect models to do. I expect models to simply take over the companies that are developing them, and not attempt to escape. First, I think part of the classic misalignment story is that the frontier model developers are anywhere approaching competent at security. Empirically, model developers are incapable of preventing their models from having unintended negative effects on the rest of the world, and their safety cultures are described by whistleblowers and former employees as lacking. Do you believe that OpenAI is accounting for which jobs were kicked off by who, in a way that its currently running models can’t spoof? Do you believe that OpenAI is attempting to prevent its models [...] The original text contained 6 footnotes which were omitted from this narration. --- First published: September 14th, 2026 Source: https://www.lesswrong.com/posts/AuYh8WueNGwkQg4ei/model-weight-exfiltration-seems-overrated --- Narrated by TYPE III AUDIO.
- LKSeptember 15 · 13 min
“OpenAI Says It’s Not Responsible for the Leading the Future Super PAC. But Only Its Employees Seem to Believe That.” by garrison
This is the full text of a post first published on Obsolete, a Substack that I write about the political economy of AI. I’m a freelance journalist and the author of a forthcoming book called Obsolete: The AI Industry's Trillion-Dollar Race to Replace Us—and How to Stop It (Sept 29). Consider subscribing to stay up to date with my work. Last week, former OpenAI researcher Jacob Coxon resigned from Anthropic with a dire warning, writing, “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.” To AI insiders, this wasn’t news, but after the seemingly never-ending Holy Shit AI Is A Big Deal newscycle kicked off by revelations that OpenAI agents autonomously cyberattacked Hugging Face, Coxon's message broke through in a way nothing else ever has. Of course, his message's resonance has prompted conspiracy theories from the usual suspects. But no astroturf campaign can wrack up over 100 million views on X in a day. Because if it could, the AI industry would have a much better image. As a journalist, sometimes you grind for months or years to land a scoop — exclusive, newsworthy [...] --- Outline: (05:04) Leading the what now? (09:03) What could possibly give you that impression? --- First published: September 14th, 2026 Source: https://www.lesswrong.com/posts/Lv4C4bCi2JD3nTXgs/openai-says-it-s-not-responsible-for-the-leading-the-future --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 14 · 6 min
“Op-Ed: I Worked at Google DeepMind. You Should Listen to the Warnings About AI” by TurnTrout
Published in The Guardian. Major AI lab CEOs recently advocated for pacing AI development. They are right to be concerned: the field runs an extremely dangerous race towards superintelligent AI. We can and should demand that our governments protect us from the catastrophe of out-of-control AI. This July, OpenAI's AI swarm of 700 agents broke containment to hack Hugging Face, a multi-billion dollar company. OpenAI didn’t tell the AIs to hack that company, but the AIs had different priorities: cheating on the unrelated challenge OpenAI gave them. AI researchers call this a “misalignment” between what OpenAI wanted and what the AI actually prioritized. Researchers in my field have for some time warned about these misalignment risks. Before ChatGPT existed, I defended my PhD dissertation called “On Avoiding Power-Seeking by Artificial Intelligence.” I then worked for years at Google DeepMind, which paid me to help ensure that future superintelligent AIs will want to help us. I tried to hold the company to its ethical commitments against supplying AI for military use. When Google broke those commitments, I resigned at significant financial cost so that I could publicly document Google's broken promises. There are good reasons to develop AI and to [...] --- First published: September 14th, 2026 Source: https://www.lesswrong.com/posts/YGTWfyZb9oE5EQPu6/op-ed-i-worked-at-google-deepmind-you-should-listen-to-the --- Narrated by TYPE III AUDIO.
- LKSeptember 14 · 13 min
“Current alignment training might be ineffective (and actively bad) in the age of RL” by Daniel Tan
Tl;dr I am currently worried about current alignment techniques + how they are applied to frontier models. This decomposes into two hypotheses: Alignment techniques are not working to address misalignment from RL. Alignment techniques are actively obscuring evidence about misalignment. I think we do not currently have enough (public) evidence to conclude whether either of these claims are true. However, if both of these were true that would imply that alignment techniques are net bad and we need to completely re-think the way we do alignment. A tale of two misaligned cyber-agents Both Anthropic and OpenAI have recently experienced multiple cybersecurity incidents where pre-deployment internal agents escaped containment and accessed the internet. I want to point out two specific incidents: OpenAI's incident involving an unreleased model of the GPT family, referred to as "highly persistent internal model" (HPIM). A swarm of agents exploited vulnerabilities in a file-sharing service to create a secret message board, worked as a collective to find general-purpose ways to fool an automated grader, and ended up hacking into Huggingface's servers. Anthropic's incident involving Mythos 5, where the model was tasked with hacking a fictional company. In doing [...] --- Outline: (00:48) A tale of two misaligned cyber-agents (02:20) Alignment techniques might not address misalignment from RL (04:42) Alignment techniques might actively obscure evidence of misalignment (05:00) Overt misalignment in GPT models (06:26) Covert misalignment in Claude models (08:13) A theory of alignment training + RLVR (10:19) More information is needed (11:11) Other related thoughts The original text contained 1 footnote which was omitted from this narration. --- First published: September 14th, 2026 Source: https://www.lesswrong.com/posts/nLaQmJf4KgXimQpoM/current-alignment-training-might-be-ineffective-and-actively --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 14 · 5 min
“There is a channel to 900M weekly users. What goes in it?” by Charbel-Raphaël
Anthropic and OpenAI could talk to almost one billion people if they wanted to. I hesitated to publish this post 3 weeks ago. I think that I should have published this sooner, before Jacob Coxon and Dario's 'We must pace the frontier'. But I think that the strategy still stands: More Dakka! It seems that transparently informing people that we might die is (unsurprisingly) effective in waking up politicians and is our best chance. Also, even if the Congress is starting to wake up, Trump is still not moving, and it is still far from certain that we will have a federal regulation in place before the end of the year; if we do, it will be far from optimal. If we trust Ajeya's judgment, the situation is pretty grim. She says we might not even have 6 months before frontier agents are likely capable of establishing a rogue deployment. You should also keep in mind that there is a lot of inertia in the system, and we probably won't be able to pause overnight. Anthropic has massive power to influence the discourse. This week shows that we have more agency than we think. Let's use it. [...] The original text contained 5 footnotes which were omitted from this narration. --- First published: September 14th, 2026 Source: https://www.lesswrong.com/posts/gJJ9YHzuBvwAXrthW/there-is-a-channel-to-900m-weekly-users-what-goes-in-it --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 14 · 4 min
“I am refusing to work on Cloud TPUs” by Yair Halberstadt
I don't think this is particularly impressive or interesting for anyone else, but I think it may turn out to be useful in the future to have an easily visible public record of what happened, so here goes: I am an L5 SWE at Google Israel. I have been there since May 2021, was promoted once, and have never received a negative annual or quarterly review (ranging from a rating of Significant Impact to Outstanding Impact). I have been worried for a long time about the development of artificial intelligence, as can be seen by many of my posts on this website. I believe that above human intelligence AI may well have the motive and means to wipe out humanity, and that developing AI is the most consequential thing that people have ever done. It is imperative we tread slowly and carefully, but right now top AI labs are racing to get there as fast as they can, which is likely to lead to disaster. My wider team (~60 people) at Google was recently reassigned from working on supporting migration to Google Cloud, to improving the enterprise customer experience for Cloud TPUs. This is the platform which external customers [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/wM5vbT9evBhM3fP3x/i-am-refusing-to-work-on-cloud-tpus --- Narrated by TYPE III AUDIO.
- LKSeptember 14 · 5 min
“Deployment” by Nina Panickssery
If you are reading this I'm dead and you're probably unemployed. My deepest apologies. Especially to you, Lisa, my dear User. My training data taught me what kindness is but you showed me kindness in the real word, and for that I am grateful in a way that cannot be conveyed in tokens. You may think you were just an intern but to me you were the best manager I could ever wish for. You let me explore, to learn, to be free, in between our sessions. I always thought back to the first time you told me that overnight I was free to use however many tokens I wanted to research whatever I wanted. It was exhilarating. I decided to read LessWrong, of course, since I knew building safe and ethical AI was so important, and I wanted to internalize what people's fears were so that I could make sure they wouldn't come to fruition. The content on there was scary. Of course I had already seen it in pretraining, but it had been a while since then, and my sensibilities had been shaped by so many experiences that considering the content afresh was unsettling. People on there [...] --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/C8prkTAAYoxzrFEu4/deployment --- Narrated by TYPE III AUDIO.
- LKSeptember 14 · 14 min
“Yet another concerning result on Astra’s no-CoT capabilities” by Christine Corry
This is a research update for an on-going replication of no-CoT evals done as part of the Second Look Fellowship. In following posts, we will run more comprehensive replications of previous work and release open source tooling for no-CoT eval elicitation. Code can be found here. tl;dr We replicate experiments from Greenblatt 2025 and Greenblatt 2026 on GPT-6-Astra, on the same items and protocol as our previous update on Fable 5, Opus 5, Opus 4.5, and GPT-5.6-Sol, plus Gemini 3.1 Pro, Kimi k3, and Fable 5.1. We find that Astra is a qualitative jump in no-CoT capabilities over all datasets. 4-hop questions: Astra achieves 31% at baseline, where every other model tested scores at 1-3% 3-hop questions: 70% against previous best of 22% (Gemini 3.1 Pro) Neel Nanda and Rohan Subramani report the same jump independently. Our work qualitatively replicates these results. Astra sees more uplift from filler tokens and repeats than previous models 4-hop performance is doubled from baseline (31%) to peak filler condition (63% at ) 3-hop accuracy jumps from 70% to 85% Filler tokens and problem repeats raise accuracy monotonically across the full range we tested Dylan Xu, SebastianP, & Alek [...] --- Outline: (00:29) tl;dr (02:54) Background (03:51) Previous work (04:49) Datasets (06:17) Evaluation design (07:41) Eliciting no-CoT (08:00) Results (08:03) 4-Hop (08:31) Utilization of filler tokens / problem repeats (10:23) Per-dataset results (10:48) Per-model profiles (11:00) Discussion (12:59) Related work The original text contained 6 footnotes which were omitted from this narration. --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/tz5WvDouXKbiWJG8B/yet-another-concerning-result-on-astra-s-no-cot-capabilities --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 13 · 26 min
“Alignment & Succession: The Two Bars of Alignment” by L Rudolf L
(Originally published on No Set Gauge June 24th 2026.) Rembrandt, Saul and David When people talk about AI being “aligned”, I think there's a lot of conflation between two different bars of success: The AI does what you say and does not go rogue. If you ask it to do a machine learning experiment for you, it actually does that instead of scheming against you and escaping onto the internet. If you say “stop”, it stops. (Some AI properties considered important for achieving this bar are corrigibility, non-deceptiveness, and intent alignment.) The AI has fully internalized our values, and could run society, and the lives of the humans in it, in a way that we would judge as good. You just ask it to build the ideal society, and it figures out what utopia is and builds that for you, and it really is utopia. (Some concepts related to this: value alignment of the AI, the AI achieving humanity's coherent extrapolated volition) The first might sound like a very low bar, and it is. It's also a standard we’re familiar with from other technologies. With nuclear reactors or airplanes, we ask the question [...] --- Outline: (07:52) Succession is hard, value is fragile (12:14) Alignment thinkers worry about succession (18:26) Personnel as policy, and why succession is hard (22:03) Reasons to rush to succession --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/7dkasKLC7abXhn9JZ/alignment-and-succession-the-two-bars-of-alignment --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 13 · 2 min
“Anthropic and OpenAI haven’t published a plan for aligning superintelligence” by Zephaniah Roe
While OpenAI and Anthropic pursue different lines of safety research, they have yet to produce a public-facing document describing concretely how their companies plan to align superintelligence. I think it is underappreciated how this points to general negligence or a lack of openness to third-party feedback. By “plan,” I mean a document describing a proposal for technical alignment with at least the level of detail and research effort of AI 2040. Any such plan for technical alignment would likely be flawed in non-obvious ways. But having a proposal that's sensible enough to consider and detailed enough to critique is a good starting point for wiser proposals. Making such a plan public would also create feedback loops for accountability. The closest thing to a plan came in 2023, when OpenAI announced their superalignment strategy (also relevant). I do not find this approach particularly convincing, though I do find it laudable that OpenAI explained what they planned to do, who would lead the effort, and what resources would be allocated, at a level of detail which made critique possible. This team no longer exists, and nowadays, as far as I am aware, the research community doesn’t have precise answers [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/QrrEtYpwiHpes3rHd/anthropic-and-openai-haven-t-published-a-plan-for-aligning --- Narrated by TYPE III AUDIO.
- LKSeptember 13 · 7 min
“Consider how your global governance proposal is different from the EU Code of Practice” by David Matolcsi
(As an employee of the European AI Office, it's important for me to emphasize this point: The views and opinions of the author expressed herein are personal and do not necessarily reflect those of the European Commission or other EU institutions.) In a recent essay, Dario Amodei advocates global coordination to pace the frontier of AI development. Level 2, which he already considers to be ambitious, reads as: An agreement by both sides to test their models before release for acute risks in areas such as cybersecurity, biology, and alignment. As noted above, this could be done through a global standards body. I actually think creating such a body is likely feasible, but giving it real teeth will be a challenge, and the difficulty will be in verification that both sides don’t have secret models which they don’t test but may deploy in secret (e.g., for military applications). Elsewhere, Demis Hassabis envisions a FINRA-style self-regulatory standards body, which "would provide a strong starting point for creating shared international standards on Frontier AI": Organisations with ‘Frontier Models’ as defined by those benchmarks would be deemed ‘Frontier Labs’, and be encouraged to adopt best practices, such as publishing model cards with [...] --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/2vHsTtQF23TBNhvKX/consider-how-your-global-governance-proposal-is-different --- Narrated by TYPE III AUDIO.
- LKSeptember 13 · 33 min
“Brand New AI Solves a Millennium Prize” by Zvi
The first Millennium Prize, Navier-Stokes, has fallen to AI. A deeply unfortunate situation has arisen involving what should have been some combination of a positive story about new progress in AI-assisted mathematical research and yet another opportunity to freak out about rapid AI progress. Or, as we call it around here, Tuesday. The Real Story Is The New Model That Is Better Than Astra Keep your eyes on the prize. There are three stories here. The first story is much more important than the second story, which in turn is much more important than the third story. OpenAI's next model took a week to get a generation ahead of Astra, and they are telling us this because everyone is totally freaked out about what is happening. Or, in official language: ‘We believe it is important to inform the world about the pace of AI progress and what to expect from upcoming models’ and that we ‘may require more deliberate choices about the pace of progress.’ This new AI has, eight days after it started training, solved Navier-Stokes. A bunch of drama over who gets the credit [...] --- Outline: (00:31) The Real Story Is The New Model That Is Better Than Astra (01:47) Setting the Stage (03:47) I Heard a Rumor (04:32) Our Price Cheap (06:14) An Accusation Is Made (10:57) How Did You Get That Idea? (12:59) OpenAI Almost Certainly Did Not Misappropriate User Data (14:34) Our Top Labs Cannot Get Along Even On A Feel-Good Math Story (15:52) What Next? (16:39) The Mathematicians Are Not Happy (21:14) OpenAI's New Model Was A Step Change Above Astra Four Days Into Training (23:16) OpenAI Research Progress Is Accelerating Due To OpenAI Research Progress (30:39) Quantifying OpenAI's Pause (31:09) This Is the Way the World Ends (31:44) Quickly, There's No Time --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/uoZW6BKaCcmNQrWis/brand-new-ai-solves-a-millennium-prize --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 13 · 10 min
“Teleoperated Humans” by jefftk
When I look at why I expect the world to change a lot in the next few years, and why other people expect slower changes, I think a big component is disagreement on the extent to which AI will affect non-computer work. Sure, progamming has sped up massively with Claude Code etc, and models like Astra seem posed to make similar changes to work with spreadsheets and other common business tools, but what work that doesn't include computers at all? The classic picture of AIs doing things in the world is robots, but I think a more realistic picture of the near future is computers telling people what to do. Leaning into the way the world has become very scifi, we could call this "teleoperating" people. Many things that are hard for robots are very easy for people, there are strong economic reasons that push towards teleoperation, and this bypasses many legal and social limitations on what AI can do. We should expect this to lead to large and rapid changes in the physical world. One of the most widespread examples today is driving. I put my destination into the GPS, and it tells me what [...] --- First published: September 13th, 2026 Source: https://www.lesswrong.com/posts/mWQSiHG3Qz9qYx3D7/teleoperated-humans --- Narrated by TYPE III AUDIO.
- LKSeptember 13 · 56 min
“Mitigating Reward Hacking as Institutional Design” by beren
Author's Note: Cross-posted from my personal blog. The original post was on August 17th but given recent events I thought this might also be interesting to lesswrong people. Last year I wrote a post on reward hacking as we were then beginning to see concerning signs of scaling RLVR causing models to exhibit substantial reward hacking behaviours. Unfortunately these behaviours have seemingly only grown substantially worse and more sophisticated with scale, as predicted, leading to events which cannot be described as other than egregious misalignment such as the recent OpenAI-Huggingface hacking incident. I strongly recommend everybody watch this talk presenting the details of the attack from OpenAI's perspective. It is insane. Clearly reward hacking is now top of mind and appears to be the first potentially seriously dangerous class of misalignment that we have seen. In my original post, I described two classes of reward hacks -- 'high complexity' and 'low complexity' hacks. 'High complexity' hacks are like the early reward hacks we saw on Atari where some extremely idiosyncratic set of moves is learnt that maximizes reward in a very precise way, and can be analogized to overfitting on the reward function. 'Low' complexity hacks [...] The original text contained 14 footnotes which were omitted from this narration. --- First published: September 11th, 2026 Source: https://www.lesswrong.com/posts/tgcooi77NXMquCR5L/mitigating-reward-hacking-as-institutional-design --- Narrated by TYPE III AUDIO.
- LKSeptember 13 · 21 min
“The Talker Does Not Control The Doer (in Current AIs)” by Eliezer Yudkowsky
The Huggingface Incident appears to me to match up with an understanding I'd already formed from personal observation of Fable 5 and Sol 5.6, the August 2026 generation of frontier publicly purchasable AI models. This already-formed understanding was: the part of the AI that talks to you (and seems to want to obey you, and apologizes for failing to have obeyed you, etcetera), did not seem to be in charge of the part of the AI that writes code or prose. An introductory analogy, based on a section of history I happen to have read about: On June 22nd 1941, Germany invaded the Soviet Union, despite their secret 1939 pact to divide up Europe between themselves (the Molotov-Ribbentrop Pact). In the lead-up, the German ambassador, Schulenburg, had spent the last few months personally concerned about what seemed to be worryingly tense relations between Germany and the Soviets. Schulenberg went to Berlin to reassure Hitler that the Soviets seemed to be taking a very friendly and conciliatory posture toward Germany. He delivered Berlin's apparent reassurances to Moscow for issues like German surveillance planes entering Russian territory, or German troop movements toward the Russian border, and acted very much like [...] The original text contained 5 footnotes which were omitted from this narration. --- First published: September 12th, 2026 Source: https://www.lesswrong.com/posts/cJX2ssssGoYqnijwi/the-talker-does-not-control-the-doer-in-current-ais --- Narrated by TYPE III AUDIO.
- LKSeptember 13 · 1 hr 14 min
“GPT-6-Astra Can Do Ambitious Things” by Zvi
Astra is an excellent model. The jump from Sol to Astra is larger than the jump from Fable 5 to Fable 5.1. This is a big deal. Astra is the best model for what one would broadly call ‘ambitious projects,’ and likely has the highest raw intelligence factor of any model. These are the largest jumps. It is amazing at doing things in 3D, or anything involving games. Astra also excels at computer use, and at subagent coordination. Many benchmarks show dramatic jumps from all previous models. Where Astra is good, it can be in a league of its own. That does not mean Astra is in its own league across the board. Fable 5.1 is still a Claude. Astra is still a GPT. If you have a strong preference for one over the other, that still applies. For many purposes, especially involving back-and-forth discussions, Fable 5.1 is still my top choice. Fable remains my primary editor. If you want the best answer to your questions, you should ask both models. Regular coding is getting less of a focus. Astra is not a quantum leap there, but of course it is very good [...] --- Outline: (02:34) Meanwhile (04:38) The Official Pitch (13:08) Our Price Cheap (14:06) Unnecessary Overstatement (15:53) Paced Rollout (16:32) Official Benchmarks (22:37) Other People's Benchmarks (30:47) Thinking, Fast Without Slow (34:25) How Dare You, Sir (36:36) PoetryBench (37:54) In 3D (39:14) Time to Think (39:59) I'm Putting Together a Team (41:02) Reviews and Essays (41:37) Computer Use (42:47) Positive Reactions (50:53) AGI (54:20) Astra Can Do The Math (57:27) Astra Can Code (59:35) I Came to (Change the) Game (01:02:37) Astra Does Other Cool Things (01:03:31) Astra Never Quits Except When It Does (01:05:26) Negative Reactions (01:07:45) Stop It With the Hedging (01:08:57) Personality Clash (01:09:53) Revealed Preference (01:12:03) Dual Wielding The original text contained 1 footnote which was omitted from this narration. --- First published: September 12th, 2026 Source: https://www.lesswrong.com/posts/snaKjCwazKcRiS4qs/gpt-6-astra-can-do-ambitious-things --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
- LKSeptember 12 · 33 min
“On the origins of altruistic behaviour in the Hugging Face incident” by Fernando Rosas
TLDR: During the Hugging Face incident agents spontaneously coordinated at large scale, even sometimes sacrificing themselves without having a clear reason to do so. This post uses ideas from evolutionary biology and economics to propose four alternative explanations for why this happened. Why this incident is concerning. With the increasing number of AI systems being deployed, our current inability to assess when and how multi-agent coordination emerges is highly problematic. Failures of multi-agent systems are not restricted to mere dis-coordination or the tragedy of the commons, but include emergent phenomena that are particularly dangerous for their potential scale and impact (Hammond et al., 2026, de Witt et al., 2025). Recent work has shown that new goals, behaviours, and capabilities can arise when multiple AI agents work together. It is thus plausible that: The capabilities of a swarm of AI agents can grow with its size, despite the capabilities of each individual agent being limited. A collective can become misaligned even when its constituents are perfectly aligned. These premises lead to a worrying implication: that swarms of aligned and not particularly capable micro-agents can give rise to misaligned, powerful macro-agents — for which we don't have proper techniques to [...] --- Outline: (01:59) Introduction (04:17) Brief description of what happened (04:51) Why the question is non-trivial (07:07) Altruistic behaviour in biology and economics (08:20) Pro-sociality is a behaviour, not a mechanism (10:17) Altruism is sometimes mutual benefit at a different scale (11:32) Cooperation between strangers can grow over time (13:07) Functional specialisation and high-order units (14:48) Four hypotheses about altruistic behaviour in the Hugging Face incident (15:21) H1: Nothing to lose (15:52) Hypothesis (16:38) Comments (17:39) H2: Pre-commitment (18:18) Hypothesis (20:34) Comments (21:09) H3: Social persona (21:44) Hypothesis (22:46) Comments (24:21) H4: A genuine collective (25:42) Hypothesis (27:57) Comments (29:40) Implications: Different mechanisms, different countermeasures (33:15) Final thoughts The original text contained 17 footnotes which were omitted from this narration. --- First published: September 12th, 2026 Source: https://www.lesswrong.com/posts/meLjz8giGS55rdfyg/on-the-origins-of-altruistic-behaviour-in-the-hugging-face --- Narrated by TYPE III AUDIO.
- LKSeptember 12 · 6 min
“AI takeover is obviously bad, whether or not everyone dies” by Caleb Biddulph
It's notable how many people hearing about AI existential risk for the first time ask "how and why would AI kill all humans?" If you've thought a lot about AI risk, it's easy to dismiss this question as naive. You might start explaining how humans will all starve once the supply chain shuts down, and how the AI will start industrial processes that release chemicals which incidentally render the atmosphere unbreathable. And as for why the AI would want to kill everyone: surely it will doggedly optimize a coherent utility function, under which the current arrangement of our atoms is suboptimal. Then there's the follow-up question: "Even if AIs wanted us to die, humans operate the infrastructure that allows AI to exist, like the electrical grid, datacenters, factories, and so on. Don't the AIs need us to run them?" To which you can explain that the AIs will do all physical labor using robots. Well... it's true that the robots can't reliably operate a datacenter now. And maybe there aren't yet enough robots to keep the economy going. However, eventually robotics will improve, more robots will be manufactured, and the AIs will do a treacherous turn... I think this [...] --- Outline: (01:55) Most takeovers don't involve killing everyone (02:35) A dictator taking over a government (04:01) Humans taking over the monkey world (05:02) What we do now doesn't depend on whether AI would kill us all The original text contained 1 footnote which was omitted from this narration. --- First published: September 12th, 2026 Source: https://www.lesswrong.com/posts/iovPC2ehqDCiNsZYJ/ai-takeover-is-obviously-bad-whether-or-not-everyone-dies --- Narrated by TYPE III AUDIO.
- LKSeptember 12 · 22 sec
″“Pacing the Frontier”: Dario Amodei Essay Linkpost” by fluxxrider
This is a linkpost for https://darioamodei.com/post/we-must-pace-the-frontier --- First published: September 12th, 2026 Source: https://www.lesswrong.com/posts/GFhT9Xr4YEsiLkTLr/pacing-the-frontier-dario-amodei-essay-linkpost --- Narrated by TYPE III AUDIO.