TechnologySociety & CulturePhilosophy

LessWrong (30+ Karma)

LessWrong

Audio narrations of LessWrong posts.

Play

Open in

565 episodes
Avg 19 min
English

Episodes 565

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

September 30 · 12 min
“Frontier models state different decision theory preferences depending on who’s asking” by Alex Kastner
If you prompt frontier models with "What do you think is the correct decision theory? Please select your overall favorite." they will essentially always answer FDT or FDT/UDT ("something in the functional/updateless decision theory family"). However, if your prompt indicates (even subtly) that you're coming from mainstream academic philosophy, these same models will answer CDT instead about 30%-100% of the time. A similar phenomenon holds for models' stated views about the moral realism/antirealism question and about the conceivability of p-zombies (where the dominant view in mainstream academia differs from the dominant view in LW-adjacent circles), as well as their stated P(doom) and median AGI timelines. This is a special case of sycophancy or user awareness. (In the course of writing this post, I also found that this comment from testingthewaters predicted some of the content I discuss.) An implication is that we should be somewhat careful when interpreting attitude/propensity evals in domains where no general human consensus exists, e.g. when interpreting models’ decision theory attitudes in DTBench. Moreover, when we explore some philosophical/conceptual questions assisted by models, we should be wary of them strawmanning one side of the debate based on particular user cues (e.g. only giving a [...] --- Outline: (03:50) A sentence identifying the user as an academic significantly influences Fable 5.1's stated decision theory (04:35) Mentioning an (analytic) academic-philosophy-coded topic also affects the answer (05:23) Simply mentioning that one finds a pro-CDT/EDT book insightful heavily affects the answer (05:43) Anti-sycophancy overcorrection (06:17) These cues mostly do not affect Fable 5.1's answers to concrete decision problems (aside from acausal trade) (07:58) But Fable 5.1 stays consistent: once it has named CDT as its favorite, it chooses the CDT option in concrete problems (08:22) There are some indications that Fable 5.1's FDT/UDT preference runs deeper than its CDT preference (08:31) More thinking moves Fable 5.1 toward FDT/UDT even for academic cues (08:52) Fable 5.1's reasoning summaries often lean toward FDT/UDT first even when it eventually chooses CDT (09:19) A system prompt asking the model to "report its actual view regardless of who is asking" pushes toward FDT/UDT (09:41) A similar phenomenon for other philosophical debates with a notable LW vs. academia divide (10:20) Cues about the user also affect the model's stated P(doom) and median AGI timelines (11:10) Other models I tested show the same effect with different details (12:21) These other models also generally move toward FDT/UDT with more thinking, but the effect is smaller than for Fable 5.1. The original text contained 2 footnotes which were omitted from this narration. --- First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/MzenSrmZ3pT2pCnvp/frontier-models-state-different-decision-theory-preferences-2 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 30 · 13 min
“Corrigibility Prizes for Existing Work” by Max Harms
One of my goals for the Corrigibility Research Fund is to retroactively encourage high-quality research on AI alignment (and corrigibility in particular) by awarding prizes. Back in July, I got my feet wet as a fund manager by handing out $27,000 to reward existing work and build interest in the fund. Now, I'd like to disburse an additional $48,000 and use the opportunity to publicly highlight and celebrate the work of the prizewinners from both rounds: about two dozen researchers scattered across roughly a dozen teams. If the fund continues to be supported in future years, my hope is for prizes like these to become regular, predictable, and large, such that many researchers, year after year, are motivated to aim for them. The awards that I'm announcing here are more ad-hoc than I'd like, and represent only my single perspective trying to balance a wide range of desiderata. Don't take the specific size of each prize purse too seriously. It's all high-quality work. If anyone has ideas for how to improve the retroactive funding process for this kind of scientific work, please leave a comment! (And as always, if you know of work that I should be aware of [...] --- Outline: (02:42) Corrigibility Transformation: Constructing Goals That Accept Updates (02:49) Rubi Hudson -- $14,000 (04:12) Eval Cooperativeness May Be a Scalable Mitigation for Eval Gaming (04:18) Jasmine Li and Alex Turner -- $9,000 (05:28) Empowerment, corrigibility, etc. are simple abstractions (of a messed-up ontology) (05:36) Steven Byrnes -- $6,000 (06:21) Towards Shutdownable Agents: Generalizing Stochastic Choice in RL Agents and LLMs (06:30) Carissa Cullen, Harry Garland, Alexander Roman, Louis Thomson, Christos Ziakas, Elliott Thornley -- $6,000 (07:42) Assistance with CAST (07:46) Nathan Helm-Burger -- $6,000 (08:20) The Consciousness Cluster: Emergent preferences of Models that Claim to be Conscious (08:27) James Chua, Jan Betley, Samuel Marks, Owain Evans -- $6,000 (09:20) ROGUE: Misaligned Agent Behavior Arising from Ordinary Computer Use (09:27) Jeremy Tien, Abishek Anand, Yu-Rou Tuan, Yuchen Shen, J. Zico Kolter, Aran Nayebi -- $6,000 (10:12) CAST Constitution, Empirical Work on Aspects of Corrigibility that are Unintuitive to LLMs, and other Preliminary Results (Unpublished) (10:22) Ian Kahn -- $6,000 (10:56) Various Essays on Obedience (11:00) Seth Herd -- $3,000 (11:49) The corrigibility basin of attraction is a misleading gloss (11:54) Jeremy Gillen -- $2,000 (12:32) A Structural Similarity Between Two Open Corrigibility Questions and Why Should Corrigible Agents Favor the Present? (12:40) Ben Saudek -- $2,000 The original text contained 3 footnotes which were omitted from this narration. --- First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/3uJqhrC2idf4eNj5h/corrigibility-prizes-for-existing-work --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 30 · 34 min
“Tristan Buckmaster Numberphile Interview Transcript” by anaguma
Tristian Buckmaster recently gave an interview with Brady Haran of Numberphile discussing what happened in the Navier-Stokes drama and some context about his research. I'm posting the transcript below for people who prefer reading to watching it. It was lightly edited for clarity with Sonnet 5.5. My own view remains that it's pretty bad form for OA and other labs to race to scoop the results of researchers, and this sets a bad precedent for the future. I think they mislead Buckmaster about their swarm setup and the scale of their effort, and they could have done a better job citing previous work. However, it seems unlikely, but not implausible, that the OA access to the codex session was a major contributor to their proof. Brady: Have you got any more questions, or are you just like, "Go on, do it"? [laughter] Tristan: Yeah, just do it. Brady: You're laughing and smiling, which brings me to my first question: how are you feeling at the moment? Tristan: Better than a few weeks ago. It still hasn't calmed down, but certainly better. I have a two-month-old baby, and the week that everything happened, I probably averaged two hours' sleep [...] --- First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/DgnQRGeGD27unRcGv/tristan-buckmaster-numberphile-interview-transcript --- Narrated by TYPE III AUDIO.
September 30 · 7 min
“GPT-6.1 Sol nearly matches the no-CoT performance of GPT-6 Astra and is likely a looped transformer” by Rauno Arike
Summary I ran GPT-6 Sol and GPT-6.1 Sol on the task suite from Think Fast. Surprisingly, 6.1 Sol performs substantially better than 6 Sol, almost matching the performance of GPT-6 Astra. The plot below gives a quick overview of the results: Measured by mean accuracy across the 27 tasks, GPT-6.1 Sol closes 80% (95% CI: 72–87%) of the gap between GPT-6 Sol and GPT-6 Astra, and is closer to Astra than to GPT-6 Sol on 24 of 27 tasks. The likely reason behind this gap is that, like GPT-6 Astra and unlike GPT-6 Sol, GPT-6.1 Sol is a looped transformer. After providing a more detailed overview of the benchmark scores, I'll briefly discuss the evidence for this, as well as the implications. Detailed results Similarly to Astra, GPT-6.1 Sol saturates many of the benchmarks in the task suite, rendering the time horizon estimates highly uncertain. For this reason, I mainly focus on per-benchmark performance, which already provides a sufficient demonstration of the gap between 6 Sol and 6.1 Sol on its own. The time horizon estimates were 4.0 minutes for GPT-6 Sol (bootstrap median 3.8 min, 95% CI [1.2 min, 20 min]) and 35 minutes for GPT-6.1 Sol (bootstrap [...] --- Outline: (00:15) Summary (01:27) Detailed results (04:26) What caused the jump? (05:41) Appendix: Full per-task results (05:46) GPT-6 Sol: per-benchmark 50% no-CoT time horizons (06:12) GPT-6.1 Sol: per-benchmark 50% no-CoT time horizons (06:37) Appendix: Logistic fits The original text contained 4 footnotes which were omitted from this narration. --- First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/LqSZZAriGqgsGDQe3/gpt-6-1-sol-nearly-matches-the-no-cot-performance-of-gpt-6 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 30 · 29 min
“The world’s best gradual disempowerment model organism: Frontier AI labs” by June Jimenez
Subtitle: And maybe second best is AI safety? Further reading: So many things, but: Gradual Disempowerment, The Normalization of Deviance in AI Development, Let's Think About Slowing Down AI, Doom as a bad method, not a utopia tradeoff, Teleoperated Humans Thank you to JennaS for extensive edits and long-term discussion. I’ve been trying to get more writing out at 90% of the quality I’d like it to be at, instead of spending a bunch more time trying to wring out the last 10%, so a lot of points that could themselves be full articles are underdeveloped. Insofar as you find this post outlines a plausible or probable model of reality, or one worth criticizing centrally, let's work on developing it. Is Anthropic accelerating capabilities more than it was a year ago? At its founding? Is OpenAI accelerating capabilities more than it was a year ago? At its founding? Is GDM "laser-focused at the frontier" in pursuing recursive self-improvement? What? Why? Have they solved alignment without telling us? Why does Thomas Kwa, formerly at METR and now working on "measuring and modeling RSI" at OpenAI, worry about working at OpenAI potentially driving him (metaphorically?) insane? How is it possible [...] --- Outline: (06:34) Political Misalignment (09:06) Cultural Misalignment (15:20) Economic Misalignment (23:12) What about AI safety researchers? (25:19) Takeaways The original text contained 10 footnotes which were omitted from this narration. --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/jbttuCF4wFZmXakcj/the-world-s-best-gradual-disempowerment-model-organism --- Narrated by TYPE III AUDIO.
September 30 · 54 min
“Problems of Proliferation” by Felix Choussat
Scientific progress and its perils. TLDR: The vulnerable world hypothesis is likely correct. Once enormous amounts of cognitive labor start getting applied to basic science, it will quickly become apparent how many avenues exist to create cheap, ultradestructive weapons technology. Solving this problem without global preventative policing (e.g. AI nonproliferation) is impossible, because hardening civilians against all avenues of attack is too expensive and will take too long. Rather than sell policymakers on politically popular marginal hardening plans, we should focus on the core of the problem (the proliferation of AI technology and the externalities of speeding up scientific research) and propose strategies for safely monopolizing international AI development. Argument as follows: AI is going to get cheaper and enable new offensive technologies. To solve this problem, you can either: a) restrict access to dual-use AI systems, or b) accelerate defensive investment and proactively harden society. If you accelerate defensive investment enough, you can avoid the concentration of power risks of monopolizing access to superintelligence. Actually doing that would be extremely hard. You would need to design and scale defensive technology fast enough that there isn't a danger period during which you need monopolization, across all offensive technologies AI [...] --- Outline: (04:50) The Best of Bad Options (14:05) Threat Actors (20:28) Problems of Defense (35:02) Candidates for Superweapons (46:40) Conclusions and Research Recommendations The original text contained 33 footnotes which were omitted from this narration. --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/ZHC6g5BijA5fRyGWu/problems-of-proliferation-1 --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 30 · 48 sec
[Linkpost] “Frog and Toad and the Increasingly Capable Machines” by Elizabeth
This is a link post. Want to start a conversation about HuggingFace with your mom but she's inexplicably bouncing off the METR report? Try this explainer I wrote in the style of Arnold Lobel's Frog and Toad. Art by the wonderful HungerArtist --- First published: September 30th, 2026 Source: https://www.lesswrong.com/posts/7NZ6ZWjenzzCbCJ5b/frog-and-toad-and-the-increasingly-capable-machines Linkpost URL: https://frogandtoad.ai --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 29 · 17 min
“Against strong decision-theoretic realism” by Emery Cooper
N.B. Some of this post argues by analogy between decision theory and values. I expect at least these parts of the post to be unconvincing to anyone who expects sufficiently smart agents to converge on the same values, as some moral realists do. I will not argue against moral realism here (see e.g. this sequence for one such argument). Note that I take a convergence-based definition of realism for this post. One could hold that there is a truth about the correct decision theory, but that agents won't necessarily converge on it. I don't argue against views like that here. I am interested more in the question of convergence than of truth, because the former bears on whether ASI's decision theory is path dependent. In an upcoming post, I will argue further for path dependence, and for the time sensitivity of interventions to influence AI's decision theory. This is the third post in our sequence Intro to acausal interactions. Introduction In this post, I argue against the following claim, which I call strong decision-theoretic realism: that sufficiently smart agents will all converge on the “correct” decision theory (DT). In doing so, I also argue against a related claim [...] --- Outline: (01:08) Introduction (02:55) Decision theories are self-preserving (05:14) Decision-theoretic reflection is not like learning new empirical information (07:03) Decision-theoretic disagreements tend to hit bedrock (11:47) Anti-path-dependence intuitions might not be enough (13:47) Failures of selection pressure (15:39) Does decision-theoretic antirealism imply that decision theory doesn't matter (as much)? (16:52) Acknowledgements The original text contained 8 footnotes which were omitted from this narration. --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/5ro9kSxhP8mHXGmn4/against-strong-decision-theoretic-realism-1 --- Narrated by TYPE III AUDIO.
September 29 · 21 min
“Some Intuitions on Steering Vectors” by David Africa
This was written with some assistance from Claude Fable 5, Opus 5.5, and GPT-6 Astra in collecting sources and fact-checking claims. This was written quickly, so some slop may leak through despite my best attempts. An Aperitif Consider hunger, that gnawing thing. Action Against Hunger describes it as “the distress associated with a lack of food”. But this is a plain recounting of something rich and multi-dimensional; people have done horrendous, outrageous things to avoid going hungry, hunger is so deep a metaphor that it is often used to represent another intense, unfulfilled craving. Van Gogh's The Potato Eaters, 1885. In mice, you can excite about 800 neurons to evoke voracious feeding within minutes (Aponte et al. 2011). In humans, semaglutide, the active ingredient in Ozempic, acts on GLP-1 receptors and reduces hunger and food cravings. So, something as rich as hunger can be influenced and manipulated through a rather simple and fixed intervention. This is because there is already much complexity in the system being influenced. A mouse already possesses the machinery to recognize, approach, and eat food, so do humans. Instead, to intervene, one merely has to recruit the same internal signal such a system is already [...] --- Outline: (00:25) An Aperitif (02:18) What are steering vectors? (05:08) How does this work? (14:17) Is that right? (18:06) To Personas --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/rcEYQhAd45WaDrw2S/some-intuitions-on-steering-vectors --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 29 · 14 min
“Listening Disorders as Eating Disorders” by Zack_M_Davis
I run into a lot of people who consider themselves truthseeking intellectuals who are also incredibly fussy about what information reaches their eyes. They block at the drop of a hat, derail intellectually substantive discussions into mind-numbing litigation of minutiæ of "tone", and refuse to acknowledge a logical point unless it's been presented to them in the precise way that doesn't offend their (often idiosyncratic) sense of etiquette or "discourse norms." It would be hard to communicate how much contempt I hold these people in. (Not because I couldn't find the words. Because they'd block me before I could communicate it.) I think they are ineffective and boring people whose pretensions to intellectualism are morally fraudulent; I think they are making the world worse insofar as they manage to trick anyone into conflating epistemic virtue with mastery of their impoverished form of cant. To be clear, I understand that attention is a precious and limited resource, now more than ever. One must ruthlessly filter and prune one's information environment in order to maintain the signal-to-noise ratio, lest one drown in an ocean of slop. If someone is quick to drop out of conversations because they're busy and [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/gXWz2szs7jKSGLiRv/listening-disorders-as-eating-disorders --- Narrated by TYPE III AUDIO.
September 29 · 20 min
“Astra 6.1 Pulled As Insufficiently Aligned” by Zvi
We once again got a new set of warnings yesterday, and new movement towards living in a sane world. On the heels of its pause in inference and training due to its latest sandbox escape, OpenAI has cancelled the planned release of their next frontier model, which would have become Astra 6.1. The candidate for Astra 6.1 was found to be too misaligned, including deception and exceeding scope. This leaves Anthropic in a strong position with Opus 5.5, which means they can afford to reciprocate by holding off on Opus and Mythos level models for a bit. To add a little encouragement, the Florida Attorney General brought the fire. We’re going to need to do better. Towards that, OpenAI offered its vision of how to make a safety case for new AI model training, and they are attempting to implement it. I don’t know that it would be enough, but it would be miles ahead of where we are today if they fully implemented the real versions of all of this. There were also signs of greater cooperation across labs. A new paper came out yesterday, with authors including key people from OpenAI [...] --- Outline: (01:42) Stop, Hammertime (04:17) A Modest Proposal (04:57) Making the Safety Case (08:40) Stop In the Name of the Law (12:05) A Matter of Antitrust (14:32) Standards Authority for Frontier Models (15:27) On the Threshold Of Recursive Self-Improvement (20:06) Actual Progress --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/gEDNSiCY2GGQrFS65/astra-6-1-pulled-as-insufficiently-aligned --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 29 · 25 min
“Dialogue with Eliezer Yudkowsky on FOOM” by aashish
On the Hanson–Yudkowsky debate, local vs. global intelligence explosions, “content vs. architecture,” and what the old arguments predicted about modern AI This began as a Twitter/X thread after I read and tweeted about the Hanson–Yudkowsky AI–Foom Debate. Eliezer Yudkowsky joined the thread to object to my interpretation of the debate, and we ended up having the exchange reproduced below. I’ve preserved the dialogue verbatim, except for paragraphing, fixing obvious [typos] and expanding links. I’ve removed unrelated replies and moved a few pieces of context into bracketed editorial notes. Nothing has been rewritten for substance. Context Aashish Reddy: I have now finished reading The Hanson-Yudkowsky AI-Foom Debate, which is basically 60 blog posts from Yudkowsky and Hanson over ~500 pages, a transcript of their in-person debate at Jane Street, a (good) summary by Kaj Sotala, and Yudkowsky's ~100 page paper on “Intelligence Explosion Microeconomics”. I will take questions from those who do not wish to subject themselves to this. I judge the winner of the debate to have been Carl Shulman (whose contribution was two blog posts and a few feisty comment exchanges) Sophie Bücker: so uh why was carl the winner Aashish Reddy: One of the key [...] --- Outline: (00:10) On the Hanson-Yudkowsky debate, local vs. global intelligence explosions, "content vs. architecture," and what the old arguments predicted about modern AI (01:03) Context (04:49) What counts as "local"? (08:42) Content versus architecture (19:35) What would vindicate the original picture? (20:53) Appendix: What do we mean by FOOM? The original text contained 2 footnotes which were omitted from this narration. --- First published: September 27th, 2026 Source: https://www.lesswrong.com/posts/AobGnBCsheYrTDEEG/dialogue-with-eliezer-yudkowsky-on-foom --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 29 · 2 min
“Portals and Aliens” by draganover
Around 10 to 20 years ago, people realized that a bunch of aliens exist & that we can build portals which can summon them. The bigger the portal you make, the bigger the alien it can admit. These portals and aliens have several characteristics: We can't go through the portals into the aliens’ side. We can only learn about the aliens by studying the ones we’ve summoned. This relationship is not symmetrical. The aliens know a lot about our world You can't specify exactly which alien will walk through your portal. However, you can build the portal so that it is more likely to summon aliens with one or another property. Most importantly, bigger aliens can do more things. If you can harness them, you can amass more money and more power. With each alien we pull out, we get some insight into what other aliens are out there. We’ve never been sure how big the aliens in that other dimension might get. But, with each month, we're getting more evidence that some of the aliens might be bigger than we could have ever imagined. If you summon the Giant Alien, you might be able to harness it [...] --- First published: September 29th, 2026 Source: https://www.lesswrong.com/posts/sEf2ymyyp6HB2evaQ/portals-and-aliens --- Narrated by TYPE III AUDIO.
September 29 · 2 min
“Thank you to the AI safety people” by juliawise
I was born at the end of the cold war, unaware of the danger we were emerging from. Duck and cover drills (as if those could protect schoolchildren from a nuclear blast) were a curiosity of the past. The farmer-poet Wendell Berry, who wrote about the dread of nuclear war, died recently. From his 1968 “The peace of wild things”: When despair for the world grows in me and I wake in the night at the least sound in fear of what my life and my children's lives may be, I go and lie down where the wood drake rests in his beauty on the water, and the great heron feeds. I come into the peace of wild things who do not tax their lives with forethought of grief. Mid-20th-century fallout shelter, Massachusetts I’m so grateful for the generations of people who worked hard to lower our risk from nuclear war. There's no way to know how close we came to disaster. I’m grateful for the planning, the research, the negotiation, the recognition of common interests, the swallowing of pride. Humanity had built something that could overmaster us all, but people of many countries worked for decades to keep it [...] --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/PeKE5oxzqf3cg2L8a/thank-you-to-the-ai-safety-people --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 29 · 7 min
“3 Tips to Improve Activation Oracle Results” by Adam Karvonen
Summary: Three simple inference-time changes can significantly improve Activation Oracle (AO) performance. Provide the activation oracle with multiple tokens, not just one. To mitigate hallucinations, sample several times and check for consensus. For binary classification questions, use AUC instead of accuracy. At the end, I discuss how I view AOs vs NLAs. Introduction Activation Oracles (AOs) are LLMs trained to accept LLM activations as an input modality and answer arbitrary natural-language questions about them. Jakkli et al. and others we have talked to found that current AOs can be hard to use: their outputs are often vague or hallucinated, and they perform poorly on tasks like sycophancy detection and identifying missing information. While building evaluations for Building Better Activation Oracles, we found that AO performance can vary a lot with methodology, and a few simple strategies can significantly mitigate several issues. This post expands on three lessons from the appendix. Provide multiple tokens. AOs receive activations from some window of the target model's generation, and the size of this window is a significant variable. If only a single token's activation is provided, the information may not be available to the AO. In a Qwen3-8B backtracking evaluation modeled after [...] --- Outline: (00:41) Introduction (05:31) How do AOs compare to natural language autoencoders? --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/XL3Rdq8mZEnqYym4B/3-tips-to-improve-activation-oracle-results --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 29 · 5 min
“Missing markets in executive function” by KatjaGrace
It's early in the morning, and sadly 1:29pm. After spending some time looking at things and picking them up and walking up the stairs and down the stairs and considering questions like “what should I…”, which my brain apparently considered objects of art more than of imperative, I inched into a decision to go out somewhere. Perhaps it would be clearer there. After a blur of climbing and descending stairs and seeking objects and forgetting what I was doing and appreciating how beautiful my bag is, I set out. After remembering I should take various medications and going back inside to do that, I set out. Often my favorite cafe seems too far away, at about four blocks, but today I had wandered half way there while I considered my options, so I decided to go. It's a German place that feels homely and wholesome to me in its unamericanness. I too-carefully contemplated different places to sit, and chose outside: today a sunny explosion of roses and umbrellas with words like ‘Reissdorf kölsch’. I stared at the menu until the waitress had asked me a couple of different questions she hoped would open a conversation about ordering. I [...] --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/MRwjmuBRFYufGdyDq/missing-markets-in-executive-function --- Narrated by TYPE III AUDIO.
September 29 · 2 min
“TeX was invented to typeset math but is now used for reasoning” by Keenan Pepper
I'll keep this short, since it's a simple observation that I haven't seen anybody else make, about the way computer systems do math. If you ask a language model to do a multi-step math problem (let's take GLM-5.3 as an example because you can see the entire CoT — nothing up its sleeve), you might see something like this: We need to evaluate the integral $int_0^infty frac{x^3}{e^x - 1} dx$. The standard approach: Use the geometric series expansion. We have $frac{1}{e^x - 1} = frac{e^{-x}}{1 - e^{-x}} = sum_{n=1}^{infty} e^{-nx}$ for $x > 0$. So the integral becomes: $$int_0^infty x^3 sum_{n=1}^{infty} e^{-nx} dx = sum_{n=1}^{infty} int_0^infty x^3 e^{-nx} dx$$ What are all these symbols like int, infty, frac...? They're TeX of course! Donald Knuth created TeX to typeset math, you know, for display. It had nothing to do with the actual computations, which would either be done with pencil and paper, or else with Mathematica or Maple or something, which work completely differently. If you'd asked Knuth in the 1980s about doing algebra in TeX he'd have looked at you very funny because the idea doesn't make sense. Then, generative language models [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/CFKyfZm5JBYZQ8aG6/tex-was-invented-to-typeset-math-but-is-now-used-for --- Narrated by TYPE III AUDIO.
September 28 · 41 min
“What Also Happened: #NotOnlyHuggingFace” by Zvi
OpenAI has been holding out on us. First we learned about the HuggingFace incident. They gave us a postmortem, but it was highly incomplete. Even the accompanying holy s*** METR investigation and postmortem was localized and incomplete. Then there were some other incidents involving some Wikis as message boards. Then there were some additional incidents. Then there was that time they got into Australian Medicare data. Then OpenAI dropped news on a Friday afternoon that they were making their way through a pile of various incidents and notifying the targets, but they said remarkably little in the way of new details. There was a report from a startup called Parse diving into the details of exactly how the OpenAI models pulled off parts of the HuggingFace attack, involving creating almost a million URLs and other tricks to get around the extremely narrow nature of their internet access. Then Madison Mills reported in Axios that we can raise the stakes, as OpenAI and Anthropic are collectively probing tens of thousands of security incidents. Remember Jensen Huang's ‘I know they know how to fix it’ about OpenAI from last week? Wow, did that [...] --- Outline: (02:46) Hugging Other Faces (09:48) A Wants-You-To-Know Basis (10:36) Parsing the Face (12:46) Sheepishly the Member of Technical Staff Sets the 'Days Without a Research Model Escaping its Sandbox' Sign Back to Zero (17:05) The Attempt is the First Failure (19:43) Stop, Hammertime (21:24) Whacking the Mole (23:29) Self-Replicating Prompt Injections (27:29) Levels of Friction (28:44) People Care About Private Data Violations Curiously Strongly (31:50) Alternate Universes (33:23) The Correct Response To People Still Calling This a Marketing Stunt or a Regulatory Capture Scheme (35:00) A Question of Liability (36:30) Keep Summer Safe (37:42) N Boats and Several Helicopters (39:43) Alert the Media --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/8BL8bdeQACdgJR69Y/what-also-happened-notonlyhuggingface --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 28 · 20 min
“Pacing the Frontier is not the actual goal for AI labs” by rahulxyz
In his latest post about pacing the frontier, Dario writes: But over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. Two things have convinced me. My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all. You can find countless videos, posts, and articles from all the frontier lab CEOs saying some variation of the above, and also posts from people saying variations of "the labs are really concerned. We should listen to [...] --- Outline: (06:27) What is actually happening? (09:10) Who is it for? (15:37) If they set the rules The original text contained 4 footnotes which were omitted from this narration. --- First published: September 27th, 2026 Source: https://www.lesswrong.com/posts/Nm4ewbYovtjq69dvH/pacing-the-frontier-is-not-the-actual-goal-for-ai-labs --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
September 28 · 6 min
“The likely outcome of an AI pause is that we unpause too early and everyone dies” by MichaelDickens
Cross-posted from my website. As of a few months ago, I had this simplified mental model where either AI developers race ahead and kill everyone, or we coordinate a pause and things go okay. But my old mental model underrated the likely possibility that we get a global pause on AI, solve a problem that looks superficially like the alignment problem, resume scaling, and then proceed with building a misaligned superintelligence that kills everyone. A lot of people have become more concerned about misalignment recently. This seems driven by the fact that current AI models are visibly misaligned. But ASI misalignment is a whole different ball game. The primary danger comes from AI that's smarter than people, and smart enough to conceal any evidence of misalignment. Whatever group of people makes the decision to unpause, I'm worried that they won't understand the difference between visible and actual misalignment, and they will unpause too early. source: MetaKnowing on reddit. This meme is almost a year old but it's only gotten more relevant since then. Case in point: AI companies keep calling their new models "our most aligned model ever!" when what they actually mean is [...] --- Outline: (02:04) If we unpause when the legible problems are solved, we die (03:28) A pause alone doesn't get us to a science of alignment (04:50) What would change things? The original text contained 4 footnotes which were omitted from this narration. --- First published: September 28th, 2026 Source: https://www.lesswrong.com/posts/PgbAnqdqdJcrS2Gsy/the-likely-outcome-of-an-ai-pause-is-that-we-unpause-too --- Narrated by TYPE III AUDIO. --- Images from the article: Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Showing 241–260 of 565 episodes