Nick Cammarata of Goodfire makes a bold claim: Neural Networks are Already Clean. This was his follow up after various resulting online discussions.
This is clearly good work that tells us new and important things. The question is, how big an update is this, how much progress does it represent and what is the outlook going forward?
The reactions here range from the super optimistic takes of ‘this means mechanistic interpretability is solvable,’ or even ‘all but solved,’ with comparisons to the famous paper Attention is All You Need.
Dean W. Ball (OpenAI): There is so much to say about this, and I will, but WOW, congratulations to Nick and Goodfire. This may be one of the most important single days in deep learning history, and perhaps much more than that.
No matter what, this is a victory for the dogged pursuit of understanding.
roon (OpenAI): when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even Eliezer will have to lower his pdoom. we can start making alignment into a ‘normal science’
This paper is solid and meaningful [...]
---
Outline:
(02:46) Good Smoke, Potential Fire
(04:01) One Neuron, One Central Concept
(06:40) I Can See Everything
(08:55) What a Concept
(13:51) The More You Know
(15:54) Attention Is Still What You Need
(18:15) Occam's Razor
(19:53) The Neurons Sometimes Share Clear Themes
(20:29) Hype
(28:39) Anti-Hype
(30:56) Theme and Variations
(37:15) Is This How Minds Work?
(39:15) But We Can Open The Box, Said Toad
(46:55) That Was Not the Hard Problem
(49:31) More Skepticism
(54:34) That's All There Is, There Ain't No More
(57:47) The Default Outcome of Mech Interp is Goodhart's Law
(01:00:37) Interpretability is Dual Use
(01:01:52) Now The Work Can Begin
---
First published:
October 19th, 2026
Source:
https://www.lesswrong.com/posts/w2tBHwNERDoSawznx/i-can-see-everything
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
