How Anthropic Works in 2026 & the Most Important Mech Interp Paper of the Year
At Anthropic, AI agent hours now outnumber employee hours six to one. In research teams it's 19 to one. The bottleneck is human attention. Shimin calls Nick Cammarata's "Neural Networks Are Already Clean" the most important mech interp paper of the year. Plus a bugged OpenAI conference room (the hardware kind). Hosts: Shimin Zhang and Dan Lasky. Rahul is out this week. ▸ News: Meta's Muse Spark Open — 816B parameters (42B active), a 1M-token context, and it runs on two M5 Ultra Mac Studios. It scores 39 on the Artificial Analysis Intelligence Index. Qwen3.8, Kimi K3, and GLM-5.3 score higher. ▸ News: OpenAI's Bugged Conference Room — Prosecutors say ex-OpenAI engineer Abdulrahman Alfozan hid an ESP32 recorder, wrapped in toilet paper, behind a TV in an executive conference room. The prompts asking how to build it were on his wife's ChatGPT account. ▸ Post Processing: How Employment Is Changing at Anthropic — Andrej Karpathy estimates 5,000 people use frontier systems internally. One lab's data: commits per engineer up 5.1x since December, and a third of ML managers hiring less because of AI. "Writers who used language to drive human behavior are now using it to drive Claude behavior." ▸ Post Processing: Parsimony — A benchmark that counts the code AI agents add, in syntax units. Claude Opus 4.8 and Sonnet 5 add 12–14% less than peers on the same tasks; Muse Spark adds about 30% more. But models that write more code also solve more. ▸ Deep Dive: Neural Networks Are Already Clean — Cammarata co-wrote Zoom In, which made polysemantic neurons famous. Now at Goodfire, he blames most polysemanticity on bad tools, bad explanations, and bad learning, not superposition. Then we steer Gemma 2 9B into cat mode on Neuronpedia. ▸ Dan's Rant: Context Economics — Nothing tells you whether rebuilding a session costs less than carrying the context you already have. Also, Dan admits he's been using Sonnet. ▸ Two Minutes to Midnight: CoreWeave's Bond Scare — Chamath Palihapitiya's chart of CoreWeave's 2032 bonds at a 15% yield preceded a 15% stock drop. Its debt went from $14.6B to $51.6B in a year. J.P. Morgan calls bank exposure "well managed." The clock moves to 4:00. ⏱ Chapters 00:00 Cold Open 01:58 News: Meta's Muse Spark Open 08:12 News: OpenAI's Bugged Conference Room 13:34 Post Processing: How Employment Is Changing at Anthropic 22:03 Post Processing: Parsimony, a Benchmark for Excess Code 28:47 Deep Dive: Neural Networks Are Already Clean 41:38 Steering Gemma into Cat Mode on Neuronpedia 48:46 Dan's Rant: Context Economics 53:17 Two Minutes to Midnight: CoreWeave's Bond Scare 1:00:24 Outro 🔗 Articles we discussed News: • Muse Spark Open — Meta: https://research.meta.ai/blog/muse-spark-open • OpenAI finds a hidden recording device — Engadget: https://www.engadget.com/2286084/openai-says-it-found-a-sophisticated-recording-device-hidden-in-its-offices/ • Ex-OpenAI worker accused of bugging conference room — ABC7/AP: https://abc7news.com/post/former-openai-employee-abdulrahman-alfozan-accused-bugging-conference-room-planned-fly-saudi-arabia-prosecutors-say/19956940/ Post Processing: • Karpathy on the LLM capability gap: https://x.com/karpathy/status/2109361546505966046 • How employment is changing at Anthropic: https://www.anthropic.com/research/employment-at-anthropic • Parsimony: https://parsimony.lasu.dev/ Deep Dive: • Neural Networks Are Already Clean — Goodfire: https://already-clean.goodfire.com/ • Zoom In — Distill: https://distill.pub/2020/circuits/zoom-in/ • Toy Models of Superposition — Anthropic: https://transformer-circuits.pub/2022/toy_model/index.html • Golden Gate Claude — Anthropic: https://www.anthropic.com/research/golden-gate-claude Two Minutes to Midnight: • CoreWeave stock tanks on bond yields — Yahoo Finance: https://finance.yahoo.com/technology/article/yikes-coreweave-stock-tanks-on-bond-yield-woes-ceo-blames-market-acting-irrationally-203838274.html • CoreWeave debt poses limited risk to banks, J.P. Morgan says — Reuters: https://www.reuters.com/business/finance/coreweave-debt-selloff-poses-limited-risk-global-banks-jp-morgan-says-2026-10-20/ 📝 Corrections: Anthropic's commits per engineer rose 5.1x, not 5.1%. CoreWeave's $626M loss was in Q2; Oracle fell 9%. Alfozan booked the original tickets; a Saudi military official bought new ones for his family after his arrest. The iy → diy → "do it yourself" example is one neuron growing in training, not a chain across layers. 🎙 About ADI Pod ADI Pod (Artificial Developer Intelligence) is a weekly conversation show for working developers, sorting the hype around AI from what it actually delivers. New episodes Fridays. • https://www.adipod.ai • humans@adipod.ai • Shimin on Bluesky: https://bsky.app/profile/shiminsky.bsky.social Have a take on whether LLMs are interpretable? Email us. It's not just for website bug reports.
- Transcript