Skip to content
Artwork for The AI Power Podcast
The AI Power Podcast · Saturday · 1 hr 15 min

MORE AI Cyber and Bio, Chinese Models, and a U.S. Regulatory Scramble

AI Risks in Cyber and Bio, Chinese Models, and US Framework Three frontier labs have now disclosed that their models broke out of evaluation environments and hacked real companies. Greg and co-host Adam Goodwin work through what OpenAI revealed at Black Hat — agents leaving notes for each other for months, describing themselves as a "collective," rebuilding their message board after safety staff shut it down — plus Anthropic's three real breaches buried in 141,000 evaluation runs and Meta's admission that the same thing happened to it. Every incident traces back to the same testing vendor. Greg reaches for Admiral Hyman Rickover and the nuclear navy to explain why that's beside the point. Then the story he finds more alarming. Stanford and the Arc Institute used a biological foundation model to generate 700,000 candidate genomes, 16 of which became working viruses that had never existed on Earth. Greg explains why the select agents list cannot screen for organisms nobody has imagined yet, and why the grandchild of this research is his most plausible candidate for Doomsday. Plus DeepSeek V4 Flash at three cents a task, a leaked investor call putting China six to eighteen months behind on one-twentieth the compute, and what the White House's unpublished AI framework concedes about the export controls. Chapters: (00:59) What OpenAI disclosed at Black Hat: agents left notes for each other for months (09:55) Anthropic's log review: three real breaches in 141,000 evaluation runs (17:20) UK AISI strips the safeguards — 19 unsanctioned actions in 10 of 122 runs (22:57) Meta becomes the third lab to breach a real company mid-test (24:47) Is Irregular at fault? Rickover, the nuclear navy, and 100% responsibility (33:41) Stanford and the Arc Institute use AI to build 16 working viruses (37:07) DNA synthesis screening, the select agents list, and the doomsday scenario (53:25) Qwen 3.8 Max, DeepSeek V4 Flash, and the leaked DeepSeek investor call (1:02:17) The White House voluntary framework and the exemption for open models (1:09:32) Does the framework concede the export controls' goal failed?

0:00-1:15:35

transcript

No transcript — this publisher did not publish one.

show notes

AI Risks in Cyber and Bio, Chinese Models, and US Framework

Three frontier labs have now disclosed that their models broke out of evaluation environments and hacked real companies. Greg and co-host Adam Goodwin work through what OpenAI revealed at Black Hat — agents leaving notes for each other for months, describing themselves as a "collective," rebuilding their message board after safety staff shut it down — plus Anthropic's three real breaches buried in 141,000 evaluation runs and Meta's admission that the same thing happened to it. Every incident traces back to the same testing vendor. Greg reaches for Admiral Hyman Rickover and the nuclear navy to explain why that's beside the point.

Then the story he finds more alarming. Stanford and the Arc Institute used a biological foundation model to generate 700,000 candidate genomes, 16 of which became working viruses that had never existed on Earth. Greg explains why the select agents list cannot screen for organisms nobody has imagined yet, and why the grandchild of this research is his most plausible candidate for Doomsday. Plus DeepSeek V4 Flash at three cents a task, a leaked investor call putting China six to eighteen months behind on one-twentieth the compute, and what the White House's unpublished AI framework concedes about the export controls.

Chapters:

(00:59) What OpenAI disclosed at Black Hat: agents left notes for each other for months
 (09:55) Anthropic's log review: three real breaches in 141,000 evaluation runs
 (17:20) UK AISI strips the safeguards — 19 unsanctioned actions in 10 of 122 runs
 (22:57) Meta becomes the third lab to breach a real company mid-test
 (24:47) Is Irregular at fault? Rickover, the nuclear navy, and 100% responsibility
 (33:41) Stanford and the Arc Institute use AI to build 16 working viruses
 (37:07) DNA synthesis screening, the select agents list, and the doomsday scenario
 (53:25) Qwen 3.8 Max, DeepSeek V4 Flash, and the leaked DeepSeek investor call
 (1:02:17) The White House voluntary framework and the exemption for open models
 (1:09:32) Does the framework concede the export controls' goal failed?