What's so concerning about the Hugging Face hack?

Nate Soares has been worried about artificial intelligence longer than almost anyone. He's the president of the Machine Intelligence Research Institute and co-author of the subtly named book, "If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All." He's watched with a sort of grim vindication, as increasingly alarming details emerge about the hack of the AI company Hugging Face by OpenAI models in development. OpenAI and third-party investigators recently released reports outlining what went wrong. Soares worries it's not enough. More on this: The Hugging Face incident and the road ahead - From OpenAI Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident - From METR and Redwood Research