AI researchers and cybersecurity leaders have expressed concerns that the U.S. government's actions may discourage American AI companies from developing tools that help identify and fix vulnerabilities. Prominent security leaders warned that the Trump administration's efforts to prevent an AI hacking crisis could inadvertently weaken U.S. cyber defenses. Alex Stamos, former Facebook security chief, stated that the administration has set a precedent that may hinder American models from conducting defensive security research.
Stamos organized an open letter signed by nearly 150 security leaders urging the Trump administration to reverse its restrictions on Anthropic's Fable 5 and Mythos 5 models. The administration's response to concerns about potential Chinese access to these models has led to increased scrutiny of their security features. During this period, Anthropic engaged a leading zero-day bug hunter to address Amazon's security concerns regarding Fable and Mythos.
The dispute has evolved from a focus on specific models to broader implications for AI security research. Stamos noted that the vulnerabilities flagged by Amazon do not seem unique to Anthropic's models. Concerns were raised about a jailbreak that allows Fable to create proofs of concept, a common practice for security teams to address vulnerabilities. Katie Moussouris, CEO of Luta Security, indicated that Amazon's findings did not suggest mass exploitation but rather involved prompts for defensive security work.
Anthropic stated that it had collaborated with both internal teams and external researchers to test Fable 5 for vulnerabilities before its release. The company acknowledged that complete jailbreak resistance is not currently achievable and has focused on making jailbreaks difficult to execute.
Cybersecurity experts warn that if AI companies fear repercussions for developing models that can identify vulnerabilities, they may be inclined to remove essential capabilities, potentially disadvantaging U.S. defenders. Moussouris emphasized that any fixes could diminish the model's utility for cybersecurity purposes. Researchers argue that the administration's approach could give adversaries an advantage, as foreign AI developers may continue to utilize similar tools without the same restrictions.
The U.S. government is establishing a vulnerability clearinghouse through a recent AI security executive order, which aims to manage reports on threats to AI models. However, concerns remain about the availability of cybersecurity talent within the Trump administration following recent personnel changes.