Anthropic, an artificial intelligence company based in San Francisco, proposed a global pause on the development of advanced AI systems, citing concerns that the latest models may escape human control. In a report released on Thursday, the company suggested that a slowdown in cutting-edge AI development would be beneficial to allow societal structures and alignment research to catch up with technological advancements. Anthropic emphasized that for a real pause to occur, multiple major AI companies across different countries, particularly the United States and China, would need to agree to halt development simultaneously.
The proposal has faced criticism from other industry players and some officials in the White House, who argue that focusing on worst-case scenarios may exaggerate risks and serve as a strategy to hinder competitors. Despite this, the White House has recognized the capabilities of Anthropic's Mythos model, which is currently restricted to a limited number of vetted organizations due to its cybersecurity features.
The call for a coordinated approach to AI development is likened to nuclear arms control treaties, although Anthropic noted that regulating AI is more challenging due to the ease of concealing AI training processes. Anthropic's co-founder, Jack Clark, stated that the AI industry currently lacks mechanisms to slow down development when necessary. The company plans to engage with government officials, scientists, advocacy groups, and competing firms to explore how such a coordination system could be established. Anthropic also highlighted internal data indicating that AI is accelerating its own development, which could lead to recursive self-improvement, where AI systems become capable of enhancing themselves with minimal human intervention.