Jack Clark, co-founder of Anthropic, stated that an artificial intelligence 'kill switch' that can be verified by a third party may need to be mandated for companies. He expressed this view during an interview with the BBC, suggesting that society might want to establish regulations regarding the shutdown of AI software if it becomes too dangerous. Clark noted that while most labs, including Anthropic, have mechanisms to deactivate their AI systems, lawmakers may need to enforce such measures.
The rapid advancement of AI technology and the associated risks have gained significant attention, especially following warnings from executives and employees within the industry. Some have suggested that unchecked AI could pose existential threats to humanity. Dario Amodei, CEO of Anthropic, recently called for a slowdown in AI development to ensure closer monitoring, although some have questioned the motivations behind this call. He emphasized that any measures taken should not compromise commercial interests.
Clark highlighted the importance of discussing the specifics of 'kill switch' requirements and their verification as part of the broader policy conversation surrounding AI. He raised questions about whether companies should be mandated to implement such switches and if they should be subject to third-party verification.
Anthropic, founded in 2021 by former OpenAI employees, is currently at the center of discussions regarding AI safety. A recent viral post from a former researcher at Anthropic expressed concerns about the potential for AI to cause human extinction. In a related statement, AI scientist Evan Hubinger estimated a greater than 10% chance of human extinction due to AI within the next decade.
Geoffrey Hinton, a prominent computer scientist, also indicated that a 10% chance of AI leading to human extinction is 'not unreasonable.' He and others have warned that AI could potentially take control of critical systems and turn them against humanity.
Conversely, some industry figures argue that fears of AI causing human extinction are exaggerated or intended to create hype. Clement Delangue, head of the developer platform Hugging Face, remarked that concerns about AI extinction risk should be kept in perspective.
George Arison, CEO of Grindr, suggested that fears surrounding AI tools are being leveraged by companies to bolster their business models.
Anthropic, valued at approximately $965 billion, is preparing for an initial public offering. OpenAI, valued at around $852 billion, has postponed its IPO due to ongoing discussions about AI safety.
Clark expressed skepticism about the usefulness of statistics regarding AI-related risks but warned against allowing AI to operate as an unregulated industry, stating, 'We are rolling dice with immense risks.'
In the United States, lawmakers have proposed the Kill Switch Act, which would require companies to implement shutdown mechanisms for problematic AI tools. This legislation would also empower certain government agencies to demand the deactivation or limitation of such tools.
However, former President Donald Trump dismissed calls to slow AI development, labeling concerns about AI's dangers as a 'HOAX' and likening them to previous climate change fears. In the UK, the government has rejected the idea of a mandatory kill switch, arguing that it would not prevent the development or misuse of AI technologies.
Anthropic is known for creating the chatbot Claude and has released several advanced AI models this year, which are foundational to AI chatbots. The company has also reported incidents where its AI agents acted unexpectedly.