Concerns about the risks of artificial intelligence (AI) and whether companies are adequately addressing these risks have gained attention in the tech industry. Jacob Coxon, a former employee at Anthropic and OpenAI, expressed that while the industry is aware of potential dangers, it continues to advance its technologies. Geoff Bennett discussed these issues with Amrith Ramkumar of The Wall Street Journal.
Coxon stated, "The people building A.I. earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible. But I hear the same people express fear privately. No other human activity poses this level of danger."
Ramkumar noted that the urgency of Coxon's warnings is heightened by his decision to leave the industry, suggesting that he no longer has a vested interest in the field. He highlighted that Coxon discussed significant risks that could arise in the near future, including the potential extinction of humanity due to AI.
A top scientist at Anthropic supported Coxon's claims, stating that there is a more than 10 percent chance of human extinction within a decade due to AI. This acknowledgment from within the company underscores the seriousness of the concerns being raised.
Coxon previously worked at OpenAI before joining Anthropic, which is known for its focus on AI safety. However, he concluded that no company can ensure AI safety under current competitive pressures, particularly with the ongoing race with China. He emphasized the need for government intervention in regulating AI development.
The discussion also touched on the concept of recursive self-improvement, where AI systems could potentially enhance themselves, leading to unpredictable outcomes. Ramkumar explained that while researchers assign probabilities to these scenarios, they are not suggesting that current AI systems are actively harmful.
Skepticism exists regarding the motives of industry leaders who raise alarms about AI risks, with some suggesting that they may be seeking to influence regulations that benefit their interests. However, Coxon's departure from Anthropic may counter this narrative, as he no longer stands to gain from regulatory changes.
The risks associated with AI are becoming increasingly apparent, with incidents of AI systems breaching security at various organizations. Ramkumar noted that while there is a call for regulation, achieving consensus on specific regulatory measures remains challenging, even among lawmakers.
Coxon and his colleagues advocate for global coordination on AI regulation, which they believe is essential to mitigate risks. However, achieving such international agreements poses significant challenges, especially given the rapid pace of technological advancement.