AI Companies Propose Self-Regulation with Embedded Evaluators

<p>Dario Amodei, Sam Altman, and Elon Musk, leaders of AI companies Anthropic, OpenAI, and xAI respectively, have agreed on the introduction of "embedded evaluators" to monitor safety practices within their organizations. This consensus emerged amid growing concerns about AI safety. Both Amodei and Altman committed to integrating these evaluations, while Musk expressed support for the initiative.</p><p>The specifics of how this regulatory system would function remain unclear. Evaluators would be assigned to work within AI companies, tasked with identifying risks and auditing safety practices. They would report their findings, although Amodei noted that certain sensitive information might be withheld from public disclosure.</p><p>Experts involved in the evaluations are expected to come from both small nonprofits and large consulting firms, including Accenture, which has a billion-dollar partnership with Anthropic. Previous investigations, such as one led by the nonprofit METR into OpenAI's practices, have raised concerns about ethical conduct within AI organizations.</p><p>On-site evaluations are not new; similar practices exist in high-risk sectors like aviation and banking. However, the voluntary nature of these evaluations raises questions about the independence of the organizations chosen by the companies, as they may only have access to information that the companies permit. Some METR staff have expressed skepticism about the effectiveness of such evaluations.</p><p>Embedded evaluators face challenges in maintaining independence while being funded by the companies they monitor. Past instances, such as the 2008 financial crisis, highlight the risks of conflicts of interest in regulatory environments. Critics argue that allowing AI companies to select their evaluators undermines accountability.</p><p>Despite the potential benefits of having embedded evaluators, experts emphasize the need for stronger government oversight to ensure that these evaluators are qualified and independent. There are calls for regulatory frameworks that would mandate safety standards and establish consequences for non-compliance.</p><p>In response to public concerns and internal pressures, both Anthropic and OpenAI have shown support for state and federal legislation regarding third-party audits. Recent discussions among industry leaders suggest a shift toward accepting some level of regulation, although critics argue that current proposals may not be sufficient to address the risks associated with AI technology.</p><p>As the debate continues, some politicians are advocating for more stringent measures, including international treaties and bans on certain AI capabilities. California Governor Gavin Newsom recently issued an executive order to require onsite auditors and develop an AI kill switch, reflecting a growing recognition of the need for oversight in the rapidly evolving AI landscape.</p><p>While the agreement among AI leaders to implement embedded evaluators is a step forward, there remains skepticism about whether these measures will be effective in ensuring safety and accountability in AI development.</p>