Recent warnings from researchers at major artificial intelligence labs have raised concerns about the rapid pace of AI development and the potential risks it poses to humanity. These warnings stem from competition between the US and China, leading to fears that labs are advancing towards superintelligent models that could exceed human capabilities. Father Paolo Benanti, an advisor to the Catholic Church on AI, argues that the focus on superintelligence detracts from the need for public discourse on how to regulate these companies. He emphasizes that ethics in AI should not be solely determined by corporate boards but should involve public debate. Benanti advocates for democratic regulation of AI technologies.
In an essay following the safety warnings, Anthropic CEO Dario Amodei suggested that new AI models should undergo mandatory third-party evaluations before their release. He also called for restrictions on distillation, a process where labs train their AI on outputs from proprietary models. Others have proposed a total ban on recursive self-improvement, where AI enhances its own development.
Benanti, however, is more focused on establishing standards to ensure that AI remains under human control rather than restricting development methods, which he believes could hinder innovation. He criticizes the narrative from leading labs as self-serving, suggesting it resembles cartel behavior when a few companies dictate industry rules.
He also notes the complexities in assessing claims about AI capabilities due to various economic and political incentives. While companies like OpenAI and Anthropic have supported calls for a slowdown in AI development, they are also preparing for public offerings, indicating significant financial stakes in the market. The competitive landscape between the US and China further complicates the situation.
Benanti critiques the reverent language used in discussions about superintelligence, arguing it misrepresents AI as something beyond statistical computation. He believes that the emergence of AI presents an opportunity for religious institutions to engage in public debate about technology's implications for society. Although Benanti is a priest, he does not officially represent the Holy See but advises the Church as a scholar.
In May, Pope Leo published an encyclical addressing AI, warning against a technocratic approach that prioritizes efficiency over human values. He called for international cooperation among AI companies and superpowers to ensure that technology serves the common good. Benanti warns that the focus on superintelligence could distract from the goals outlined in the encyclical, urging for human oversight in AI development.