Some AI labs are currently testing advanced mathematical problems using proprietary models that are not accessible to the broader scientific community. The authors of this report express their disapproval of this practice and urge these labs to cease testing advanced mathematical problems in this manner. The mathematical community has established norms regarding the dissemination and peer review of results, which are essential for the reliability and effectiveness of mathematical work. One key norm is that authors should fully understand and verify the correctness of their papers and take responsibility for their content. Furthermore, authors who make significant advancements in the field typically present their work at seminars and conferences to foster understanding among peers.
The authors emphasize the importance of human understanding in mathematics, especially as AI can produce mathematical arguments that the prompting human may not comprehend or verify. They seek to establish a new paradigm that incorporates human understanding into scholarly output. To this end, they solicited feedback from the mathematical community regarding responsible release practices for AI-generated results, receiving over 600 responses. Based on this feedback, they formulated a set of recommendations for AI labs whose models may significantly impact mathematics.
The recommendations are divided based on the level of human understanding accompanying the results:
1. For papers with a responsible mathematician, traditional norms should be followed, including posting preprints, submitting for peer review, and presenting findings at conferences.
2. For AI-generated results that lack human understanding, the following actions are recommended: - AI labs should enhance the initial written versions of results by citing relevant literature and producing proofs in a traditional mathematical style. - Results should be deposited in scholarly repositories that are independent of AI labs, ensuring proper citation and modification tracking. - AI labs should disclose the model name, prompts used, thought processes, computation time, and costs associated with the results. - Proofs should be formalized according to community standards, with clear documentation of the formalization status. - Each solution should include documentation on how AI was utilized and a summary of other problems attempted by the models.
The authors argue that AI labs have a responsibility to support the development of human understanding of their AI-generated outputs. They suggest that funding for this support should be managed by existing nonprofit institutions rather than the AI labs themselves. The report warns that reliance on proprietary models could create a two-tier system that alienates the mathematical community and exacerbates inequalities in access to mathematical resources.
To promote equitable access, the authors recommend that AI labs provide broad access to their publicly available models, facilitating collective verification and shared understanding within the mathematical community.