This will be, at least for the foreseeable future, a dynamically updated section on my views on AI use in general, and its interaction with mathematics in particular. I am a signatory to all the three commitments of the Association for Human Mathematicians, which are as follows.
1. Members commit to not working with commercial AI companies by providing technical labour, mathematical knowledge, consultation, or publicity.
2. Members commit to not publishing mathematical texts (including papers, reviews, referee reports, lecture notes, teaching materials, and books) generated by AI models. Members only publish their own writing, as opposed to outputs of AI tools. We affirm that mathematical writing should have human authorship.
3. Members (of the AI-free caucus) do not use AI models in a research capacity. Members (of this caucus) engage in core mathematical activities, such as finding proofs, exploring examples, and posing research questions, without prompting Al tools.
Note: The third commitment is optional to which I am a signatory.
I endorse, what is in my experience the prevailing view of mathematicians, of seeing mathematics as a social activity, supported for and by a community with no primary role to be played by machines (let alone ones that lead to no betterment of understanding). Years of practice with mathematics has convinced me, and many others, that the development of true mathematical understanding lies in the toil to get to answers -- not in looking up, not asking at the first opportunity, and certainly not in using machines all-too-willing to pass for answers anything that sounds smart enough.
That the big AI corporations have chosen to punch the culture and practice of mathematics with their PR stunts is but one of the smaller problems the world faces from them. The absolute disregard shown towards the environment (both social and physical), intellectual property rights, and the increasing potential for misuse that cannot even be pinned down precisely only goes on to prove that these corporations must be done away with entirely. Maybe arson isn't always a bad thing?
If at all the AI arms race is to continue (and I don't see why it has to -- the necessity and inevitability all appear to be fabricated by some enthusiasts, while major AI corporations continue to struggle with their finances and an ever-intensifying environmental assault), I would support the growth of smaller-scale, open-source models which are expressly used for problems of major concern. This, in my opinion, is one way the technology actually does something for the betterment of humanity, something that dumping hundreds of claimed but unverified solutions to problems in pure mathematics certainly hasn't.