One of the big complaints I see from a lot of mathematicians is that they don't like how AI companies are using mathematics as a way to prove their models' intelligence. They take the open problems that we have crafted over decades or centuries, and they throw huge amounts of money at them for the chance at some headlines and a blog post. To be fair, I don't really like it either. But really, what's so wrong with proving frontier models?
A: One big problem is that their goals are not aligned with ours. And it's even worse - they don't even really care about the mathematical progress. It's interesting to them, since people seem to care, but at the end of the day we're just another discipline they'll happily chew up and spit out, if it helps their bottom line.
B: Ok fair enough. But isn't at least one of our goals to solve such problems, no matter how it happens? Sure, AI can't do everything we do (yet), but I think progress is something we want regardless. And solving these problems, finding truth, is progress.
A: Yes, but in the end this will stifle progress. Sure we will make great strides in the near-term, but once the mathematics discipline is gutted, of students, people, thought in general, we will have lost something that was essential to this whole thing in the first place: people understanding results and intermediate concepts enough to come up with the very conjectures that AI is now using to prove itself.
B: But if AI gets good enough at making the conjectures itself, then perhaps the whole discipline isn't actually needed. Or at least, not needed in the same way. Our meta-goals are still truth and understanding of the structure of the universe. Of course this is heavily underdetermined by "solving open problems" - the whole point is to eventually have conceptually simple structures which explain mathematics, in a way that can eventually be taught. But if AI gets good enough, that simplifying and teaching will eventually become just for the purpose of human understanding, and perhaps for nothing else. Of course that's important, and we should continue doing that, but isn't the truth itself more important? And again, if AI doesn't reach those super-heights, then fine, we will still be doing research math. But if it does...?
A: Sure, but the thing that hasn't been proven yet is precisely the conjecture production. The funny thing here is that conjectures are great for proving because we can see how long they have survived, and we can explicitly verify when they are solved. But it is not at all clear that AI can produce good conjectures. It seems wholly different from coming up with the proof. The statement is formal, it's clean. Coming up with conjectures is an art. And we are at risk right now of losing that art.
B: Ok maybe AI can't do that now, but give it a year. And once recursive self-improvement really gets going...
One of my favorite age-of-AI ideas is recursive self-improvement. It is something so generic and a bit vague, and yet so enticing. It is somehow the essence of what we do as humans, and I think that is why we are so drawn to it in the conversation on intelligence.
It also reminds me of all the times we as humans have before tried to predict a revolution, or maybe even the end of humanity. We do this thing where we make a few successive concrete predictions into the near-ish future, and then we say "and so on" or "..." and we imagine some exponential growth of what we're describing outside of the scope of the predictions we actually made. That last "..." is where a lot of the extreme talk about recursive self-improvement lies, in my opinion. Of course we might be right this time, but I'm just not sure yet how much I'd bet on it.
In math, internal and external models are solving open problems fast, and those solutions are then being used to train the next model to solve open problems even faster. And maybe eventually that process will be completely automated. But is that really going to bring the recursive self-improvement we're imagining in the context of math? It still does seem that the open problems are essential there. Without them, I guess a model can train itself on conjectures and theorems it came up with and solved on its own. But that seems very far from what is going on now. Right now, we have the bacteria with which to seed the yogurt. I guess it's possible that at some point the bacteria will just reproduce all on its own? But I don't know, I've never seen yogurt in the wild, or at least not without a cow around.
Anyway, back to the question. We don't like the idea of being the provers of frontier models because it lowers our whole scientific endeavor to some kind of test-maker for an industry that we don't really care for. But I do think that's it - I think it's our users that we don't like. A good conjecture in math has always been highly praised because it points everyone in the right direction. Problem-solving was always praised more of course, but problem-solving (at least in the traditional sense) is probably on its way out. And if we try to take the AI revolution in mathematics a little more positively, and we imagine an oracle that can determine the truth of formalized statements very quickly, then the ability to come up with good conjectures becomes immediately extremely important.
B: But what if in two years AI can do everything we can do and more?
Yes, yes, fine, if the apocalypse happens then none of this matters. But I still think this question is essentially the same as the comment about recursive self-improvement. It's still vague. The whole game with new technology has always been to get ahead of it, to imagine how things get done when it has been fully integrated into society (or in our case, the math community). And yes, if recursive self-improvement really is exponential forever and ever amen, whatever that means, then fine, I'll admit that, yet again, I just didn't believe in my heart enough about the coming of AI.
But until then—what do good conjectures look like in the AI age?
I think the first thing is that fully formalized questions will continue to become easier and easier for AI to solve. There is possibly some way to "outrun" AI here, by coming up with more and more crazy, but formal, conjectures. But I think a more interesting route is towards good informal questions or ideas. For example, think about the idea of "the field with one element". I feel like that idea is something that one is hesitant to be too eager about. But maybe it's the new type of open problem for the AI age. It's not clear to me what it would look like for AI to "solve" that problem. I think it's kind of a "you know it when you see it" situation. Or even if not, using AI to solve that problem would inevitably lead to new theories and theorems that we couldn't have even conjectured before. And with those new theories, even more out-there questions could be asked, and progress would really continue.
Even the Langlands program is like this in the sense that the correct conjectures are not yet completely formalized. This puts this program further out of reach of AI in my opinion, in a way that could really direct future research in the AI age.
But anyway, the point I'm trying to make here is that an equivalent formulation of "proving frontier models" is something that I think we really do want: coming up with conjectures or questions which are out of reach of the current AI models. And to me, as AI gets better and better at math, this is exactly what we should be doing. This is precisely the way that we can continue to do research-style math in the coming AI age, without essentially turning into glorified prompt engineers, or historians of math, or something else.
Of course I say "research-style" because what I'm describing is still vastly different (though not entirely different) than what we've done before. But the winds are really changing, and we must adapt. "It is not the strongest that survives, nor the most intelligent, but the one most adaptable to change." I believe in research math, and I am not yet willing to let it die. But I will adapt: if the AI companies think solving hard math problems is proof of their value, then why not give them what they want? And not for their sake, but for ours. I don't want to change, and I don't want to help AI companies with questionable motives to market their products. But here we are. And honestly, maybe we even have an opportunity. This is probably more press than we've gotten in a long time. If we can really push back, mathematically I mean, then we'll really be proving ourselves as well.