When I was in grad school, I had the opportunity to take a course from my adviser in which he discussed his current research and some open questions. It was a relatively accessible subject area and the questions were sometimes easy enough that we could meaningfully contribute.
On one particular Friday afternoon, he stated a conjecture that he hoped was true, and invited us to try to help him prove or disprove it. It was the sort of thing that he really wanted to be true; he liked things smooth and beautiful. I, on the other hand, hoped it was false as I like the weird and exceptional in mathematics. It was also the case that I had absolutely no command of the sort of machinery that one would use to prove such a thing, but I could certainly look for a counterexample.
I learned on Monday that he had spent the entire weekend trying and failing to prove it. I, on the other hand, had put all my energy into finding a counterexample and had one within an hour.
My single (quite small) contribution to mathematical research was a counterexample because it was all I could do. The story does illustrate that it can be helpful to have people with different tools, hopes, and motivations working on a problem, though. I was not, and will never be, even a shadow of that great mathematiciam I studied under, but on that occasion, I had reason to look in a different direction than he did.
bananaflag 2 hours ago [-]
> It was also the case that I had absolutely no command of the sort of machinery that one would use to prove such a thing, but I could certainly look for a counterexample.
Hm, as a mathematician, my experience feels opposite. A proof would be an adaptation of a proof I know, some tweaking it here and there. A counterexample would require some deep understanding of the structure of the objects involved, which frequently is beyond my comprehension.
But probably this is because I think of quite abstract objects which are harder to grasp. For numbers or polynomials, this would be the other way round.
parl_match 40 minutes ago [-]
> I learned on Monday that he had spent the entire weekend trying and failing to prove it. I, on the other hand, had put all my energy into finding a counterexample and had one within an hour.
He spent an entire weekend before having the wisdom to pause, and let someone else contribute their time to finding a counter.
codemog 57 minutes ago [-]
There’s a story in How to Solve It that’s basically the same.
hintymad 3 hours ago [-]
> The Jacobian Conjecture
Interestingly, Yitang Zhang of the twin-prime-conjecture fame spent 7 years working on the Jacobian conjecture under the advisor Tzuong-Tsieng Moh at Purdue. A key step in his thesis used a corollary of Moh's. It turned out that the corollary was incorrect. As a result, Moh refused to write any recommendation letter for Zhang, and Zhang couldn't find any teaching or research job and ended up spending years working at a Subway[1].
Imagine Zhag had ChatGPT in 1986 when he started working on the Jacobian Conjecture.
[1] Of course now this has become an inspiring story. That said, the story definitely invokes complex emotions. The best way to describe it is probably this Chinese poem, which I have no idea how to translate: 庾信平生最萧瑟，暮年诗赋动江关
tchalla 2 hours ago [-]
I was once in a presentation for a math PhD thesis. During the thesis, the evaluator of the thesis noticed a flaw in their proof. The student understood and then asked “What now?” The evaluator prof simply shrugged.
simonreiff 2 hours ago [-]
That sounds like the worst "exam nightmare" scenario imaginable, but did the student get the PhD in the end?
abdullahkhalids 1 hours ago [-]
Mistakes in proofs are relatively common, but such a mistake doesn't automatically mean that the proof is entirely wrong. Many times, the mistake is just in the exposition, and can be fixed easily. Other times, the mistake is fixable and the fix is apparent. Perhaps the author forget to treat a relatively trivial edge case. In the first two cases, the student would likely just pass with minor corrections to be submitted soon.
Sometimes, of course, the proof is just wrong. That is the dangerous case, which will cause either major corrections or failure.
angry_octet 1 hours ago [-]
Proof fix-ups are quite common, but if it was not possible, then no, not for that topic.
angry_octet 1 hours ago [-]
A recording of a car crash: discovering on live radio/podcast that the central tenet of your book is wrong, and amateurishly so.
Naomi Wolf 'death recorded' on BBC[1], skip to 5:51. After this the book was pulped and she had some sort of psychotic break during COVID and allied with ultra-right and COVID denialist loonies.
Is the video available not through a proprietary player?
> After this the book was pulped and she had some sort of psychotic break during COVID and allied with ultra-right and COVID denialist loonies.
That's quite an extreme shift considering she had previously been a leading figure in third-wave feminism and an OWS activist.
2 minutes ago [-]
globnomulous 59 minutes ago [-]
This is astonishing. How could a person go to the lengths of writing an entire book without ever looking up this sort of thing?
brohee 10 minutes ago [-]
The most astonishing thing is that she still holds the PhD whose thesis was the basis of the book. A stain on Oxford reputation.
jandrese 51 minutes ago [-]
> allied with ultra-right and COVID denialist loonies
That makes sense. She wouldn't want to be burned by people doing fact checking again. Write books for the crowd that will never do the legwork to discover that you're full of shit.
derbOac 2 hours ago [-]
Inspiring? Because of the twin prime conjecture success following his time in the wilderness? I suppose so.
I'm tired of tales like this in academics though. That's not a criticism of you for telling the tale, I'm just so tired of this kind of thing in academics in general. So, so, so much politics and public reputation management. Zhang should have never had to suffer like that.
As my own research has drifted more into math, I've been surprised at how many assertions in the literature turn out to be false. Not just false, but propagated into the applied literature extensively, and even when you point out the problems a lot of defensiveness and denial about it along the lines of Zhang's story.
I agree about wondering what would have happened if LLMs had been around in 1986. My guess is the outcome would have been the same for the same reasons?
My experience with LLMs in proofs is they can be very helpful, but also very wrong. It's like having another person with another set of hunches about what path to go down.
hintymad 2 hours ago [-]
> So, so, so much politics and public reputation management. Zhang should have never had to suffer like that.
Very true. Unfortunately, when there are people, there will be politics. I remember when reading Yau's autobiography, I kept marvel how much calculation, or "politics" if you will, that Yau mentioned or implied in the book.
> My guess is the outcome would have been the same for the same reasons?
At least Zhang didn't have to spend 7 years working on the Jacobian conjecture. He said in an interview that he always wanted to work on number theory. Moh asked him to work on Jacobian, and he obliged.
OG_BME 2 hours ago [-]
ChatGPT's idiomatic translation of the poem:
Yu Xin’s was a life of utter desolation;
in old age, his poems and rhapsodies stirred the riverlands.
zahlman 7 minutes ago [-]
Yu Xin's past was exquisitely tragic
Ages passed; now his works betray magic
Eufrat 1 hours ago [-]
I guess that works, it is always difficult to capture the cultural and linguistic melancholy of such poetry.
satvikpendem 4 hours ago [-]
That's a good thing. It saves people wasting time trying to prove something they now know to be false, so that they can move on to other things to prove, it's a more fruitful use of humanity's time overall at least in the field of mathematics.
parpfish 3 hours ago [-]
proofs by counterexample are effective but ultimately unsatisfying. they get you to an answer but they don't help help you understand and bend you r mind into seeing how the math works and lead you on to the new set of questions.
and for now as long humans are going to judge of what counts as an elegant or illuminating proof, there's going to be work for human mathematicians
remus 2 hours ago [-]
I would only agree partially. There are counterexamples that are not illustrative, but it is fairly common that in thinking about how to construct a counterexample you gain a more thorough understanding of the original problem and at least one fundamental issue which prevents the conjecture from being true.
mb7733 2 hours ago [-]
Are you thinking of proof by contradiction, which is rejected by constructionism?
[Dis]proof by counterexample is the most straightforward way to show a statement to be false. What better way is there to disprove a general statement like 'all x are y' than finding an 'x' that isn't 'y'?
SpicyLemonZest 2 hours ago [-]
It’s very straightforward, but it often doesn’t (and here didn’t) fully satisfy the curiosity that was embedded in the original problem. Why did the Jacobian conjecture seem to be true? Is there some underlying symmetry that’s very slightly broken? Or perhaps there’s all kinds of counterexamples, and the intuitive pattern is only real for certain kinds of functions which happen to predominate in our intuition. Then how should we adjust our intuitions to better capture the space of possible polynomial functions?
mb7733 2 hours ago [-]
Those are all good questions, but I don't really understand what alternative you or the OP are looking for actually resolving an untrue conjecture, besides a counter example.
SpicyLemonZest 1 hours ago [-]
I would frame it differently. The existence of compact counterexamples to a true-seeming conjecture suggests that there’s some deeper understanding waiting to be discovered. Fuzz testing for theorems, if that makes sense. I hope mathematicians in 2036 will be able to explain in detail why the Jacobian conjecture was false and identify which similar, true conjectures the community’s intuition was pointing towards.
delecti 2 hours ago [-]
You could spend the rest of your life coming up with conjectures that look elegant but are ultimately false. Disproof by counterexample only works if it's false, and we shouldn't be satisfied with a false conjecture to begin with.
moralestapia 2 hours ago [-]
Not much worth in understanding a statement that is wrong and has been shown wrong.
Unless you want to spend time "proving" that 2 * 2 = 1.
taneq 3 hours ago [-]
Maybe I’m just not pure enough but I find the whole concept of proof by counterexample to be elegant, and I don’t see why proving that something must be true is superior to proving that it can’t be false.
chowells 2 hours ago [-]
It's elegant if all you're concerned with is whether a conjecture is true or false. Answered, move along!
But mathematics is not a collection of facts. Mathematics is the study of abstraction. And what do you learn from a single data point? What can you abstract from that?
That's why just being a counterexample isn't really interesting. There has to be more than "counterexample" for there to be something to abstract. Was it generated from an analysis of the problem? Can the counterexample be generalized to explore the problem further? Is the counterexample a surprise in a way that suggests something is missing from current understanding?
Being a counterexample doesn't mean that something isn't interesting to a mathematician. But it's also not the interesting part.
bananaflag 2 hours ago [-]
You mean proof by contradiction, which is something different.
DiscoDays 3 hours ago [-]
It is also a good thing, because it helps to refine the theorem statement. At least, my humble experience in CS theory research is that I’d try to prove a theorem I want to be true, find a counterexample, refine the statement, and continue.
P.S. It helps that in CS lots of theorems are about either inductive or coinductive definitions.
soupspaces 2 hours ago [-]
Except you can't possibly know that. New insight can arise regardless of whether mathematicians are trying to prove or disprove a statement, and regardless of whether the statement ultimately turns out to be true or false.
dzdt 2 hours ago [-]
I suppose it will fall to AI as well to compose the mathematical equivalent of The Ballad of John Henry. Who will be the human champion, the last great hero who can deliver proofs "from the book" that a machine cannot outperform?
I wish I had LLM-built Lean formalisations in university, so much of the math in the slides had errors, and some professors are very bad and ungracious admitting it, while simultaneously rejecting requests for clarifications by saying "the proof is in the slides".
Of course Lean proofs are rarely a good way to understand proofs, but hopefully they can be used to generate more human understandable arguments.
1 hours ago [-]
VladVladikoff 3 hours ago [-]
A lot of this math is beyond my comprehension, but it often seems to talk of proofs of theorems. What I want to know is if we continue on this accelerated AI mathematics trajectory, will we eventually be discovering new forms of math that will in turn have some applications down the line in engineering or biomedicine etc? I guess what I’m asking is are we on the cusp of a huge breakthrough for humanity, or largely just proving what was already known?
koolba 3 hours ago [-]
> will we eventually be discovering new forms of math that will in turn have some applications down the line in engineering or biomedicine etc?
If we do it probably won’t be for a long while. We’re barely using math from a couple hundred years ago for most applied usage.
hgoel 2 hours ago [-]
Engineering and biomedicine, probably in the long term (if at all). But accelerated development of new mathematical methods has a possibility of proving to be relevant for fundamental physics research.
Occasionally large improvements in our models of the universe have been associated with the development of mathematical tools that allow those models to be expressed and/or tested.
the AI doesn't even gloat. a rival mathematician would at least title their paper 'a remark on the falsity of...'
wizzwizz4 4 hours ago [-]
Human mathematicians have been being out-counterexampled for at least two decades. The main difference, as I understand, is that (A) we now have a lot more compute to throw at such things, and (B) it is currently trendy to do so. But the sizes of counterexample we're seeing are around about what I'd expect pre-generative-AI counterexample search systems to be able to find.
It's not easy to find a counterexample to the Jacobian conjecture, by any means – by which I mean to say that naïve brute-force search will take too long – but the scope of existing searches listed on Wikipedia[0] suggest that many tricks are already known, and that people just hadn't looked, systematically, for a counterexample in three variables before. Wikipedia writes:
> Tzuong-Tsieng Moh checked the conjecture for polynomials of degree at most 100 in two variables.[17][18]
where reference 17 is from 1983, and reference 18 is a preprint with no given date. Knowing very little about this problem, my impulse is to side with the unnamed faculty member cited in the article:
> [who] said to me that the fact that the counterexample was so easy to find just indicated that humans had not spent enough time thinking about the problem,
For context, the auto-generated counterexample is in three variables, has degree 7, and was discovered in 2026.
paulpauper 4 hours ago [-]
mathematicians have been using computers for well over half a century, but this was after "bounding" the problem first and then running through the cases with a computer. Now AI is doing the first part. However, mathematicians are still needed at crafting prompts, and knowing where to look, still. The prompt for the Jacobian conjecture was obviously not random. the search space is too big to just try all the combinations of 3 variable polynomials.
skinner_ 2 hours ago [-]
Okay, I'm not sure about the original one, but here is the prompt of a successful reproduction:
Obviously it is not random, but it's very generic. No mention of search space or how to reduce it.
hobonation 4 hours ago [-]
This is the best take. Computers don't care about this stuff. A computer could make a movie, but only a human can appreciate it.
We're a good team, and that's ok.
criddell 3 hours ago [-]
For now. I wonder if we will ever get to the point where the computer starts doing mathematics that we just can't understand. Surely there must be some limit to what we can understand (like how a gorilla will never understand prime numbers, there are probably limits to our intelligence as well).
zeroonetwothree 3 hours ago [-]
Mathematics only really matters insofar as humans can understand it.
hgoel 2 hours ago [-]
Aren't deep learning models themselves a case where we have hints of some deeper underlying logic to why some things are more effective than others, but we lack the mathematical tools to properly work it out for anything of practical size?
All we're able to do is apply flawed analogies, generic information theoretical models, trial and error, post-hoc rationalizations and benchmarks without really understanding why.
drivebyhooting 2 hours ago [-]
Doesn’t this generalize?
Mathematics matters less than less as fewer people are capable of understanding it.
So whatever cutting edge, deep insight about the nature of groups matters less than different equations which matters less than solving linear equations, etc.
stabbles 3 hours ago [-]
Not really, a theorem with a hard proof can have simple but important corollaries. It's also not unthinkable that theorems exist with proofs that cannot reduce to something simple/short.
nullsanity 4 hours ago [-]
[dead]
skinner_ 4 hours ago [-]
> The prompt for the Jacobian conjecture was obviously not random. the search space is too big to just try all the combinations of 3 variable polynomials.
Maybe the prompt contained a part like this: "the search space is too big to just try all the combinations of 3 variable polynomials, so be clever about it". Or maybe this part was omitted from the prompt, because modern LLMs are smart enough to figure this out without us having to mention it.
jknoepfler 3 hours ago [-]
If someone has written that in a paper or article they've ingested, as they no doubt have, then sure.
jameshart 1 hours ago [-]
The ability of LLMs to solve problems is not confined to the training data they ingested. Claude knows how to read mathematical papers because of its training data, but it can and will pull in literature relevant to a specific problem into its context.
We really need to stop thinking about LLM training data as the knowledgebase they build from and instead consider it more the skillset they start with.
sesm 3 hours ago [-]
I don't think there was a 'prompt' for it, rather a long and dedicated work of a professional mathematician which involved LLM in some capacity. I'm sure the search step wasn't an ad-hoc script running in a Claude Code session (as somebody would naively assume), it was an optimized numerical code running in Anthropic's compute cluster. Note that details are not published yet and `__alpoge__` is officially working at Anthropic.
QuesnayJr 4 hours ago [-]
If the poster's (is it Kevin Buzzard?) suggestion works out and AI finds a counterexample to the Hodge conjecture, that would be a really big deal. It's one of the Millenium problems, for example.
One thing that he mentions that already quite surprising is that AI was able to autoformalize the Golod-Shaferevich theorem and proof.
mcshicks 3 hours ago [-]
It is Kevin Buzzard. It's kinda small font on my phone but if you look at the "about xena" link it says it's his site.
williamstein 2 hours ago [-]
Agreed, it's definitely Kevin. His writing style is unmistakable.
OG_BME 2 hours ago [-]
I think he was being provocative and maybe a bit tongue-in-cheek when he said that. A candidate object alone doesn't resolve the Hodge Conjecture. Any apparent counterexample would have to prove that no algebraic cycle exists, no invariant subspace exists, or that every element of an infinite ideal is nilpotent. Much harder, but not impossible.
nephihaha 3 hours ago [-]
"Outcounterexampled": there's a neologism worthy of German.
FabHK 2 hours ago [-]
Übergegenbeispielt.
soupspaces 2 hours ago [-]
vibe counterexamplemaxxing
prmph 53 minutes ago [-]
Better as one word "vibecounterexamplemaxxing"
riazrizvi 2 hours ago [-]
The framing in these posts is nonsense. ChatGPT isn't doing shit. Human mathematicians using ChatGPT are breaking boundaries.
Rendered at 00:25:46 GMT+0000 (Coordinated Universal Time) with Vercel.
On one particular Friday afternoon, he stated a conjecture that he hoped was true, and invited us to try to help him prove or disprove it. It was the sort of thing that he really wanted to be true; he liked things smooth and beautiful. I, on the other hand, hoped it was false as I like the weird and exceptional in mathematics. It was also the case that I had absolutely no command of the sort of machinery that one would use to prove such a thing, but I could certainly look for a counterexample.
I learned on Monday that he had spent the entire weekend trying and failing to prove it. I, on the other hand, had put all my energy into finding a counterexample and had one within an hour.
My single (quite small) contribution to mathematical research was a counterexample because it was all I could do. The story does illustrate that it can be helpful to have people with different tools, hopes, and motivations working on a problem, though. I was not, and will never be, even a shadow of that great mathematiciam I studied under, but on that occasion, I had reason to look in a different direction than he did.
Hm, as a mathematician, my experience feels opposite. A proof would be an adaptation of a proof I know, some tweaking it here and there. A counterexample would require some deep understanding of the structure of the objects involved, which frequently is beyond my comprehension.
But probably this is because I think of quite abstract objects which are harder to grasp. For numbers or polynomials, this would be the other way round.
He spent an entire weekend before having the wisdom to pause, and let someone else contribute their time to finding a counter.
Interestingly, Yitang Zhang of the twin-prime-conjecture fame spent 7 years working on the Jacobian conjecture under the advisor Tzuong-Tsieng Moh at Purdue. A key step in his thesis used a corollary of Moh's. It turned out that the corollary was incorrect. As a result, Moh refused to write any recommendation letter for Zhang, and Zhang couldn't find any teaching or research job and ended up spending years working at a Subway[1].
Imagine Zhag had ChatGPT in 1986 when he started working on the Jacobian Conjecture.
[1] Of course now this has become an inspiring story. That said, the story definitely invokes complex emotions. The best way to describe it is probably this Chinese poem, which I have no idea how to translate: 庾信平生最萧瑟，暮年诗赋动江关
Sometimes, of course, the proof is just wrong. That is the dangerous case, which will cause either major corrections or failure.
Naomi Wolf 'death recorded' on BBC[1], skip to 5:51. After this the book was pulped and she had some sort of psychotic break during COVID and allied with ultra-right and COVID denialist loonies.
[1] https://www.bbc.com/news/av/world-us-canada-48639663
> After this the book was pulped and she had some sort of psychotic break during COVID and allied with ultra-right and COVID denialist loonies.
That's quite an extreme shift considering she had previously been a leading figure in third-wave feminism and an OWS activist.
That makes sense. She wouldn't want to be burned by people doing fact checking again. Write books for the crowd that will never do the legwork to discover that you're full of shit.
I'm tired of tales like this in academics though. That's not a criticism of you for telling the tale, I'm just so tired of this kind of thing in academics in general. So, so, so much politics and public reputation management. Zhang should have never had to suffer like that.
As my own research has drifted more into math, I've been surprised at how many assertions in the literature turn out to be false. Not just false, but propagated into the applied literature extensively, and even when you point out the problems a lot of defensiveness and denial about it along the lines of Zhang's story.
I agree about wondering what would have happened if LLMs had been around in 1986. My guess is the outcome would have been the same for the same reasons?
My experience with LLMs in proofs is they can be very helpful, but also very wrong. It's like having another person with another set of hunches about what path to go down.
Very true. Unfortunately, when there are people, there will be politics. I remember when reading Yau's autobiography, I kept marvel how much calculation, or "politics" if you will, that Yau mentioned or implied in the book.
> My guess is the outcome would have been the same for the same reasons?
At least Zhang didn't have to spend 7 years working on the Jacobian conjecture. He said in an interview that he always wanted to work on number theory. Moh asked him to work on Jacobian, and he obliged.
Yu Xin’s was a life of utter desolation; in old age, his poems and rhapsodies stirred the riverlands.
and for now as long humans are going to judge of what counts as an elegant or illuminating proof, there's going to be work for human mathematicians
[Dis]proof by counterexample is the most straightforward way to show a statement to be false. What better way is there to disprove a general statement like 'all x are y' than finding an 'x' that isn't 'y'?
Unless you want to spend time "proving" that 2 * 2 = 1.
But mathematics is not a collection of facts. Mathematics is the study of abstraction. And what do you learn from a single data point? What can you abstract from that?
That's why just being a counterexample isn't really interesting. There has to be more than "counterexample" for there to be something to abstract. Was it generated from an analysis of the problem? Can the counterexample be generalized to explore the problem further? Is the counterexample a surprise in a way that suggests something is missing from current understanding?
Being a counterexample doesn't mean that something isn't interesting to a mathematician. But it's also not the interesting part.
P.S. It helps that in CS lots of theorems are about either inductive or coinductive definitions.
[1] https://en.wikipedia.org/wiki/John_Henry_(folklore)
[2] https://en.wikipedia.org/wiki/Proofs_from_THE_BOOK
Of course Lean proofs are rarely a good way to understand proofs, but hopefully they can be used to generate more human understandable arguments.
If we do it probably won’t be for a long while. We’re barely using math from a couple hundred years ago for most applied usage.
Occasionally large improvements in our models of the universe have been associated with the development of mathematical tools that allow those models to be expressed and/or tested.
It's not easy to find a counterexample to the Jacobian conjecture, by any means – by which I mean to say that naïve brute-force search will take too long – but the scope of existing searches listed on Wikipedia[0] suggest that many tricks are already known, and that people just hadn't looked, systematically, for a counterexample in three variables before. Wikipedia writes:
> Tzuong-Tsieng Moh checked the conjecture for polynomials of degree at most 100 in two variables.[17][18]
where reference 17 is from 1983, and reference 18 is a preprint with no given date. Knowing very little about this problem, my impulse is to side with the unnamed faculty member cited in the article:
> [who] said to me that the fact that the counterexample was so easy to find just indicated that humans had not spent enough time thinking about the problem,
For context, the auto-generated counterexample is in three variables, has degree 7, and was discovered in 2026.
https://aaronlou.com/jacobian_counterexample_prompt.pdf
Obviously it is not random, but it's very generic. No mention of search space or how to reduce it.
We're a good team, and that's ok.
All we're able to do is apply flawed analogies, generic information theoretical models, trial and error, post-hoc rationalizations and benchmarks without really understanding why.
Maybe the prompt contained a part like this: "the search space is too big to just try all the combinations of 3 variable polynomials, so be clever about it". Or maybe this part was omitted from the prompt, because modern LLMs are smart enough to figure this out without us having to mention it.
We really need to stop thinking about LLM training data as the knowledgebase they build from and instead consider it more the skillset they start with.
One thing that he mentions that already quite surprising is that AI was able to autoformalize the Golod-Shaferevich theorem and proof.