Realizing more and more how modern-day AI taps into some very old urges.
-
@johncarlosbaez
I keep wondering.
In every field, there are early reports of astounding usefulness of LLMs.Followed by debunkings and finally a realization that the bullshit generator can generate plausible-sounding bullshit, and that's it.
In coding, potentially the closest thing to maths in application, it shows itself as a shortcut to replace typing, but with a cost in review that outstrips the functionality entirely, ending up slower than without the tool.
In any field, the only application where there appears to be a genuine benefit (albeit partial and fallible) is as Search.
So what's up with Maths? Is it an application as Search, and somehow magically able to discern things the thing does not and cannot understand? Or is it once again overhyped bullshit?
@androcat - math is different from most subjects in that it doesn't need to be "about" anything: it can be a game of following rules. Treating it as a game is superficial, because that attitude can't tell the difference between important good math and silly bad math. But if you know what theorems you want to prove, something that's good at playing the game can help you.
Nonmathematicians often don't realize that the hard part is figuring out what theorems are worth proving.
-
@johncarlosbaez from which book is this fragment?
@FrohlichMarcel - check out the alt text! I put a lot of work into alt text.
-
@androcat - math is different from most subjects in that it doesn't need to be "about" anything: it can be a game of following rules. Treating it as a game is superficial, because that attitude can't tell the difference between important good math and silly bad math. But if you know what theorems you want to prove, something that's good at playing the game can help you.
Nonmathematicians often don't realize that the hard part is figuring out what theorems are worth proving.
I definitely see it as a game. Or a set of games, really, because one of the fun parts is changing the rules and seeing what that new game will be like.
... but I don't see how an LLM can do any of that.
It's not a brain, it doesn't understand what it intakes, and it can't even follow rules : it only apes randomized parts of the (stolen) corpus it's working from.So either, examples of success are
1) cherry-picked from an ocean of failures
OR
2) actually failures that have been manually reworked in entirety...unless something makes math very different from every other field we've seen trampled by the GenBS so far.
-
@petealexharris @johncarlosbaez I do find it really interesting that this idea of a uniquely capable yet maliciously compliant non-human entity is one of our ancient and universal cross cultural archetypes.
@marshray @petealexharris - hmm, now I really want to know why we have that archetype. Given that we have it, it's not so surprising that when we got the ability to make it real, we did. But why were we so interested in all-powerful trickster genies long before we could really create them?
-
I definitely see it as a game. Or a set of games, really, because one of the fun parts is changing the rules and seeing what that new game will be like.
... but I don't see how an LLM can do any of that.
It's not a brain, it doesn't understand what it intakes, and it can't even follow rules : it only apes randomized parts of the (stolen) corpus it's working from.So either, examples of success are
1) cherry-picked from an ocean of failures
OR
2) actually failures that have been manually reworked in entirety...unless something makes math very different from every other field we've seen trampled by the GenBS so far.
@androcat - In math it's fun to change the rules and seeing what the new game will be like. But the reason math is actually important to nonmathematicians is that if you choose the rules right, the results help you do magic: for example invent calculus, which lets you invent classical mechanics, which lets you predict how objects move, etc. etc. To me, the art of mathematics is choosing rules that help us understand the universe.
But so far I believe people have only used LLMs to win games with pre-established rules. Much harder than Go, but not really what makes math fun or important. -
@androcat - In math it's fun to change the rules and seeing what the new game will be like. But the reason math is actually important to nonmathematicians is that if you choose the rules right, the results help you do magic: for example invent calculus, which lets you invent classical mechanics, which lets you predict how objects move, etc. etc. To me, the art of mathematics is choosing rules that help us understand the universe.
But so far I believe people have only used LLMs to win games with pre-established rules. Much harder than Go, but not really what makes math fun or important.@johncarlosbaez
But are the machines good enough to do harm, or merely an annoyance?Because some people here have been talking about an "abundance of proof" caused by these text generators.
-
@johncarlosbaez
But are the machines good enough to do harm, or merely an annoyance?Because some people here have been talking about an "abundance of proof" caused by these text generators.
@androcat - I expect it'll get easier and easier to prove things using LLMs. So, as long as people think math is mainly about proving things - things that some mysterious men have already decided are worth proving - we'll hear people say that math is getting taken over by LLMs.
At some point people will be forced to realize that there's much more to math than proving things. But will this happen before or after lots of mathematicians get fired?
-
@marshray @petealexharris - hmm, now I really want to know why we have that archetype. Given that we have it, it's not so surprising that when we got the ability to make it real, we did. But why were we so interested in all-powerful trickster genies long before we could really create them?
Maybe every time we met another group of humans or other hominins who thought differently about the world, negotiations with them were complicated.
-
@androcat - I expect it'll get easier and easier to prove things using LLMs. So, as long as people think math is mainly about proving things - things that some mysterious men have already decided are worth proving - we'll hear people say that math is getting taken over by LLMs.
At some point people will be forced to realize that there's much more to math than proving things. But will this happen before or after lots of mathematicians get fired?
@johncarlosbaez
I also expect that the worth of the proving, for whatever worth proofs might have, lies in the effort put in, not the results put out.But you don't think this will all turn out to be hype, like in other fields?
-
@johncarlosbaez
I also expect that the worth of the proving, for whatever worth proofs might have, lies in the effort put in, not the results put out.But you don't think this will all turn out to be hype, like in other fields?
@androcat - I know @wtgowers and @tao fairly well, and trust that what they write is factually correct - though they may be overestimating the importance of "solving previously posed conjectures" and underestimating all other aspects of math. So I read what they write and ignore a lot of second-hand stuff written by journalists.
Here's a recent article by Gowers:
https://gowers.wordpress.com/2026/05/08/a-recent-experience-with-chatgpt-5-5-pro/
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
That is a very apt metaphor!
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez oh god yes. Also add game cheaters to that bucket.
-
Maybe every time we met another group of humans or other hominins who thought differently about the world, negotiations with them were complicated.
@petealexharris @johncarlosbaez It does seem to fit right there at the dawn of “using language to coordinate with people other than those you share food with every day”.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
Having actual mastery (deep integrated understanding or skill) without effort would be great.
Delegated mastery is just another word for having helpers, or using a tool. You can buy mastery of others and tools, but you can't buy the judgment that comes from deep understanding.
That's why AI supercharges subject matter experts, but can cripple novices. It's a great junior assistant, but needs someone with judgement to boss it around.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez Bring on the hellhounds.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez It's a very elegant way to say “weakness and incompetence are the source of all evil”.
-
@petealexharris @johncarlosbaez I do find it really interesting that this idea of a uniquely capable yet maliciously compliant non-human entity is one of our ancient and universal cross cultural archetypes.
@marshray @petealexharris @johncarlosbaez
I think it's built into humanity.We are aware of malice because it is a common failure mode in cooperative endeavours, especially in childhood when people have not learned to manage strong emotions like jealousy.
Humans develop internal complex predictive models of not only their environment in general but of the behavior of other humans. We default to modeling others at our own level of competence.
Which makes modeling toddlers extra hard, because we desperately need to model them, but they are nowhere near our level of competence.
Both toddlers and robots often seem "maliciously compliant". They do *precisely* what you ask, especially when that's not what you intended.
Of course all humanity has maliciously compliant trickster legends. Anyone with children would love those stories.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez Agree. AI is indeed the demon being summoned, but all it gives you in return, is unsurpassed crap.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
-
@Beggarmidas - everything is more complicated than what anyone says, but I really do sense a line of tradition running from medieval magic to many people's attraction to AI. I wrote about it earlier here:
https://mathstodon.xyz/@johncarlosbaez/116855445718992437
How are your fertile inspirations coming to life? With work you can create something solid, but I get one or two emails a day from people trying to invent new theories of physics with the help of AI... almost all of which are nonsense, because they don't know enough physics.
