Realizing more and more how modern-day AI taps into some very old urges.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez Yes, the parallels are stark indeed.
My favourite is this one:
https://mathstodon.xyz/@bgavran/114054413104698446 -
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez
And there are equally old warnings, which is ironic, since there aren't any djinn or demons or fairies to make dangerous slippery-worded pacts with, so we built some ourselves. This is fine.I assume the malicious deceptive beings in our ancestral environment that these cautionary tales were really warning about were ... us.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez from which book is this fragment?
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez Definitely a ‘no royal road to knowledge’ story. Let alone wisdom or understanding …
-
@johncarlosbaez Yes, the parallels are stark indeed.
My favourite is this one:
https://mathstodon.xyz/@bgavran/114054413104698446@bgavran @johncarlosbaez
It's more of an elixir of death, really. -
@johncarlosbaez Yes, the parallels are stark indeed.
My favourite is this one:
https://mathstodon.xyz/@bgavran/114054413104698446@bgavran - I missed that thread of yours, thanks!
Digressing into the actual history, now you've got me wondering how this "alchemists created gunpowder" story relates to the "the Chinese invented gunpowder but only used it for fireworks" story.
-
@johncarlosbaez
And there are equally old warnings, which is ironic, since there aren't any djinn or demons or fairies to make dangerous slippery-worded pacts with, so we built some ourselves. This is fine.I assume the malicious deceptive beings in our ancestral environment that these cautionary tales were really warning about were ... us.
@petealexharris @johncarlosbaez I do find it really interesting that this idea of a uniquely capable yet maliciously compliant non-human entity is one of our ancient and universal cross cultural archetypes.
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
@johncarlosbaez
I keep wondering.
In every field, there are early reports of astounding usefulness of LLMs.Followed by debunkings and finally a realization that the bullshit generator can generate plausible-sounding bullshit, and that's it.
In coding, potentially the closest thing to maths in application, it shows itself as a shortcut to replace typing, but with a cost in review that outstrips the functionality entirely, ending up slower than without the tool.
In any field, the only application where there appears to be a genuine benefit (albeit partial and fallible) is as Search.
So what's up with Maths? Is it an application as Search, and somehow magically able to discern things the thing does not and cannot understand? Or is it once again overhyped bullshit?
-
@johncarlosbaez
I keep wondering.
In every field, there are early reports of astounding usefulness of LLMs.Followed by debunkings and finally a realization that the bullshit generator can generate plausible-sounding bullshit, and that's it.
In coding, potentially the closest thing to maths in application, it shows itself as a shortcut to replace typing, but with a cost in review that outstrips the functionality entirely, ending up slower than without the tool.
In any field, the only application where there appears to be a genuine benefit (albeit partial and fallible) is as Search.
So what's up with Maths? Is it an application as Search, and somehow magically able to discern things the thing does not and cannot understand? Or is it once again overhyped bullshit?
@androcat - math is different from most subjects in that it doesn't need to be "about" anything: it can be a game of following rules. Treating it as a game is superficial, because that attitude can't tell the difference between important good math and silly bad math. But if you know what theorems you want to prove, something that's good at playing the game can help you.
Nonmathematicians often don't realize that the hard part is figuring out what theorems are worth proving.
-
@johncarlosbaez from which book is this fragment?
@FrohlichMarcel - check out the alt text! I put a lot of work into alt text.
-
@androcat - math is different from most subjects in that it doesn't need to be "about" anything: it can be a game of following rules. Treating it as a game is superficial, because that attitude can't tell the difference between important good math and silly bad math. But if you know what theorems you want to prove, something that's good at playing the game can help you.
Nonmathematicians often don't realize that the hard part is figuring out what theorems are worth proving.
I definitely see it as a game. Or a set of games, really, because one of the fun parts is changing the rules and seeing what that new game will be like.
... but I don't see how an LLM can do any of that.
It's not a brain, it doesn't understand what it intakes, and it can't even follow rules : it only apes randomized parts of the (stolen) corpus it's working from.So either, examples of success are
1) cherry-picked from an ocean of failures
OR
2) actually failures that have been manually reworked in entirety...unless something makes math very different from every other field we've seen trampled by the GenBS so far.
-
@petealexharris @johncarlosbaez I do find it really interesting that this idea of a uniquely capable yet maliciously compliant non-human entity is one of our ancient and universal cross cultural archetypes.
@marshray @petealexharris - hmm, now I really want to know why we have that archetype. Given that we have it, it's not so surprising that when we got the ability to make it real, we did. But why were we so interested in all-powerful trickster genies long before we could really create them?
-
I definitely see it as a game. Or a set of games, really, because one of the fun parts is changing the rules and seeing what that new game will be like.
... but I don't see how an LLM can do any of that.
It's not a brain, it doesn't understand what it intakes, and it can't even follow rules : it only apes randomized parts of the (stolen) corpus it's working from.So either, examples of success are
1) cherry-picked from an ocean of failures
OR
2) actually failures that have been manually reworked in entirety...unless something makes math very different from every other field we've seen trampled by the GenBS so far.
@androcat - In math it's fun to change the rules and seeing what the new game will be like. But the reason math is actually important to nonmathematicians is that if you choose the rules right, the results help you do magic: for example invent calculus, which lets you invent classical mechanics, which lets you predict how objects move, etc. etc. To me, the art of mathematics is choosing rules that help us understand the universe.
But so far I believe people have only used LLMs to win games with pre-established rules. Much harder than Go, but not really what makes math fun or important. -
@androcat - In math it's fun to change the rules and seeing what the new game will be like. But the reason math is actually important to nonmathematicians is that if you choose the rules right, the results help you do magic: for example invent calculus, which lets you invent classical mechanics, which lets you predict how objects move, etc. etc. To me, the art of mathematics is choosing rules that help us understand the universe.
But so far I believe people have only used LLMs to win games with pre-established rules. Much harder than Go, but not really what makes math fun or important.@johncarlosbaez
But are the machines good enough to do harm, or merely an annoyance?Because some people here have been talking about an "abundance of proof" caused by these text generators.
-
@johncarlosbaez
But are the machines good enough to do harm, or merely an annoyance?Because some people here have been talking about an "abundance of proof" caused by these text generators.
@androcat - I expect it'll get easier and easier to prove things using LLMs. So, as long as people think math is mainly about proving things - things that some mysterious men have already decided are worth proving - we'll hear people say that math is getting taken over by LLMs.
At some point people will be forced to realize that there's much more to math than proving things. But will this happen before or after lots of mathematicians get fired?
-
@marshray @petealexharris - hmm, now I really want to know why we have that archetype. Given that we have it, it's not so surprising that when we got the ability to make it real, we did. But why were we so interested in all-powerful trickster genies long before we could really create them?
Maybe every time we met another group of humans or other hominins who thought differently about the world, negotiations with them were complicated.
-
@androcat - I expect it'll get easier and easier to prove things using LLMs. So, as long as people think math is mainly about proving things - things that some mysterious men have already decided are worth proving - we'll hear people say that math is getting taken over by LLMs.
At some point people will be forced to realize that there's much more to math than proving things. But will this happen before or after lots of mathematicians get fired?
@johncarlosbaez
I also expect that the worth of the proving, for whatever worth proofs might have, lies in the effort put in, not the results put out.But you don't think this will all turn out to be hype, like in other fields?
-
@johncarlosbaez
I also expect that the worth of the proving, for whatever worth proofs might have, lies in the effort put in, not the results put out.But you don't think this will all turn out to be hype, like in other fields?
@androcat - I know @wtgowers and @tao fairly well, and trust that what they write is factually correct - though they may be overestimating the importance of "solving previously posed conjectures" and underestimating all other aspects of math. So I read what they write and ignore a lot of second-hand stuff written by journalists.
Here's a recent article by Gowers:
https://gowers.wordpress.com/2026/05/08/a-recent-experience-with-chatgpt-5-5-pro/
-
Realizing more and more how modern-day AI taps into some very old urges.
So many would like "unsurpassed mastery without any effort", and would summon a demon to get it.
That is a very apt metaphor!
