When will there be AI superintelligence?

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 04:44:36 GMT

malte@radikal.social — Thu, 10 Sep 2026 04:44:36 GMT

@evan I agree we don't know much about intelligence - in the way that we tend to confuse it with other potentially very different phenomena. One thing that often gets confused with it is self-consciousness. Another thing that is not synonymous with either of those is intention. Psychologists working on those concepts have interesting things to say about all three, but it doesn't get through in the public yet.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 04:06:56 GMT

bignose@sw-development-is.social — Thu, 10 Sep 2026 04:06:56 GMT

@evan
> LLMs are great at a lot of things that computers haven't done well previously.

I hear this claim a lot, but it's never substantiated verifiably.

In fields where I understand the LLM output, it is unreliable plagiarism or unreliable garbage.

In fields where I need to rely on others more knowledgeable, those who *aren't incentivised to hype the LLM* report LLMs are unreliable plagiarism or unreliable garbage.

They're okay at language patterns. What are you saying they're great at?

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 03:37:14 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 03:37:14 GMT

@sandriver thanks for this!

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 03:23:23 GMT

sandriver@zoner.work — Thu, 10 Sep 2026 03:23:23 GMT

@evan@cosocial.ca just as an aside, I think it's important to distinguish cognition and intelligence, since obviously even trivial neural nets have some level of cognition, as do life forms that don't even have a nervous system, or only exist as single cells.

As to whether hardware needs to be brainlike, I lean towards the existence proof, but also with the proviso that neural net software is too abstract and deliberately elides potentially important physical properties of a brain. Some examples:
* neurons have chemical signalling that propagates at the speed of sound, in a volume, beyond the synaptic transmission.
* neurons have internal state due to their epigenome.
* neurons have complex, mesh-like connections between anatomical regions.
* neural information is inherently sensitive to time in a variety of ways, including those mesh-like connections.
* non-neural tissue in the brain also affects neuronal behaviour.
* there are unknown nonclassical properties of neurons.

Maybe you could throw enough matrix munching transistors at a neural net and get a life-like mind that is superior to life in all domains, but I doubt it, not with current software models of neural nets. They are only able to reproduce particular kinds of cognitions, and certainly nothing like a mind.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 03:22:24 GMT

chris@mstdn.chrisalemany.ca — Thu, 10 Sep 2026 03:22:24 GMT

@evan I think, and hope, the biggest impediment to that exponential growth of “intelligence” will be raw resources. Power, material, and water.

These data centres are already hoovering up precious minerals and complex components so much that it is causing serious shortages. They are already consuming magnitudes of electricity to require their own generation, or sucking power from communities. They are already consuming so much water that concerns are rising for capacity and environmental impact.

And then there is the financial capital.

We have never seen a technology this *hungry* before. Short of turning humanity itself into batteries, I don’t think there will be true intelligence before these issues are solved. So my answer would be beyond 2050.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 03:05:34 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 03:05:34 GMT

@jpaskaruk why's that?

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 03:03:50 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 03:03:50 GMT

@davep it's perfectly ok with me if this back of the envelope calculation isn't convincing to you. I'm sharing my reasoning for coming to the answer I would give to this question, not trying to convince you.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 03:02:15 GMT

jpaskaruk@growers.social — Thu, 10 Sep 2026 03:02:15 GMT

@evan I might have voted if there was a "stop drinking the poo-flavour koolaid" option

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:58:43 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:58:43 GMT

@davep I don't know if I can provide a citation that params play the same role in LLMs that synapses play in biological brains. I only used it by very bare analogy; they are the "connections" between parts of the network. And from the little I know, I think they're much less complex than biological synapses.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:52:54 GMT

mungencakes@kolektiva.social — Thu, 10 Sep 2026 02:52:54 GMT

@frederic @evan

Dude is pretty open about their methodologies and the danger.

I don't see how their monitoring can be quick and comprehensive enough. They trained Astra in a data center with 100K GPUs. A half million tflops/s.

https://openai.com/index/an-alien-mind/

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:52:50 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:52:50 GMT

@davep I'm sorry I had an angry response to this earlier. I shouldn't have flown off the handle. I read your post as much more confrontational than it is on second reading.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:33:20 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:33:20 GMT

@ricci thanks for the thoughtful reply!

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:31:38 GMT

tehstu@hachyderm.io — Thu, 10 Sep 2026 02:31:38 GMT

@evan Definitely an attempted ladder-pull, before they IPO.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:31:26 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:31:26 GMT

Ultimately, I think it kind of doesn't matter. If machine intelligence can communicate and behave as if it "really" understands its percepts, I don't know if I care whether it truly is thinking or not.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:28:59 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:28:59 GMT

Other humans at least have the same kinds of bodies as I do, so by the principle of mediocrity (I'm probably not an exception), if I have a mind other humans probably do too.

We can't assume that with machine intelligence. It is by definition very different from me. Everything I know about AI suggests that what's going on inside the machine is very different from what goes on inside me.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:28:26 GMT

ricci@discuss.systems — Thu, 10 Sep 2026 02:28:26 GMT

@evan

I'm not convinced that they will keep growing at that rate - I think we are fairly well along the road of diminishing returns for dollars-to-model-quality.

Yes, models are getting better. I agree.

But I don't think that it's currently true that if you spend 10x as much, you train a model that is 10x better. I'm not going to pretend that I know what the constants of this function are, but I think they are sub-linear. I think that a scenario of ever-increasing amounts of money spent for ever-decreasing marginal improvement is going to make further linear increase in models unlikely.

I'm also not convinced that number of neurons are a great proxy for intelligence - both in terms of comparison to synapses, and in terms of linear scaling. At some point, it used to be easy to convert more transistors on a chip into a commensurate amount of computing but now ... it's not so linear. I don't know if there is such a point of diminishing return for neural nets, and if there is, what it would be but - my hunch is that it likely exists. There are not a lot of things that scale linearly forever.

Thanks for this thought-provoking thread!

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:27:57 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:27:57 GMT

This is the Problem of Other Minds, but even worse that with other humans.

https://en.wikipedia.org/wiki/Problem_of_other_minds

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:27:26 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:27:26 GMT

Oh, one other thing that has come up is whether artificial intelligence is "true" intelligence; can it "really" think.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:16:31 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:16:31 GMT

@draken thanks, that's a really interesting hypothesis.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:15:45 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:15:45 GMT

@bignose why are you confident in that? LLMs are great at a lot of things that computers haven't done well previously.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:14:56 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:14:56 GMT

@rxp OK. I find the idea that we can't have a conversation without rigorous definitions for every term grating. We can talk about vague topics; one of the benefits of intelligence.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:12:12 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:12:12 GMT

@murph because we're getting dumber?

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:11:28 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:11:28 GMT

All that said, my gut is still pretty good with that 2040 date.

Reply to When will there be AI superintelligence? on Thu, 10 Sep 2026 02:10:41 GMT

evan@cosocial.ca — Thu, 10 Sep 2026 02:10:41 GMT

Oh, and we're going into a dangerous century. We might not have the social cohesion, much less the resources, to maintain huge AI labs through another 15 levels of doubling complexity.