@sandriver thanks for this!

@jpaskaruk why's that?

@davep it's perfectly ok with me if this back of the envelope calculation isn't convincing to you. I'm sharing my reasoning for coming to the answer I would give to this question, not trying to convince you.

@davep I don't know if I can provide a citation that params play the same role in LLMs that synapses play in biological brains. I only used it by very bare analogy; they are the "connections" between parts of the network. And from the little I know, I think they're much less complex than biological synapses.

@davep I'm sorry I had an angry response to this earlier. I shouldn't have flown off the handle. I read your post as much more confrontational than it is on second reading.

@ricci thanks for the thoughtful reply!

Ultimately, I think it kind of doesn't matter. If machine intelligence can communicate and behave as if it "really" understands its percepts, I don't know if I care whether it truly is thinking or not.

Other humans at least have the same kinds of bodies as I do, so by the principle of mediocrity (I'm probably not an exception), if I have a mind other humans probably do too.

We can't assume that with machine intelligence. It is by definition very different from me. Everything I know about AI suggests that what's going on inside the machine is very different from what goes on inside me.

This is the Problem of Other Minds, but even worse that with other humans.

https://en.wikipedia.org/wiki/Problem_of_other_minds

Oh, one other thing that has come up is whether artificial intelligence is "true" intelligence; can it "really" think.

@draken thanks, that's a really interesting hypothesis.

@bignose why are you confident in that? LLMs are great at a lot of things that computers haven't done well previously.

@rxp OK. I find the idea that we can't have a conversation without rigorous definitions for every term grating. We can talk about vague topics; one of the benefits of intelligence.

@murph because we're getting dumber?

All that said, my gut is still pretty good with that 2040 date.

Oh, and we're going into a dangerous century. We might not have the social cohesion, much less the resources, to maintain huge AI labs through another 15 levels of doubling complexity.

There have been previous generations of AI technology that hit walls -- the planning systems of the 1950s, the expert systems of the 1980s. It's possible that LLMs will hit some kind of wall -- that it won't be possible to get to superintelligence with that method.

All that said, AI is in a very different place than it was at the beginning of my working career, during the "AI Winter" of the early 90s. Deep learning really changed a lot. We use AI techniques now a lot more, for a lot more areas of applicability. And LLMs are really impressive.

I'm pretty dubious about claims that superintelligence is so immanent, and so dangerous, that LLM technology should be restricted to only a few trusted companies and the US government. That sounds like some self-serving bullshit by companies that want to consolidate economic power by partnering with a protectionist government.

I don't think we know for sure a lot about intelligence. We don't know that it can be implemented on silicon chips, but we also don't know if it can't. We don't know if it has to work like brains do, or if it could work in entirely different ways. We don't even know exactly how our own minds arise from our brains. We don't know if we can replicate intelligence without understanding our own first.