You're not wrong, but the "stochastic parrot" claim always comes with another, implicit one that we are not like that; that there's some fundamental difference, somehow, even if it is not externally observable. Chinese room etc.
In short, it's the good old religious debate about souls, just wrapped in techno-philosophical trappings.
I dont think that's the core of the objection at all. I've never seen it made by people pushing the idea that AGI is impossible, just that AI approaches like LLM are a lot more limited than they appear - basically that most of the Intelligence they exhibit is that found in the training data.
But in which way isn't most of our Intelligence not what is from training data?
1st evolutionary algorithm and then the constant input we receive from the World being the training data and we having reward mechanisms rewiring our neural networks based on what our senses interpret as good?
As someone teaching their five year old to read, I think people way underestimate the amount of training data the average human child gets. And perhaps, since we live in a first world country with universal education, and a very rich one at that, many people have not seen what happens when kids don't get that training data.
It's also not just the qualia of sensation, but also that of the will. We all 'feel' we have a will, that can do things. How can a computer possibly feel that? The 'will' in the LLM is forced by the selection function, which is a deterministic, human-coded algorithm, not an intrinsic property.
In my view, this sensation of qualia is so out-there and so inexplicable physically, that I would not be able to invalidate some 'out there' theories. If someone told me they posited a new quantum field with scalar values of 'will' that the brain sensed or modified via some quantum phenomena, I'd believe them, especially if there was an experiment. But even more out there explanations are possible. We have no idea, so all are impossible to validate / invalidate as far as I'm concerned.
You could leverage the exact same accusation against the other side - we know fundamentally how the math works on these things, yet somehow throw enough parameters at them and eventually there's some undefined emergent behavior that results in something more.
What that something more is is even less defined with even fewer theories as to what it is than there are around the woo and mysticism of human intelligence. And as LarsDu88 points out in a separate thread, there are alternative explanations for what we're seeing here besides "We've created some sort of weird internal 3D engine that the diffusion models use for generating stuff," which also meshes closely with the fact that generations routinely have multiple perspectives and other errors that wouldn't exist if they modeled the world some people are suggesting.
If there's something more going on here, we're going to need some real explanations instead of things that can be explained multiple other ways before I'm going to take it seriously, at least.
> somehow throw enough parameters at them and eventually there's some undefined emergent behavior that results in something more.
But the other side doesn't see it that way, specifically not the "something more" part. It's "just math" all the way down, in our brains as well. The "emergent phenomena" are not undefined in this sense - they're (obviously in LLMs) also math, it's just that we don't understand it yet due to the sheer complexity of the resulting system. But that's not at all unusual - humans build useful things that we don't fully understand all the time (just look at physics of various processes).
> which also meshes closely with the fact that generations routinely have multiple perspectives and other errors that wouldn't exist if they modeled the world some people are suggesting.
This implies that the model of the world those things have either has to be perfect, or else it doesn't exist, which is a premise with no clear logic behind it. The obvious explanation is that, between the limited amount of information that can be extracted from 2D photos that the NN is trained on, and the limit on the complexity of world modeling that NN of a particular size can fit, its model of the world is just not particularly accurate.
> we're going to need some real explanations instead of things that can be explained multiple other ways before I'm going to take it seriously, at least.
If we used this threshold for physics, we'd have to throw out a lot of it, too, since you can always come up with a more complicated alternative explanation; e.g. aether can be viable if you ascribe just enough special properties to it. Pragmatically, at some point, you have to pick the most likely (usually this means the simplest) explanation to move forward.
> But the other side doesn't see it that way, specifically not the "something more" part. It's "just math" all the way down, in our brains as well. The "emergent phenomena" are not undefined in this sense - they're (obviously in LLMs) also math, it's just that we don't understand it yet due to the sheer complexity of the resulting system. But that's not at all unusual - humans build useful things that we don't fully understand all the time (just look at physics of various processes).
You might not, but you don't have to look far in these very comments to be met with woo and mysticism on the emergent phenomena side, either.
> The obvious explanation is that, between the limited amount of information that can be extracted from 2D photos that the NN is trained on, and the limit on the complexity of world modeling that NN of a particular size can fit, its model of the world is just not particularly accurate.
I think the more obvious solution is that it's not modelling the world, because... why would it be? It seems obvious to me that this is a significantly more complex way to handle their given task than every other explanation that does not require them create a model of the world.
> Pragmatically, at some point, you have to pick the most likely (usually this means the simplest) explanation to move forward.
I agree completely with this, which is why the idea that this is modelling the world is absolutely bizarre to me, particularly when our understanding of their training is that it's all weighting for pixel nearest neighbors via ranking how well it does at denoising 2D images.
It's not like de-rendering is something new, either. ShaderMap has existed since the mid 2000s and could get similar results from 2D images without any AI/ML, and we have other models that can generate it without anyone suggesting they model the world, e.g. https://arxiv.org/abs/2201.02279
People like LeCun don't think that even models where the stated goal is world simulation are able to do it, e.g. sora, either - https://twitter.com/ylecun/status/1758740106955952191 - so it seems even more unlikely when that isn't the goal that this has happened due to some emergent phenomenon.
The fundamental difference is qualia, which is physically inexplicable, and which LLMs and neural networks show no sign of having. It's not even clear how we would know if it did. As far as I can tell, this is something that escapes all current models of the physical universe, despite what many want to believe.
How do we know that they don't have qualia? Qualia are by definition private and ineffable. You can kind of sort of infer that other persons have them like you do on the basis of their public responses to them; but that is only possible due to the implicit assumption that other persons function in broadly the same way that you do, and thus qualia give rise to the same visible output. If that assumption doesn't hold, then your ability to infer presence of qualia to reactions (or lack thereof) to them is also gone.
But also, the very notion of qualia suffers from the same problem as other vague concepts like "consciousness" - we cannot actually clearly define what they are. All definitions seem to ultimately boil to "what I feel", which is vacuous. It is entirely possible that qualia aren't physically real in any sense, and are nothing more than a state of the system that the system itself sets and queries according to some internal logic based on input and output. If so, then an LLM having an internal self-model that includes a state "I feel heat" is qualia as well.
> All definitions seem to ultimately boil to "what I feel", which is vacuous. It is entirely possible that qualia aren't physically real in any sense, and are nothing more than a state of the system that the system itself sets and queries according to some internal logic based on input and output. If so, then an LLM having an internal self-model that includes a state "I feel heat" is qualia as well.
Is it "possible"? Absolutely. However, I have no means by which to measure, as you say, where at least with humans and animals I posit that their shared behaviors do indicate the same feeling, so I have some proof.
With an LLM, the output is a probability distribution.
Moreover, if it is as you say it is, then computers have qualia as well, which is scary because we would be committing a pretty ethically dubious 'crime' (at least in some circumstances).
Again, I just don't see it. Anything is possible, as I said, but not everything is as likely, by my estimation. And yes, that is entirely how I feel, which is as real as anything else.
In short, it's the good old religious debate about souls, just wrapped in techno-philosophical trappings.