I think the main point is — the AI don't really read and understand, they read and remember the patterns of words, which is different from understanding. They see a chain of words often enough that it becomes statistically the next best output based on some input. (with some randomness in-between.)
"we do understand that they generate words with probabilities one by one though"
This is no more helpful in understanding AI's than is knowing that human brains operate according to the laws of physics is helpful in understanding the human mind.
Most humans don't completely understand the things that they read or the words they utter. Why would we expect different from artificial intelligence? Understanding things is computationally expensive from both a biological and digital perspective.
That's not a useful description. The AI doesn't "know" what it "knows". It's not even filling some gap in its knowledge. It's just putting words together that statistically can go together.
They do learn by reading their corpus, and they do know what's in their corpus.
If you created a temperature sensor and the AI removes itself from hot temps, then it feels pain.
The insistency that AI not be antropormorphized sometimes gets in the way of communication.
Totally, and that’s a fair point. I don’t know what understanding means, not enough to prove an LLM can’t, anyway, and I think nobody has a good enough definition yet to satisfy this crowd. But I think we can make progress with nothing more than the dictionary definition of “understand”, which is the ability to perceive and interpret. I think we can probably agree that a rock doesn’t understand. And we can probably also agree that a random number generator doesn’t understand. The problem with @FeepingCreature’s argument is that the quality of the response does matter. The ability for a machine that’s specifically designed to wait for input and then provide an output, to then provide a low quality response, doesn’t demonstrate any more intelligence than a bicycle… right? I don’t know where the line is between my random writer Markov chain text generator from college and today’s LLMs. I’m told transformers are fundamentally the same and just have an adaptive window size. More training data then is the primary difference. So then we are saying Excel’s least-squares function fitter does not understand, unless the function has a billion data points? Or, if there’s a line, what does it look like and where is it?
exactly. or even ask these AI systems to define it. Just like a parrot, they'll repeat what other say without actually understanding what understanding is.
What you are probably missing is that for the model to be competent at predicting the next the word IT HAS to learn how to reason as a human being would.
I dunno, for me "understanding" something implies an ability to reason about your reasoning. a rule-based AI can't do that, and even if interesting patterns emerge, it's still deterministic.
I think that humans abuse discovered patterns and structure in language and meaning to search through possible interpretations very quickly.
Right, but does that structure really represent a "deeper" understanding or just vast and meticulous optimizations of statistical algorithms similar to Watson's? Or is there a difference?
We feel like we know how we think, but we can't actually explain it in enough detail to reproduce. Humans have a bad history of rationalization and tunnel vision. And now we discover that all the "wrong" ways to think deeply are actually the right ways to make a working AI.
If the AI can fool us into believing that it "understands" then maybe we can fool ourselves in the same way.
I agree with the sibling. You need to say what it means to "have an idea what the meaning of text is" if you're going to use this as an argument that neural nets don't understand language.
I think what it means to understand language is to be able to generate and react to language to accomplish a wide range of goals.
Neural nets are clearly not capable of understanding language as well as humans by this definition, but they're somewhere on the spectrum between rocks and humans, and getting closer to the human side every day.
I can't help but think that arguments that algorithms simply don't or can't understand language at all are appealing to some kind of Cartesian dualism that separates entities that have mind from those are merely mechanical. If that's your metaphysics, then you're going to continue to find that no particular mechanical system really understands language, all the way up to (and maybe beyond) the point where mechanical systems can use and react to language in fully all situations that humans can.
I'm not really sure what you mean. This seems to be another instance of the weirdly persistent belief that "only humans can understand, and computers are just moving gears around to mechanically simulate knowledge-informed action". I may not believe in the current NN-focussed AI hype cycle, but that's definitely not a cogent argument against the possibility of AI. You're confusing comprehension with the subjective (human) experience of comprehending something.
All this speculation about what an AI like ChatGPT can or can't do is largely unproductive and unknown. The truth is we have no idea what ChatGPT understands or how comparable whatever it's doing is to human understanding. There are a lot of reasons to think it's not the same, but there are also a lot of reasons to suspect that the neural net of AI's like ChatGPT are not just operating at the level of next token prediction.
There's at least some good reasons to believe that through its training ChatGPT has acquired high-level models of our world such that it "understands" how to make accurate predictions.
But going further, I'd argue it also seems to have learnt how to reason. For example you can ask it logical puzzles and it will often get the answer right and explain its reasoning. Some will argue that it's not "real reasoning". which could be true, but we really don't know this with any certainty. All we know with certainty is that the output in many cases appears to have some of the characteristics of reasoning and understanding.
Ask yourself, does a fish "understand" how to swim? If you ask a fish to explain how it swims it couldn't do it. So a fish can swim, but a fish doesn't understand how to swim. So what does it even mean for an AI be able able read, but not understand how to read? Is it just that a fish doesn't understand how to fish like a human? Does this distinction even matter?
To summarise the point I'm trying to make here, there enough gaps in our knowledge and evidence to suggest to there is likely some amount of understanding and reasoning happening and that it would be arrogant to suggest otherwise.
But I suppose to more directly answer your question. ChatGPT certainly doesn't "learn" in any meaningful way from reading. AI's like ChatGPT simply doesn't have the ability to remember things, so it physically cannot learn from reading. It might understand and reason about what it reads, but it cannot learn from it. That is assuming you're talking about the implementation of the models and not the "reading" it does during its training.
It’s not thought because it lacks any understanding or any attempt at understanding. Being trained in millions of real conversations to be able to find the statistically most “optimum” characters is so incredibly far away from understanding that it’s not even the same game, let alone the same league or ballpark.
Similarly, a machine learning model needs to be trained on millions and millions of pictures of bicycles to be able to identify one in an image. A human only ever needs to see one bicycle and can forever identify all manner of bikes. This is because human possess the ability to understand what a bicycle is, (in terms of its form, materials, function, cultural meaning etc etc) whereas an AI model is just doing a reverse wave function on an image. I mean, that’s great, it’s an amazing mathematical feat, but it’s not even the same planet as intelligence.
It depends exactly what you mean by understanding. It has a map of connections between word tokens that allow it to generate an output of word tokens that are useful to us, and which has now just recently been specifically trained for this particular problem domain. So now for these sorts of questions it no longer produces gibberish. None of that works in a way at all analogous to how the human brain processes language, reasons about concepts, or connects ideas beyond just words. It’s not meaning in the sense that we generally understand meaning.
If you take a step outside its trained response envelope, it will still fail hilariously generating meaningless drivel. It’s grasp of concepts just isn’t there, it has no concrete foundation. All it knows is word frequency weightings. Don’t get me wrong it’s amazing engineering, it’s probably going to be incredibly useful.
reply