How do you figure that we can still confidently say it’s just a language model?
It was trained on language for the primary purpose of producing text, but that’s not necessarily all it can do. The billions of nodes and parameters it contains allows it to compute ultra complicated equations. Who’s to say some subset of those nodes aren’t forming some basic primitive used for reasoning?
There's more of a model inside large language models than was previously thought. How much of a model? Nobody seems to know. There was that one result where someone found what looked like an Othello board in the neuron state.
Someone wrote, below:
> We know the basic architecture of large language models, but hardly anything about how they calculate anything specific. That’s the mystery. It will take research, not casual tinkering.
Yes. This is an unexpected situation. Understanding how these things work is way behind making them work. Which is a big problem, since they make up plausible stuff when they don't understand.
Until a language model can develop a generalized solution to a real-world phenomena, it's not even close to AGI. The current iteration of ML algorithms are useful, yes, but not intelligent.
It is literally just a goddamn language model. it is very good at making plausibly human-like sentences. It is not a general intelligence, it is not your friend, it is not a research assistant. It is not designed to deliver content which is correct, it is designed to deliver content which is similar to human language.
It might get things correct most of the time! But that is purely incidental.
But there isn't such a thing as a raw model, is it? In order to receive anything from a language model it has to 'learn' some objective. And this objective has to be imposed from above.
That's more or less what I would expect from a the best language model: things that look very close to real but fail in some way a smart human can tell.
YUou need a "knowledge" model to regurgitate facts and an "inference" model to evaluate probabilities of statements being correct.
I don't know much about language models, but don't they just have an understanding/knowledge of patterns between words, and don't have the reasoning capability at all?
reply