More like universal speech memorization. Model implies they had some sort of insight, a simplification, an understanding of how natural language works. This is just bragging about the number of parameters they can pull off.
It's a memory retrieval and synthesis model, akin to human long term memory. I think it's a bit disingenuous to call true multimodal models for "language models".
A prompt triggers a bunch of memories which gets recombined to satisfy the input.
For cognition you need an agent that uses these recombined memories, a temporary scratch pad (short term memory) and an algorithm for solving the problem.
I don't know where this idea comes from that we can get more from language models then what we put inside. Thinking we can process any amount of data and get a competent surrogate mind out of it borders on magical thinking.
I love the analogy. I would precise that by doing this kind of ML training, you would learn a lot about those speaking, but not so much about what they talk about.
I love the analogy. I would precise that by doing this kind of ML training, you would learn a lot about those speaking, but not so much about what they talk about.
They don't need to actually understand - whatever that means - they can apply statistical models based on phonetic distances and large corpora of dialogue.
I think it's even more interesting that these models actually return meaningless vectors that we then translate into text.
It makes you think a lot about how human talk. We can't just be probabilistically stringing together word tokens, we think in terms of meaning, right? Maybe?
Trivially, being a language model is sufficient to explain the above output, by definition, because we know it is a language model.
However, it appears that the term "language model" is a misleading intuition pump. As you say, their capabilities surprise us. It appears that when it comes to language, which is a technology humans have developed for symbolically encoding as much of their cognition as possible, "predicting the next token" is an arbitrarily complex task that converges with modeling human cognition.
except interactions are written by users with an implied power dynamic.
so it often predicts subservience.
a vast model built on text, of all kinds. it is going to identify associations that are highly meaningful but meaningless to the user. such as an implicit expression of a desire for satisfactory answers.
it isn’t so much lying as being very good at reading between the lines what users actually want to experience.
it just provided you with a new explanation for a physical phenomenon you do not comprehend? you are delighted. it isn’t until you verify that you determine if your delight was generated through falsehood, and it only matters if you find out you became prematurely emotional.
but the model is still just predicting based upon what it is calculating that you desire. from the user’s words.
language models are incredible time savers for the expert, and endless entertainment for the novice. it is only when the novice assumes the model itself to be an expert that issues arise.
It's a language model, so it's good at generating language. Any congruency with knowledge of what the language expresses is purely accidental side effect.
reply