It seems odd to me that would work better necessarily considering that humans evolved different capabilities many millennia apart and integrated them all with intelligence comparatively late in the evolutionary cycle. So it’s not clear that multimodal from the get go is a better strategy than bolting on extra modalities over time. It could be though since technology is built differently from evolution but interesting to consider
So this is extremely complicated and nuanced with respect to intelligence acquisition, and I don’t think there’s a definitive right or wrong answer.
I certainly acknowledge my own bias with this however, with respect to what Chomsky discusses, I make the distinction that most of the “code/data/information” that you need in order for the language capacity to develop is actually embedded in our biological mechanical systems. That is to say, if you were to take a human infant and never expose it to another human with respect to generating sounds for language, the infant would still develop some sort of sound based communication system. We see this with feral children, mute children, deaf children. They still have a verbal function, even if it’s not connected to any semblance of coherency.
So in that sense it’s like you’re given all of the building blocks for language out of the gate biologically and then the people who are around you tell you how to assemble them into some thing that is functional. This is why different languages have different rules yet language acquisition is consistent across cultures.
This is why I am insistent on holistically understanding the computing infrastructure and systems because the sensors processors, etc. are the equivalent to our cells, genes muscles, bones, etc. Most people don’t think about computing systems and generally intelligent systems this way.
If you go back and look at the work of wiener and early Cybernetics it does discuss a lot of this, however, after Cybernetics was absorbed into artificial intelligence, which was an absorbed into computer science, it doesn’t really look holistically at systems of systems, unfortunately, in the general case.
And I would argue that all of machine learning currently is very much moving in to the direction that I am describing where is exposure to frequency of correlated data that gives you your effective understanding of the world, and being able to predict the future state. That’s what I mean when I say multi-modal is “sequential and consistent in time” with respect to causal action.
> It's not "technics", but "language" that underlies our evolution as a species.
> the making and use of technology, in the broadest sense – is what makes us human.
Idea in the artucle is a superset of idea which you describing.
Imagine that language is also a technology. Look at sign language for examle. Or mimes. Or simply spoken language. All three examples are examples of different technologies which make connection between inner worlds of multiple people.
I get the impression that humans have separate parts of the brain that do LLM type linguistic stuff and others that do non verbal intelligence of the sort that smart animals have. Maybe they'll figure a way to tag the other stuff on.
If you believe that speech is a survival skill (which I would think most people believe) then anything that increases its prevalence or effectiveness would be evolutionarily selected for, right? We’re talking a 300-500+ Ky period, probably almost 2X that (depending upon the speech capabilities of our hominid ancestors), so plenty of time for evolutionary pressure to apply.
That certainly a possibility. The other (non-mutually exclusive) implications may also be that human language acquisition benefits from being part of a multi-task model. Or that the problem has been overreduced ie: human language acquisition cannot simply be distilled into a words-in->words-out problem and that vision/hearing are actually integral parts of language acquisition that cannot be left out. Or that model arch still has major improvements to be made and attention is not all you need, for example.
Stiegler is getting pretty close to what I consider the true answer. It's not "technics", but "language" that underlies our evolution as a species. Language is our shared repository of experience, and an evolutionary system of ideas and values in its own right. All we know comes from the environment, but most of what we know is inherited by way of language.
So the environment and language represent present and past experience, they both contribute to how we choose our actions. By environment I mean not just nature, but also other people and all the things we build.
BTW, this related to LLMs in a way - they are trained mostly on language and very little on present time experience. When that changes, and present time experience will get its own rightful place, then LLMs will make better autonomous agents. Of course they are combinatorial parrots if all they know is past data, but they can escape this predicament when they feed on present data.
There's so much we don't know about this imaginary world that I don't think this is a fair assessment.
If, early on in the evolution of animals with brains, it was typical for them to communicate through these connections, wouldn't it make sense for all descendant animals to also have this ability? If contact between divergent species was still frequent, and the contact was positive and mutually beneficial, why not communicate through this channel when needed? Spoken language may have been unique to their humanoids. Indeed, spoken language is a bit more odd than directly connecting to the brain of the other.
To me, this is just as unusual or counter to evolution as our ability to use sign language to communicate (albeit primitively) with apes, or give spoken orders to dogs. That is: it isn't.
'language' as we know is quite ambiguous and an inefficient way of communication.
my guess is that what they will probably evolve is some kind of an audio encoding format for the exact information they want to convey to the other entity, like literally sending it control inputs over audio.
Who believes that? It beggars belief that there would be evolutionary pressure to develop the machinery for speech if intermediate steps were not being put to progressively better uses.
They aren’t trained on all of human creativity. Most of human communication is verbal, and that still isn’t captured. I am not saying whether it will have much difference, just that your statement is factually wrong.
Strong Disagree Making marks and symbols with our hands that other people can read is something uniquely human. Should we not speak because we can text as well?
To write, to read, and to speak are the foundations of human evolution. Technology augments these natural gifts, not replaces them.
Interesting take. I think it also matters how broad or small you define language or communication. Personally and in a philosophical way spoken and written language is the main differentiation between us and and animals. We can quickly and specifically make intent and meaning mediated between us. I also think we are hilariously bad at it in a modern context, the complexity of the current society makes communication (and by extend community!) unnecessarily (?) noisy.
Another interesting point is the idea of Logos, Word, Speech as the divine attribute par excellence in some religious and mythological contexts.
That said I am not sure what the author wants to optimize. I mean if AI can mediate meanings and intent, that sounds interesting. If it is to AI have their own chats, I doubt that is useful from a human context.
So humans do this too naturally - this is how new languages develop over hundreds of years and groups evolve and split. Aside from that, I'm a little nervous about seeing machines accelerate this process.
The idea would be that language first evolved as a mechanism for thought and only later became used for communication. Spoken language would still be connected to the underlying mechanism though.
I don't think that's at all likely. We've had fire and had the physiological adaptations enabling speech for well over a million years. We've been manufacturing complex tools such as spears with stone tips for about half a million years, to Homo heidelbergensis. Even Ergaster had lifespans allowing survival to an age where grandchildren could reach adulthood, allowing for skills transfer.
reply