It’d be a longer conversation that’s hard to do via HN comments, but I think the main divide is I get the impression you’re giving the AGI implicit human-like reasoning, but the idea behind the orthogonality thesis or alignment generally is that you don’t get these things for free.
It’s not that humans hate ants or apes, it’s that we pursue goals without thinking too hard about them. A house being built may destroy an ant hill but it’s not because we hate ants.
The core argument is it’s not only possible to have an intelligence that’s a lot more capable than us but with dumb goals because of our failure to align it, but that that’s the default outcome. There is no “reasoning with it” because it’s not a human like intelligence, it has a goal it’s focused on (paperclips) and if it’s a lot smarter than us then that’s game over.
I don't understand. It's true that judging humans by human intelligence should have a perfect alignment. But it's not necessarily true that judging other creatures' actions by drawing rough analogies to human intelligence is valid. Birds waiting for an opportunity to steal seeds doesn't really strike me as a universal indicator of intelligence.
That's very true. Although I think there is a good chance they could have incompatible values to us. It'd be pretty unlikely for two intelligent beings to evolve exactly the same morality. Imagine if they killed us all for eating meat, or if they were strong environmentalists like in The Day the Earth Stood Still. Or something else completely alien to us.
Why do you think they're orthogonal though? Empirically, it seems like smarter humans are more ethical. Same seems true of high intelligence vs low intelligence animals too.
Right, but that concept is itself very compatible with darwinian evolution. Humans are the current pinnacle of evolution when it comes to intelligence, but that doesn't mean everything else is entirely unintelligent.
Does it effectively keep the humans on a pedestal to consider our traits as fundamentally different?
I think it is all on a continuum, humans are animals. But putting us on that continuum makes all the animals look like absolute garbage, intelligence-wise. If we want to put humans in the game, we’re going to be absolutely dunking on, like, every animal in terms of ability to abstract and teach concepts.
Very intelligent humans have more persuasive power over less intelligent humans than they do over most animals. Even then it's very inconsistent and it's not at all clear that intelligence helps win over humans all that much.
I will worry about hyperpersuasive AI when someone manages to come up with one that can talk flat earthers out of it...
Human understanding of consciousness/intelligence is so full of bias due to all of our scientific knowledge being derived from human consciousness. We have set an impossible bar for animal and AI intelligence not on objective ontological premises, but largely on human primacy hubris. Hubris which considers the concentration and usurpation of resources more intelligent than the communal or even universal or natural distribution. Hubris which considers survival at all costs more intelligent than self sacrifice for the collective. Hubris which says, yeah, chimps can effortlessly remember way longer sequences than humans, but is that useful memory? Afterall they haven't tricked themselves into spending most of their waking hours staring at a blinking box.
i suppose, but this perspective also comes with its own anthro-centric assumptions.
For example, the assumption that we are the most intelligent creature to have yet emerged from the evolutionary process is most certainly a function of how one measures "intelligence". it might be the case that one can measure intelligence through the fossil record, but i would be interested to see a study of the selective pressure that produces advanced and intricate central nervous systems, and to understand how this could be said to increase in overall terms. i think it might not be the case, although it might, given certain selective conditions. i'm not sure we know what those are.
To me, the definition of intelligence is to produce the most appropriate response given the circumstances. So a plant that turns its leaves towards the sun is displaying intelligence. are plants more intelligent than we? by some measures yes - being able to respond to direct sunlight by producing energy is certainly appropriate in some circumstances - plants have solved a problem that we cannot, and therefore are more intelligent than we. (and i know folks will reject the idea that photosynthesis is a form of intelligence, but in my opinion that's anthro-centrism!)
is pondering the orbits of distant planets the most intelligent activity to have been performed by any organism ever? is building mega-cities and particle colliders? We don't know yet. it could be extremely inappropriate, because it could be next to impossible to travel between stars, given the reality of our existence. we could be leading ourselves to our own extinction, by dramatically altering our own planetary biosphere and chemistry, in pursuit of lofty visions such as interstellar travel. That's not so intelligent.
Another relevant point is that we don't have very much of the fossil record. i read somewhere that we had like 4% or so, although i am not exactly sure how one estimates that percentage. But assuming so, somewhere in that 96% there could be critters more freakishly intelligent that we.
suppose that in the 4-5 billion years of life on earth, of which our own record of relatively extreme intelligence is only 2,000 years old, there could in fact have been civilizations created by social creatures with as much or more intelligence than we. There could have been cities, perhaps in the millions and millions of years when dinosaurs were common. Granted this goes against commonly held assumptions, but it is certainly not impossible.
Keep in mind, a mere 150 years ago, we had a global civilization that could have vanished easily without a trace in 20,000 years, let alone 1 million. do we think of ourselves as incredibly more intelligent than our great grandparents, or theirs, or than people living 300 years ago? Better informed on certain topics for sure, but more intelligent? We are more like plants now, in that our intelligence is specialized to certain techno-powerful purposed, of which the intelligence relative to our external conditions is still yet to be decided. We might say that our ancestors were more intelligent for avoiding the kinds of environmental and social disasters that we have engendered.
Anyway, were we to go extinct now, we assume that future intelligent creatures would find remains of our great civilization. But we don't know how long these would last. There's some luck involved. it's possible that the hominid layer could degrade quickly!
On the other hand, if common assumption is correct and we really are the only potentially starfaring organisms to have evolved yet, then that argues my original point that life, and life worth communicating with, could be exceedingly rare. in 4 billion years (about 1/4th to 1/3rd of the age of the universe) life sufficiently intelligent to travel the stars has only emerged once. Those odds seem low, but they might be freakishly, abysmally low as well. Had it emerged twice, and we had some evidnce, odds would be better, but once! That's hard to quantify, especially as the longer life persists, the more likely it is to be wiped out by some kind of extinction event.
It's all kinda not-okay, and kinda icky. But we certainly seem to put intelligence on a higher tier. If we ever make another sentient species, we should probably not enslave them. Pinky swear, guys?
Like, judeo-christianity? Western societies viewpoints on human intelligence and how it relates to animal intelligence for centuries? The notion, typically, is that while humans are animals our intelligence is not just different in degree but different in kind. I think it comes from that bit in the bible where their god places humans as above all other life forms, just a guess though.
A relativistic definition of intelligence: I think it would be hard to do, e.g. your example would make dogs one of the most intelligent species on Earth, relative to humans, much smarter then chimpanzees. To be frank I find the whole concept of intelligence to be more problematic then its worth.
That said, if we are ever able to establish communication with a species other then our own that evolved elsewhere in the galaxy, intelligence might at that point become an outdated concept.
You're applying evolutionary forces to a simplistic binary plateau.
This isn't intelligence, it's a subset of maximal solutions to bounded problems.
Hint: your dog bounds you as much as you bound your dog. Your dog has some input into your modern homo sapien sapien mind. Same goes for your atmosphere, your gut fauna, your planet's iron core and so on.
There's a problem recognizing intelligent species for what they are even if we can meet them face to face. Dolphins, crows, octopus, orangutans, ... Since they don't build civilization as we conceive it, asserting their intelligence is controversial.
Orthogonal is exaggerating, I'll agree that the angle may not be exactly 90 degrees, but it's nearer that than 0.
>Empirically, it seems like smarter humans are more ethical.
"Seems" deserves some emphasis. Who is more likely to wind up in handcuffs, a smart thief, or a dumb one?
>Same seems true of high intelligence vs low intelligence animals too.
It's not exactly clear what constitutes ethics in animals. Applying typical human ethics; Chimpanzees have murdered their social rivals, and dolphins sometimes enjoy tormenting other animals.
reply