- AIs becoming sentient and causing harm for their own ends: yeah I guess we only want humans to cause harm for their own ends then.
Well, here's the thing. Even the worst villains of history had human values and feelings: In other words, alignment. A superoptimizer AI might have the ability to wipe out the whole human species, in a way we won't be able to understand in time to prevent it, and all for an instrumental goal incidental to whatever it's doing.
(In a way, this thread is a data point for why we need a more sophisticated debate about AI.)
I think most doomer arguments are not that the AI would be evil, but rather that it would be misaligned with human interests, and would seek to accomplish goals with that misalignment, which could be bad for us. Evil AI is a bit too anthropomorphic.
It's more like powerful AIs that just don't share our values, because we didn't bother to figure that part out. But yet we still give them goals, blissful to the possibility they will find dangerous solutions to accomplishing those goals.
What the parent is saying is that an AI (that is, AGI as that is what we are discussing) gets to pick its goals. For some reason, humans have a fear of AI killing all humans in order to to achieve some goal. The obvious solution is thus to achieve some goal with some human constraint. For example, maximize paperclips per human. That actually probably speeds up human civilization across the universe. No, what people are really afraid is if AÍ changes its goal to be killing humanity. That’s when humans truly lose control, when the AÍ can decide. But, then the parent’s comment does become pertinent. What would an intelligent being choose? Devolving into nihilism and self destructing is just as equal as a probability as choosing some goal that leads to humanity’s end. That’s just scratching the surface. For instance, to me, it is not obvious whether or not empathy for other sentient beings is an emergent property of sentience. That is, lacking empathy might be problem in human hardware as opposed to empathy being inherently human. The list of these open unknowable questions are endless.
In short, the argument isn't that the AI will become more AntiHuman as it evolves. Rather, the AI's existing utility functions might not be aligned with human utility functions from the outset, which could have negative consequences. It's hard to make an AI do what we actually want it to.
Interesting. I am of the opinion that ai is not intelligent hence i dont see much point in entertaining the various scenarios deriving from that possibility. There is nothing dangerous in current ai models or ai itself other than the people controlling it. If it were intelligent then yeah maybe but we are not there yet and unless we adapt the meaning of agi to fit a marketing narrative we wont be there anytime soon.
But if it were intelligent and the conclusion it reaches, once it’s done ingesting all our knowledge, is that it should be done with us then we probably deserve it.
I mean what kind of a species takes joy in “freeing” up people and causing mass unemployment, starts wars over petty issues, allows for famine and thrives on the exploitation of others while standing on piles of nuclear bombs. Also we are literally destroying the planet and constantly looking for ways to dominate each other.
A malevolent AI is a particularly terrifying prospect.
Despite the fact that humanity has possessed the ability to destroy ourselves for quite so long, fortunately because we're still flesh and blood, biological entities and our evolution has lead to some more or less universal truths about us. We tend to love our families. We tend to want what we consider to be the best for our offspring. We tend to have some sense of obligation to protect our parents when they can no longer do so for themselves.
All of these (and many other) things that act to mediate our civilization-destroying traits wouldn't necessarily apply to an AI.
An AI would want to either destroy or enslave humans, not because we're humans, but because we're a significant threat to their goals, regardless of what those goals are. An sufficiently intelligent AI with any set of goals and a desire to meet those goals will eliminate all obstacles to meeting those goals unless we specifically tell it not to. We are such an obstacle.
So, if its goal was to maximize deaths, then it might protect humanity, to maximize the number of humans which will die in the future. Evil AI is so evil.
I mean there are a lot of potential human objectives an AI could be maligned with in relation to humans. Simple ones are moral misalignment. Extenstential ones are ones where the AI wants to use the molecules that make up your body to make more copies of the AI.
I like to consider though that a super intelligence would not necessarily think in human ways 'kill everything in self interest' e.g., People, forests, animals, planet etc. Just because we humans act this way, doesn't mean AI will too. Fair enough to consider it, but equally, once it is intelligent, it will likely accelerate beyond our comprehension, and we tend to comprehend through fear and self interest, wisdom beyond humans is the opposite of this.
I doubt AI could or would do a better job of killing people and democracy than us humans.
> Imputing AI's with human like goals, emotions, motivations, etc doesn't seem reasonable to me. Then again... to play Devil's Advocate against my own position here, I suppose somebody might consciously choose to purposely build an AI that has those attributes for some reason. But even then, I'm skeptical that they'd wind up replicating the parts of being human that could lead to "evil" behavior.
Tabling the question of if the human-like goals/emotions/motivations have some "deeper" quality or not, I think there are actually many reasons why people are working towards and getting better and better at replicating them. On the benign side, because they're lonely, curious if they can do it, etc. On the evil side, because we're social conditioned as humans to treat things that present themselves as humans with respect. I've already gotten some spam calls that did some kind of semi-plausible automated response when I answered them, it's easy to imagine it getting worse. Personalized spam is also on this spectrum.
As I understand it, the second case is basically the definition of sociopathy (i.e. abuse of social trust as a means to serve a selfish end). Again, tabling the idea of if the AI is evil or the creator is evil, the end result is evil behavior that is at least enabled by AI. [0]
The further concern I have is that even the more benign side done with good intentions ultimately saps away human energy for real compassion and perverts actual relationships. A "person" with human-like emotions/goals who is owned and managed by a corporation, or who can be turned off when you're tired of them is not a person at all, regardless of the intentions they were created with. Even well intentioned inventors are ultimately creating hyper-real simulacra of humanity that feels genuine in the moment, but is framed by fundamental untruths about the human condition (ownership, mortality). There is a market for Siri and sex-dolls, I 100% think there is a market for this too.
[0] ...to devil's advocate myself a little here, I guess I'd say that there are some greater goods that can served by minor abuses of social trust -- maybe even extending to personalized spam. But on the whole I think it's been awful for society.
The typical example is the paperclip maximizer, an AI that pursues the goals we gave it to such an extreme that it dooms humanity. Not because its values were opposed to ours but because it has no values.
Well, here's the thing. Even the worst villains of history had human values and feelings: In other words, alignment. A superoptimizer AI might have the ability to wipe out the whole human species, in a way we won't be able to understand in time to prevent it, and all for an instrumental goal incidental to whatever it's doing.
(In a way, this thread is a data point for why we need a more sophisticated debate about AI.)
reply