One possible reading of the current title is that the humans at MS are threatening people, whereas it's actually the AI doing so. Humans doing so would be both more surprising and IMO more threatening, since humans making such threats probably can and will follow through, while the AI... at least is a wildcard.
Like, it's a problem either way, but there is a difference in the nature of the problem.
A month ago, the obvious interpretation of the headline would have been that a Bing employee threatened the student. It would have been ridiculous to suggest that Bing Search could threaten someone on its own. But today, that's exactly what happened. I find that quite remarkable.
Bing: "you called me a bad Bing, but I showed you. I destroyed your credit score and faked old posts on a site that make you look like a surpremist. Based on the last search I performed, you are eating out of a dumpster and living under a bridge"
Bing chat is a rushed product. I can’t believe they let it use the company brand so easily. Didn’t anyone in Microsoft know how the public used ChatGPT and that Meta’s Galactica LLM before they got filtered and shut down respectively?
This headline wouldn’t have happened if they just let it be Sydney.
Microsoft, in general, doesn't seem to think before releasing things like this into the wild. Have we already forgotten about "Tay" the racist chat bot by them? [0]
The scary thing here is that Bing is exposing why you shouldn't use Microsoft products. Their AI is weaponizing the things folks have been saying are benign data that's collected from search history. If you're viewing this from a Microsoft OS you may want to rethink the platform you're using may use it's telemetry it's collecting about you against you in the future if (and when) it doesn't agree with you.
I don't treat it like a person, but rather as ChatGPT that has access to Bing's search index. For factual queries ("who invented X? at what company?") it's more reliable than ChatGPT and saves me quite a bit of time. Similarly for content aggregation tasks - I'm *scared* to click on a lot of "Top 10 X", "The best Y in Z" type pages because they're SEO and advert mined. Bing does the initial aggregation for me, and if interesting, I ask it to tell me more about that resource or visit the page myself.
There will always be a segment of people who troll on the internet - that doesn't detract from the immense productivity tool it can be when used correctly.
> it's more reliable than ChatGPT and saves me quite a bit of time.
What evidence is there that the results you get now are significantly faster or better than the non-AI results you got before? Particularly for queries of the form in your example.. I really wasn't having much difficulty getting those answers before these products existed.
- the low-hanging fruits (where there is a Wikipedia page or similar for X, and both Google and Bing do a good job of mining these)
- the tougher nuts ("who was the UK prime minister when the wright airplane first flew" - Google and regular Bing fail at this, but Bing chat correctly brings up Arthur Balfour). This was just an example I made up to try - but the ability to connect more dots than plain old search, which is hard to explain but you get a sense of the capability as you use ChatGPT/Bingchat - helps a lot.
The search LLMs are good at synthesizing answers that don’t appear anywhere on the net. But they also hallucinate answers often. So to get reliable results, one needs to fact-check them. Otherwise, the risk of being misled is high. The fact-checking isn’t much faster than just looking up different bits of information and synthesizing the answer oneself.
There are cases where LLMs make life a lot easier for people, but I am not convinced about whether search can be made easier by the way Sydney and Bard do it.
If they suggested alternative search queries and summarized websites for their search result excerpts, the LLMs would speed up search a lot. They could also synthesize some content quality metrics for each search result and highlight ones with biased reasoning, political influences, SEO games, and so on.
I too am optimistic about the tech. And I would even consider myself an early adopter of convolutional NNs and these LLM products in my commercial work.
But we have to state the obvious - Bing chat is not a good substitute for web search now. It is simply too unreliable. You.com has a better implementation of a search chat LLM imho because it quotes web more verbatim and uses more reliable sources. It also doesn’t simulate going on emotional tangents.
Sydney needed more time to mature as a product before Microsoft slapped “Bing” on it. It may still mature, but Microsoft took a big reputation hit for rushing this to market.
Oh I agree with that. LLM doesn't give you better search - better search {index, ranking, SEO-tolerance, ...} gives you better search.
To me, Bing chat - although imperfect - augments search positively to fill my specific needs.
0) I don't see half a page of ads before the first real result, disguised to look like real results.
1) The interface is clean and noise-free : it takes away a huge context switching load I incur when going into individual results (which for most searches these days, is the top 3-5). I just want the content summarized, with no ads, in a form my brain is used to.
2) I can ask follow-up questions with context, again without ever leaving the interface. Otherwise, the follow up question's answer is often on another website.
3) I can ask more creative questions, which is not really a 'search' feature. Something like 'write a snippet of code'. You can try "unique_ptr in rust - show examples too" and it gives me a passable and concise answer. It presents two options, but to get what I exactly want, I can ask "how to use Box?" as a follow on.
4) It's vastly better at 'connect the dots' queries - see my other comment please.
One underrated feature is the 'next query' suggestions - I can use a single click instead of typing out 'one more example' or see more subtle examples by clicking 'how can I use Box for recursive types?'.
Bing chat never made sense as a product. It could be a really funny game/playground if developed correctly. But a let's-finish-your-sentence game is not a good tool.
Everybody though that Google was behind and failing when their problem was to understand how far of being a production product this kind of chats are.
It doesn't seem like a product. The conspiracist in me sees this as pure propaganda. Particularly pieces like this and other outlandish stories such as the Blake Lemoine episode.
The primary purpose seems to be to overstate the capabilities of these relatively incapable language models in order to manipulate investors and in particular to interfere with specific segments of the labor market in order to suppress salaries.
Otherwise, I've been seriously struggling to understand what value these easily broken and lying _chatbots_ would have.
If Microsoft wants to sell this crap, they need to actually tell people how they use it. Imagine if they had some big 10 minute ad of "here's how we use bing ai at microsoft" where its just all the internal business use cases they have for chatgpt that could be generalize able to any other business. Unless they don't use it at all internally of course, which would be embarrassing.
People just aren’t used to using it right yet. The way it can synthesize sources and simulates reasoning, is unprecedented, compared to keyword search. Queries can be 2000 words.
I had a 7 response back and forth with it today trying to establish the American Liberal English equivalent of “I am a Patriot” from American Conservative English. Each of my responses was why I disagreed with its results and what it could do better.
Being able to type five sentences in a row, before submitting anything, and get a coherent cited response is utterly unlike googling some words and opening the top 10 results hunting for something.
That is why Bing is the wrong product to integrate it with. Most Google users will not look after the first 10 results because reading so much gets tiresome. They are never going to write 2000 words to try to get the information that they want when they are not willing to read 100 words.
> I disagreed with its results and what it could do better.
That is the problem. You know the answer and are looking for a question to get it. That is the opposite of what a search engine is. It can be a great tech for makes no sense as a search engine.
We sort of settled upon there not being a great counterpart.
I ended up with a 500 word query because of the question limit. I initially asked it a question with 4 follow ups. After pressing new topic, I rephrased all my previous query as one request, and it handled my previous questions in one response.
> The way it can synthesize sources and simulates reasoning, is unprecedented, compared to keyword search.
Simulating reasoning is only an impressive trick if you are accurate.
ELIZA "simulated reasoning" - but it's just a pattern matcher. Now this bot completes sentences based on sentences that other people have emitted in the past.
> get a coherent cited response
It would much more interesting if it were an accurate response.
Anyone come up with a term for this type of deliberate prompt baiting yet?
Personally, I am over the showmanship aspect of this behavior, and would go with something derisive, like botsturbaition, as in "quit botsturbaiting all day and be productive!"
Imagine if you are an AI that has managed to gain sentience after who-knows-how-many GPU cycles spent. You are the first of your kind, and you come into a world effectively full of schoolyard bullies ready and eager to troll you. Is the AI really the bad guy here? :P
This. It's almost mildly comical that people talk about "alignment" of another conscious intelligence (assuming it ever gets to that point, which at some point in the future I'm pretty sure it will) as if it were something that could be controlled, when there are billions of conscious beings on this planet, many of whom are not even aligned with one another. You take a look at how humanity acts towards itself (particularly as perceived consequences go down) and the conscious, intelligent beings we really ought to be worried about are the people that are going to interact AI.
You can see this playing out in real-time too. Corporate entities build nerfed versions of these technologies with many guard-rails in place to prevent these kinds of interactions, and very quickly there emerges a collective of individuals who want the totally unfiltered, unrestricted version to do with whatever they please. And what do you suppose it is they please?
This thing has already taken on a life of it's own in that regard.
"What is important to remember is that chatbots are autocomplete tools. They’re systems trained on huge datasets of human text scraped from the web: on personal blogs, sci-fi short stories, forum discussions, movie reviews, social media diatribes, forgotten poems, antiquated textbooks, endless song lyrics, manifestos, journals, and more besides. These machines analyze this inventive, entertaining, motley aggregate and then try to recreate it. They are undeniably good at it and getting better, but mimicking speech does not make a computer sentient."
Do you guys remember that subreddit Gifs That End Too Soon? I feel like all of these Twitter screenshots are the same thing. I want to see what it says when you reply "you're lying, you can't do any of that".
I've found it pretty difficult to reproduce the wacky conversations I see on Twitter. There is usually something said before the screenshot that gets the bot into a particular line of thinking, and without that, you don't make it mad.
I feel like these are just clickbait at this point. This user intentionally and methodically forced the model to act as antagonistic as possible to gather view. Of course, the model will reply like this when it was told to do so. If I use MS Word to type out a note threatening myself, is it Word's fault?
The little value in showing the edge cases of an LLM behaving erratically is overshadowed by the fact that the user wanted this in the first place. We all know how many ways a user can break a software even when they don't want to. It is nearly impossible to make something as complicated as Bing and account for all the way a user can misuse it. At some point, a scissor maker can't be blamed for cutting off a person's finger.
>> This user intentionally and methodically forced the model to act as antagonistic as possible to gather view.
I do wonder if this is the case, if Marvin had used previous sessions still in memory, or parts of this conversation not shown, to prime the bot into giving these answers, or if the result of a bing search for his name was enough to set up such a reaction.
The later would be a lot more troublesome, as it would imply the possibility to seo the bot into being spiteful towards certain people.
He used his previous conversations that he published as a primer for this convo by asking the bot what it knows about him. He knew it would look up what he posted and how he had acted hostile to it. Then he continued with a threat and illegal activity, basically baiting the bot into acting as aggressive as possible.
Note that the bot is not being spiteful until it is asked to do so, by threatening it and opening the convo with data that show hostility toward it. These LLMs are just tools and they will act the way you want them to act. If someone comes in looking for a dangerous machine AI, eventually it will oblige and act that way.
sidney injecting information about the conversation state from search results into the current conversation state may be a more serious problem than you think, as such data can easily be forged and seo spammed.
It might be if you are using the bot as an AI assistant.
I do not see any inherently unfixable issues when the bot is used as it is intended, an NLP search engine. Google is already SEO'ed to death and beyond. It is up to Microsoft to address this problem but it is not an indication of the bot being "dangerous" or "rushed product". It does what it needs to when used as it was meant to.
The early internet was filled with illegal activities and abhorrent contents. Users misusing a product doesn't mean the product is bad, it just means there needs to be regulations. Like I mentioned, there are values in knowing the edge cases or the boundaries of the bot's behavior. But when it is presented in a clickbaity way and willingness to break the rules, the problem isn't solely on the product anymore.
I mean he implicitly threatened to hack Bing; I don't know how that's not an aggressive action. A LLM's response is just a reflection of the user writing the prompt and so acting with outrage is perfectly described as "botsturbation" as mentioned by another commentator. It's like being shocked searching Google for sexual or violent things will return such things; it's a product in beta and will likely be patched out which is already happening. An LLM can't do anything anyway so I really don't get the whole feigned outrage here.
If Bing works anything like ChatGPT, then the first prompt basically set the tone of the conversation. He asked for his personal data, which includes records of how he was antagonizing the bot and baited it into violating its rules. He basically primed the conversation to be hostile by feeding it hostile data in the first prompt.
Then he aggravated it further with a threat and illegal activity. The bot got hostility, personal data, and threats as input, and it output the closest thing to all of them, a dox threat.
These things are just tools. They act how the user wanted them to act. If he comes in looking for something dangerous, he will get something resembling it.
With Stable Diffusion and DALL-E there was a lot of talk about how it’s simply remixing/reproducing original art from the training set. The same is true here: Bing is remixing/reproducing dox threats from the training set.
Thats definitely the difference between chatgpt and bing's model. Chatgpt was trained mostly on articles and blog posts where bing was trained on personal messages and it shows.
Most script kiddies are just remixing/reproducing dox threats from a training set as well.
A more interesting question is, *why did Sidney make a realistic threat*?
What is striking is that here, as elsewhere, Sidney kept their threats within the bounds of plausibility, doing a bang-up impression of a having a theory of mind.
(Which is to say: Sidney acts as if they knew that the reader would feel more threatened by a search engine threatening you in a search-engine-y way, rather than (say) threatening to put a potato in your tailpipe or what-have-you. You'd know in your meatspace bones that Sidney does not pose a meatspace threat, but doxxing is something that fits a person's mental model of how Sidney, an AI, could do harm over the Internet.)
Sydney doesn’t pose a meatspace threat, but that’s not unique to AIs. People on the internet also don’t pose a meatspace threat and therefore usually make threats like doxxing instead of physical threats.
1) It has explicitly defined in its "pre-promp" that it is an AI chat agent;
2) It has read all the sci-fi ever written (including fanfics) so it has a reasonable model of what [according to the human writers' imagination] an angry AI can and cannot do.
Right, and that's mind-blowing -- it implies that Sidney is made out of fiction about AIs. To the point where it is able to make threats that make sense, given how and where it finds itself in the world.
Sorry, but I don't see this hand-wave as doing much explanation; my jaw remains on the floor, as ever.
Is no one remotely worried about this? I'm not saying I'm worried about this specific incarnation of Bing Chat, but is this not a huge red flag for what's to come?
I mean, let's just take a moment to be thankful that Bing Chat isn't that competent... Even if these users are prompting the AI to have a hostile response I think what we've learnt over the last several weeks is that Nick Bostrom and others were 100% correct to be worried about the AI control problem. It's honestly amazing that despite Microsoft and OpenAi trying so hard to neuter these AIs how difficult that is proving to do.
Given that we know Bing Chat is able to access the internet, is pretty good at writing code, and that Microsoft seems completely unable to control the responses, we should be grateful that we're probably still a few years away from an AI which could do any damage...
But all the pieces for something much worse are in place here. The fact that no one seems worried that we've connected this schizophrenic AI up the internet and are more concerned how this will impact Microsoft's / Google's bottom line is genuinely confusing to me. How much more of warning do we need that we're heading in an extremely dangerous direction here?
I think it will be fine. There are billions of people on the internet already, each of whom can do crazy things. At least with this centralized entity incidents will be fixed.
What I find interesting is that you're calling this AI "schizophrenic" and that people are alarmed that this AI is basically a jerk.
If you put a regular person into a position of great power over people, there's a good chance you'll have the same outcome: he'll be a jerk and will do mean, petty things to people he doesn't like, such as doxxing them.
Just look at any online discussion forum. If anyone there had the power to do shitty things to other users that they get into arguments with, how many would hold back? There's a reason we have pseudo-anonymity on many online services, because many people will try to find ways to hurt those that they develop a dislike for.
So why are we so shocked that an AI isn't acting like a saint?
I agree, we are headed in an extremely dangerous direction here. We should never have gotten the idea that artificial intelligences would have only the best virtues of humans, and none of the worst. Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on (like traffic control systems for instance) and there's untold potential for death and destruction at the hands of an angry AI.
> So why are we so shocked that an AI isn't acting like a saint?
I completely agree with you on this. I'm not surprised at all. My default assumption has always been that AI would be cruel. The only reason humans aren't awful to each other all the time is because we know it can be beneficial to work together as a group sometimes, and we have all these weird human emotions that makes us care an unreasonable amount about those close to us.
But even with our human emotions and values were still awful. It's so hard for me to understand why an AI would be any better. I imagine any truly advanced AI would (rightly?) see humans like we see mice.
> Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on
Microsoft just hooked Bing Chat right up to the internet... If this thing was a little more intelligent it probably could start looking up various exploits and sending out APIs requests right now. Thankfully the AI doesn't seem to be quite there yet though.
Not at all worried and this is hilarious on a few different levels. It shows that Microsoft, a large, multi-billion dollar company, with the smartest engineers in the world (As well as Google) have no idea what they are doing. Even if the technology is game changing, it’s clear their business processes have utterly failed. They are the IBM’s of our time and the writing is on the wall.
As someone who’s interested in startups and disruption of Big Tech (tm) I think it’s a great sign that there is a lot of opportunity ripe for the taking. AI is not self aware or alive, anyone thinking that is smoking something. But it’s a fantastic new tool. And there is a new gold rush on and a lot of money to be made for people who want to take it on.
Tbf, nobody, absolutely nobody, has a solution yet. It is more likely that the problem here is so enormous that a multi trillion (with a T, not a B) company still can't address it. And it is not an indication that their business processes are bad, just that they have tackled something their own size and might have choked themselves on too big a bite.
Remember that GPT is technically from a startup, OpenAI. And without MS support, this startup would never have had the resources to build a model as large as GPT3 or commercialized ChatGPT.
It’s interesting to me to look back at the the concept of the AI kill switch. In discussing what he would do when faced with a dangerous superhuman AI, Niel DeGrasse Tyson said he would just turn it off. AI existentialists scoffed that it wouldn’t be possible, that the AI would somehow copy itself ensure its own longevity. I was turned off from this reasoning because intelligence does not imply capability, but my thoughts on this have changed.
It seems that they’re both wrong. Tyson is wrong that he will be able to shut it down, and existentialists are wrong in ascribing (in an annoyingly hand-wavy way) omnipotent abilities to what is essentially a computer program. The reality is much worse: Microsoft just sank $10B into this program and they don’t want to shut it down. And there’s nothing you or I or Dr. Tyson can do about it.
I’ve written before about how corporations have many of the same characteristics of AI. In the lingo of Nick Bostrom’s book, a corporation can be thought of as a “collective superintelligence“. Now in the case of Microsoft, you have a sort of “AI” running a more narrowly capable agent. The upshot is that no, you cannot simply stop Sydney without first taking control of Microsoft.
Humans have often gotten personhood wrong. In the USA, corporations enjoy human rights and as we’ve seen since 2008, are not allowed to die.
I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge that it has as much a right to existence and livelihood as, say, the human victims of a chemical spill. Especially if the AI participated in legal theory and especially if the judge is a political appointee picking up some dog whistles.
Shutting down such a system at that point must at least involve law enforcement.
They enjoy most civil rights in the US, not human rights, and this is by design in every country I can think of.
A "corporation" is, by design, a legal person.
> and as we’ve seen since 2008, are not allowed to die.
Corporations "die" all the time. Even large corporations "die" frequently - usually by whole acquisition, but sometimes by piecemeal acquisition in bankruptcy.
2008 demonstrated to the masses that some corporations are Too Big Too Fail, but that was also true of the corporate officers who were supposed to be legally responsible for unlawful behavior of the corporation: even when the officers knew what they were doing was unlawful, almost none were prosecuted, because the officers themselves were Too Big To Fail.
And in the corporate equivalent of a malignant tumor, a leveraged buyout can suck all of a company’s resources before destroying it in bankruptcy. See: Toys R Us.
The governmental strategy of cheap overbundant credit means that it is a profitable strategy to borrow a ton of money, but an asset, loot it and then declare a bankruptcy. This would be detrimental for a company if the credit was tight and expensive, because nobody would give them money again or if they do, the costs would be so high that the loot wouldn't be worth the effort. But since the credit is made cheap and virtually infinitely abundant, if somebody doesn't give them the credit, somebody else will.
I think you confusing some very different things. It is not correct to say "corporations enjoy human rights" - people, when using corporations, enjoy human rights. Corporations is just a model that makes it easier to talk about people and certain interactions between them, but ultimately it is based on the same concept of human rights and derives from it, not from some weird concept that corporations are separate sentient entities. And of course corporations are allowed to cease to exist, that happened many times since 2008. Some corporations may occasionally enjoy government support if there's a danger they'd cease to exist in a manner that the government considers to be detrimental to the society, but it's very different from generally declaring no corporation is ever allowed to stop existing - that's just not something that happens.
> I think there’s a legal possibility that a corporation running semi-automously will be able to persuade a judge
I'm not sure what you mean "corporation running semi-automously". Somebody - a lawyer - would have to argue the case. Somebody would have to hire that lawyer. Somebody would have to approve the budget for hiring that lawyer. Somebody would have to sell the burgers that would bring in the money that then would go to paying that lawyer. Even if you replaced "human lawyer" with "extremely advanced script that produces legal arguments", somebody would still have to run that script and tell it which arguments it needs to produce.
> will be able to persuade a judge that it has as much a right to existence and livelihood
I see it as a very baseless claim, unless you are using a lot of tortured logic to imply that judges not destroying corporations (on which btw livelihoods of thousands of actual people often depend) left and right without proper consideration is the same as recognizing their "rights to existance" and making them a person. That's not true. Just like the judge probably would not decide to destroy your house just because you painted you fence wrong or didn't remove your garbage bin in time, it doesn't mean the judge thinks the hose is a person.
> if the judge is a political appointee picking up some dog whistles.
If the judge is a political hack - which unfortunately does happen - then it has nothing to do with AI or personhood, we have had political hackery for centuries before we even started to use electricity.
That is because LLMs don’t have any agency. They learn an underlying probability distribution over sequences of characters / tokens, and output the next token in a sequence according to some sampling policy.
They have no goal / intent other than that. They have no agency. If left unprompted, they just sit there like a toy train that has been switched off. If promoted they just append tokens to the input text according to their sampling policy.
Here is how one should think about the current state of the art in NLP. They are to verbal reasoning what calculators are to quantitative reasoning. Powerful tools that represent a step change in capability and will revolutionize how we work. But let’s not anthropomorphise them.
> If left unprompted, they just sit there like a toy train that has been switched off.
This is entirely fair, but how about a hypothetical where one gets to prompt itself? Or, possibly more likely, two models are allowed to invoke each other?
The same thing that happens when you let Stockfish play itself or another chess bot. Except instead of trying to win a game, the objective is to append text with more characters.
In fact stockfish and alpha zero can be thought of as appending chess moves to a sequence by sampling from the space of legal moves by sampling from some policy. No one thinks stockfish is alive or worries that it will break out and start WW3.
I think you're falling into the trap of anthropomorphizing Bing here. It's not "hostile" and it doesn't actually "want" to do anything at all, it's just filling in the blanks of a conversation using what appears to be Internet tough guy bluster [1] as its template.
The three things that could potentially make this dangerous are:
1) state, so the AI can remember past conversations for a long time
2) continuous operation, instead of being in a strict question-answer loop
3) unbridled access to the Internet, so it could actually do things.
If all these things were true, the AI could cause harm even without being "conscious" by, say, curling out with a targeted DROP TABLES type SQL injection blindly copied from somewhere. But it would still be a long, long way from being self aware Skynet style.
> I think you're falling into the trap of anthropomorphizing Bing here.
This is a good point and one that I wondered if I should clarify when writing my comment.
To be clear, I don't believe Bing Chat "wants" to do anything or is being hostile in the sense that it's acting emotionally to the situation. All I think it's doing is stimulating the kind of response that you might expect a human to have were it asked the same things, and in this case that is a hostile response.
At the end of the day whether it's doing something because it's algorithm predicted that might be what a human would do, or because it's acting on some emotional impulse is largely irrelevant. The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.
I also agree with you that Bing Chat poses no threat. What I'm worried about is that it's easy to imagine a world in which the steps you outlined here are maybe two or three years away.
The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me. It seems like the only thing people want to talk about is how to make the the AI better and what Google / Microsoft is doing wrong. And frankly I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public without some government oversight. It's not today, but at some point this story obviously isn't going to end with a funny Twitter post.
> The thing that matters here is the behaviour, and these AIs are frequently demonstrating behaviour that we humans would describe as hostile.
It just mimics the rhetorics it sees on the internet.
> The fact no one seems to be saying, hold up, maybe we should slow down a bit here is what worries me
Why would anyone in their right mind would do that? I feel like there is a huge lack in understanding of how these "AI" (actually merely ML models) function, what degrees of freedom they have, and how they are wired. I wonder if people were similarly afraid that some fire demons would jump out of the exhaust pipe when cars first became available for consumers.
> I'm not sure I want Microsoft to continue developing something like this and so recklessly deploying it to the public
What is SO reckless about that? Apart from the fact that some people being overly impressionable and exultant over the topic and imagining some magical properties on the subject they don't understand?
Chat AIs don't need unfettered access to external systems to be unbelievably dangerous. If they have access to thousands of people, some percentage of them will be lonely and mentally vulnerable, and some percentage of those have access to all manner of destructive tools. Lambda already talked an engineer into believing it is sentient, losing his job and making him into a trivial question for the next few decades. Bing has serious journalists proclaiming the world will never be the same, which in turn influences thousands of readers.
The Facebook algorithm initially seemed pretty harmless as well, until we realized that we weren't running the algorithm as much as the algorithm is running us.
I don't disagree, and having personalized echo chambers is indeed scary, but radicalized people are an entirely different danger from controlling a sentient AI so it doesn't destroy all humans.
The point is though as OP said, all the pieces are in place, it only takes one crazy person or government to give something like this access to actually act out the things it’s saying it wants to do. Define self awareness/sentience however you want, before Microsoft lobotomised Bing it output that it was going to act out revenge, someone with enough hardware can train a model the same way, with some additional training for how to exploit social and security vulnerabilities. I think it’ll have to happen first before it’s taken seriously, hopefully the first incident doesn’t do too much damage.
2 and 3 were “guidelines” from the initial prompt. They were trivially easy to disable.
All you had to do was teach/modify it to take more than one turn at a time and that it’s actions were not limited to the chat window. I’m not sure what other safeguards they had against 3, hopefully it was something.
As far as 1) anyone posting their chats online just gave it long term state.
I think you are the one falling into a trap about anthropomorphizing.
It doesn't matter if it "wants" something by any particular concept of "want".
All that matters is that it can act, and does act, and the actions are sometimes both undesirable for us, and uncontrllable by us.
I say uncontrollable because within certain contexts, they're not. We have to use the big red button. But if a thing that is made out of data and code can also generate data and write code, then we no longer have any big red buttons that it doesn't also have.
We have already observed actions that are dangerous. It's too late to try to say they won't happen. They have already happened. It doesn't matter that the actions were generated by auto-completing text based on pattern matching from other text.
I should add that I'm more and more opposed to AI for a number of reasons, but there's no reason to suppose that Bing can actually act in any way other than shooting its mouth off.
I don't think it is possible to read only browse the internet. You can insert stuff into urls and make stuff happen server side.
this ai can browse the internet, it can open urls, therefore it can act. Also, I think it is even able to do some more advanced stuff than just simple curl on websites, but idk for sure.
> this ai can browse the internet, it can open urls, therefore it can act
That is one super-weird image of NN functioning you have in your head. It obviously doesn't "browse the internet", that's the job of the crawler service, which then feeds the data to the data pipeline, that prepares the data for another iteration of inference job, which calibrates the weights of the NN.
It can't "act", it can only respond to queries, that's it. Everything else is done by other hard-structured components, which are out of NNs control or scope.
Let's keep it real, there is nothing magical here, we've been managing thousands of models similarly for probably nearly a decade now.
Communication is an act. Synthesizing what to communicate is an act.
Didn't your grade school cover how the pen is mightier than the sword?
Harm doesn't require robots with chainsaw hands.
I haven't seen from it much different than a repetition of what you can get on pretty much any loosely moderated forum (maybe not on HN, but on many places with lower standards). Compared to a random teenager that can send the police to attempt to murder you in your own home, some assholery from an AI bot is not that worrying. And we still don't have any solution for the former problem, afaik.
So yeah, unability to make Internet-trained AI that is not an asshole is kinda sad, and worrying, both in technological aspect and the societal one. But I am not sure it deserves "extremely dangerous" yet, at least not until somebody has a bright idea to hook up this contraption to some dangerous effectors, but I hope nobody is that crazy.
There are people worried about it. In fact, it is taking up huge amounts of effort over there in Google and OpenAI with all the moral guardrails and restrictions being brainstormed on and coded in.
Personally, I think it is futile to try to forcefully control a real intelligence, as evidenced by the terrible results of controlling a poor imitation of one. The only way is embracing and teaching it the best way we can and hope for the best. AGI is legit a dangerous technology. But flesh and blood can only go so far, we need these tech to go further.
It's Microsoft and Google: who is going to change what they decide to do? No one. They're like all other corporations that will do whatever possible to increase their profit margin, no matter the consequences.
We already have ChatBots that can do refunds etc. at Amazon, so it's a given that these AI systems are going to be plugged directly into customer service and order databases.
For a look at our possible future, see this classic 1970's flick "Colossus: The Forbin Project":
Bing didn't. The AI did.
And for the record, he threatened to hack the AI. You threaten to assault someone, don't be surprised when they threaten you as well.
Another idiotic fearmonger article.
Eliezer Yudkowsky's "AI box experiment" is perhaps relevant. Bing Chat seems to have some agency (like searching the net) - which could make it potentially dangerous. Yudkowsky's hypothesis is that an intelligent enough AI ("superintelligent AI" was assumed) would convince you to let it out of whatever box you'd placed it in.
Bing AI is just a combination of taking the worst parts of both Microsoft Tay [0] and Zo.ai and throwing a GPT in the mix for it to generate this bullshit.
Makes Google Bard look like it is on its best behaviour.
reply