I use probably about 15 cloud SASS products for my company. Over the past year many of them have switched from offering either live chat or email support to chat bots. Often they will send out a marketing email pitching it as a huge improvement in support. My experience with chat bot support ranges from frustrating to infuriating, especially when I need help with something business critical.
I'm not sure what the right adjective is to describe how it makes me feel when I encounter one, especially if it's with a company that we're doing serious business with. Disrespectful? Condescending? I think belittling works, because it definitely gives you the impression that you're concern is trivial. Now that they have your money they just need to placate you with as little effort as possible.
I mean, the same is true for phone bots, and companies have been using them as tech support for decades. At least with phone bots, however, there is usually an incantation (often involving pressing zero repeatedly) that can get you to a live human.
Personally, if a company can't provide support where you speak to a human, I'd prefer that they be honest about that - rather than having my hopes raised then dashed and my time wasted by a chat/phone/e-mail contact that doesn't go to a human.
I worked on some IVR implementations some time ago. Quite a few would interpret and understand swearing, and route you to a human via a priority call queue. Similar for multiple button presses of the zero key.
Chatbots aside, Comcast and United Airlines continue to be big consumer businesses despite horrible customer service reputations. Socially it's bad, but economically it's hard to justify spending that much on customer support. (Those companies also enjoy semi-monopoly positions, though, so I wouldn't recommend emulating that unless you enjoy a similar position in the market.)
Comcast is a monopoly, with a huge barrier to entry.
People complain that UA hasn't learned anything about customer service. I am surprised that those people haven't learned anything about UA, and are still flying it. (Perhaps it's because flyers don't actually care about customer service, and will take whichever flight saves them $15.)
> It's a new use of technology, give it time to improve.
Why is it a new use of technology? I seem to remember chatbots being around in the 90's... or am I imagining things?
Of course, they sucked, and people said "they will get better and eventually get better than humans." I don't see what has changed in that regard.
Maybe the assertion is still true, but despite advances in NLP and other areas of ML, it hasn't seem to have improved the bots, or their associated customer experience.
Exhibit A: flying cars, "yes, they suck now, but they'll get awesome Real Soon Now" - same core meta-issue: most of the problems are still unknown and haven't been explored yet.
I develop Alexa skills and I agree with the article. Another point that is missing is that the technology hasn't "closed the loop". We have great ASR and NLU available, but responses still need to be built by hand or templated. This is the major reason bots feel so unnatural. I think there's a lot of opportunity for robust Natural Language Generation solutions to help build voice apps and chat bots that can operate more flexibly.
Also, there's a need for a more declarative, high level programming language to describe how you can interact with a bot. Defining them as trees or graphs isn't flexible enough. I've had promising results from experimenting with concepts from Ceptre (https://github.com/chrisamaphone/interactive-lp). Fortunately there are a lot of startups experimenting in this space, I think we'll see the most effective practices replicated quickly.
In my experience, the major reason bots feel unnatural is that they quickly reveal their lack of actual understanding of anything, with the mean time to Turing-test failure being not much longer than the preliminaries (such as getting your account #.)
Actually, maybe 'unnatural' is not the quite right word, as I have had some human customer service that is no better in this regard (or perhaps they were actually some of the better chatbots?)
I would also posit, that the chatbots are not armed with the ability to actually solve problems.
I'd use a chatbot, if it could understand that X is a fraudulent transaction, and to have it cancelled. Instead I get , "Would you like to return your item?" -- NO, it's fraud.
It seems either the engineers responsible, or the upper management have made it an order to not allow higher functioning bots. Im guessing they're using the bots as yet another layer of obfuscation to get people to go away, just as companies used "longer than expected" delays on phone lines to get people to leave.
I agree with the conclusion. Voice input instead of text and better understanding and language generation would help.
However the current status is not so grim. In a different application domain, a customer of mine uses a chatbot to collect leads of credit requests. Their very basic chatbot converts much better than their form. The difference is so big that it pays for the extra work to manually extract data from those conversations that went to the end but we can't automatically get all the information from (funnily formatted dates, addresses, etc)
Does anyone know of a single good chatbot? Every single one I have had the misfortune to interact with has been the best advertisement for their owner's competitors that I know of.
Currently our intents match banking and insurance questions related to the nordic market. This type of question needs training data that is specific to the US region. There is no problem supporting it technically, it is just lack of training data.
You right about that, you ask a difficult question that a even human could not answer correctly as it isn't precise enough.
All I actually do know from that question is that it is related to insurance cover abroad. But do you mean car insurance? Travel insurance? Health insurance? As we do not have an intent for "unknown insurance cover abroad" yet, we fail to help you further. This happen sometimes for many contexts, but we work hard to improve this. We do cover this kind of questions very well when people have login issues, as this is the most common inquiry we have in production. It can be a range of problems from token issues, cell phone issues, technical issues on the website or that the account is simply blocked. But we often manage to help people find the final solution without involving a human.
My point is, we can solve this today, we just need more training data. Chatbots are not perfect, but with more training data they will become better, and be a highly praised service. Finding information using natural language is the future. I'm confident that we will solve most of the edge case inquires soon.
But....but....they shipped something? However awful it is, they ticked that box and have their awful POS UI live for the world to see. And backed by fancy technology and expensive devs.
That is a good question, I don't understand why it isn't already covered, but I told our AI trainers that they should add training data for this asap :-)
Wendy: Hello, I am Wendy our Virtual Agent. I am here to help you with your support questions on High-Speed Internet, Digital TV and Home Phone.
You: Can I get cheap internet without tv and phone?
Wendy: To inquire about or change your current service plan, please contact a customer service representative at (800) 347-1991, or find us at your local retail store.
These are pretty well-worn arguments against chatbots. There's an overuse of bots for tasks that do not benefit from a conversational interface (checking the weather, surveys, promotions, etc.). But if you stop thinking of bots as sentient beings and instead treat them as domain-specific command line interfaces, you may get more utility out of them. For instance, it's possible just typing/saying "Send me a pizza with sausage and black olives" is the most efficient way to order a pizza.
I think that the pizza analogy is really useful. We've had intelligent voice-capable agents for pizza ordering for ages: calling the store. People hate it! They want a specialized UI for ordering pizza.
Chat API's usually offer inline keyboards and regular custom keyboards. At least in Telegram you can even replace inline keyboards based on interactions, meaning you could trivially construct a pretty elaborate UI specifically for stuff like ordering pizza's.
And even better, the pizza joint can update the conversation with status updates, special deals if you so desire, and you automatically have a history of previous orders that you could simply repeat at a later time.
A chat bot strikes me as the perfect UI for these types of things, and at this point, with Messenger, Messages, and Telegram supporting all this, a large number of your users already have 'your' app installed!
Right, I think that WeChat basically exemplifies this too. But at that point, is the user really interacting with a "bot", or is the chat platform just an app delivery mechanism like an OS or a web browser?
I'm honestly asking, and I think the pizza example is really illuminating - how DO people order somewhat-customizable food via these platforms?
> Right, I think that WeChat basically exemplifies this too. But at that point, is the user really interacting with a "bot", or is the chat platform just an app delivery mechanism like an OS or a web browser?
I'm assuming the latter. I mostly avoided WeChat as an example because i haven't used it myself. But from what I understand WeChat goes much further than just chat interfaces, which kind of is out of the scope of what I'm really interested in for now (but still curious about).
EDIT: I'd add that many of the advantages that come with using chat-apps as a platform are still there even in apps that don't offer all of what WeChat does.
> But if you stop thinking of bots as sentient beings and instead treat them as domain-specific command line interfaces, you may get more utility out of them.
That's my thinking exactly, and while I agree with the article, I feel it would've been much more interesting if it explored what kinds of ways chat interfaces and bots can be used.
The first example mentioned is illustrative:
> Press 1 to report missing credit card... Press 2 to report a fraud... (...) Press 8 to activate a credit card...
> It's far from an enjoyable experience. When you call them and hear a recorded voice, you can tell that your call is going to take a while. You also know that there's a high chance you'll end up talking to a real person anyway.
> Chatbots are no better than these robots.
I strongly disagree with this. The problem with this interface is not the 'robot' part, the problem is that you have to patiently listen to a robot describe the options before you get to choose.
If I could have a chat with a bot that immediately provides me with an inline keyboard or a list of options so I could choose an option right away, or type it out, or perhaps even type out a command-line style multi-argument command, I'd prefer that over pretty much any other interface, including clicking around their clunky website. And most chat API support inline keyboards that can even 'replace' themselves, allowing for a pretty powerful mini-ui.
The problem, as I see it, is trying to make it natural and adding all sorts of NLP and 'human' elements. Let bots be bots and at most augment them with these things.
I truly believe that chat bots can end up being a 'CLI for non-geeks'. They're not appropriate for everything, but what with custom keyboards, inline keyboards, and other inline doodads, I wouldn't be surprised if they'll end up being successful even for the non-CLI or non-chat type use cases. Everyone has Telegram or Messenger or Messages, and 'everyone' seems to be moving to their phone or phablet as a primary interface these days. They don't want you shitty custom app, they hate your clunky barely-mobile website. But their favorite chat app has authentication, essential UI stuff, 'native' services (location, etc.), and push notification and sync built in, so why not use that?
For a growing number of people around me, the 'de-facto' chat interface is the one they have open much of the time (plus snapchat's weirdness, perhaps). While I feel the current interfaces are still lacking for these non-person-chat use cases, already now I find that they can be much more useful than people realize. It wouldn't be too difficult, I suppose, to optimize these chat apps to handle various bots better (and perhaps WeChat already figured this out well enough?).
I run a Facebook page, and they've recently started pushing chats through Messenger every time you post something asking you to spend money to boost the post. Every single time. And if you do boost, they'll beg you to boost again through Messenger. I tried "stop", I tried "cancel", I tried "unsubscribe", and I even tried cursing at it to see if it would stop. It didn't. Eventually I had to go into Messenger settings and block Facebook's chatbot. I reported it as spam, since there's no way to make it stop bothering you.
And now I see they're automatically opening Messenger when you visit someone's page to get you to message that business for... I'm not exactly sure what reason.
It still is "all the rage" for a lot of companies who have invested a lot of money to push this UI. What you're seeing here is backlash from people who are forced to use it and found it lacking. The only people who like it are businesses who paid for it and want to force people to use it.
Going a little sideways it's not like at any Call Center you usually get to talk (without having first gone through all their standard "script" and possibly become upset and manage to escalate to a higher level of support) with someone that actually can provide you with assistance beyond what is already on the FAQ of the correspondent site.
(provided the site FAQ's are actually frequently asked questions and not - as often happens - a bunch of questions that the original website developer jolted down because he/she imagined they will be asked and that noone actually updates with "real" questions asked by customers).
So you get to talk with a human, but very often this human is either clueless or cannot really do anything about the issue at hand. (not of course fault of the human, but rather of the way he/she was mis-trained or because of directives coming from the company)
It's not that easy on many support Call Centers in my experience, not before having gone through the checklist with a number of unuseful suggestions (already tried because listed in FAQ).
> When you're filling a signup form, you instantly know what information they'll need from you. You can decide if you're comfortable sharing them before you engage in filling it.
If only that were true! The trend is to ask for a little bit info at a time, then take you to the next page where it asks for other info, and so on. You don't see at a glance the whole form anymore.
I resort to filling in fake info on sign-up forms, order pages, employment applications, airline/hotel reservations, etc., just to find out what questions they'll ask so I can decide whether to refuse before I tell them anything.
And also to find out key information they tell you only at the end (sales tax, shipping cost, etc.).
Along similar lines, I detest a trend I've noticed where login forms show the username/email & password on separate forms. It's annoying to have to click through more than is necessary and additionally breaks my password manager's auto login.
Worse yet, they have very similar (except for color) buttons for "Log in" and "Sign up". Often if you have bad internet and the last layer of the page hasn't loaded yet, you don't know which button to press.
The worst offender here is Microsoft, if you have a personal and a business account under the same email address (that alone is a major UX fuckup). But they add insult to injury by showing you a typical email+password form, and when you press [Tab] after the email address and start typing your password, the form is replaced under your eyes with a dialog to choose which account you want to log in, only then followed by a password prompt.
They (and we, I admit to doing this with Level) do this because statistically more people make it through the information collection if we stagger it out.
People see a fleet of fields and just go. People see 2 fields and they tend to just enter in data. Even if they leave and come back, mobile apps and websites can much mor easily checkpoint this (digression: I think it's easy to checkpoint a form but lots of people don't and only web toolkits make it drop-dead easy to do).
Sorry you're on the right hand side of the computer competency power law distribution.
>I resort to filling in fake info on sign-up forms, order pages, employment applications, airline/hotel reservations, etc., just to find out what questions they'll ask so I can decide whether to refuse before I tell them anything.
That's a good strategy, particularly since there are websites that send everything you type, even if you erase it or don't submit.
If you can spend the time to write good documentation for your product then you don't need a chatbot, just good search and good formatting.
If you can't be bothered to write good documentation then you won't be bothered to feed useful information into the chatbot, which is way more clumsy than a searchable manual anyway.
A great many people won't read your documentation and will become "stuck" until they have a person explain things. If they're customers you pretty much have to tolerate it unless you're willing to lose them as customers.
Chatbots are extremely hard. Many of us are jumping on the bot wagon and leverage AI with NLP & intent recognition. But they miss the important bit: It's all about the conversation itself.
"If content creation isn't right, the conversation won't delight" (Adrian Zumbrunnen)
The technical aspect behind chat bots is only the tip of the iceberg. We need Conversation Experience Designers instead of AI Engineers & UI Designers.
Many chatbots start with questions like "how can I help you? Type something". What should I as a customer expect from the bot? When I visit a website, it's pretty soon clear how it works and what to expect. This doesn't apply to bots.
Great article! I have been working on my own Chatbot solution based on IBM's Watson (supposedly cutting edge tech)... it is cumbersome... it cannot do something simple like extract a full address from a conversation (I think cities are still in beta)... The good news, there is a definite opportunity for some start up. AI is all the rage and for solving a specific problem with clearly defined parameters (or if you have a team of a few 100 people working on it) it is getting pretty good. But interacting with a human is many things... "defined" is not one of them.
I suspect what's great about assistants like Siri, Alexa, and even Google Now is less that they have voice interfaces and more that they have well-funded support teams behind them to deliver exponentially more efficient routing of queries into appropriate knowledge-bases based on the best of contemporary research driven by their respective corporate thinktanks.
Compare and contrast with data mining the logs of call centers into RNNs and hoping for the best. A fun exercise here is to take one's SMS logs and do the same. In my experience, one will achieve an amazing Max Headroomization of one's cliches and trademark expressions, but little else.
When confronted with a voice menu system, I used to scream "OPERATOR! OPERATOR! OPERATOR!" at the top of my lungs and bang on the 0 key repeatedly until I got connected to a human. Recently, I've noticed they hang up on this behavior and even force one to wait through the entire braindead list of choices before allowing pro-users to bypass all the inanity because, after all, one must "listen carefully as our options have changed!"
> Recently, I've noticed they hang up on this behavior
I've definitely had them ignore any sensible operator prompts, but actually hanging up? That's a step beyond. Care to name and shame any offenders?
(I've always enjoyed the claim, and once or twice believe I've observed the behaviour—though I am sceptical—that a lot of these systems scan for obscenity, and route to an agent. When I get a robocall while I'm out walking (phone for audiobooks), I always curse at it for a while, just in case. Even though it almost never gets an agent, it's a good stress reliever!)
I've worked phone support for a major cell carrier. Among the things I was told there by a manager.
- That people that spam zero should be hung up.
- We could do a fake accent if we could maintain it through the whole call.
- After they had deleted the feature for support to activate your new device as a cost saving measure we were to point users to the proper procedure to do it themselves on the web and tell them that feature was broken despite the fact that leads could still do this.
- To make up fake things we had in common to establish a relationship with the customer so they would give us better surveys.
That sounds terrible, but I'm confused by this one:
> - After they had deleted the feature for support to activate your new device as a cost saving measure we were to point users to the proper procedure to do it themselves on the web and tell them that feature was broken despite the fact that leads could still do this.
Does it mean that you were told to give instructions that you knew wouldn't work, or 'just' that you were told to direct users to the hard way of doing something even though there was an easier way of doing it?
We were told to tell to customers that the feature was broken and that they had to do it themselves rather than pass it to team leads who in fact could do it in order to encourage users to do it themselves. Meanwhile we were providing support for the segment of customers the company considered most valuable. Most were paying $100-$250 a month.
Frontier airlines absolutely does this. I think it's only during hours when there are not agents working, because I was able to reach an agent when I called again during more normal business hours.
But it did feel really weird hearing the automated system say "that is not a valid option. goodbye" and hang up on me. No indication that I could call back during certain hours to reach a human.
I really hate when they don't treat 0 (unlisted option) as what it actually is.
The user is telling the menu tree that those who programmed the tree didn't include this use case or that poor descriptions routed the user to the wrong area. Have a human cleanup pass.
Often I do end up pressing 0 when I have a strong suspicion that I'm going to need to speak to a human to get my actual problem fixed and am pretty sure nothing's going to change that.
Siri is not essential. Every time it fails to deliver a feature of my phone, I can thumb my way to it in seconds. So it's actually fun whenever it works.
Also: because it's not essential, one gets to adjust to its failures. There's a broad range of queries for which I know Siri will just say "this is what I found online for..."; I tend to just open the browser and use Google's autocomplete instead.
So... I have a problem when the chatbot is trying to perform an essential service, like getting a technician to come to my home because cable internet is not working. Imagine medical triage done by chatbots, jesus christ.
An MTV chatbot that tries to do interactive music recommendation? This could work. Spotify's algo is already very good, but you can't refine "yeah, I wanted something more like X, but a little like Y. And not X exactly" -- just "give me more like X including lots of X".
And nobody gets hurt by a bad song recommendation.
That's my feeling of Siri usage as well, that it's not reliable enough to just sort of take for granted.
But, what's really interesting to me is that Alexa has surpassed that level (at least in my household). So that for routine things, it's not a novelty to use it, but an actual improvement.
This is still obviously for non-essential services like music, games, weather, etc. but it's also reliable enough to handle things like turning on/off lights in awkward spots.
Alexa isn't really a chat bot. You issue commands to her. She does an amazing job at it. Seriously, the voice recognition is truly great. But you are still issuing a command.
This article is... weird. It basically assumes the Facebook pseudo-menu-driven chatbots people fat-finger into their systems with almost no consideration for UX are the state of the art.
It then compares it to a personal agent that actually has a similar model and the worst voice recognition and search precision of the current crop agent software which... isn't even really the same kind of software?
What is this article even trying to say? Poorly written bots are bad, well-written agents are good, and voice interfaces aren't as bad as people make them out to be?
I've been spending a good part of a last year working on a chatbot with a very domain-specific scope, with that goal of arriving a product that delivers a modest but effective user experience. It's hard and it takes an unbelievable amount of work to handle myriad of edge cases you get into when a human on the other end tries to treat your bot like a real albeit limited interlocutor in the conversation.
The real problem I see is that the last decade of user experience improvements that work well on the web don't really translate into a chat, as an industry we're hopelessly bereft of best practices at this point, and our users notice this and experience it as the frustration of having no idea what to expect from a bot. The title article and comments cite a lot of good examples.
NLP and other applications of machine learning will make bots better at delivering correct answers, but making bots feel not-horrible around the edges is about user experience design. Here are some suggestions that have helped me a lot.
* Design for failure first
Just like mobile-first design gets the brain out of the pattern of tacking on mobile interactions as second class citizens, failure-first design focuses on the primary experience the users have of your bot, it not working. Don't delude yourself into thinking that your NLP intent parsing is going to result in more hits than misses down your happy path to user delight. A human will always sidestep your intended flow by accident, and that human will form judgements about your product based on it's ability to gracefully recover. Luckily the bar is incredibly low here.
* Be careful with conversational niceties and over-humanization of tone
It's easy to think that friendly banter and emojis can help personify a bot and smooth over the above-mention failure paths, but the novelty of these wear off quickly for a user, and the user is likely to experience more frustration if the conversational tone doesn't match their frustration. It is also extremely easy to end of in the uncanny valley when using friendly conservational copy in the bot messages. Repetition of a robotic message feels benign-if-annoying, but repetition of a cute emoji-laden phrase can feel very off-putting.
* Fall back to being a CLI with visibility and helpers
If you've ever been stuck working with a bot, you know that all you want is to know what it can do, and how you can get it to do that thing. If you notice the user is in a failure state through keyword matching or repeated failed routing attempts, fall back to a high-visibility list of actions. Having quick-action buttons can make this even smoother.
* Train the user on consistent hooks and keywords.
When speaking in a human conversation, utterances of 'stop' or 'wait' are almost always respected as context-independent keywords that escape or pause the context of the conversation. If I asked you what you wanted for dinner, and you responded 'stop', I wouldn't try to figure out what kind of food that was. In my project, 'help' 'quit' and 'back' are all respected as keywords, and every context of the conversation implements callbacks to respond in context to each of these.
* Ask a lot of questions that are easy to answer
Handling raw language is super hard. Routing language into a finite set of options is a lot easier, plus humans feel listened to when asked for clarification or if they have been understood correctly. When taking user input and routing to an action, ask for yes/no confirmation, and provide options like "This is totally wrong". Opportunities like this could be great to collect data about how users are stuck to improve on the experience. It's also validating for the user to feel like they can specify that they were not heard correctly.
The issue with customer service is that a lot of companies end up with awful CS because 1) they outsource it to braindead monkeys who hate their job and are paid pennies and 2) there is no way for someone to complete their task without talking to CS (no documentation, etc).
So instead of pouring endless money into chatbots, allow people to do whatever they wanted without involving CS. Provide good, up to date and easy to find documentation and a web interface if the task needs data entry (porting your number, changing your billing details, etc). This will reduce the load on your CS which means you can now bring this back in-house and no longer rely on an outsourced disgruntled workforce.
To the surprise of nobody who has ever gone through automated telephone system hell. Whenever I bump into one, I impulsively shout 'zero' 'zero' 'fucking operator' into the phone.
I usually just jam the zero key. Most of the time it works.
Have you ever used one that emits "typing" sounds in to the phone? AT&T started doing this some time ago on their small business wireless support line. I jam zero until it says, "Ok, enter the number you are calling about", I enter it, and it says, "Ok, let me look that up for you", then keyboard taping sound... I can't even describe my reaction the first time I heard it. So, so stupid.
So, in contrast to the article I personally feel excited about the potential of chatbots - especially as custom and user-friendly CLI's/UI's more than as AI-driven 'human' interfaces, at least for now - in a way I've not felt excited about anything in a long while (both from a business and ideological perspective, but I care more about the latter, all things considered).
I'd just like to ask that if anyone is seriously pursuing chatbots, beyond just helping businesses or consultants sell the 'next big thing', I'd really, really love to talk. My email address is on my profile page.
Chatbots, home IoT, and to some extent VR all strike me as things Silicon Valley wants to be "the next big thing" but are not and likely will not be because few people want them.
I absolutely despise chat and voice for any kind of trivial task. I want a UI. Chat and chat bots are annoying. Voice in any form if infuriating. Having to actually call a number on the phone for any reason always sends me into a minor little rage. "Oh, you have to f'ing call for it... sigh." Why would I want more of this kind of experience? Maybe a few people do but I can't see it being a Big Thing(tm).
I really do not want an Internet-connected fridge, stove, light switch, or microwave. When I buy such things I look for non-IoT-encrusted versions since those are going to be more reliable and are not going to be conscripted into a botnet or spy on me. IoT features in these kinds of products deliver nothing of value to me. I also really do not want DRMed food. That includes newer Keurigs. There is a healthy vintage Keurig market on eBay for a reason.
VR still seems like a fad or a niche product for certain subsets of gamers. There's some market for it but it's no "E-Commerce," "Social," or "Mobile."
It's sort of funny to see the SV investment culture producing so many solutions in search of problems after hearing that same culture lecture against this practice for decades. "Do the market research first!" "MVP MVP!" etc. They're not following their own advice.
Yep chatbots sucks. The trick for good customer support is to enhance a human agent with AI and not trying to replace it. Suggesting answers to the agent and automatically replying when the confidence is very high.
reply