Understanding how Answer Bot works with your content

This article offers a high-level overview of how Answer Bot works. It includes the following topics:

• How does Answer Bot process natural language?
• How does Answer Bot Article Recommendations feature decide which article to recommend?
• Common misconceptions: What Answer Bot doesn’t do

How does Answer Bot process natural language?

Answer Bot is powered by Artificial Intelligence which means that it is able to mimic human behavior. Answer Bot uses natural language processing (NLP) to read every article in your help center and to understand the main concept behind each article. Answer Bot then takes all the concepts from all the articles and places them onto a map. Each concept gets its very own “address” on the map so that it lives near other, similar concepts. However, instead of just city, street, and zip code, this address has 500 parts. Whenever a new question comes in, Answer Bot does its best to understand the concept that the question is asking about and use the map to determine the closest existing article.

For example, here are some concepts that Answer Bot might extract from a few questions:

Question	Possible concept
How do I dump my tickets to a file?	Exporting Data
I’m locked out of my account	Account Access / Password Reset
How do I create a crane?	Folding Origami Birds

How does Answer Bot Article Recommendations feature decide which article(s) to recommend?

When an incoming question closely matches with an existing article, they become “neighbors” on the map (as described above) and it’s clear that Answer Bot should recommend the article. However, when the closest match is a few streets over, or in a nearby neighborhood, it becomes less certain that the concepts are related.

The data science team at Zendesk carefully monitors Answer Bot’s performance and has finely tuned this over time by adjusting a “threshold knob”. This threshold is not adjustable by admin or agents, it’s only accessible to the Zendesk development teams. The threshold knob is a global control, meaning it affects all Answer Bot accounts, and is used to determine how closely two concepts must be on the concept map to be considered similar concepts. If the threshold knob is turned up, Answer Bot becomes more conservative and will recommend fewer articles that are more likely to be relevant to the question. However, this means there will also be more questions where Answer Bot does not make any recommendations at all. If the threshold knob is turned down, Answer Bot will recommend more articles, but there’s a higher chance that some of the articles will appear irrelevant to the end user.

Common misconceptions: What Answer Bot doesn’t do

There are some common misconceptions about Answer Bot, and machine learning in general, that can lead to confusion over how they work. In this section, we’ll address these misconceptions and hopefully give you a clearer understanding about what Answer Bot does -- and doesn’t do -- with your data.

It includes the following questions:

• Does Answer Bot learn based on end user feedback? Isn’t that where the machine learning comes in?
• Is Answer Bot’s AI-powered search always better than a keyword search?
• Can I “train” Answer Bot by asking the same question and answer over and over again, and responding with “Yes” or “No” to mark an article as relevant or irrelevant?
• If I add labels to my articles, is that like adding a keyword to the article? Can this be done to boost how often an article is suggested?
• If I can’t improve Answer Bot with the “improve answers” button, how can I improve Answer Bot’s performance?

Does Answer Bot learn based on end user feedback? Isn’t that where the machine learning comes in?

Although Answer Bot is powered by a machine learning model, this does not mean that Answer Bot is constantly learning. Answer Bot’s model does not incorporate feedback in real-time from end users or agents. Therefore, the feedback has no influence on which articles Answer Bot will recommend.

The end user feedback is captured and used in a number of ways:

• It is displayed to agents to provide additional context on what articles were viewed, marked as “not helpful,” or used to resolve a case
• It is exposed in reporting for admin to track Answer Bot’s performance
• It is evaluated by the data science team at Zendesk

If you see that Answer Bot is repeatedly recommending incorrect articles, the best thing to do is modify the title and the first 75 words of the articles to make the main concept more clear. You can also create a list of articles for Answer Bot to draw from by using labels so that Answer Bot’s suggestions will only draw from a sub-set of articles.

Is Answer Bot’s AI-powered search always better than a keyword search?

Overall, we’ve found that Answer Bot’s AI-powered recommendations are more accurate and relevant than a keyword search, especially when the question is asked as a full sentence (instead of one to three words).

However, there are times when a keyword search may work better. For example, when a user asks a single-word question via Web Widget (Classic), Answer Bot defaults to using a keyword search, as this is generally more accurate for single-word queries. The exception to this is languages, like Chinese, that do not have explicit word boundaries like spaces.

Can I “train” Answer Bot by asking the same question and answer over and over again, and responding with “Yes” or “No” to mark an article as relevant or irrelevant?

No. Answer Bot will consistently recommend the same articles regardless of any feedback from agents or end users. Answer Bot is specifically built so it doesn’t require any training to get started. It’s already pre-trained to understand natural language. If you test out a phrase/question and Answer Bot is making incorrect recommendations, the best thing to do is modify the title and the first 75 words of the articles to make the main concept more clear.

If I add labels to my articles, is that like adding a keyword to the article? Can this be done to boost how often an article is suggested?

Labels are a great way to create a list of approved articles that Answer Bot can pull from. However, labels do not have an influence on the weighting that Answer Bot gives to each article. For more info on labels, check out this article.

If I can’t improve Answer Bot with the “improve answers” button, how can I improve Answer Bot’s performance?

The best way to improve Answer Bot’s performance is to consider the following:

• Analyze your Answer Bot Activity - Use Explore to see which articles are you best and worst-performing.
• The Structure of Existing Articles - Look at your help center articles and make sure that the content is concise and well organized. Each title should be phrased as a short sentence or a question.
• Content Cues - Use machine learning technology and Guide article usage data to help you discover opportunities and tasks that will improve the health of your knowledge base.