This article includes the following topics:
- How is natural language processed?
- How are articles selected for recommendations?
- Common misconceptions
Related articles:
How is natural language processed?
AI agents use artificial intelligence to evaluate articles, which means that it is able to mimic human behavior. The AI agent uses natural language processing (NLP) to read every article in your help center and to understand the main concept behind each article. It then takes all the concepts from all the articles and places them on a map. Each concept gets its very own “address” on the map so that it lives near other, similar concepts. However, instead of just city, street, and zip code, this address has 500 parts. Whenever a new question comes in, the AI agent does its best to understand the concept that the question is asking about and use the map to determine the closest existing article.
For example, here are some concepts that might be extracted from a few questions:
Question | Possible concept |
---|---|
How do I dump my tickets to a file? | Exporting Data |
I’m locked out of my account | Account Access / Password Reset |
How do I create a crane? | Folding Origami Birds |
Note that the AI agent automatically detects the language used in an email by combining the subject and description and using language prediction. This may cause suggestions to appear in a language that doesn't match the one set in the end user's profile.
How are articles selected for recommendation?
When an incoming question closely matches with an existing article, they become “neighbors” on the map (as described above) and it’s clear that the AI agent should recommend the article. However, when the closest match is a few streets over, or in a nearby neighborhood, it becomes less certain that the concepts are related.
The data science team at Zendesk carefully monitors and has finely tuned this over time by adjusting a “threshold knob”. This threshold is not adjustable by admin or agents, it’s only accessible to the Zendesk development teams. The threshold knob is a global control, meaning it affects all accounts. It's used to determine how closely two concepts must be on the concept map to be considered similar concepts.
When the threshold knob is turned up, the AI agent becomes more conservative and will recommend fewer articles but the recommendations are more likely to be relevant to the question. However, this also means there will be more questions without any recommended articles or help center content. When the threshold knob is turned down, more content is presented, but it's less likely to be relevant to the end user.
Common misconceptions
There are some common misconceptions that can lead to confusion. In this section, we’ll address these misconceptions and hopefully clear some things up.
- Does the AI agent learn based on end-user feedback? Isn’t that where the machine learning comes in?
- Is AI-powered search always better than a keyword search?
- Can I “train” the AI agent by asking the same question and answer over and over again, and responding with “Yes” or “No” to mark an article as relevant or irrelevant?
- If I add labels to my articles, is that like adding a keyword to the article? Can this be done to boost how often an article is suggested?
- If I can’t use the “improve answers” button to improve performance, how can I improve performance?
Does the AI agent learn based on end-user feedback? Isn’t that where the machine learning comes in?
Although it's powered by a machine learning model, this does not mean that the AI agent is constantly learning. The model does not incorporate feedback in real-time from end users or agents. Therefore, the feedback has no influence on which articles are recommended.
The end user feedback is captured and used in a number of ways:
- It is displayed to agents to provide additional context on what articles were viewed, marked as “not helpful,” or used to resolve a case
- It is exposed in reporting for admins to track performance
- It is evaluated by the data science team at Zendesk
If you see that the incorrect articles are repeatedly being recommended, the best thing to do is modify the titles and the first 75 words of the articles to make the main concept more clear. You can also create a list of articles to draw from by using labels so that suggestions come from a sub-set of articles.
Is AI-powered search always better than a keyword search?
Overall, AI-powered article recommendations are more accurate and relevant than a keyword search, especially when the question is asked as a full sentence (instead of one to three words).
Can I “train” the AI agent by asking the same question and answer over and over again, and responding with “Yes” or “No” to mark an article as relevant or irrelevant?
No. The AI agent will consistently recommend the same articles regardless of any feedback from agents or end users. It is specifically built so it doesn’t require any training to get started. It’s already pre-trained to understand natural language. If you test out a phrase or question and the wrong articles are recommended, the best thing to do is modify the titles and the first 75 words of the articles to make the main concept more clear.
If I add labels to my articles, is that like adding a keyword to the article? Can this be done to boost how often an article is suggested?
Labels are a great way to create a list of approved articles to pull from. However, labels do not have an influence on the weight given to each article. See Best Practices: Using labels to optimize your article recommendations.
If I can’t use the “improve answers” button to improve performance, how can I improve performance?
The best way to improve AI agent performance is to consider the following:
- Monitor your autoreply with articles activity: Use Explore to see which articles are your best and worst-performing.
- Consider the structure of existing articles: Look at your help center articles and make sure that the content is concise and well organized. Each title should be phrased as a short sentence or a question.
- Use Content Cues: Use machine learning technology and article usage data to help you discover opportunities and tasks that will improve the health of your knowledge base.
30 comments
Rich Andersen
Hi everyone,
I've just created an article. How long will it take the bot bot to detect the new article and it's content?
0
Arwen Griffioen
@...
It should only take a couple of minutes for the article to be integrated into Answer Bot
0
Lauren Mulkern
Does Answer Bot impact the keyword search results with its machine learning, or is that still another algorithm when AB is enabled?
0
Josh
Thank you for messaging us.
Keyword search results greatly impacts the Answerbot as it learns how people search. It mimics human behavior. You can lean more on this article: Understanding how Answerbot works.
0
Lauren Mulkern
Hi Josh. The link you provided directs back to this page, which I've read, and unfortunately, there is no answer to my question here.
Does the Answer Bot machine learning algorithm impact the results you see in non-answer bot locations, or are these locations still using the regular Zendesk search algorithm? Examples: the Instant Search results that show when you are typing in the search bar, and the search results in pages like this: https://support.zendesk.com/hc/en-us/search?utf8=%E2%9C%93&query=answer+bot
Thanks!
0
Arwen Griffioen
Hello Lauren,
Answer Bot does not impact standard search methods, optimisations or results. It is a separate algorithm that is only used in Answer Bot enabled channels.
Cheers!
1
rasmus.kjeldgaard
This article has a dead link -> Analyze your Answer Bot Activity
0
Nikki
"If I add labels to my articles, is that like adding a keyword to the article? Can this be done to boost how often an article is suggested?"
The answer here seems to indicate NO. That adding a label is not like adding a keyword and does not influence Answer Bot's results. However, other articles seem to contradict that, such as this one.
It says that
Don't these first two bullet points very much sound like creating a keyword that boosts how often an article is suggested?
What am I missing here?
2
Gustavo Oliveira
That is a very good question.
The article mentioned, says indeed that you can influence article search relevance and Answer Bot results, but that doesn't mean it is going to change how often the article/community post will be suggested.
The points mentioned says:
And it also says:
Please see also the article “Understanding the relevance score in search results” for a deeper explanation on the subject, as at the end of the day, the articles/posts suggestion are based on that score.
Best regards,
1
Nikki
Gustavo Oliveira - thanks for the tip about the article “Understanding the relevance score in search results” -- that was very informative!
0
Gustavo Oliveira
You're welcome. I am really glad to hear it was helpful :D
Have a lovely day ahead.
0
Fiona Witham
Hi
Is there a way to tell Answer Bot to NOT suggest articles from some categories -we have internal only information in our Support Center which is marked for viewing by agents and admins only and we do not want those articles suggested to our end users.
Thanks
Fiona
0
Cheeny Aban
If the article is only available for agents and admins, those articles will not be suggested to your end-users. If you have an experience where an article was suggested to an end-user, I suggest that you check the permission of the article. Moreover, you can also create user segments to make sure that articles will be accessible by specific users. More information about user segments can be found here: Setting view permissions on articles with user segments
0
Fiona Witham
Hi @...
Thanks for your reply.
Answer bot was suggesting articles even though they were marked as agent/admin only. I actually found the answer elsewhere - I have added an article label to all the 'public' articles so that answer bot only looks in those, otherwise it suggests even articles that are only for admin/agent.
Thanks
1
Riah Lao
Hi,
FYI, the link for Analyze your Answer Bot Activity is not working.
How can we view the tickets where answer bot failed to give suggestions and also where the customer rejected the suggested articles?
Thanks
0
Cheeny Aban
Thank you for flagging this article. Will send feedback to the proper team to have it fixed!
0
Riah Lao
Hi @...,
How can we view the tickets where answer bot failed to give suggestions and also where the customer rejected the suggested articles?
0
Josh
Thank you for reaching out to us. For this, you can create a report for answerbot using Explore: Explore recipe: Analyzing Answer Bot activity
Here's another for reviewing the last 100 tickets your answerbot encountered: Explore recipe: Analyzing the last 100 Answer Bot tickets
0
Crystal Chenier
Hello,
I am having trouble with my AnswerBot generating articles after entering keywords in the search. After entering something as simple as a Password Reset, all it does is start the AnswerBot over, it doesn't generate a article about password resets. Any suggestions?
0
Dave Dyson
I see you posted this as a comment as well - I've asked our customer care team to have a look. Thanks for your patience!
0
Crystal Chenier
Thank you
0
Allen Lai | Head of CX at Otter.ai
I created an answer bot and one of the conditions is if the end-user selects a specific ticket form. But in many of my tests, the email that's sent back to the end-user is missing the recommended article I want it to send. Is it because the request's subject or description doesn't match the article? How do I have it send a specific article and not require the ticket subject or description to have to match the article?
0
Acacia Voynar
I understand answer bot searches all help center articles. Does this include search articles that are set for agent & admin permissions and are otherwise unavailable to signed in end users?
0
Dane
End-users will not be served with articles restricted to the user segment Agents and Admins.
0
Sophia Rakowsky
If the brand that a bot is associated with has additional brands specified as search sources in its help center, will the bot search all those sources when suggesting articles? Or does it only suggest articles in that brand's help center?
0
Noly Maron Unson
Hi Sophia,
Help Center is unique to the Brand it is associated with. This means that if the inquiry is being submitted as part of Brand A, Answer Bot will only look for articles within Help Center A.
Hope this helps.
0
Sophia Rakowsky
Right, but since search settings allow us to enable cross-brand searches, one would assume the bot would have access to all those sources. We have found a workaround to this via API but it would be nice if there were an easier way. One solution would be to allow us to save articles to more than one brand.
0
Yassar Saleem
I am facing this weird issue where the bot doesn't understand the question but if I ask again it suggests the correct articles.

0
Matt Farrington Smith
Question for Jeremy Korman and others (although this discussion seems dead!)
Does the Answer Bot just search articles, or can it also search the Gather discussion forum (community) too?
I know we can return forum posts in site search so I was wondering.
If not, can the API be utilised to add these in?
0
Destiny
Thanks for getting in touch. Currently, the bot isn't equipped to pull data from Gather forums or the Community sections. Our Product team is aware of this gap, and they're considering it for future updates, although we don't have a set timeline for when this feature might be implemented. For the best impact, I encourage you to post your detailed feedback and specific use case here: Include Community Discussion content in the bot, which can help our Product managers understand and possibly prioritize your needs.
In the meantime, you may find these articles useful for refining your bot's ability to conduct article searches:
Regarding the use of APIs, you can indeed utilize the Answer Bot Recommendations API.
I trust you'll find this information useful.
0