Understanding how Answer Bot works with your content

Have more questions? Submit a request

13 Comments

  • Hannah Voice
    Comment actions Permalink

    Hi Jeremy Korman

    Reading over this article, it sounds like Answer Bot is not designed to get more intelligent.

    "Although Answer Bot is powered by a machine learning model, this does not mean that Answer Bot is constantly learning. Answer Bot’s model does not incorporate feedback in real-time from end users or agents. Therefore, the feedback has no influence on which articles Answer Bot will recommend."

    "Answer Bot will consistently recommend the same articles regardless of any feedback from agents or end users."

    In that case, I am wondering how the Improve Answers feature comes into play. Can you explain more? If I'm understanding this article correctly, it seems like having agents mark articles as off-topic wouldn't be of any value?

    Many thanks,

    Hannah

     

    2
  • Jeremy Korman
    Comment actions Permalink

    Hi Hannah Voice, thanks for reaching out. Earlier this year when we upgraded the model to support additional languages and jargon, we also evaluated the “improve answers” feature. The analysis showed providing this feedback was eating up agent time, but wasn’t delivering a substantial impact to how well Answer Bot performs.  After much consideration, we decided to remove the “improve answers” button on July 31st 2020.  We expect that this will save you and your agents time, and allow you to focus on other best practices for maximizing Answer Bot’s performance. We’ve been reaching out to accounts that are using this feature to let folks know about the upcoming change, I’ll double-check that you’re on the distribution list

    0
  • Hannah Voice
    Comment actions Permalink

    Thanks for clarifying Jeremy Korman. Could you confirm my understanding here that there are no circumstances under which Answer Bot will get more intelligent the more it's used? Does anything at all impact it (other than optimizing Help Center content)? Thanks.

    0
  • Jeremy Korman
    Comment actions Permalink

    Hey Hannah Voice, that's correct, Answer Bot won't automatically improve on its own. However, the more you use Answer Bot, the more data you'll have available in Explore and via Content Cues (if you're on Guide Enterprise).  The out of the box dashboard makes it super simple to discover opportunities to improve your help center and boost Answer Bot's performance.  Here's an overview of the dashboard

    -1
  • Dawn Anderson
    Comment actions Permalink

    Hi Team

    We did a search using a certain keyword and it's returning an article that has no mention of this word or label - do we know how the answerbot knows what to return?

    Thank

    0
  • Jeremy Korman
    Comment actions Permalink

    Hi Dawn Anderson, thanks for the comment.  There are 2 quick things I'd like to point out:

    1. If Answer Bot suggested a particular article then the algorithm detected that the inquiry and article were in the same neighborhood (if we're using the analogy from the article above).  Unfortunately, there's not an easy way to expose exactly what words or phrases made this happen for each inquiry.
    2. Answer Bot uses labels to create a "whitelist" of articles, but it doesn't use labels like keywords (check out the misconceptions above).  Here's another article with more info.
    0
  • Ibrahim
    Comment actions Permalink

    Hi,

    Finally a bit of explanation on how Answer Bot works. Can I summarize this as follows:

    Answer Bots suggests (almost arbitrarily) 3 answers.

    It will not learn that certain answers are correct/incorrect.

    There is absolutely no way to teach Answer Bot.

    It is us, the agents, that have to learn how each article in the Help Center may have to be written to hopefully fit Answer Bot's (black box) Algorithm.

    If we don't adapt the articles to Answer Bot's algorithm then it may keep suggesting the incorrect articles..forever.

    0
  • Jeremy Korman
    Comment actions Permalink

    Hi Ibrahim, thanks for the comment.  Our data science team has done a lot of work to reduce the amount of energy agents and admin should need to spend managing Answer Bot, so I certainly hope the article recommendations don't appear arbitrary!  If you haven't done so already, I'd recommend you check out this article on Optimizing your Content for Answer Bot.

    Also, I'd add that Answer Bot won't always recommend 3 articles.  If Answer Bot only finds 1 relevant article, it'll only recommend the 1 article.  Similarly, if there are 0 relevant articles, there will be 0 suggestions.  So there could be 0, 1, 2, or 3 suggested articles.

    Lastly, you're correct that the best way to influence Answer Bot's recommendations is to adjust the article's title and the first 75 words.  However, there are other ways you can guide Answer Bot to search from a more relevant sub-set of articles by using Labels.  Here's a comprehensive list of additional resources that you might find interesting!

    0
  • Ibrahim
    Comment actions Permalink

    Hi Jeremy,

    agent's time is not reduced by forcing them to figure how to adapt every article to all variations of customer queries which are processed by a black box.

    Adapting every article (which are also processed by the black box) to preprocessed queries (by a black box) is like shooting in the dark.

    1. Is there a way to aiding/leading agents to write articles which will map to respective query-'neighbourhood'?

    2. Is there any way at all of teaching Answer Bot where to search? 

     

    0
  • Arwen Griffioen
    Comment actions Permalink

    Hello Ibrahim,

    I'm sorry to hear that you feel like Answer Bot is a black box. The machine learning team has worked extensively with Jeremy to write a clear and understandable article on how the model behind Answer Bot works. If there is a particular part of the article that isn't clear enough we'd be more than happy to try to expand it.

    Given that Answer Bot is only processing the first 75 words of the article it is important that each article covers only a single concept OR that the opening paragraph summarises all of the concepts that are covered. I see that your articles tend to cover multiple topics and that the first paragraph doesn't generally elaborate on the topic indicated by the title. This simple addition to your articles would boost Answer Bot performance and may help your users as well. 

    The Content Cues Product can help you discover areas where the content of your articles might not address your customer's questions. This will help to focus content management efforts and ensure that less time is spent making changes without impact.

    Do please submit a ticket if none of this information helps and we can look at your specific use cases, articles and issues.

    Arwen
    Machine Learning

     

    0
  • Ibrahim
    Comment actions Permalink

    Thanks for the clarification.

     

    Content Cues is only available to Enterprise plan, right? If so, is there any other similar tool or approach?

    0
  • Jeremy Korman
    Comment actions Permalink

    Hi Ibrahim, yes, Content Cues is a Guide Enterprise feature.  If you're not on Guide Enterprise, you can build a report to see transcripts of what your customers are asking Answer Bot, check out this Explore recipe: https://support.zendesk.com/hc/en-us/articles/360048645274

    1
  • Frédéric
    Comment actions Permalink

    sad news that the "improve answer bot" button has been removed without notice.

    It was helpful.

    0

Please sign in to leave a comment.

Powered by Zendesk