About Answer Bot (Early Access Program) Follow

Comments

75 comments

  • Avatar
    Amie Barder (Edited )

    Does this feature work with the mobile SDK?

    Also, the article says it will provide "up to three articles". What happens if there are no good matches?

  • Avatar
    Patrick Harland-Lee

     

    I've noticed that since we implemented it today, when I try to show the first article in the email the images from the article all fail to load. Any idea what's happening?

  • Avatar
    Amy

    I've just set this up today, loving it so far! One question I had was from a reporting perspective... Is there a report I can generate to know how many tickets out of our total volume have been solved with automatic answers going forward?

    Are there long term plans to add localisation support for it - we're a language learning company with 12 supported languages in CS so having this across the board would be amazing!

     

  • Avatar
    Pete LeRoy

    Hi Amy, I'm glad to hear that you're loving this so far. I'm also a Zendesk client and I just activated Automatic Answers and have begun testing. However, my results have not been desirable. Is your Knowledge Base public facing? How many articles do you have? Thanks, Pete  

  • Avatar
    Amy

    Hi Pete!

    Out of the few tests I did yesterday we had about 75% success rate.

    We have about 60 articles or so in our KB which is public facing - https://help.busuu.com/hc/en-us

    Our challenge comes as we get English contact from users who are not native English speakers which is where automatic answers doesn't seem to work so well.

    I'm wondering if there's a way to help the system learn quicker if irrelevant articles are suggested by marking them as inappropriate?

  • Avatar
    Pete LeRoy

    Amy, 75% success rate sounds fantastic! Has your rate been that high from the start? Are you including labels in the "List of article labels to be included by Automatic Answers for this trigger" field in your Automatic Answers auto-respoder trigger? Thanks, Pete 

  • Avatar
    Alex Bakula-Davis

    +1 to Amy's question about reporting!  It would be great to quantify the time savings this gets us.

    Any tips on how to structure articles to pull better results?  We use Answer Suggestion, and the articles pulled there are a lot more accurate than the ones given through Automatic Answers, even on VERY basic questions (ex. how do I delete a user?)

     

  • Avatar
    Greg Pope

    We just set up Automatic Answers and it's working well in early testing.Two questions for Mike:

    1. Is there a way to edit the color of the View Article buttons so they match our Help Center branding?
    2. It would be great if the end user's Help Center ticket activity included a link to the article that answered their question so they could go back to it if needed. Is there a way to do that?
  • Avatar
    Tim Cambra

    We noticed that in our trigger the # commands arnt working. We had "(#{{ticket.id}})" but its not actually linking the end user to the ticket in zendesk as it used to. We had to use a full URL for the end user to get to there ticket from the email which does work(https://{{ticket.url}}). Is this a known issue or am i doing something wrong?

  • Avatar
    Gadi Vered (Edited )

    Hi - I like this feature and have a question about setup.

    I have followed the setup instructions here, they are in addition to the basic "notify requester of received request".

    as a result - 2 emails are sent to the end user (customer).

    This is a bit excessive - can i bake this into my first response?

    If so - why were the instructions for 2 seperate triggers? 

  • Avatar
    Mike Mortimer

    Hi all, going to try cover all the points, questions and feedback. Let me know if I missed anything! :)

    Amie, Mobile SDK integration - not currently, although we will be looking at how to best offer this feature in other channels.

    Patrick, images not loading - can you open a request with us by emailing automatic-answers@product.zendesk.com with the ticket IDs and any other information?

    Localisation (multi-language support) - this is complicated and much harder to do with this type of AI feature than normal features, as the AI works by understanding the language model for English. That said, we are evaluating the best options to support as many languages as possible in the future

    Reporting - There is detailed reporting in Insights. Head to the "Insights - view only" dashboard then to the "Deflection" tab... Feel free to build your own reports and metrics (and let me know if you have a report that you think would be worth sharing with everyone else)

    Flagging irrelevant suggestions to help aid the AI - maybe... stay tuned.

    Irrelevant suggestions - send us the example ticket IDs and the article you think should have been suggested so we can investigate (automatic-answers@product.zendesk.com)

    Tips to get best results - (1) A single article should represent the solution to a single question / problem, (2) If you have too many articles, perhaps try whitelisting a subset of the articles you know are good FAQ style articles so only these can be used for suggestions, (3) If you have segments of customers (and are able to target those segments on ticket creation, as a trigger condition) then you can also use article labels to whitelist subsets of articles that would be best suited to each segment.

    Greg, Customisations to colour / style, etc - we're currently focussing on functionality and ensuring the feature works as it's meant to. We do plan to offer some level of customizability in the future.

    Greg, integration into requests portal - assuming that's what you mean? Appreciate the feedback, we may look at this in the future.

    Tim, would you be able to send that issue in to automatic-answers@product.zendesk.com so we can take a look into the trigger and debug it? 

    Gadi - we would expect that you will run either the Automatic Answers trigger OR the Notify requester of received request - not both. The instructions allow you to setup the Automatic Answers trigger separately, mainly so you can test it and once happy, simply activate it for a wider audience. The instructions are a guide, you can certainly update the Notify requester of received request trigger instead - you just run the risk of breaking it if something is not right. If you encase the automatic answers placeholders within the automatic_answers.article_count logic check (as in the example in the guide) you can easily have the AA trigger serve as a normal Notify requester of received request even if there are no articles returned.

    Thanks everyone!

  • Avatar
    Leif (Edited )

    Is there a way to set up AA to not send out based on certain words/phrases? Part of our group uses Zendesk for tickets, but they're specialized in what they do and we don't have articles for them. Thanks!

  • Avatar
    Gage Nestor

    I have my triggers set up but the articles that appear are not relevant to my test emails. I made 6 separate triggers because I have 6 different helpdesk subject but I'm seeing that when I get the article recommendations that aren't even related to the trigger I had set up. They seem to be mis matching. For example, I have a trigger for Shipping and one for RFID but when I send a test email trying to get shipping articles it gives me RFID articles instead. Any tips on how to fix this?

  • Avatar
    Mike Mortimer

    Hi Gage, I created a ticket to look into your issues.

    Leif, you can definitely setup the trigger to look for specific subject or comment text and (when present) exclude these tickets from the trigger (and catch them with a different trigger).

  • Avatar
    Jennifer Haight

    I was notified that I have early access, but when I look under Admin > Business Rules, I don't see "Automatic Answers" listed.

  • Avatar
    Mike Mortimer

    Hi Jennifer, for some reason we had missed your account! I have enabled this for you and you should have also just received the welcome email. Let me know if there are any questions / issues.

  • Avatar
    Antti Rajakangas

    When solving the test tickets through the article, it solves the tickets without filling the mandatory fields or adding any tags to the ticket. Is there a way of tagging the ticket with the labels the article has? I can create a trigger to try to pre-fill some fields but I can't figure out how we can get the detail of what the contact was about without keeping the ticket open and manually adding the data we track? I know the contact is deflected but for me it would be great to see that detail on the tickets in addition to the detail of the article.

     

     

  • Avatar
    Mike Mortimer (Edited )

    Hi Antti,

    That's great feedback - it's obviously pretty difficult to replicate the classification that your agents may be doing currently and have this be automatically done for these ticket, but your suggestion about overlaying additional article information such as labels might be a good, simple step. We'll have a look at including this as an extension to the Insights article dataset in future

  • Avatar
    Jennifer Haight

    Hi there,

    I'm seeing articles that are in Draft mode (and therefore shouldn't be visible to end-users) returned on Automatic Answers. I am testing this as myself (who can see that article) - perhaps that's why?

    Also, is there a way to control for brand? We have two brands and a separate help center for each. I only want the corresponding brand's content to be returned for incoming tickets associated w/ that brand.

  • Avatar
    Thomas Bouchard

    In testing this feature, I found that the three articles suggested to the end-user were in no way related to the test ticket Subject field. But, this got me thinking. What data points is the Automatic Answers feature using to find suggested articles - Subject, Description, something else?  Please help me understand this. Thanks!

  • Avatar
    Mike Mortimer

    Hi Jennifer, draft articles should be getting excluded, have they always been draft? Can you send us a bit more info to automatic-answers@product.zendesk.com so we can check that out? The requester will always only be given articles suggestions for articles they have access to, however, even for agents, draft articles should be excluded. In terms of brands, the feature already considers this. It'll only suggest articles from the brand associated to the ticket.

    Thomas, can you please also send us some example tickets to automatic-answers@product.zendesk.com so we can review? 

    The deep learning model uses both the subject and description to understand the meaning and context of the ticket and compares this to your articles to find the closest meaningful match. It's not a keyword searcher - its optimised for long form sentences as you'd expect to see in normal email questions. That does tend to throw off some customers when testing, as they send in a ticket with one keyword in the subject and description and expect certain articles containing that keyword. When testing, we do recommend using as realistic examples of normal tickets (subject and description) as possible.

    Thanks!

  • Avatar
    Patrick Harland-Lee

    Hey Mike, does the Automatic Answers product search for ticket tags in the article as well?

  • Avatar
    Elisa Reggiardo

    Hi Mike,

    still testing Automatic Answers internally got an internal comment in Korean (!).

    Email sent to the requester with article links was sent in English though, any clue what is going on?

    Thanks!

  • Avatar
    Mike Mortimer

    Hi Patrick, no article labels are only used to create subsets for the solution to then analyse within.

    Elisa, we fixed a bug the week before last that created some translation issues and inconsistencies for end-users. The agent experience wasn't changed because we are about to rollout a completely new experience for agents this week. Stay tuned!

  • Avatar
    Aja Varney

    Hi Mike, 

    So far, we're loving this Automatic Answers feature... however, I saw referenced above that there is supposed to be detailed reporting in the Insights Tab. 

    Any reason I would not have that "deflection" tab? I tried logging out/in and such, but still no deflection tab. Ideas?

  • Avatar
    Jennifer Haight

    What condition(s) could I add to my trigger so it doesn't fire when an agent starts a ticket with a requester?

    The assumption being that if we start the ticket, we don't need Automatic Answers to suggest articles.

  • Avatar
    Mike Mortimer

    Aja, we'll get that sorted out for you - it likely hasn't activated yet as there is no data flowing through. You'll need to have some attempted Automatic Answer email sends / solves before the dashboard will activate

    Jennifer, you can use the condition: Current user: is not (agent).

  • Avatar
    Sam

    Hi ...also loving this feature so far.  3 things,

    1. although it asks the requester if this has solved their issue in the email with nice buttons, it none the less still solves every ticket that was answered automatically immediately. Is that because of another rule we have or is there something in the auto answer trigger that needs to be setup differently?
    2. The agent says ZENDESK as the resonding agent name. Is there a way to customize this and show PUMP! Support (or something) ?
    3. Is there a way to just deal with English tickets, like "ticket must be english" and we use whatever detection you guys do whether AI or browser href-lang to figure out or guess the language? 

    Thanks!

     

  • Avatar
    Jennifer Haight

    Did the UI change for how the Automatic Answers show up to the agent in the ticket? Rather than an internal note, I'm seeing this - http://d.pr/i/rh0a

    Will the article names be coming back here? I found the names more helpful than the raw URL. Also don't know why I'm getting a permission error when I hover over the URL (see screenshot above).

  • Avatar
    Greg Pope

    We're noticing the same thing this morning - url rather than the article name (not a huge deal but the name would be best) and all of the articles show a permission error.

Please sign in to leave a comment.

Powered by Zendesk