Understanding Answer Bot (EAP) Follow

Comments

33 comments

  • Avatar
    Jason Howard

    Very excited about this feature!  We've been using it for about 2 weeks now, and wanted to offer some feedback, for currently our deflection rate is 1.8% and the CTR is 9.6%.  I believe this could / should be better.  We are trying to train the system by utilizing the "Improve Answers" functionality but this doesn't seem to be helping.  One thing I've noticed is this:

    We have an app our customers use, on both android and iOS phones.  Our Help articles are therefore designed for android or iOS users.  There are many things in a ticket that indicates which platform the customer is on, i.e. "iOS" "iPhone" etc., both in the content of the email and via tags that are auto-populated.  Often times the suggested articles for tickets like this are all for android, not iOS.  Our articles very clearly say either "android" or "iOS" in the title, so the system should be able to pick up on this and suggest the appropriate articles accordingly.

    Probably a known issue - I understand we have early access to this - just wanted to give some feedback accordingly.

  • Avatar
    Josh Greenwald

    @Jason we are participating in this early access program as well. Where do you find those stats for deflection rate and CTR?

    I'd love to learn more about how to evaluate the effectiveness of this.

    @Aimee if you know please do share!

  • Avatar
    Jason Howard

    @Josh - all the metrics associated with Answer Bot are in Insights / GoodData.  Create a new report and you'll see an entire section for Answer Bot values.  Cheers.

  • Avatar
    Josh Greenwald

    @Jason, this was a great pro tip but for some reason when clicking around in Good Data none of the reports will populate.

     

    @Aimee is there a bug going on now or maybe am I adding filters in wrong?

  • Avatar
    Paul

    I would like to enable this on our account but I don't want the requester to have the option to solve/close  the ticket.

    My feeling is that we could still have the ticket open through the way our trigger was formatted but my preference would be wording that says something along the lines of - Some articles to help with your enquiry whilst you await an agent response.

    We use zendesk to manage inbound sales enquiries and as such want to close enquiries when we decide and I think currently this would be a confusing response for the requester.  Is this possible ?

  • Avatar
    Nicole Relyea (Edited )

    Hey Paul - 

    Thanks for sharing the details of your use case; that's really helpful. Answer Bot was designed with the idea that it could suggest topics and people could therefore resolve their question with self-service, but you're right works better in a support implementation and not as well for a sales one. 

    Let me check around and see if this is configureable.

  • Avatar
    Natalie Mercuri

    We are also piloting this feature, rolled out yesterday (06/09/17). I really appreciate the "Improve Answers" feature, however I think it may be beneficial to also have the ability to add suggested articles so that key words in future tickets will trigger more relative Help Center articles.

    I realize that a part of this function will include updating the tags and parsing out our Help Center articles, which we are currently undertaking, but we would also like to improve the Answer Bot algorithm in the process.

  • Avatar
    Nicole Relyea

    Thanks for the feedback, Natalie, and welcome to the Zendesk Community!

  • Avatar
    Paul

    @nicolerelyea

    I think you have summarised our predicament perfectly the the comparison between the support and sales use. 

    Let me know if you have any thoughts on configuring this functionality for a sales environment. 

  • Avatar
    Nicole Relyea

    Hey Paul - 

    Given that Answer Bot is still in early access, I'd assume that it's been developed with just it's primary use case in mind for the moment. That being said, if anyone in the community has ideas on how to tweak the configuration for a sales implementation, please do! 

    I'll be sure to update you if and when any internal development or customization in that direction happens. 

  • Avatar
    Rhonda Koerner

    Is this configurable by requester? Meaning, can we configure this so that the Answer Bot does not shoot out articles to internal people? Our Success team will forward emails to the support desk, on behalf of a client. We do not want the Answer Bot to send out articles to our Success Team as they don't need the article.

  • Avatar
    Nicole Relyea

    Hey Rhonda - 

    That's a good question. I'm looking into this for you. Standby. 

  • Avatar
    Nicole Relyea

    Hey Rhonda - 

    A couple of follow up questions for you: 

    - Are the folks on your Success team agents in Zendesk? 

    - How many people are we talking about (fewer or greater than 50 individuals?) 

    - Do you use any kind of SSO?

  • Avatar
    Nicole Relyea

    Rhonda - 

    A suggested solution is to tag the agents' profiles and exclude them from the trigger. But, this only works if the people you're wanting to exclude are agents, and if there's a reasonable number of them since you'd have to do this on a case-by-case basis. 

  • Avatar
    Rhonda Koerner

    Hi Nicole, thanks for the response. No, they are not agents. They are Success Coaches who submit tickets on behalf of the client. Once we get the ticket from the coach, we change the requester to be the client and work directly with the client. They are hand-off tickets. Is there a way to exclude them? Probably not, since they are not agents. Can you exclude an Organization?

  • Avatar
    Nicole Relyea

    Hey Rhonda - I don't think you can, but I'll double check. 

    Do they have profiles in your system? If so, you should still be able to tag and exclude them even though they're not agents. 

  • Avatar
    Joe Fedric

    I'm not sure how large of a problem this is for anyone else, but I've noticed a regular occurrence of users submitting tickets, getting their answer suggestions, then selecting "Please close my request".  Then, they leave a bad review, because we "never got back to them".  This is frustrating, because they're "solving" their own ticket, but they don't seem to understand they are opting out of a personal response from us.

    Obviously, ticket deflection and customer self-service is the goal with this tool.  So, does anyone have any ideas for improving this particular UX?  I do not see any way to target only people that selected the "Please close my request" in the Automations or Triggers area, so I cannot exclude them from customer surveys.  Maybe the team could add an "are you extra-sure you don't want to hear from our team?!" prompt or something of that nature.  I'd hate to try to solve this with my own HTML+CSS+JS solution if it's coming down the pipe, or there's a better way I just don't know of yet.


  • Avatar
    Jason Howard

    +1 to Joe's request above, for I've had this happen twice as well.

  • Avatar
    Joe Fedric

    Only twice?!  I'm jealous.  Man, I'm sitting on about 3 dozen bad reviews thanks to this (our volume is ~300-500/week).  I've reached out to these people to educate them on what that button does, but I can't do that for the thousands of people that reach out to us per month.  

  • Avatar
    Rhonda Koerner

    Maybe think about changing the wording of your trigger. This app is buried within a trigger. You can't change the wording of the app but put something leading into it in the trigger. Something like....."Thank-you for reaching out to our support team. We have listed below articles that may provide you an answer while you wait for an agent to address your ticket. 

    If the article answers your question and you do not need an agent to respond, please choose, "Yes, close my Request". This will automatically close your ticket and you will not receive any updates from an agent. If you still want to speak to an agent, do not choose any article."

     

     

    It sounds a bit wordy and clumsy but it may help with people giving bad surveys

  • Avatar
    Jason Howard

    @Joe - I guess we're lucky - our volume currently is ~50-60/week. 

  • Avatar
    Joe Fedric

    @Rhonda,

    I suppose changing the wording is one way to do it, assuming they are reading :)

    My customers have a bad habit of not reading :p

    I'm going to work on the trigger a bit - I just wanted to see if anyone else had arrived at a solution that was working for them, on this specific aspect of the self-service model in the Answer Bot.

  • Avatar
    Rhonda Koerner

    We have AnswerBot turned off right now. Until we can figure out if there is a way to not send the surveys to clients that read the article and closed the ticket without any agent assistance. Not keen on surveying people that had no agent interaction.

    Best of luck Joe!

  • Avatar
    Joe Fedric

    @Rhonda, so you've faced the same challenge.  I'm not sure if I should disable it, as we're a photo editing software company that has many technical processes to follow should something not work.  So, the value we see delivering relevant articles is a bit higher than the takeaway from people getting upset because they closed the ticket themselves.

    I suppose what we need, @Zendesk, is a boolean field in the triggers/automations:

    Requester Solved Ticket through AnswerBot = true/false

    This is one of the few things bugging me right now - my satisfaction rating should be higher!!!  Let's get this added, I'll be more than happy to test it :)

  • Avatar
    Nicole Relyea

    hey guys - glad you're having such a productive conversation. 

    I checked in with the Answer Bot team, and they suggested having the Answer Bot trigger add a tag, and then tell the CSAT automation not to fire if that tag is present. 

    I haven't had a chance to test it, but you could try that out and see if it takes care of this issue for you. 

  • Avatar
    Joe Fedric (Edited )

    @Nicole, The problem I have with that approach mirrors ZD's own logic when it comes to automatic ticket merging: it is unpredictable. Do I really want only customers that didn't get an automatic answer to provide feedback? There are only a small subset of users doing this, and they're just not aware that they're doing it. I'm afraid of stopping feedback from all customers that received even a single article suggestion. It's like cutting off the nose to spite the face.  Nearly 2/3 of my requests have some automatic answer attached to them.  This solution is a bit overkill, IMO.

  • Avatar
    Rhonda Koerner

    also, from what I can see (and please correct me if I am wrong) there is no way to create a tag for Answerbot that states that the ticket was closed by a client who read the article. Every one of our tickets has a link to the articles provided by AnswerBot. I can't put in a tag called "answerBot" because then no one would get a survey. Can the AnswerBot team please provide the tag instead of just saying.....add a tag? Help us a little bit during this Beta Testing. thanks

  • Avatar
    Nicole Relyea

    Hey Joe - hm. Let me dig into this a bit more. Standby. 

  • Avatar
    Nicole Relyea

     

    It occurs to me that we might get some better answers by having this conversation over in the Answer Bot Community Forum, where other users and the Product Managers hang out. I've pinged some folks internally about this conversation so they can jump in, but if you guys want to pick it up (or ask any other questions you have) over in the other forum, you may get some better responses from other users as well. 

  • Avatar
    Joe Fedric (Edited )

    @Nicole, I'll move my convo over there! :)

    Edit: Do you think you could save us a bit of time and let me know if there is already a topic on this subject?  I'd like to search for it, but I have customers to answer :D

     

    Edit 2: Nevermind, I found it: https://support.zendesk.com/hc/en-us/community/posts/115007189668-Preventing-Users-who-solved-tickets-via-AnswerBot-from-leaving-satisfaction-ratings

Please sign in to leave a comment.

Powered by Zendesk