Working with Satisfaction Prediction Scores Follow

Comments

15 comments

  • Avatar
    Wouter van Gessel

    Wow, this seems like an amazing tool! Customer service is our USP, therefore we are very eager to use this. 

    In this article you mention "you need a minimum of 500 satisfaction ratings", however in another article (https://support.zendesk.com/hc/en-us/articles/218050717-Introducing-Satisfaction-Prediction) you mention "this feature also requires a minimum of 500 satisfaction ratings per month".

    We do have more than 500 ratings in total, but only around 150 per month. We really hope we can use this feature. Can we and if not, will it be possible in the future?

  • Avatar
    Omari Whyne

    I second this comment!

    Look forward to hearing a reply.

  • Avatar
    Josh Greenwald

    Maybe there is a way to train the model on historical data? Maybe an ask for one of their internal teams ?

  • Avatar
    Tsvetan

    Same here, if we have that minimum of 500 in total, but each moth these are less than 500, is this going to work?

  • Avatar
    Aimee Spanier

    Hi, all. Sorry for the error - as I understand it now, you need 500 *per month*, and should have at least 1,000 ratings total if possible.

     

  • Avatar
    Charles Magnuson

    It seems this feature isn't working as described. We receive around 2,000 CSAT ratings per month, ~500 per week, yet I am told we don't have enough ratings: http://ca.mag.cm/2o1D2L0G1V2c

     

    My interpretation is that we all need to wait at least one month for Zendesk to build up enough data about our accounts for this tool to begin working correctly. @Zendesk, can you confirm this?

  • Avatar
    Leandro Gomes

    Only enterprise? or not!.... 

     

    What you doing zendesk?

  • Avatar
    Alfred Diez

    This is very cool. We get escalations sometimes and it's great to be able to have the system predict which ones may go side ways before it actually occurs.

  • Avatar
    Jon Thorne

    We've enabled this and didn't get the same message Charles did, but we haven't got the predictions showing yet. As per Charles' message, my questions is also:

    "My interpretation is that we all need to wait at least one month for Zendesk to build up enough data about our accounts for this tool to begin working correctly. @Zendesk, can you confirm this?"

    Thanks

  • Avatar
    Jessie Schutz

    Hi Jon! Sorry about the delayed response here.

    Support is currently working with Product to find out what's going on here. I'll be sure to post here when we get it figured out!

  • Avatar
    Jessie Schutz

    Hey Jon!

    Sorry for the delayed response. Here's what I was able to find out about this:

    "Satisfaction Prediction uses a Machine Learning process to train a model for your account. Although you meet the minimum initial criteria to have a model built, there is a validation process that is applied to the model to ensure it can accurately predict the outcome.

    If the model doesn't perform well enough, then the feature cannot be enabled. To overcome this problem we simply need more data. It is possibly due to the types of tickets you have - if they are too varied, with insufficient good and bad ratings then the model won't perform well. The automated model building process will continue to attempt to build a new model for your account. Once it is able to successfully build a model that meets the threshold, the feature will become active and your account administrators will receive an IPM and email to notify them."

    Please let me know if you have any other questions!

  • Avatar
    Andrea Brown

    I'm curious about how satisfaction prediction works with Hub and Spoke. Will the prediction come over to my Hub account with the ticket? I see that I need to turn it on at the Spoke level, but I'm curious about whether I'll need to also log into the Spoke to see this feature.

  • Avatar
    Megan Howell

    @Andrea, The satisfaction prediction stays inside the Spoke accounts and isn't carried to the Hub, so you will need to log into the Spokes to see those predictions! Sorry for any inconvenience this might cause, but satisfaction is generally separated so one Spoke doesn't have a great influence on all the others. Let us know if we can answer anything else!

  • Avatar
    Wouter van Gessel

    We were very enthusiastic about this feature at start. After using this for 1 month, we find that the predictions are way off, making this function useless. Therefore we've removed the function. Perhaps it works well for larger companies with thousands of tickets per day.

    CAUTION: We did find out that clicking "Enable satisfaction prediction scores", automatically enabled the System Automation "Request customer satisfaction rating".

    This caused a month of unwanted satisfaction rating requests emails to our customers. Bad bug Zendesk...

    Automations should never be activated without the user knowing...!

  • Avatar
    Mike Mortimer

    Hi Wouter, thanks for sending your feedback through. I'll try answer the 2 parts of your experience, first - the prediction scores: Because of how we normalise the prediction scores (to ensure stability for your automations / triggers) the score itself may not seem accurate - we are currently reviewing alternative options for showing the score but the best way to interpret it is to take your long term CSAT rating (i.e. your current CSAT score overall, over a period of time). 100 - this score should give you an accurate threshold for where a ticket moves between good and bad, e.g. 100 - 88 = 12. Scores below 12 are high risk for negative ratings. We've validated this across many accounts using the feature.

    In terms of the second issue - this is actually not related to satisfaction prediction but rather an unfortunate bug related to the core CSAT feature where saving that form again will cause the system to interpret this as activating CSAT even though it was already active. This is now a known issue and should be resolved shortly. Indeed, if you simply updated a standard CSAT related setting then the re-activation of the trigger also would have occurred. 

Please sign in to leave a comment.

Powered by Zendesk