Zendesk on Zendesk: How we make the most of our NPS data

Comments

7 comments

  • Avatar
    Andrew Gealy

    Thanks for sharing this, Lori!

    You mentioned that "These text analysis programs are great at synthesizing what survey responses say, but fail miserably at discerning why they are saying it."

    It sounds like you now do individual ticket analysis by hand to gain insight into the "why," but do you still use text analysis software earlier on in the process to sort through or categorize raw survey responses?

     

  • Avatar
    Lori Gauthier

    Hi, Andrew! Thanks so much for your comment. :)

    We do still use text analytic tools to (1) understand the customer perspective and (2) identify comments that most typically require some degree of human interpretation. 
     
    For example, we might find from a text analysis program that 10% of customers' comments fall into a "feature gap" category. This first analytical pass helps us see that a notable proportion of customers think their pain points are driven by our features, or lack thereof.
     
    Then, when resources are available, we have Zendeskians take a second pass at comments in that "feature gap" category, because we know that that category tends to include comments where perception and reality are out of sync. 
     
    Now, notice I said "when resources are available." That second level of analysis can be time-consuming and, thus, quite costly. The resources aren't always available. 
     
    Fortunately, Zendesk has a whole team of engineers focused on developing machine learning tools for the customer support industry. My hope is that those same tools can one day be used to replace (or at least supplement) the human activities currently required at getting to the why behind the what of customers' NPS comments. 
     
    What about you? Are you running NPS surveys? Do you use text analytics tools (for NPS or any other surveys)? I'd love to hear about your experience. Please share! 
     
    And thanks again for the comment! 
    :) Lori  
  • Avatar
    Andrew Gealy

    Thanks for the additional explanation, Lori. :)

    We do have NPS surveys in place—one that appears while students use our product and another that’s sent out once a student has taken their exam and finished using our services. 

    We’re small enough that we haven’t yet established an ongoing system for analyzing these comments. We’ll go through a batch from time to time, but always by hand. We haven’t used any text analytics tools so far.

    I’m actually working to systematize and hopefully at least partially automate our process for reviewing this kind of feedback. Can you share what specific tools you use at Zendesk? 

  • Avatar
    Lori Gauthier
    Eep! I meant to respond so much sooner than today. So sorry for the delay, Andrew.
     
    Okay, what text analysis tools do we use for NPS data at Zendesk? We used Clarabridge for about 18 months and then trialed several other tools including QDA Miner Litetm (for R)SPSS Text Analytics, and Discover Text. Clarabridge performed slightly better than the other tools but at a substantially higher cost. We haven't decided which, if any, of these tools we'll use going forward. 
     
    My hope is that we choose a different path altogether: developing a tool internally that can analyze customer comments within the context of customer account histories. I'm meeting with a wonderful team of engineers at the end of February to discuss this possibility. Fingers crossed! :)
  • Avatar
    Matti Airas

    Hi Lori, You probably have come up with a solution already. I just wanted to let you know that our company Etuma provides an automatic feedback text categorization service to e.g. Zendesk users (http://www.etuma.com/embedded-analytics). It is a simple and feasible API service. We have commercial users and they are very happy. I would love to have a quick talk with you if this issue is still open.

  • Avatar
    Patrick Hogan

    I just want to commend you, Lori, for a great job on this post. Concise and clear explanation on how to maximize our NPS data. I also like to agree on doing follow-ups. It can really be a huge help where you can get additional insight that the system cannot give.

  • Avatar
    Peter (Edited )

    Hi Lori, 

     

    Interesting post. I was wondering if you had any further updates regarding Zendesk's own text analysis solution, and if you'd considered expanding to a 3 question NPS, namely "what's the score?", "why is that?", and "how can we improve?".

     

    Cheers!

Please sign in to leave a comment.

Powered by Zendesk