This article provides answers to some of the most frequently asked questions about Zendesk QA. 

This article contains the following topics:

  • AutoQA categories
  • Autoscoring
  • General

AutoQA categories

Why are some conversations not evaluated by AutoQA categories?

After AutoQA categories are set up, AutoQA evaluates conversations based on the following criteria.

  • Evaluated
    • New conversations imported with a Closed or Solved status are evaluated. Historical data is not evaluated by AutoQA.
    • Conversations that have at least one message by the customer and agent, with a minimum of word count of 10.
    • The participant (agent) in the conversation has to be a user in Zendesk QA and a member of the same workspace that has AutoQA categories set up.
  • Not evaluated 
    • Historical data
    • Conversations marked as "Trash" (i.e. they are spam or do not have enough context).

Why do some categories have N/A as an evaluation?

AutoQA may sometimes return an "N/A" evaluation for certain categories. This occurs when there is not enough information to provide an accurate rating. For Azure categories, at least two messages must be sent by both the agent and the customer.

If you disagree with an AutoQA score, you can change the rating to better suit your process. Admins or Account Managers can customize the following AutoQA categories: Spelling and Grammar, Greeting, and Closing.

Can we add custom rules for any category?

At this time, you can add custom rules for Spelling and Grammar, Greeting and Closing. You can also create custom AutoQA categories from scratch. 

Can I whitelist words for the Spelling and Grammar category so that they are ignored?

Yes, Admins and Account Managers can whitelist words.

To whitelist words

  • Open Account Settings > AutoQA > Spelling and Grammar > Add exemption.

Autoscoring

What can I do if I disagree with a score?

You can always change the AutoQA rating to a rating you think suits your process more. Admins and Account managers can only customize the following AutoQA categories: Spelling and Grammar, Greeting and Closing. You can also create custom AutoQA categories from scratch. 

To customize AutoQA categories

  1. Open Account Settings > AutoQA
  2. Select the category.
  3. Enter the customization in the search field then click Add Exemption/Closing/Greeting.
    We are doing exact matching, so ensure you add different variations of the expression/sentences you want to include in the list.
  4. Apply the relevant languages for each customized listing.

I would like some evaluation cases to be stricter/less stricter, what can I do?

We can’t change the autoscore.

Please share any feedback that you have about autoscoring. To help gather feedback about category accuracy, please mention context, category name, given score, suitable score, conversation link, and reasoning for long-term score change. We cannot guarantee the change but we will improve the accuracy over time based on the feedback.

Are autoscores counted towards the Internal Quality Score (IQS) of a conversation?

Category autoscores are provided as helpful information. They only count towards IQS after a reviewer has submitted the review.

Highlighting

Why does the highlight toggle not appear for some categories?

All Zendesk QA–based AutoQA categories have highlighting capabilities. However, highlighting for Azure-based AutoQA categories is still in development. When a category is selected and is present in both the Azure and Zendesk QA lists, the Azure one is preferred (for better accuracy). Therefore, the highlight toggle and highlights will not be present until this feature is released.

Why are some categories highlighting a specific place in the conversation?

When highlighting is present, it can be in the form of highlighting words (e.g. the Spelling and Grammar category) or messages (e.g. the Greeting category).

General

Can AutoQA detect and evaluate chatbots?

Yes. AutoQA evaluates Zendesk and Ultimate bots but does not offer a combined Auto Quality Score (AQS) for bots in the Dashboard. This is to avoid penalizing bot builders in potential mismatches for human categories. The combined AQS is only available for agents.

What languages are supported for the AutoQA categories?

Category name Languages/OpenAI-enabled Languages/OpenAI-disabled
Closing All English, German, Spanish, French, Italian, Dutch, Polish, Portuguese
Empathy All English
Grammar & Spelling English (US), German, French, Polish English (US), German, French, Polish
Greeting All English, German, Spanish, French, Italian, Dutch, Polish, Portuguese
Solution All Not available*
Tone All Not available*
Readability English English
Comprehension All Not available

*Solution, Tone and Issue Understanding are only available if you opt in to OpenAI processing.

Powered by Zendesk