What's my plan?
Add-on Quality Assurance (QA) or Workforce Engagement Management (WEM)

Verified AI summary ◀▼

Rating categories for scorecards help you evaluate customer conversations by focusing on areas like spelling, comprehension, and tone. You can create custom categories, including manual and exact text-match types, to suit your needs. AutoQA can automatically score new conversations, providing insights to improve support quality. Remember, conversations must include messages from both parties to qualify for analysis

Location: Zendesk QA > Settings > Scorecards

There are countless ways to build a rating system, but the key is finding what works best for your team. Rating categories allow you to effectively evaluate customer conversations.

Well-defined rating categories help streamline the review process and ensure consistent quality assessments in Zendesk QA.

The Internal Quality Score (IQS) is calculated based on these categories, providing valuable insights to help you assess and improve your customer support.

This article contains the following topics:

  • Understanding rating categories for scorecards
  • Creating manual custom rating categories
  • Creating exact text-match custom rating categories

Related articles

  • Viewing and managing rating categories for scorecards
  • Viewing and managing scorecards

Understanding rating categories for scorecards

Some of the most commonly used rating categories in Zendesk QA include:

  • Spelling and grammar: Did the agent use correct spelling, grammar, and punctuation?
  • Comprehension: Did the agent understand the issue and its root cause?
  • Solution offered: Did the agent follow the correct process and provide an appropriate solution according to internal guidelines?
  • Tone of voice: Was the response personal and aligned with the brand’s voice?
  • Closing: Did the agent anticipate potential issues and go the extra mile to resolve them before closing the ticket?
  • Documentation: Did the agent accurately document all necessary information internally?

In addition to using the existing Zendesk QA system categories, you can create custom rating categories tailored to your organization’s specific needs. Specifically, you can:

  • Create manual custom rating categories
  • Customize select system categories
  • Create exact text-match custom rating categories
  • Create AI prompt-based categories

Before creating categories, it’s important to understand the following information about categories and how they’re used in Zendesk QA:

  • Enabling automatic scoring with AutoQA allows Zendesk QA to automatically evaluate and score agents. Autoscoring categories are indicated by a hologram icon ().
  • When a new category is created, only new incoming conversations with a Closed or Solved status are automatically analyzed and rated based on the category conditions. It may take some time for the scores to load due to necessary calculations.
  • To qualify for analysis, conversations must include at least one message from both the customer and the agent, with a minimum word count of 10.
  • AutoQA doesn’t evaluate existing conversations, spam conversations, or historical data on dashboards. However, existing conversations with a status of Closed or Solved that are updated after the category is added are also automatically analyzed and rated. As a result, you may notice a conversation that has already been analyzed and rated being analyzed and rated again in a new workspace.
  • If no matches are found, "N/A" is assigned. AutoQA may also return "N/A" for certain categories when there is insufficient information to provide an accurate rating.

Creating manual custom rating categories

Besides taking advantage of the existing Zendesk Quality Assurance system categories, exact text-match custom rating categorie, and AI prompt based categorie (EAP), you can also create manual custom rating categories tailored to your organization's needs.

Manual categories are scored by human reviewers.

Admins and account managers can create new manual custom rating categories.

To create a manual custom category

  1. In Quality Assurance, click your profile icon in the top-right corner.
  2. Click the Settings icon ().
  3. In the sidebar () under Account, click Scorecards.
  4. Click Create and select Manual category.

  5. Enter a Category name and Description for reviewers (optional).

    This is displayed when hovering over a category name while rating a conversation.

  6. Click Create category.
  7. After creating it, you must add the category to your scorecards to start using it.

Creating exact text-match custom rating categories

Exact text-match categories use autoscoring to identify specific keywords or phrases and evaluate whether agent responses meet your quality standards.

Admins and account managers can create exact text-match custom rating categories.

You can create up to 20 custom AutoQA categories per account. If you reach this limit, you must mark an existing category as inactive or delete it before creating a new one.

To create a new exact text-match custom category

  1. In Quality Assurance, click your profile icon in the top-right corner.
  2. Click the Settings icon ().
  3. In the sidebar () under Account, click Scorecards.
  4. Click Create and select Category.
  5. Select the Exact text-match custom category type.

  6. Enter a Category name and Description for reviewers (optional). This is displayed when hovering over a category name while rating a conversation.
  7. Enter the keywords and phrases you want autoscoring to identify in agent messages, with one word or phrase per line.

    You can add multiple rows and use conditions for complex findings. See Exact text-match conditions.

    (Optional) Enter '/' to add the variable {..}. You can use this dynamic content placeholder {..} to represent any name. For example, “Hello {··}!”

  8. Select how you want to automatically rate the agent. For example, if the agent did not offer a demo, and the text does not include “let’s schedule a demo,” the agent receives a negative () score for this category. Otherwise, you can choose to score them positively () or use "N/A" if unsure.
  9. Click Create category.
  10. After creating it, you must add the category to your scorecards to start using it..

Exact text-match conditions

You can add multiple rows and use the following conditions for complex findings:

Condition name Condition type Search query example Do you get a thumbs up as a result for this conversation message?

"Thank you for contacting Acme Corporation, how can I help you today?"

is is any of

"Thank you for contacting"

"how can I help you today?"

No, because we look for entirely equal matches.
is all of

"Thank you for contacting Acme Corporation, how can I help you today?"

"how can I help you today?"

No, although the first query matches, the second does not.
contains contains any of

"Thank you for contacting us”

“how can I help you today?"

Yes, because the second query is part of the message.
contains all of

"Thank you for contacting”

“how can I help you today?"

Yes, because both queries are part of the message.
is not is not any of

"Thank you for contacting us”

“we are unavailable today?"

Yes, because neither of these queries are an exact match of the message.
is not all of

"Our business is closed"

"Please contact us tomorrow"

Yes, because none of these queries are an exact match of the message.
does not contain does not contain any of

"Our business is closed"

"how can I help you today?"

No, because the second query is part of the message.
does not contain all of "Thank you for contacting” “how can I help you today?" No, because both queries are part of the message

Powered by Zendesk