About rating categories for scorecards

Add-on

Quality Assurance (QA) or Workforce Engagement Management (WEM)

Verified AI summary ◀▼

Rating categories for scorecards help you assess customer conversations by evaluating key aspects like spelling, comprehension, and tone. You can create custom categories, including manual and exact text-match types, to fit your needs. Use these categories to derive an Internal Quality Score, which offers insights to improve support quality. AutoQA can automatically score new conversations based on these categories.

Location: Zendesk QA > Settings > Scorecards

Rating categories are used to evaluate customer conversations. Well-defined rating categories streamline the review process and ensure consistent quality assessments in Zendesk QA. From the rating categories, you can derive the Internal Quality Score (IQS), which provides valuable insights you can use to assess and improve your customer support.

This article contains the following topics:

Understanding rating categories for scorecards
Creating manual custom rating categories
Creating exact text-match custom rating categories

Understanding rating categories for scorecards

There are countless ways to combine rating categories to build your scorecard rating system. It may take some trial and error to identify the right factors to rate in order to accurately assess and improve your customer support. The following rating categories are among the most commonly used in Zendesk QA and provide a good starting point:

Spelling and grammar: Did the agent use correct spelling, grammar, and punctuation?
Comprehension: Did the agent understand the issue and its root cause?
Solution offered: Did the agent follow the correct process and provide an appropriate solution according to internal guidelines?
Tone of voice: Was the response personal and aligned with the brand’s voice?
Closing: Did the agent anticipate potential issues and go the extra mile to resolve them before closing the ticket?
Documentation: Did the agent accurately document all necessary information internally?

In addition to using the existing Zendesk QA system categories, you can create custom rating categories tailored to your organization’s specific needs. Specifically, you can:

Before creating categories, it’s important to understand the following information about categories and how they’re used in Zendesk QA:

Enabling automatic scoring with AutoQA allows Zendesk QA to automatically evaluate and score agents. Autoscoring categories are indicated by a hologram icon ().
When a new category is created, only new incoming conversations with a Closed or Solved status are automatically analyzed and rated based on the category conditions. It may take some time for the scores to load due to necessary calculations.
To qualify for analysis, conversations must include at least one message from both the customer and the agent, with a minimum word count of 10.
AutoQA doesn’t evaluate existing conversations, spam conversations, or historical data on dashboards. However, existing conversations with a status of Closed or Solved that are updated after the category is added are also automatically analyzed and rated. As a result, you may notice a conversation that has already been analyzed and rated being analyzed and rated again in a new workspace.
If no matches are found, "N/A" is assigned. AutoQA may also return "N/A" for certain categories when there is insufficient information to provide an accurate rating.

Creating manual custom rating categories

In addition to the standard Zendesk Quality Assurance system categories, exact text-match custom rating categorie, and AI prompt based categorie (EAP), you can also create manual custom rating categories tailored to your organization's needs.

Manual categories are scored by human reviewers.

Admins and account managers can create new manual custom rating categories.

To create a manual custom category

In Quality Assurance, click your profile icon in the top-right corner.
Click Settings ().
In the sidebar () under Account, click Scorecards.
Click Create and select Manual category.
Enter a Category name and Description for reviewers (optional).
This is displayed when hovering over a category name while rating a conversation.
Click Create category.
After creating it, you must add the category to your scorecards to start using it.

Creating exact text-match custom rating categories

Exact text-match categories use autoscoring to identify specific keywords or phrases and evaluate agent responses that meet or do not meet your quality standards.

Admins and account managers can create exact text-match custom rating categories.

You can create a maximum of 20 custom AutoQA categories per account. When you reach the limit of 20 active custom rating categories, you must deactivate or delete one before you can create a new category.

To create an exact text-match category

In Quality Assurance, click your profile icon in the top-right corner.
Click Settings ().
In the sidebar () under Account, click Scorecards.
Click Create and select Category.
Select the Exact text-match custom category type.
Enter a Category name and Description for reviewers (optional). This is displayed when hovering over a category name while rating a conversation.
Under If agent message, select the operator (default: contains) and enter one keyword or phrase per row that you want to score automatically if they occur in agent messages. You can:
- Click Add row for each additional keyword or phrase you want to autoscore in this category.
- Click Add condition to specify more complex keywords and phrases. See Exact text-match conditions.
- Enter '/' to add the variable {..}. You can use this dynamic content placeholder {..} to represent any name. For example, “Hello {··}!”
Under Then score and Otherwise score, select how you want to automatically rate the agent. For example, if the text does not include “let’s schedule a demo,” the agent receives a negative () score for this category. Otherwise, you can choose to score them positively () or use "N/A" if unsure.
Click Create category.
After creating it, you must add the category to your scorecards to start using it.

Exact text-match conditions

You can add multiple rows and use the following conditions for complex findings:

Condition name	Condition type	Search query example	Do you get a thumbs up as a result for this conversation message? "Thank you for contacting Acme Corporation, how can I help you today?"
is	is any of	"Thank you for contacting" "how can I help you today?"	No, because we look for entirely equal matches.
is	is all of	"Thank you for contacting Acme Corporation, how can I help you today?" "how can I help you today?"	No, although the first query matches, the second does not.
contains	contains any of	"Thank you for contacting us” “how can I help you today?"	Yes, because the second query is part of the message.
contains	contains all of	"Thank you for contacting” “how can I help you today?"	Yes, because both queries are part of the message.
is not	is not any of	"Thank you for contacting us” “we are unavailable today?"	Yes, because neither of these queries are an exact match of the message.
is not	is not all of	"Our business is closed" "Please contact us tomorrow"	Yes, because none of these queries are an exact match of the message.
does not contain	does not contain any of	"Our business is closed" "how can I help you today?"	No, because the second query is part of the message.
does not contain	does not contain all of	"Thank you for contacting” “how can I help you today?"	No, because both queries are part of the message