Improving AutoQA accuracy

Autoscoringsupports your manual review efforts by automatically evaluating and scoring customer interactions for 100% of your ticket volume based on predefinedcategories. This ensures consistent quality assessments, reduces subjectivity, and saves reviewers time, allowing you to focus on categories that need more attention.

What's my plan?

Add-on

Quality Assurance (QA) or Workforce Engagement Management (WEM)

Summary: ◀▼

AutoQA helps you score conversations against predefined categories, compare AI ratings with human reviews, and spot where quality checks need more attention. You can submit a manual review to improve accuracy tracking, see how well categories align with your team’s ratings, and use accuracy scores as a confidence check for agent performance and quality assurance.

Location: Zendesk QA > Conversations

Autoscoring supports your manual review efforts by automatically evaluating and scoring customer interactions for 100% of your ticket volume based on predefined categories. This ensures consistent quality assessments, reduces subjectivity, and saves reviewers time, allowing you to focus on categories that need more attention.

Autoscores can vary for different agents. You can’t change an autoscore however, as a reviewer, if you disagree with the autoscore, you can submit a different rating for the conversation.

Although AutoQA doesn’t learn directly from your feedback, you can monitor system performance by checking the accuracy score on the AutoQA dashboard.

The accuracy score measures the consistency and level of agreement between ratings generated by AutoQA and those provided by human reviewers for the same conversation.

AutoQA accuracy helps you:

• Understand which AutoQA categories align best with human reviews and how reliable AutoQA is.

  For example, if Comprehension has an accuracy score of 90%, that means human reviewers agreed with AutoQA's rating on 9 out of 10 conversations they both reviewed.

• Use the AI’s scoring as a confidence indicator when evaluating agent performance.
• Reduce time spent manually checking every conversation to identify categories that may need more review attention.

Automated reviews are tracked using the Auto Quality Score (AQS), while manually submitted grades contribute to the agents’ Internal Quality Score (IQS).

AQS counts toward the IQS of a conversation only after a reviewer manually submits the review.

Note: AQS is only available for agents. AutoQA evaluates bots but does not offer a combined AQS for agents and bots in the dashboard. This is to avoid penalizing bot builders in potential mismatches for human categories.

To manually submit a review on an AutoQA category to improve accuracy

1. In Quality assurance, click Conversations in the sidebar.
2. Select the conversation you want to review.
   Tip: You can use custom filters to find conversations using AutoQA categories to review.
3. Click Review.

   

4. Select a scorecard.

   Make sure the AutoQA categories you want to evaluate are included on your scorecard. This ensures that when you submit your review, your rating can be compared against AutoQA’s rating for the same category.

5. Select your new rating using the AutoQA category ratings that are already filled in for reference.
6. Click Submit.