Autoscoring supports your manual review efforts by automatically evaluating and scoring customer interactions for 100% of your ticket volume based on predefined categories. This ensures consistent quality assessments, reduces subjectivity, and saves reviewers time, allowing you to focus on categories that need more attention.

What's my plan?
Add-on Quality Assurance (QA) or Workforce Engagement Management (WEM)

Summary: ◀▼

AutoQA helps you score conversations against predefined categories, compare AI ratings with human reviews, and spot where quality checks need more attention. You can submit a manual review to improve accuracy tracking, see how well categories align with your team’s ratings, and use accuracy scores as a confidence check for agent performance and quality assurance.

Location: Zendesk QA > Conversations

Autoscoring supports your manual review efforts by automatically evaluating and scoring customer interactions for 100% of your ticket volume based on predefined categories. This ensures consistent quality assessments, reduces subjectivity, and saves reviewers time, allowing you to focus on categories that need more attention.

The accuracy score measures the consistency and level of agreement between ratings generated by AutoQA and those provided by human reviewers for the same conversation.

This article contains the following topics:
  • Understanding AutoQA accuracy
  • Manually submitting a review on an AutoQA category to improve accuracy
Related articles:
  • Understanding autoscoring categories
  • Understanding the AutoQA dashboard in Zendesk QA

Understanding AutoQA accuracy

Autoscores can vary for different agents. You can’t change an autoscore however, as a reviewer, if you disagree with the autoscore, you can submit a different rating for the conversation.

Although AutoQA doesn’t learn directly from your feedback, you can monitor system performance by checking the accuracy score on the AutoQA dashboard.

The accuracy score measures the consistency and level of agreement between ratings generated by AutoQA and those provided by human reviewers for the same conversation.

Ratings vary based on the scale used. For example:

  • Binary scale:
    • If you agree, the accuracy is 100%.
    • If you disagree, the accuracy is 0%.
  • Five-point scale:
    • A rating of 5 out of 5 has 100% accuracy.
    • A rating of 4 out of 5 has 75% accuracy.
    • A rating of 3 out of 5 has 50% accuracy.
    • A rating of 2 out of 5 has 25% accuracy.
    • A rating of 1 out of 5 has 0% accuracy.

AutoQA accuracy helps you:

  • Understand which AutoQA categories align best with human reviews and how reliable AutoQA is.

    For example, if Comprehension has an accuracy score of 90%, that means human reviewers agreed with AutoQA's rating on 9 out of 10 conversations they both reviewed.

  • Use the AI’s scoring as a confidence indicator when evaluating agent performance.
  • Reduce time spent manually checking every conversation to identify categories that may need more review attention.

Automated reviews are tracked using the Auto Quality Score (AQS), while manually submitted grades contribute to the agents’ Internal Quality Score (IQS).

AQS counts toward the IQS of a conversation only after a reviewer manually submits the review.

Manually submitting a review on an AutoQA category to improve accuracy

To improve AutoQA accuracy, ensure that your scorecard includes the AutoQA categories you want to evaluate before you manually submit a review. This lets the system compare your rating with AutoQA’s rating for the same category.

To manually submit a review on an AutoQA category to improve accuracy

  1. In Quality assurance, click Conversations in the sidebar.
  2. Select the conversation you want to review.
    Tip: You can use custom filters to find conversations using AutoQA categories to review.
  3. Click Review.

  4. Select a scorecard.
  5. Select your new rating using the AutoQA category ratings that are already filled in for reference.
  6. Click Submit.
Powered by Zendesk