Evaluating the performance of reviewers

What's my plan?

Add-on

Quality Assurance (QA) or Workforce Engagement Management (WEM)

Location: Zendesk QA > Settings > Scorecards

Evaluating the reviews completed by your reviewers in Zendesk QA is an important part of a consistent and accurate QA process. By performing spot checks, you can monitor and identify any discrepancies in your reviewers' work. You can think of this as “grading the grader.”

This article describes how to create and use a unique scorecard to review your reviewers. This method gives your reviewers their own Internal Quality Score (IQS) that is traceable in the dashboard, without the need for immediate calibration sessions.

This article contains the following topics:

• Creating a grade the grader scorecard
• Reviewing your reviewers
• Monitoring your reviewers performance

Related articles

• Creating scorecards

Creating a grade the grader scorecard

To create a scorecard to evaluate reviewers

1. Create a separate workspace in your account. For example, you can name it “Grade the grader”. This is where the reviews for the reviewers will go.
2. Create a unique scorecard for reviewing your reviewers. For example, you can name it “Spot-check”.
3. Under Workspaces, select the workspace for reviewing reviewers that you created earlier.
4. Under Workspaces with AutoQA enabled make sure No workspaces is selected.
5. Click Add a group section.
6. Add three binary categories to the Spot-check group. For example:
   • Comment quality
   • Review accuracy
   • Review time
   
7. Select Add root causes to explain rating and add Root causes for Negative ratings () for each category in the scorecard used by the reviewer to perform reviews:
   • [Category name]: Score too high
   • [Category name]: Score too low

   Where [Category name] is replaced with the name of the respective categories in the scorecard used by the reviewer to perform reviews. For example: Tone Score too high; Tone Score too low.

8. Configure any other settings for each category within the scorecard as needed.
9. Click Apply.
10. Click Publish.

Reviewing your reviewers

To review your reviewers

1. In Quality Assurance, click Conversations in the sidebar.
2. Filter your conversation list for conversations that have been reviewed in the past week.
3. Click Review.
4. Select the scorecard you have created for reviewing the reviewers. For example, “Spot-check.”
5. Leave a review for the reviewer using the scorecard.

   

Monitoring your reviewers performance

Use dashboards to understand your reviewer's performance.

To monitor your reviewers using dashboards

1. Filter your Reviews dashboard specifically for this scorecard to visualize how your reviewers are performing.

   

2. Use the Categories dashboard to dive deeper into where the reviewer is misaligned (for example, scoring too high or low for a specific category).