Evaluating the reviews completed by your reviewers in Zendesk QA is an important part of a consistent and accurate QA process. By performing spot checks, you can monitor and identify any discrepancies in your reviewers' work. You can think of this as “grading the grader.”

What's my plan?
Add-on Quality Assurance (QA) or Workforce Engagement Management (WEM)

Verified AI summary ◀▼

Evaluate your reviewers' performance by creating a unique workspace and scorecard to monitor discrepancies in their work. Creating a separate workspace ensures your reviewers can't see the reviews left on their peers. This approach also provides traceable Internal Quality Scores and insights into areas needing improvement, without immediate calibration sessions. Filter dashboards to visualize and analyze reviewer performance effectively.

Location: Zendesk QA > Settings > Scorecards

Evaluating the reviews completed by your reviewers in Zendesk QA is an important part of a consistent and accurate QA process. By performing spot checks, you can monitor and identify any discrepancies in your reviewers' work. You can think of this as “grading the grader.”

This article explains how to create a unique workspace and scorecard to limit review visibility among peers and generate a reviewer-specific Internal Quality Score (IQS), which is traceable in workspace dashboards, without the need for immediate calibration sessions. It also covers how to use the browser extension to review your reviewers.

This article contains the following topics:

  • Creating a grade the grader workspace and scorecard
  • Finding reviewers' conversations to review
  • Reviewing your reviewers using the browser extension
  • Monitoring your reviewers' performance

Related articles

  • Setting up calibration in Zendesk QA

Creating a grade the grader workspace and scorecard

To monitor and identify discrepancies in your reviewers' work, start by creating a unique workspace and a custom scorecard to review their performance.

To create a workspace to evaluate reviewers

  1. Create a unique workspace in your account. For example, you can name it “Grade the grader”.
  2. Add reviewers as agents to your "Grade the grader" workspace. This limits their visibility to their own reviews only.
To create a scorecard to evaluate reviewers
  1. In Quality assurance, click your profile icon in the top-right corner.
  2. Click Settings ().
  3. In the sidebar under Account, click Scorecards.
  4. Click Create and select Scorecard.
  5. Enter a Name for the scorecard, for example, you can name it “Spot-check”.
  6. Under Workspaces, select the workspace for reviewing reviewers that you created earlier.
  7. Under Workspaces with AutoQA enabled make sure No workspaces is selected.
  8. Click Add a group section.
  9. Add three binary categories to the Spot-check group. For example:
    • Comment quality
    • Review accuracy
    • Review time
  10. Select Add root causes to explain rating and add Root causes for Negative ratings () for each category in the scorecard used by the reviewer to perform reviews:
    • [Category name]: Score too high
    • [Category name]: Score too low

    Where [Category name] is replaced with the name of the respective categories in the scorecard used by the reviewer to perform reviews. For example: Tone Score too high; Tone Score too low. This lets you track in dashboards where the reviewers are diverging.

  11. Configure any other settings for each category within the scorecard as needed.
  12. Click Apply.
  13. Click Publish.

Finding reviewers' conversations to review

To find conversations to review, select the appropriate workspace and filter your conversation list to show those reviewed by your chosen reviewers in the past week, for example.

To find your reviewers' conversations using filters
  1. In Quality assurance, click Conversations in the sidebar.
  2. Filter your conversation list to show conversations your selected reviewers reviewed in the past week, for example.
  3. Open the reviewer's conversation you want to review.

Reviewing your reviewers using the browser extension

Using the browser extension and the new scorecard, you can now leave a review for the reviewer. The extension lets you keep the scorecard open alongside completed reviews for easy reference.

To review your reviewers

  1. Install the Zendesk QA browser extension.
  2. Open the browser extension and make sure it’s set to your new separate "Grade the grader" reviewers workspace.
  3. Under Review this conversation, select the reviewer you’re reviewing as the Assignee (not the agent), and enter details to identify this review.
  4. Select your “Spot-check” scorecard.
  5. Keep the conversation you want to review open in the browser so you can see its details, then complete the review using the extension.

  6. Click Submit.

Monitoring your reviewers' performance

You can now filter your dashboards for your ‘Grade the grader’ workspace scores and your “Spot-check” scorecard.

To monitor your reviewers using dashboards

  1. Filter your Reviews dashboard specifically for this scorecard to visualize how your reviewers are performing.

  2. Use the Categories dashboard to dive deeper into where the reviewer is misaligned (for example, scoring too high or low for a specific category).
Powered by Zendesk