Calibration is the practice of having all your reviewers rate the same batch of conversations and compare their scores and comments. This ensures that your reviewers are aligned in their evaluations, providing consistent feedback to agents regardless of who conducts the review.

Admins, Account Managers, Workspace Managers, Leads, and Workspace Reviewers can perform calibrations. See Understanding roles and permissions in Zendesk QA.

This article contains the following sections:

  • Understanding calibration
  • Setting up calibration
  • Conducting calibration
  • Viewing results
  • Actions not possible in calibration reviews

Understanding calibration

Calibration ensures that your reviewers provide consistent feedback by having them rate the same conversations and compare their evaluations. Regular calibration helps maintain uniformity in reviews, ensuring agents receive similar feedback regardless of the reviewer.

Setting up calibration

Calibration can be done within the existing Workspace, and the calibration scores won't affect the existing Internal Quality Score (IQS).

To set up calibration

  1. Click your profile in the bottom left corner, then select Users, bots and workspaces.
    If needed, click the toggle sidebar icon.
  2. Select the workspace and click Calibration.
  3. Select the visibility of the calibration reviews. You can choose to allow Workspace Reviewers to see only their own reviews or to see their peers' reviews after submitting their calibration review. Workspace Managers or Leads can always see all reviews.

Agents do not see any conversation-related information in Calibration. No notifications are sent to reviewers, so it's up to the Workspace Manager or Lead to inform reviewers about the calibration session and its due date.

Conducting calibration

Calibration can be conducted within the same Workspace where you perform regular reviews.

To conduct calibration

  1. Navigate to the Tasks section.
  2. Click the add (+) icon and select  Create calibration session. You can also view the list of previous Calibration sessions here.

  3. Set a deadline for the Calibration session.
    Deadlines can be extended if needed.

When you find a conversation that you want to add to the Calibration session, use the Add to/remove from calibration session button on the top. 

Once reviews have been submitted, you can mark one review as the baseline for others by clicking the options menu () next to it. This makes it easier to compare scores against it. The baseline is marked with the label 'Baseline review'. 

Workspace managers can add and remove baseline reviews.

 

Viewing results

The IQS of the selected Workspace will remain unaffected by Calibration reviews.

You can view the list of conversations and their associated reviews in the Calibration Dashboard.

No notifications are triggered, and calibration reviews, comments, and scores will not appear on the main dashboard; they are exclusively available in the Calibration Dashboard.

The Baseline review is marked with the label 'Baseline'.

Actions not possible in calibration reviews

Note that some actions are not possible in calibration reviews.

Here is a list of them:

  • Leave a comment without score
  • @ mention a user
  • Add a # hashtag
  • Highlight conversation for later
  • Review draft
  • Notification sent to reviewee
  • Start a dispute
  • See "review seen" status
  • Reply to a review
  • Leave a comment for reviewee
  • Review several reviewees
  • See IQS
  • See review count
Powered by Zendesk