Creating AI prompt-based rating categories (EAP)

Add-on

Quality Assurance (QA) or Workforce Engagement Management (WEM)

Verified AI summary ◀▼

With AI prompt-based rating categories, you can create custom prompts to automatically score customer conversations and detect risks. Use the AI prompt library or craft your own prompts to focus on specific behaviors. Ensure both AutoQA settings are enabled, and manage up to ten active categories. Add categories to scorecards to start evaluating conversations based on your criteria.

Location: Zendesk QA > Settings > AI

Prompt-based categories are currently in an early access program (EAP). You can sign up for the EAP here.

Zendesk QA prompt-based AI insights leverage the latest AI models, allowing you to customize AI-powered prompts for quality autoscoring and risk detection.

In addition to the system categories, manual categories, and exact text-match categories, admins and account managers can create AI prompt-based rating categories.

All rating categories allow you to effectively assess customer conversations. However, by using AI prompt-based categories, you can ask targeted questions about conversations using natural language and automatically score them based on your criteria.

This article describes how to use AI prompt-based rating categories from our ready-to-use AI prompt library and how to create your own custom AI prompt-based categories.

Tip: To ensure proper functionality, both the Automatic scoring with AutoQA and the LLM-based AutoQA account settings must be turned on.

Admins and account managers can have up to ten active AI prompt-based rating categories and AI prompt-based spotlight insights per account. When you reach this limit, you must either mark one as inactive or delete it before you can create or activate more.

To create a custom AI prompt-based rating category

In Quality Assurance, click your profile icon in the top-right corner.
Click Settings ().
In the sidebar () under Account, click AI.
Click Create AI insight.
Select the Prompt-based AI insight type.
Enter an AI insight name.
Under AI insight description click Prompt library to select a prompt from our ready-to-use AI prompt library. Click Use prompt.
You can also customize any prompt to fit your specific needs or create a new prompt from scratch:
1. In What do you want to detect?, clearly describe what this insight should identify and accomplish.
  Specify whether the focus is on the behavior of agents or customers, and use objective, measurable language. You can enter up to 2,000 characters.
  
  Be sure to follow our best practices for writing prompts for AI insights to ensure your insights are accurate and actionable.
2. In Detection conditions, enter an additional scoring prompt, up to 128 characters, that describes what the agent must do to receive a thumbs-up () rating in a category. Be sure to follow our scoring prompt-based AI insights best practices to ensure accurate evaluations.
3. In Absence conditions, enter an additional scoring prompt, up to 128 characters, that describes things that should cause the agent to receive a thumbs-down () negative rating score for this category. For example, if the agent did not offer a demo, and the text does not include “let’s schedule a demo”, the agent receives a negative rating score for this category.
  Note: If AI can't find a clear answer based on the prompts, the category is automatically rated as N/A.
Click Validate to receive feedback and tips on how to improve your prompt.
Under Suggested usage, select Use as a category on a scorecard.
Click Create category.
After creating it, you must add the category to your scorecards to start using it.

Tip: To confirm that your prompt is working correctly and finding your prompted criteria, filter the conversations based on the prompted category results.