Making sense of unexpected intelligent triage predictions

You must have the Advanced AI add-on to use the features described in this article.

In certain situations, a ticket’s intent or sentiment prediction from intelligent triage is not correct. This article defines those situations and offers suggestions to prevent them from impacting your workflows.

This article contains the following topics

• The ticket's language isn't supported by intent or sentiment
• An integration adds non-human content to the first comment

Related articles:

• Automatically detecting customer intent, language, and sentiment
• Analyzing intelligent triage results and taking action

Note: When creating triggers in Admin Center, intelligent triage prediction values are available only in English. However, intelligent triage is capable of evaluating content in the languages listed here.

The ticket's language isn't supported by intent or sentiment

Issue

While intelligent triage can detect a vast number of languages, the intent and sentiment predictions are supported only in the subset of languages defined in Viewing all possible intent, language, and sentiment values. If a ticket arrives in a language unsupported by the intent and sentiment detection, the intent often appears as Spam and the sentiment often appears as a value that doesn’t make sense for that language.

Solution

If you notice a pattern in tickets being categorized as spam or sentiment mismatches due to an unsupported language, you can make adjustments to your business rules to accommodate this behavior. Before your intelligent triage–based business rules run on your tickets, you can implement a trigger to remove the sentiment and intent values. These tickets would then fall to your other business rules or routing rules.

Below is an example trigger that removes those values and adds a tag to any affected tickets. That tag can then be used to track any tickets that have been overridden in this manner. Place this new trigger before any other triggers that use sentiment or intent as conditions to prevent unexpected routing scenarios.

To create a trigger that removes sentiment and intent

1. In Admin Center, click Objects and rules in the sidebar, then select Business rules > Triggers.
2. Click Add trigger.
3. In Trigger name, enter a name for the trigger (such as Exempt from intent and triage rules).
4. (Optional) Enter a Description for your trigger.
5. (Optional) Select a Category for your trigger.
6. In the Conditions pane, under Meet ALL of the following conditions, add the following conditions:
   • Status | Is | New
   • Tags | Contains none of the following | triage_override, language__de, language__en, language__es, language__fr, language__ja, language__pt, language__ru, language__nl, language__it, language__ko, language__tr, language__zh-cn, language__sv, language__no, language__da, language__pl, language__ar, language__ro, language__cs
   • Agent replies | Less than | 1
7. In the Conditions pane, under Meet ANY of the following conditions, add the following conditions:
   • Intent | Present
   • Sentiment | Present
8. In the Actions pane, add the following actions:
   • Intent | -
   • Sentiment | -
   • (Optional) Add tags | triage_override
9. Click Create.

An integration adds non-human content to the first comment

Issue

Intelligent triage is trained on human-submitted information, but some integrations (often via API) add other content to the message submitted. A few examples could be extra metadata, chat transcript formatting, or a system-generated message from a third-party telephony integration.

This non-human content can lead to unexpected behavior. There are a couple ways to handle this. If you consistently see spam intents or incorrect sentiments, these could be the solutions for you.

Diagnosing the issue

First, determine whether the API channel (or other applicable channel) is receiving incorrectly predicted intent or triage for all tickets or only for a subset of tickets.

To see a list of API-generated tickets for evaluation, do one of the following:

• Create a view for tickets that were created via web service (API) and add some columns for intent and sentiment.
• Use via:api in Search (See Zendesk Support search reference).

Solution

If all tickets in a particular channel are receiving incorrectly predicted intent or triage

If all of your API tickets come from a single source like a telephony integration, or if there’s a common issue for all API-generated tickets, you might want to disable intent and sentiment detection only for that channel. To learn how to configure the channels for intelligent triage, see Turning on and configuring intelligent triage.

If only some of the tickets via a particular channel are enriched incorrectly

Because the API channel can be used for many purposes, it’s possible that tickets are created from that channel from a “good” source (like a custom web form) as well as a “bad” source (like an external chat system that adds system-generated content).

In this case, look for some predictable attribute that would allow you to exclude or include a subset of tickets. Here are some things to look for:

• All tickets get the same tag
• The integration sets a ticket field
• There is some consistent text in the ticket subject (see How do I create a trigger based on a ticket subject?)

After a common trait has been identified, you can use that to create a trigger that removes the sentiment and intent values (similar to the example above) when the ticket is created. Placing that trigger before all other triggers based on intent and sentiment will prevent the ticket from being misrouted.