If you find yourself asking the question "Why is this message being recognized as intent A and not B?" then you're in the right place. Particularly if your bot has been live for a while.
The reason why a bot is confused between two Intents is usually that the trained expressions under this pair of intents are overlapping. This is when Confusion Matrix comes in.
Confusion Matrix is a great tool to help you understand if the AI model of your bot is performing well in terms of intent recognization, and solve the confusion by moving trained expressions between an Intent pair. It is there to show possible inconsistencies in manually trained expressions.
This article covers the following topics:
Read more about it in Confusion Matrix Explained.
Where is Confusion Matrix?
From the left side menu, go to Training Center > Confusion Matrix. Once there, you will see two tabs on top: List of Issues and Confusion Matrix. We suggest starting from List of Issues.
How to use Confusion Matrix?
In List of issues, sort by Priority and start from the high and medium ones. Click Solve Issue > Manage Expressions to get to the expressions management view (see below).
The purpose here is to make sure the expressions under the two intents are not overlapped by moving expressions between intents. They can be moved in bulk or one by one.
With our AI superpower, we have made this easier for you by highlighting the expressions that need to be managed. Here are the options of actionable needed from you after going through the highlighted messages carefully:
- Fix 1: Untrain them or move them to the other intent
- Fix 2: Create a new intent with those expressions
- Fix 3: Merge the intents if you realize they should be the same
- Fix 4: Help your AI model learn by training more expressions to those two confused intents. Only do this if you are absolutely certain the two intents are very different and should be separated
Once you're done, click Mark as Solved to keep track of your progress.
Checkmarks are gone once a new model is trained.
Highlighted expressions
Highlighted expressions are the expressions that confuse your AI model. Expressions are highlighted when:
- The highlighted expressions were trained to incorrectly
- The highlighted expressions were trained with the correct intent but their words are very similar to another intent’s expressions.
Search for a specific intent
If you want to know if and how a specific intent causes confusion with other intents, you can search for it in the search bar in the top right corner.
Advanced filters
You define the confusion level here more precisely. In addition, if there is more than one intent that you would like to view, they can all be selected here.
All the features can be accessed via the Confusion Matrix tab as well by clicking on one of the cells
List of Issues vs. Confusion Matrix
Both List of Issues and Confusion Matrix offer a clear overview of which intents are "confused", meaning potentially having overlapping expressions.
The only difference is that List of Issues gives you a clear idea of which ones should be tackled first by automatically sorting the issues based on priority High, Medium, and Low.
Whereas in Confusion Matrix put an emphasis on how well the model is. A good model, like the example below, should have a dark line running across the table diagonally. You can read more about it in Confusion Matrix Explained.