What is the Confusion Matrix?
The Confusion Matrix is a visualization of the performance of an AI model, by measure of its intent recognition. In other words, it shows you if your AI model is able to distinctly recognize similar expressions under different intents, or if it is "confused" by it.
The Confusion Matrix offers you access to the same tools that first class AI scientists are using, allowing you to easily monitor your own bot's performance.
Every Tuesday night, a new confusion matrix is automatically generated, meaning that you will never be looking at one that is more than a week old if not manually updated. Generally, this is just fine and not cause for any concern.
If your bot has recently undergone extensive training, or has had major alterations made (changes to expressions or intents), we recommend doing a new training sooner rather than later.
To retrain a Confusion Matrix, first navigate to Training Center > Confusion Matrix, and then click Re-train in the top-right corner.
When should I use it?
You should use it when you notice Bot Answered Rate in Analytics is decreasing, we recommend a baseline of 80%, or when you notice that your bot is regularly categorizing messages under the wrong intent.
How to read the Confusion Matrix
The two dimensions, X and Y axis, represent the actual and predicted intents, respectively.
In the example above, column 1 and row 1 are the actual and predicted intent Affirmative, row 2 and column 2 are the intent Negative, so on and so forth.
The darker a cell is, the more a predicted intent overlap with an actual one.
A good model, like the example above, should have a dark line running diagonally across the table.
However, what you should pay attention to are the colored cells outside of the diagonal as they represent how often messages in an intent are predicted to another. Whenever you see cells with darker colors outside of the diagonal, this is an indicator of a problem in the accuracy of the AI model caused by potential overlaps of expressions between two intents. These can be tackled in a systematic manner in the List of Issues view where all overlapping issues are listed according to priority automatically.
Related Articles: