With the latest upgrade to the Analytics page we have broken it down into 3 key sections:
Each section is decided to give you insights on your analytics data so you can make data-driven decisions on where the bot is performing well, which Intents are bringing you the biggest return on your time investment of setting up that dialogue flow, and which can be improved and are bringing your more escalations.
Our analytics are currently covering data from the last 180 days. We will soon be able to display data from the past 12 months. Reach out to us should you need analytics for a longer period of time.
Before we cover the sections, let's cover the filter criteria that can be defined on the top navigation.
Filters
| Filter | Description |
| Timeframe | This is a date-picker to define the range of dates you would like the analytics and widgets to reflect. |
| Language | A drop-down to filter options based on one or select languages. |
| Conversation Status | This is based on the auto-defined conversation statuses, such as bot handled or escalated. |
| Label | Based on the labels you define, you can filter to show conversations related to them, so this could be Intent-specific labels or related to your A/B testing to compare results. |
| Intent | To have a filtered view, you can drill into one or select Intents to see how they are performing on all metrics. |
| Advanced | Remove test data from the Analytics to have an exact understanding of visitor data, especially useful during any period where you may be testing new flows with the Test Bot functionality. |
Performance Overview
This section will be the first thing you see and there's a reason for that - it will give you quick insights on key volume metrics focused on understanding, impact, and areas to monitor and potentially troubleshoot. The focus will highlight the differences over time to identify trends either positively based on changes you have made or based on seasonal flows, or negatively which means some work can be done to change the virtual agent's performance, same as how you would have a metric dashboard to understand your human agents' performance, this is the same for the virtual agent.
| Section | Description | Visual |
| Visitor Volumes |
Understand the number of messages and conversations the bot handled with a focus on understood rate and handling meaningful intents vs just doing a greeting and dropping off. Conversations Fully Understood: Where no message in the communication thread was under the confidence threshold to be not understood.
|
|
| Fully Managed Conversations |
Understand the impact the virtual agent has on the success of the support function as it highlights all metrics focused on auto-detected conversation statuses, such as deflection and bot handled rate, as well as the automation rates based on the conversation, flows you have designed and where it indeed reached a positively impactful moment. Use the drop-down to change between bot statuses and resolution states |
 |
| Areas to Investigate |
Dive into any issues that have occurred such as failed errors or escalations as well as monitor if the escalation rate is within the normal range or if there is a noticeable change that requires investigation. Use the drop-down to change between escalated conversations and conversations with unsuccessful activity. |
 |
All widgets have drop-down options for you to filter different key information that impacts that respective area. We like to think of the metrics as almost have an equivalent to what you may have within your human customer support team.
- Visitor volumes section is similar to tickets per hour/day/month
- Fully managed conversations is similar to first contact resolution (FCR)
- Areas to investigate is similar to escalations to other teams or where an issue happened that needs a person focused on quality assurance to ensure that everything was handled correctly and where things can be improved.
Bot Performance Analysis
This is where your visualization of data takes place - On the left, you can filter for volume-related metrics, and then on the right you have the option of time, language, resolution, and Intent focused metrics.
These are based on the filters you have defined at the top. Then use the drop-down to change between the points that are of most interest to you
Intent Performance
At the bottom of the Analytics page, you'll see a list of your intents and some of their key performance metrics. To sort by any one of them, click either Conversations, First Intent, Automation Rate, Bot Handled Rate, Escalations, Failed Escalation, or Technical Errors to arrange these ascending or descending, or Intent to sort alphabetically by name.
The conversations icon will direct you to the Conversation Logs for those Intent conversations filtered for you.
This table can have its logic adjusted to either work on Intent Prediction Based or Intent Reply Based.
The difference is based on how visitors navigate your replies. If you use a lot of buttons to divert people to Intent replies, in which case, when looking at Intent Predicted Based you will see N/A as an Intent contributing to conversations. This is because replies are triggered without the AI and therefore there was no Intent predicted.
By toggling to Intent Reply Based, you won't see this N/A, but instead which replies were triggered.
One thing to note is that data for Intent Reply Based is that the data is only available from 2023.
We cannot recommend which is one you should use, as it really depends on your reply designs. If you do have mostly button and routing chat designs, then Intent Reply Based may be better.
However, this N/A does give you insights into how your users are navigating the chat.
This toggle is only available for Chat Automation