Analyzing UltimateGPT performance for advanced AI agents

What's my plan?

Add-on

AI agents - Advanced

UltimateGPT Insights is broken down into 3 key sections:

• Top KPIs
• Knowledge Base Articles Insights
• Knowledge Base Gaps Discovery

Each section is designed to give you insights on your UltimateGPT conversation data so you can make data-driven decisions on where the AI agent is performing well, which topics are bringing you the most inquiries and traffic, as well as the biggest return on your time investment of creating knowledge base articles.

Before we cover the sections, let's showcase the filter criteria that can be defined on the top navigation.

Filters

Filter	Description
Date	Select a timeframe
Language	Filters out the conversations in the selected language

Top KPIs

Conversation Widget

The conversation widget is reporting the chat volumes held by your AI agent. Chat volumes are broken down by UltimateGPT Conversations and Intent-Based Conversations

Widget	Description
UltimateGPT Conversations	Conversations that include at least 1 UltimateGPT response or at least one activation of UltimateGPT node in the designed dialogue
Intent-Based Conversations	Conversations designed in the dialogue builder based on intents that didn't include any responses from UltimateGPT

UltimateGPT Answers Widget

The UltimateGPT Answers widget reports the rate where UltimateGPT node yielded a scenario. The widget highlights two possible scenarios of UltimateGPT, Response Generated and Not understood Scenarios.

Widget	Description
Response Generated	The number of times the AI agents generated a response whether small talk or sourced from the imported knowledge base. You can review the breakdown of the UltimateGPT responses type by hovering over the metric bar. Small Talk: Total number of responses generated by UltimateGPT that the answers are not based on the imported KBs KB Answers: Total number of responses generated by UltimateGPT that are sourced from the imported KBs
Not understood	The metric reviews the number of times the AI agents could not find a relevant context in an imported KB in order to answer a customer inquiry

The breakdown of the metric Response Generated is available from 18th August 2023.

Knowledge Base Articles Insights

The knowledge base articles section provides more insights on the response generated where the answer was sourced from the knowledge base.

There are two views of the knowledge base articles' insights, the tree map view and the table view.

You can switch between the two views from the button on the top right corner

Visual	Description
	The treemap view highlights the top most and least used articles by the AI agent to generate a response. You can view the most and least used articles by enabling the quick filters on the top. The tiles are ordered ascendingly or descendingly based on your filter's selection. Each tile represents an article where: The percentage on the top left corner is the rate where the article was used to generate a response The tile name reflects the article's title More info can be viewed about the article when hovering over it Rate of Usage: the rate at which an article is used to generate a response.  KB Type: the type of the imported knowledge base that the article is part of. KB Name: the name given to the knowledge base when being imported Article URL: external link to the article. Only provided if found
	The table view lists all the imported articles. Each row in the table presents an article where: Article name: is the article title KB Type: the type of the imported knowledge base that the article is part of. KB Name: the name given to the knowledge base when being imported Rate of Usage: the rate at which an article is used to generate a response. Functionalities: Each column can be sorted by clicking on the column header Search for a specific article through the search bar The default max number of articles is 10. You can use the pagination options to suit your preferences

Knowledge Base Gap Discovery 

The Knowledge Base Gap discovery section is designed to help you find out the topics your AI agent could not find a relevant context in the imported KB(s) in order to answer a customer inquiry. It groups the messages under identified clustered topics where you can make data-driven decisions on the best approach to close these knowledge gaps.

On the left side, the bubble chart presents clusters of your customer messages that were not understood by your AI agent. The bubble chart shows the top 10 topics that are frequently asked by your customers.

The not-understood customer messages are clustered every Sunday at 23:59:59 UTC. Every week at that time, your AI agent will cluster relevant not-understood messages under topics represented by a bubble. You can navigate through the clusters by clicking on the bubble where you view the not-understood messages and the number of messages that belong to the selected cluster.

You can improve the AI agent's knowledge for each topic by either:

1. Creating a new intent through the Intent Assistant feature
   This option is ideal if you have no intent to cover the clustered topic. You can simply follow the steps to create a new Intent, generate expressions for this Intent, and design a flow.
2. Add Messages to an Existing Intent
   If you believe you have an Intent already that covers this topic, you can add more expressions to it and train your model
3. Create an article that covers the missing topic
   You can also create an article in your help center that covers the missing topic.