Custom resolution states unlock a greater analytic potential to have granular and actionable insights to track the performance of the AI agent more accurately and in a transparent manner.
You may be wondering how many conversations end in an escalation. How many didn't receive a resolution and just dropped off? Was the customer informed of the self-service guidance or instructions but didn’t finish the flow?
By having the answers to these questions you are empowered to make changes to the conversation flow to provide the best experience, as you will have better visibility on which conversations are problematic or include insights.
In this article:
- Resolutions
- How to Set Resolutions
- Recommendations
- Resolutions in Analytics
- Benefits & Differences to Default Analytics
- Resolutions in Conversations Logs
- FAQ
Resolutions
By default, we have 5 custom resolution states and 1 default setting, which feed into different analytics.
| Custom resolution state | Definition |
| Undefined | AI agent conversations that ended without a custom resolution state, often due to drop-off or use cases missing a custom resolution state. |
| Informed | AI agent conversations where the AI agent provided guidance to the customer. For more information, see Using the informed state. |
| Resolved | AI agent conversations that ended with a meaningful resolution and no further questions from the customer. |
| Unresolved |
AI agent conversations where the issue was not answered or resolved. |
| Escalated to agent |
AI agent conversations that were escalated to a human agent. This state is automatically set when the conversation flow reaches an Escalation block in a dialogue with the “Forward to an agent” or “Custom escalation” option set. |
| Escalated via email |
AI agent conversations that were converted into an email ticket. This state is automatically set when the conversation flow reaches an Escalation block in a dialogue with the “Send an email” option set. |
How to Set Resolutions
From the dialogue builder, you can add the applicable States to any message block to highlight whether a user was informed, or if the query was resolved/unresolved or escalated. The final resolution of a conversation can be reviewed in the conversation logs afterward.
- Navigate to an Intent or Template Reply
- Select the block you would like to apply the state to
- On the details tab, scroll to Resolutions and select the appropriate state from the drop-down
- Press Save Draft or Publish to save your changes.
An important thing to note is that each block can only have one custom resolution state associated with it as when a user passes that block that custom resolution state will be applied. The resolution of a conversation will stay the same after passing the selected block until a new resolution is set. Therefore there is no need to cover all the following blocks with the same resolution type.
Recommendations
Structuring your Resolutions
Use the conversation funnel model from your onboarding as inspiration as you will likely be following a linear path of informing the user of information, checking as to whether this resolves their issue and can apply the resolved / not resolved state based on the response, then the escalation status based on the escalation type from the non-resolved or process-based escalation.
Using the Informed State
The informed state should be used when a message is sent that gives the customer information that enables them to handle the situation themselves. Alternatively, where the AI agent is providing the customer with information that they must then come back to that furthers the conversation, for example, please send me a picture of the defect or the invoice number. This way if they drop off the chat, perhaps to go retrieve that information and come back later, then it still counts as the AI agent giving them key information that helps the process along for next time.
When not to use the Informed State
Don't use informed when changing the language as there hasn't been a "real" Intent handled.
Depending on the solution, you might not want to use the informed state to say that there is no solution available unless it would be the same as what a human agent would do.
How to use the Unresolved State
Use this as part of the resolution check or feedback flows to indicate whether the user is satisfied or received an answer that didn't help. This then helps with identifying those who drop off before escalation or where conversations can be improved. This makes reviewing conversations within the conversation logs faster.
We would recommend not using this as the first custom resolution state as then this will include everyone who drops off the chat early making it very manual to differentiate truly unresolved/dissatisfied users and those who just drop off.
Resolutions in Analytics
With the introduction of custom resolution states, there is a new metric that will be available in the Analytics dashboard as well as a new visualization widget to give you a visual overview of information at a glance.
Custom Resolution Rate
The custom resolution rate is the percentage of chats that end in Informed or Resolved State divided by the number of chats.
This in the future will replace AI agent-handled rate as a more impactful metric, however, it does require custom resolution states to be used across all dialogues to provide an accurate result as all conversations that are undefined will be counted as conversations that weren't automated.
Visualizations
There is a new visualization widget - conversation resolutions - to show the distribution of which states are applied to conversations over the time frame.
Intents Table
Within the Intents table, there is a new column to highlight the custom resolution rate per intent.
To navigate quickly to the Conversation Logs filtered to that Intent to review the conversations that are associated with it, you can click on the conversations Icon next to the Intent name
Benefits & Differences to Default Analytics
The custom resolution rate is more specific and transparent as it's based on what you define as what is automated, rather than just processed based on what is identified and not escalated - like AI agent-handled rate.
For a conversation to be considered AI agent-handled, it must meet the three criteria below:
- Contains at least one meaningful intent
- Has no escalation attempt whatsoever, meaning if an escalation fails in a conversation, it will not be considered AI agent-handled
- The last customer message was understood, meaning the default reply is not used as the last reply
Whereas the custom resolution rate is based on the last recognized state in the conversation.
Revert to the AI agent-handled rate widget
To change the visualization back to the AI agent-handled rate rather than custom resolution rate, you can do this by clicking the down arrow next to the metric and then selecting the AI agent-handled rate from the drop-down.
Resolutions in Conversations Logs
Within the Conversation Logs, you have the ability to review the different states associated with a conversation by using the toggle at the bottom of the table to switch between the original states of AI agent-handled or escalated. Using the status filter toggle at the bottom of the conversations table.
To review conversations associated with a particular state you have the ability to do this by clicking Add Filter, navigating to Resolutions, and then selecting the specific state(s) you would like to filter by.
FAQ
Can I create my own status?
In time - we would like feedback to know which states you would like added if these are common and can be added for everyone and if we find everything is super specific per group and necessary we will introduce custom resolutions for people to set.
If there is a particular state you would like please let your CSM know.
Can I have a setting that removes the last set state?
No, the last set state always remains so we recommend that you include states in all your conversation so that it gets a new state based on the flow it is going down.
Can I see all the states a conversation goes through?
Currently, you cannot see a progress bar of a conversations states, only the last most state is displayed in the analytics and conversation logs, however, we plan to add this in the future to show how conversations flow - which will be especially impactful for longer or multi-intent conversations.
Can I use custom resolution states in conditional logic?
Yes, there is a system parameter called lastResolution which enables you to separate based on the values: Undefined , Informed , Resolved , Not Resolved , Escalated To Agent , Escalated via Email