Understanding metrics can seem intimidating, so we want to break them down into simple terms and give you recommendations as to which you should place the most focus on.

Deflection Bot Handled Automation
Screenshot_2023-01-11_at_11.35.16.png Screenshot_2023-01-11_at_11.35.25.png Screenshot_2023-01-11_at_11.35.05.png

Deflection Rate

Any conversation that does not end in escalation is viewed as deflected. These are the number of conversations that come to the virtual agent and do not make it to a human agent. This does also include conversations that end with the Default Reply.

Bot Handled

The definition of bot handled, on the other hand, is stricter and a conversation must match the following 3 criteria:

  1. Contains at least one meaningful intent
  2. Has no escalation attempt whatsoever - meaning if an escalation fails in a conversation, it will not be considered bot handled
  3. Last visitor message was understood - meaning the default reply is not used as the last reply

However, this can include conversations where visitors drop off before an escalation or were unhappy with the resolution, so requires checking the conversations to see if they ended in a positive outcome.

Automated Resolution Rate

Automated resolutions apply to both chat and ticket AI agents, and are calculated using the following logic:

  • For chat AI agents: An automated resolution is consumed for any conversation that is considered bot-handled and passes the verification performed by our LLM.
  • For ticket AI agents: An automated resolution is consumed for any conversation that is considered answered (because a reply was sent) and passes verification performed by our LLM.

The verification process performed by the LLM evaluates the text of the conversation to ensure that the customer’s request was actually satisfactorily resolved without human-agent intervention. Conversations that don’t pass this verification are not considered automated and do not consume an automated resolution.

For additional information, see About automated resolutions for AI agents (Ultimate).

Automation Rate

This is the most powerful metric, however, it requires some work upfront as this is based on the Resolution States, specifically conversations that end in the state Informed or Resolved. 

This is then going to be specific and provide actionable insights into how the conversations actually ended and whether there are flows that can be improved.

Check out Resolutions States Explained for more details and recommendations on how to use these states.

Powered by Zendesk