You must have the Advanced AI add-on to use this feature.

In this Explore recipe, you’ll learn how to create two reports that help you analyze the effectiveness of auto assist, which is part of agent copilot. 

This article contains the following topics:

  • What you’ll need
  • Reporting on tickets where auto assist was used
  • Reporting on which auto assist procedure was used on a ticket

Related articles:

  • Turning on and configuring auto assist
  • Using auto assist to help agents solve tickets
  • Workflow recipe: Canceling and refunding a Shopify order with auto assist

What you’ll need

Skill level: Beginner

Time required: 10 minutes

  • Zendesk Explore Professional or Enterprise
  • Editor or Admin permissions, see this article: Giving users access to Explore
  • Auto assist, see this article: Turning on and configuring auto assist
  • Messaging or email ticket data

Reporting on tickets where auto assist was used

This report gives you quick insight into the tickets where auto assist was used. You can add this report to a dashboard and filter by attributes such as the agents for whom you have enabled auto assist, their groups, the specific channels where you’re using auto assist to serve customers, and more.

Create custom metrics to track auto assist and non-auto assist tickets

As part of reporting on auto assist, you’ll want to be able to slice auto assist and non-auto assist tickets separately. To do so, create a standard calculated metric for each set of tickets:

  • Number of tickets where auto assist was used:
    • Name: Auto assist tickets
    • Formula: IF (INCLUDES_ANY([Ticket tags], "agent_copilot_enabled")) THEN [Ticket ID] ENDIF
  • Number of tickets where auto assist was not used:
    • Name: Non-auto assist tickets
    • Formula: IF (NOT INCLUDES_ANY([Ticket tags], "agent_copilot_enabled")) THEN [Ticket ID] ENDIF
Tip: If you use Explore in another language than English, temporarily set the Language field of your Support profile to English to create that formula, and save it.

You can add these standard calculated metrics to any report created in the same dataset as the metrics. For a simple example report, see the instructions below.

Create the report

  1. In Explore, click the Reports () icon.
  2. In the Reports library, click New report.
  3. On the Select a dataset page, select one of the following datasets depending on which type of tickets you want to report on:
    • Chat - Messaging tickets
    • Support - Tickets
  4. Click Start report. The report builder opens.
  5. In the Metrics panel, click Add and select the following calculated metrics created above:
    • Auto assist tickets
    • Non-auto assist tickets
      Tip: If you don’t see these metrics, make sure you created them in the correct dataset.
  6. Click the Visualization type menu () and select Pie.

Your report should look similar to the one below.

Reporting on which auto assist procedure was used on a ticket

With a little setup, you can create reports that show the specific procedures that auto assist used to help agents solve tickets. This gives you the tools you need to dig deeper into the efficiency gains provided by auto assist.

Setup recommendations for reporting on procedures

At the very least, we recommend adding procedure-specific tags during auto assist setup. If you have a little more time, we also recommend creating a procedure-specific custom field that improves the reporting experience (custom field values are more human-readable than tags).

Whichever you choose, tag or custom field, you can use this information to analyze how procedures are performing across common metrics and attributes in the Support - Tickets and Chat - Messaging tickets datasets.

Basic recommendation: Add procedure-specific tags during auto assist setup

During auto assist setup, if you configured ticket tags to identify the specific procedure that auto assist should follow for a given issue (such as order cancellations), you can report on these tags to see which procedure auto assist used for a given ticket. For details on configuring these ticket tags, see this setup step for answers or this setup step for triggers.

This information, in combination with other Explore metrics and attributes, can help you analyze which issues auto assist is most successful in helping agents resolve, as well as troubleshooting issues where auto assist is following procedures you didn’t expect.

When reporting with tags in Explore, there are some considerations to keep in mind to ensure your reports are as useful as possible. If you haven’t worked with them in Explore before, see Reporting with tags first.

Advanced recommendation: Create a procedure-specific custom ticket field

You can also create and configure a custom drop-down ticket field to indicate which auto assist procedure was used on a ticket.

For example, the image below shows a custom drop-down field with several field values tied to specific tags. Each pair of values and tags indicates which procedure auto assist will follow for a customer request.

So in this example, if a customer requested help buying a car, the car_buying tag would be added to the ticket, either via the conversation bot Transfer step or via a ticket trigger setup that an admin had previously performed. This tag would then in turn set the value of the drop-down field to Car Buying Inquiry.

When you specify the tags in your custom drop-down field, make sure they match the procedure-specific tags you used during auto assist setup. Also, remember to add any new procedures to the ticket field as you create them over time.

This drop-down field doesn’t have to be added to your ticket forms unless you want your agents to be able to see its value. Hidden dropdown fields will still sync to Explore.

When you report on auto assist in Explore, this field makes it easier to visualize, break down, and filter the procedures used on your auto assist tickets.

As with tags, when reporting with custom fields in Explore, there are some considerations to keep in mind to ensure your reports are as useful as possible. If you haven’t worked with them in Explore before, see Reporting with custom fields.

Create the report

  1. In Explore, click the Reports () icon.
  2. In the Reports library, click New report.
  3. On the Select a dataset page, select one of the following datasets depending on which type of tickets you want to report on:
    • Chat - Messaging tickets
    • Support - Tickets
      Note: For tickets where auto assist mode is used via messaging, any replies accepted and sent via auto assist do not appear in the Messaging tickets dataset, as bot replies are excluded from this dataset. This means, for those tickets, metrics such as number of agent replies and first reply time will not align with when the autopilot was used to respond to the end user. However, you can still use this dataset to report on non-agent-centric metrics, such as resolution time.
  4. Click Start report. The report builder opens.
  5. In the Metrics panel, click Add and select the Auto assist tickets calculated metric created above.
    Tip: If you don’t see this metric, make sure you created it in the correct dataset.
  6. In the Columns panel, click add and select the Auto assist procedure custom field created above.
  7. Click the Visualization type menu () and select Pie.

Your report should look similar to the one below. Note that the report includes tickets with an Auto assist procedure value of NULL (in other words, tickets where auto assist was used but no specific procedure value was captured).

Tip: Additionally, consider cloning the Efficiency tab of the prebuilt Zendesk Messaging dashboard or Zendesk Support dashboard and adding the Auto assist procedure custom field as a filter to compare the efficiency of auto assist vs. non-auto assist tickets.
Powered by Zendesk