Recently, the Zendesk Community hosted our second Post-Relate Deep Dive on Zendesk AI Agents, "the most autonomous bots in CX." Our expert panel delved into AI agents' core functionalities and showcased an interactive product demo. Attendees were shown best practices, real-world use cases, and received resources to enhance their service experiences with AI agents.
The event highlighted AI agents' potential to improve automated resolution rates, elevate customer interactions, and boost operational efficiencies. The live Q&A session provided personalized insights for participants. In this article, you'll find our event resources, a recording of the event, and a curated list of questions we addressed.
Event Recording
Q&A Summary
What are some of the best ways to build out this functionality if you don't have a bot yet?
Launch an AI agent in minutes > Set guardrails with intents > Add more personalization and control in your interactions > Analyze and optimize your AI agent performance > Automate your emails and web forms. We would recommend allowing a 2-week learning period post-deployment for your AI agent to recognize your customers’ conversation patterns. Opt for “generate a reply” for intents that are best resolved using your knowledge base. Consider utilizing this for FAQs that can be effectively addressed with info from your help center articles. Build conversation/answer flows to personalize your customer experience (eg check order status, authentication, or creating conditional scenarios).
How do you measure the success of AI Agents?
Start slowly and build as you go: Reach up to 20% automation rates just by using generative replies. Choose a couple of popular intents to use GenAI with, and every couple of days, continue to automate more intents. Review and adjust on an ongoing basis: Review AI agent conversation transcripts to identify knowledge or conversation gaps. Optimize your flows to ensure customers don’t need to repeat themselves.
What metrics should I look at in my first month of rollout?
Monitoring AI Agents is best done via the Insights dashboard. You can obtain detailed information about active users, the percentage of transfers to agents, and automated resolutions. The dashboard performance metrics offer you a quick glance at the overall performance of the AI Agents.
How do I prevent the AI Agent from giving the wrong answer?
The information your AI agent surfaces is only as good as the information within your knowledge base. Ensure that your KB content is up to date & verified on a regular cadence to ensure no legacy answers are given. Focusing on the first 75 words leads to better article retrieval. Optimize your knowledge base by using tips from this article to make each article about a single topic rather than multiple issues.
How do I get my AI Agent to prfill the ticket form to save time for the agent and increase efficiency?
You'll want to leverage an "ask for details" step within "ai agents" settings to collect certain data points
What are the best KPI's to look at when determining success with AI Agents?
We recommend the following: Self Service Rates. Where are you starting from? Bot Insights 7 Day Performance. "Next Steps to Improve Performance". Surfacing topics that are escalating into a ticket/unresolved. Actionable next steps on improving automation levels such as intent suggestions. Combine that with CSAT.
Conversations just with bots are only historically available for 7 days. Is there any way to export or store this data? We would want it for quality assurance of the bot, to make sure it's giving the right info to customers.
We are thrilled to announce the release of our AI agent QA (bot QA) feature, designed to enhance chatbot quality assurance. This new functionality allows you to identify blind spots in your AI agent support quality. Analyze and review 100% of your AI agent interactions, identify errors for human intervention, and gain insights into key performance areas.
Resource: Announcing quality assurance (QA) for AI agents
Is there an obvious sign to the end-users that they are speaking to an AI agent?
If so, is it optional for them to elect to speak with an agent easily?
Yes, there is a indicator that the comments from the Ai agent are "generated via Ai" upon reply. Ai agents can be equipped to handoff to a live agent and understand when a user requests this.
Do we need to use Sunshine for the AI Agent to access our customer data from our database?
No, you can leverage the Make API call step in the Native AI Agent to collect the relevent customer data from your back end system, store, pass variables. and then return it in a BOT message within the Conversation without using Sunshine Conversations API access.
Resource: Using the Make API call step in a conversation bot
Can the AI Agent integrate with HubSpot to create sales and marketing workflows from support tickets?
The Make API call step allows you to configure an API call out to another system, such as an internal CRM or ERP, or to push conversation details to an external endpoint, like Amazon Event Bridge or Google Analytics. I have not worked directly with the HubSpot API, but assuming they have an API endpoint we could access, we could determine how to leverage within the AI Agent Bot Flow.
Resource: Using the Make API call step in a conversation bot
Where is the information coming from to answer user's questions - Zendesk Guide articles? If so, how do we best prepare our knowledge base when we tend to have a lot of "snowflake" (one off) tickets?
Correct, information is coming directly from your help center articles. That means you don't need to worry about your AI agents handing out inaccurate info that's out of left field -- because any info they give is something your teams have already written and approved in your help center. Our reporting should be able to help you identify themes among questions that are going unanswered by your AI agent. Using these insights, you can create help center articles to address these topics. I'd say to pay special attention to the title and first 75 words or so of your article, as those are very powerful for helping your AI agent surface the right information
What are some of the most challenging use-cases ZenDesk teams are seeing out there?
Let's interpreting this question as meaning the most complex use cases that we see our customers solving using AI agents. We've seen customers find really great sucess with complex processes by using those end-to-end integrations. As a reminder, you can use these to push, pull, and parse data from all sorts of internal and external systems like your OMS, your CRM, your shipping provider. This has been transformative when it comes to automating processes that used to be a guaranteed ticket to a human agent. Now, things like returns or exchanges can be completely automated, which we know is a very complex process when it comes to all of the data required to finalize it.
Do you have any suggestions for increasing adoption of chat bots for less tech savvy customers?
Start with the basics, have content/ knowledge your ai agents can use. This will be the core foundation of how you can get immediate value with minimal setup!
Resource: Using AI to generate replies in a conversation bot