
Generative replies deliver automated, AI-generated answers in response to customer requests, with information from your own trusted help center content, without needing to build custom answers.
Through the use of generative AI, when a relevant help center article is identified, generative replies are displayed as a single message with a link to the source article. These messages are short (generally under 100 words) and reflect the AI agent persona if it's being used.
This article includes the following sections:
About generative replies in the AI agent
If you have an active help center, generative replies can immediately begin to respond to questions from your end users. These responses use generative AI to evaluate articles in your help center, then use that knowledge to provide concise answers within the ongoing conversation. If the article referenced to generate the response includes links to relevant articles, those might be included in the generated response. Generative replies enable end users to get the information they need without leaving the conversation to read an article.
Generative replies can respond to a customer question when:
- No intents are detected for that question.
- An intent matching that question is configured to generate a reply.
These replies are always followed by a standard request for end-user feedback (“Was this helpful - yes/no”). You can configure AI agent responses to this feedback.
If you’re using a restricted help center, the AI agent response respects the article view permissions set in Guide, which means:
- If the end user, agent, or admin is authenticated via the messaging authentication protocol and user mapping established with an external ID, the AI agent can suggest relevant restricted articles.
- If the end user is unauthenticated, the AI agent will suggest only public articles.
Activating and configuring generative replies
You can activate generative replies when you create a new AI agent or you can add generative replies to an existing AI agent.
To turn on generative replies in an existing AI agent
- In Admin Center, click
Channels in the sidebar, then select AI agents and automation > AI agents.
- Click Manage AI agents for messaging, then click the AI agent you want to use with generative replies.
- In the AI agent's Behavior tab, click to expand the section If the AI agent finds relevant articles.
- Make sure Generate a reply is selected. When you select this option:
- Any intents without an assigned answer setting are updated to Generate a reply.
- After the AI agent replies to an end user’s comment, it sends a request for feedback (“Was this helpful? Yes/No”). If the end user selects Yes or No, it’s followed by a customizable response.
- Click Test AI agent, then enter phrases the AI agent can match to existing help center content to view samples of replies your end users might receive.
- Click Publish AI agent to apply your changes.
Every reply the AI agent generates is followed by a request for feedback (“Was this helpful? Yes/No”). After the end user selects an option, the AI agent replies. You can customize the AI agent's feedback responses if you'd like.
Using generative replies with intents
When you turn on generative replies, replies are generated for questions that match intents without assigned answers and for questions without a matching intent.
Updating the generative replies setting for specific intents
By default, any intent without an assigned answer will generate a reply. For each intent, you can leave this setting as-is, update it to not generate a reply, or add the intent to an answer in your AI agent for messaging.
To manage generative reply settings for a specific intent
- In Admin Center, click
Channels in the sidebar, then select AI agents and automation > AI agents.
- Click Manage AI agents for messaging.
- Click the AI agent you’re using with generative replies.
- Click the Intents tab, then click the drop-down in the Answer column to select an option for each intent you want to update:
- Generate a reply lets the AI agent create a response to a customer’s matching free-text entry.
- Don’t generate a reply prevents the AI agent from responding to a customer’s matching free-text entry. Instead, it replies with the standard fallback response (“Sorry I couldn't find a good answer to your question. Here are some topics that might help though”).
- Show an answer lets you connect the intent to an existing answer or create a new answer for it.
Remember to publish the AI agent to make these changes available.
Updating the generative replies setting for questions without matching intents
You can choose whether you want to generate a reply for any question asked by an end user that does not have a matching intent. By default, a reply is generated for any customer free-text entry that matches an intent if that intent is not associated with an answer in the AI agent for messaging.
To manage generative replies to questions without matching intents
- In Admin Center, click
Channels in the sidebar, then select AI agents and automation > AI agents.
- Click Manage AI agents for messaging.
- Click the AI agent you’re using with generative replies.
- Click the Intents tab, then use the drop-down at the top of the page to select Generate a reply or Don’t generate a reply.
Remember to publish the AI agent to make these changes available.
96 comments
Deiaa Eid
We'd appreciate it if the bot could gather information from articles using web crawling. Is this something you're planning to include in your future developments?
1
Daniel Aron
Hi Håvard Skare and Tatiana Christensen i'll reach out to you separately to help. If your help center isn't restricted and you've enabled generative replies then it should work.
0
Daniel Aron
Hi Deiaa Eid, yes supporting external content (including via web crawling) with generative replies is on the roadmap. No ETA at this stage but it is a top priority for 2024.
0
Daniel Aron
Hi Sean Morrissey Alan Pugh, we're currently rolling our restricted content support with generative replies and it should be available across all accounts by January 17th. Here's the announcement with more info. Please let us know how you go!
1
Mary Ann Sin
What is a good way to measure how well this feature is performing? We are seeing a bit of drop off in chat tickets, but hesitant to consider this a win due to deflection. Any suggestion from the community would be fabulous. TIA!
0
Mary Ann Sin
If I select No to answer 'was this helpful' the below prompt appears. Where can I edit this since there is nothing to choose from currently?
I'm sorry, I didn't quite catch that. Could you please rephrase your question or select one of these options?
0
Daniel Aron
Hi Mary Ann Sin thanks for the feedback. We bot reporting and insights roadmap planned for 2024, including for the first release late in Q1 that will provide visibility of bot conversations that included generative replies, and some basic metrics to measure performance. In the meantime, an option available is to browse bot escalated tickets to gauge generative reply performance including ability to find relevant content and return accurate replies. In regards to the message following clicking the 'No' feedback option, it can currently be edited in the "If the bot can't answer the question" setting on the setup tab, however we have an enhancement coming in the next few weeks that will give you more flexibility to customize this experience so stay tuned.
1
Robin
We really can't make it work. We have tried everything, we have a help center that is live without permissions but when we ask the bot something it says that it can't answer that question. If we select "Recommend articles" it works but "Generate a reply" doesn't.
1
Håvard Skare
Hi Robin.
I was in contact with customer support in December to try and fix the same issue. Turned out the bot GenAI reply only works in English. I'm not sure if your articles and bot language are both english, but that might be it.
1
Robin
Hey Håvard Skare
Thanks a lot for your quick answer. That was disappointing... So the bot can only answer in English, isn't it even possible to translate the answer?
If not - Is this something you are working on? Daniel Aron
1
Tatiana Christensen
Hello,
We have both an open and a restricted help center, and this comment is about the open one.
Our open help center has articles in Dansh and English, as these are the two enabled languages in the help center. With regards to Håvard and Robin's comments, I can confirm that the bot can generate the answers based on articles in Danish.
At the same time, it couldn't answer to all questions even though the articles provide answers, replying that it doesn't understand the request.
Why can the bot generate answers about some articles, but not others? Is it because some articles somehow don't match the intents? This is important for us to understand.
It would be good to have a guide for how to optimize articles to increase the generative reply success rate. And the same for multi-lingual help centers, how to optimize generative reply rate across articles in different languages.
With regards to languages: For example, we have some articles set up in Danish and in English. Some have only one version, e.g. only English. If the bot conversation is in Danish, will the bot generate a Danish reply from the article that's only in English or only look at the Danish articles?
Also, when I'm on US help center page of my help center, the bot still writes me in Danish. Does it mean that it will translate the US articles and generate reply? Or as per above, it uses knowledge for all articles disregarding the language?
2
James Molina
+1 for Tatiana Christensen Comment
1
Elina
First of all +2 for Tatiana Christensen s Comment! We are experiencing the same issue. Some articles in German are recognized, others/new ones not. It would be very helpful to understand why this is the case.
Furthermore, our generative Bot often switches from informal language "Du" in German to the formal "Sie", which we do not want. All of our articles are written with the informal "Du" and our Chatbot personality is set to friendly. Is there a way that we can fix this?
1
Zach Gilbert
Happy to chat with a PM on this and maybe do a remote session, but my experience with the bot replies using KB was really bad, I had to turn this feature off moments after it was turned on. The replies it was generating were very very wrong. I did my own test and gave GPT a PDF of my KB, and it was quite accurate.
0
Nicole Rinker
Daniel Aron hi, any update on reporting for generative replies?
0
Kris
I recently received a survey on this EAP. We haven't used it, and I wanted to share why, but selecting that option immediately ended the survey which doesn't seem like a great way to collect feedback. The thing holding us back is indexing external content, and our preference would be the ability to connect it to an OpenAI assistant per-bot so we can customize the prompt and files it uses versus using an out-of-the-box model without customization options.
0
Nick
I understand that the Zendesk bot generates answers based on the Zendesk Guide knowledge base. Specifically, I am interested in knowing whether the Zendesk Advanced Bot can search for content from specific pages outside the Zendesk Guide, such as our main website.
Furthermore, I am curious to know if the bot has the capability to recognize content within images on item pages. This information is crucial for our assessment of its suitability for our needs.
4
Mike DR
We've taken note of your feedback about the AI EAP.
0
Stefan
+1 for Tatiana Christensens Comment and +1 for Elinas Comment
Would be great to change these in settings.
1
Justin Pelham
2
Jules
Hi there, I can't find the generative AI in the admin center, here Agent tools > Generative AI tab
How to enable the feature?
0
Fiona
Can the bot not see content block information? I just tested the bot by typing what is literally the article title (and some other variations) and it gave the "please rephrase" reply. the only thing i can think is that the entire article is a content block.
0
Tatiana Christensen
Hi, we are having some success with the bot generating answers.
In an article, that the bot uses for it's generative reply, we link to the request form with prefield fields related that specific topic.
The generative reply includes this link as an url, which doesn't look so good. It would be nice if it could rename the link to "link" instead of showing the full url that the user can click on to go to the request form.
3
Daniel Aron
Hi Robin, in regards to your question
Generative replies can work in any of the languages listed in this article (the ones that don't have "not supported in autoreplies with articles" in brackets) however relies on there being a localized/translated version of the article for the relevant language. Does that exist in your case? BTW bot articles are currently being reviewed and updated for clarity in regards to language support.
0
Daniel Aron
Hi Tatiana Christensen, unfortunately Danish is one of those languages that are not supported for article retrieval features (including generative replies) with Zendesk bots currently. It's possible that the model that has been trained for article retrieval on similar languages (e.g. German) and that's why it works on some occasions in Danish but not consistently. In regards to your specific question
We would not expect it to generate a reply if the article is only in English. We would generally not expect it to generate a reply if the article is Danish either, however there are exceptions as I just mentioned.
Yes the bot can speak Danish because it's one of the languages broadly supported. However, we would not expect it to translate US articles and generate a reply. That said, generative AI can sometimes go outside the bounds of the guidelines we give it in the prompt so there may be exceptions.
The best practises documented here for article recommendations and autoreplies should help for generative replies as well. The solution currently uses the same model to find relevant help center articles prior to involving OpenAI to generate a reply. We plan to improve the guidance around this in documentation soon, prior to making this feature generally available.
Elina I'm surprised to hear you are getting inconsistent results with German however given that's a supported language. I'm going to loop in Monika Vogel, our data science expert to share more insight on the issues discussed in recent comments relating to language support.
0
Daniel Aron
Hi Zach Gilbert I'm sorry to hear generative replies didn't work for you. I'll reach out to you privately to follow up on the issues you encountered.
0
Daniel Aron
Hi Nicole Rinker, yes there will be some basic reporting on generative replies in the bots actionable insights release schedule for April. There are also plans to provide further insights to measure generative replies performance in the ongoing actionable insights roadmap later this year. Looping in Lisa Tam who is leading that part of the roadmap.
0
Daniel Aron
Hi Kris, apologies for your issue with the survey - i've shared your feedback with our research team. Adding external content support with generative replies is on the roadmap for later this year. I'm curious about your comment
We don't currently plan to allow any manipulation of the prompt itself that gets sent to OpenAI as this can introduce security issues. However, I'm curious to understand your use-case further. Can you share examples of how you might want to customize the prompt and why? I'm also happy to follow up with you offline if you prefer.
0
Daniel Aron
Hi Justin Pelham thanks for your feedback, it isn't possible to provide specific instructions or rules for your account, however I'm going to record your use-case for consideration. We may be able to improve the generic prompt we provide OpenAI to improve the formatting of replies. Also, no the bot can only provide replies based on published articles. I'm curious to hear if anyone else has feedback on that idea. I imagine there would have to be an explicit setting to allow this given you wouldn't want unfinished article draft content to be exposed to live customers?
0
Daniel Aron
Hi Jules, to enable generative AI for Zendesk bots, you can follow the instructions in this article - you'll find it in admin center in your bot admin behavior tab. This article provides more info specifically about setting up generative AI for agents.
0