Recent searches


No recent searches

Using AI to generate replies in an AI agent for messaging (Legacy)



image avatar

Aimee Spanier

Zendesk Documentation Team

Edited Feb 03, 2025


1

96

96 comments

We'd appreciate it if the bot could gather information from articles using web crawling. Is this something you're planning to include in your future developments?

1


image avatar

Daniel Aron

Zendesk Product Manager

Hi Håvard Skare and Tatiana Christensen i'll reach out to you separately to help. If your help center isn't restricted and you've enabled generative replies then it should work.

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Deiaa Eid, yes supporting external content (including via web crawling) with generative replies is on the roadmap. No ETA at this stage but it is a top priority for 2024.

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Sean Morrissey Alan Pugh, we're currently rolling our restricted content support with generative replies and it should be available across all accounts by January 17th. Here's the announcement with more info. Please let us know how you go! 

1


What is a good way to measure how well this feature is performing? We are seeing a bit of drop off in chat tickets, but hesitant to consider this a win due to deflection. Any suggestion from the community would be fabulous. TIA!

0


If I select No to answer 'was this helpful' the below prompt appears. Where can I edit this since there is nothing to choose from currently?

I'm sorry, I didn't quite catch that. Could you please rephrase your question or select one of these options?

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Mary Ann Sin thanks for the feedback. We bot reporting and insights roadmap planned for 2024, including for the first release late in Q1 that will provide visibility of bot conversations that included generative replies, and some basic metrics to measure performance. In the meantime, an option available is to browse bot escalated tickets to gauge generative reply performance including ability to find relevant content and return accurate replies. In regards to the message following clicking the 'No' feedback option, it can currently be edited in the "If the bot can't answer the question" setting on the setup tab, however we have an enhancement coming in the next few weeks that will give you more flexibility to customize this experience so stay tuned. 

1


We really can't make it work. We have tried everything, we have a help center that is live without permissions but when we ask the bot something it says that it can't answer that question. If we select "Recommend articles" it works but "Generate a reply" doesn't. 

1


Hi Robin. 
I was in contact with customer support in December to try and fix the same issue. Turned out the bot GenAI reply only works in English. I'm not sure if your articles and bot language are both english, but that might be it. 

1


Hey Håvard Skare
Thanks a lot for your quick answer. That was disappointing... So the bot can only answer in English, isn't it even possible to translate the answer?

If not - Is this something you are working on? Daniel Aron

1


Hello,

We have both an open and a restricted help center, and this comment is about the open one. 

Our open help center has articles in Dansh and English, as these are the two enabled languages in the help center. With regards to Håvard and Robin's comments, I can confirm that the bot can generate the answers based on articles in Danish.

At the same time, it couldn't answer to all questions even though the articles provide answers, replying that it doesn't understand the request.

Why can the bot generate answers about some articles, but not others? Is it because some articles somehow don't match the intents? This is important for us to understand.

It would be good to have a guide for how to optimize articles to increase the generative reply success rate. And the same for multi-lingual help centers, how to optimize generative reply rate across articles in different languages. 

With regards to languages: For example, we have some articles set up in Danish and in English. Some have only one version, e.g. only English. If the bot conversation is in Danish, will the bot generate a Danish reply from the article that's only in English or only look at the Danish articles?

Also, when I'm on US help center page of my help center, the bot still writes me in Danish. Does it mean that it will translate the US articles and generate reply? Or as per above, it uses knowledge for all articles disregarding the language?

2


+1 for Tatiana Christensen Comment

1


First of all +2 for Tatiana Christensen s Comment! We are experiencing the same issue. Some articles in German are recognized, others/new ones not. It would be very helpful to understand why this is the case. 

Furthermore, our generative Bot often switches from informal language "Du" in German to the formal "Sie", which we do not want. All of our articles are written with the informal "Du" and our Chatbot personality is set to friendly. Is there a way that we can fix this? 

1


Happy to chat with a PM on this and maybe do a remote session, but my experience with the bot replies using KB was really bad, I had to turn this feature off moments after it was turned on. The replies it was generating were very very wrong. I did my own test and gave GPT a PDF of my KB, and it was quite accurate.

0


Daniel Aron hi, any update on reporting for generative replies? 

0


image avatar

Kris

Zendesk Luminary

I recently received a survey on this EAP. We haven't used it, and I wanted to share why, but selecting that option immediately ended the survey which doesn't seem like a great way to collect feedback. The thing holding us back is indexing external content, and our preference would be the ability to connect it to an OpenAI assistant per-bot so we can customize the prompt and files it uses versus using an out-of-the-box model without customization options. 

0


I understand that the Zendesk bot generates answers based on the Zendesk Guide knowledge base. Specifically, I am interested in knowing whether the Zendesk Advanced Bot can search for content from specific pages outside the Zendesk Guide, such as our main website.

Furthermore, I am curious to know if the bot has the capability to recognize content within images on item pages. This information is crucial for our assessment of its suitability for our needs.

4


image avatar

Mike DR

Zendesk Customer Care

Hi Zach!
 
We've taken note of your feedback about the AI EAP.

0


+1 for Tatiana Christensens Comment and +1 for Elinas Comment

Would be great to change these in settings.

1


Hi, I've gotten around to setting up the bot and I was curious if it's possible to give it instructions/rules? An example of what we'd like to see is for it to use bullet points in its responses where appropriate because the current answers are a bit of a wall of text, if that makes sense. A lot of our answers will likely be step-by-step based and that usually goes well with bullet points.
 
Also, is it possible for testing purposes to feed the bot KB articles that are marked as drafts in the Zendesk Guide? Or do they need to be published for the bot to use them?
 
Thanks!
 
Thanks!
Justin

2


Hi there, I can't find the generative AI in the admin center, here Agent tools > Generative AI tab

How to enable the feature?

0


Can the bot not see content block information? I just tested the bot by typing what is literally the article title (and some other variations) and it gave the "please rephrase" reply. the only thing i can think is that the entire article is a content block.

0


Hi, we are having some success with the bot generating answers.

In an article, that the bot uses for it's generative reply, we link to the request form with prefield fields related that specific topic.

The generative reply includes this link as an url, which doesn't look so good. It would be nice if it could rename the link to "link" instead of showing the full url that the user can click on to go to the request form. 

3


image avatar

Daniel Aron

Zendesk Product Manager

Hi Robin, in regards to your question

So the bot can only answer in English

Generative replies can work in any of the languages listed in this article (the ones that don't have "not supported in autoreplies with articles" in brackets) however relies on there being a localized/translated version of the article for the relevant language. Does that exist in your case? BTW bot articles are currently being reviewed and updated for clarity in regards to language support. 

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Tatiana Christensen, unfortunately Danish is one of those languages that are not supported for article retrieval features (including generative replies) with Zendesk bots currently. It's possible that the model that has been trained for article retrieval on similar languages (e.g. German) and that's why it works on some occasions in Danish but not consistently. In regards to your specific question

If the bot conversation is in Danish, will the bot generate a Danish reply from the article that's only in English or only look at the Danish articles?

We would not expect it to generate a reply if the article is only in English. We would generally not expect it to generate a reply if the article is Danish either, however there are exceptions as I just mentioned.

Also, when I'm on US help center page of my help center, the bot still writes me in Danish. Does it mean that it will translate the US articles and generate reply?

Yes the bot can speak Danish because it's one of the languages broadly supported. However, we would not expect it to translate US articles and generate a reply. That said, generative AI can sometimes go outside the bounds of the guidelines we give it in the prompt so there may be exceptions. 

It would be good to have a guide for how to optimize articles to increase the generative reply success rate

The best practises documented here for article recommendations and autoreplies should help for generative replies as well. The solution currently uses the same model to find relevant help center articles prior to involving OpenAI to generate a reply. We plan to improve the guidance around this in documentation soon, prior to making this feature generally available.

Elina I'm surprised to hear you are getting inconsistent results with German however given that's a supported language. I'm going to loop in Monika Vogel, our data science expert to share more insight on the issues discussed in recent comments relating to language support. 

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Zach Gilbert I'm sorry to hear generative replies didn't work for you. I'll reach out to you privately to follow up on the issues you encountered.

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Nicole Rinker, yes there will be some basic reporting on generative replies in the bots actionable insights release schedule for April. There are also plans to provide further insights to measure generative replies performance in the ongoing actionable insights roadmap later this year. Looping in Lisa Tam who is leading that part of the roadmap.

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Kris, apologies for your issue with the survey - i've shared your feedback with our research team. Adding external content support with generative replies is on the roadmap for later this year. I'm curious about your comment

customize the prompt and files it uses versus using an out-of-the-box model without customization options. 

We don't currently plan to allow any manipulation of the prompt itself that gets sent to OpenAI as this can introduce security issues. However, I'm curious to understand your use-case further. Can you share examples of how you might want to customize the prompt and why? I'm also happy to follow up with you offline if you prefer. 

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Justin Pelham thanks for your feedback, it isn't possible to provide specific instructions or rules for your account, however I'm going to record your use-case for consideration. We may be able to improve the generic prompt we provide OpenAI to improve the formatting of replies. Also, no the bot can only provide replies based on published articles. I'm curious to hear if anyone else has feedback on that idea. I imagine there would have to be an explicit setting to allow this given you wouldn't want unfinished article draft content to be exposed to live customers?

0


image avatar

Daniel Aron

Zendesk Product Manager

Hi Jules, to enable generative AI for Zendesk bots, you can follow the instructions in this article - you'll find it in admin center in your bot admin behavior tab. This article provides more info specifically about setting up generative AI for agents.

0


Please sign in to leave a comment.