Recent searches


No recent searches

We are excited to launch AI agents and automated resolutions!

Pinned


image avatar

Shawna James

Community Product Feedback Specialist

Posted Apr 25, 2024

Dear Customers,

We are excited to launch AI agents and preview the upcoming launch of automated resolutions! For ease of collecting  feedback on this launch, and to make sure we can provide you with quick  answers to your questions, we are providing this space for all feedback to be shared. Please comment below to share your thoughts and feedback, or ask questions about this release.

The product feedback forum in the Zendesk Community is always the best place to share information with our Product Teams around ideas you have for features or feedback you have about what we are building. This is where our product managers monitor and engage in conversation, and our aim is to aggregate all feedback here. 


Before posting, please take a moment to review the community guidelines and code of conduct. These outline how our product feedback forums work, how we review this feedback, and what you can expect. Take a look and let us know if you have any questions! 

Thank you for being a part of the Zendesk Community and sharing your feedback with us, we sincerely value your input!


-8

68

68 comments

Hi, I would like to know how far Zendesk AI Agents differ from Ultimate AI's current solution. Could you provide some details? Thank you!

2


Hi Ariadna,

Thanks for the question! We're excited to have Ultimate on board, and we're eager to bring our combined strengths to our customers.

Over time, we intend to integrate Ultimate. While we work towards this, the focus remains on delivering exceptional value through these platforms. Our goal is to ensure we match our customers with the right solution that aligns with their specific needs and use case scenarios.

Our native Zendesk AI agents are ready to use out of the box. By setting up AI agents on the Zendesk platform, you can start using pre-trained AI agents, answer FAQ questions using your Zendesk knowledge base, and even set up conversation flows. They are a great fit for businesses looking to automate common requests with minimal setup.

If you need even more customization like full multilingual support and deeper integrations, Ultimate may be a good fit. Ultimate also provides a more hands-on implementation approach with a required services package.

In the spirit of providing the best possible service, we encourage you to connect with your Zendesk account manager or a Zendesk advocate who would be delighted to provide you with a detailed discussion on how we can address your unique requirements and set you up for success with the right AI agent solution.

Thank you for being a part of our community, and we look forward to continuing to support your business with our evolving AI agent capabilities.

0


image avatar

Brandon (729)

Zendesk LuminaryUser Group LeaderThe Humblident Award - 2021Community Moderator

Hey Shawna James - Thanks for this!  As I understand it, customers won't start to see a billing impact from these changes until after their next renewal date beyond July 10th, 2024 yes?  And even then, there are checks & balances in place should a customer get an unexpected surge in tickets due to an incident.  Looking forward to watching the evolution of this product in the coming months.  PS - For anyone following along, Shawna put together this awesome template for leaving feedback.

0


My concerns so far are:
 

1. You're going to charge $1.5 per unresolved conversation where a client just left the conversation with a bot. I checked our data that's about 60% of all the “automated resolutions”. Unless a client confirms that his issue is resolved, it is not resolved and he probably just switched channels.

 



2. How do we determine in advance how many resolutions we will need during the month? We have big spikes in volume due to releases, bugs etc. Now instead of focusing on business operations, we will have to guess the quote of “automated resolutions” we need every month.

 

3. The current cost per conversation is extremely high. We work in South Asia and with current Zendesk rates, it will be 4 times cheaper to resolve a ticket via human than via Zendesk bot.

 

4. You should give the option to old clients to use the chatbot without paying for each conversation. You can exclude generative AI and all other “AI features” from it. But at least, for the money that we pay, we should be able to use a basic decision tree chatbot.
 

22


Hallo, 

 

You show this picture in your article:

 

If I have understood correctly, this would be extremely expensive for this company with over 500k automated resolutions. Right? Could you use the example to explain the costs involved? 

 

Thank you in advance, Melanie

6


image avatar

Brandon (729)

Zendesk LuminaryUser Group LeaderThe Humblident Award - 2021Community Moderator

Hi Melanie Frey -

I'm sure Zendesk will reply with additional thoughts in this thread, but as I understand it, each instance will include a set number of resolutions per agent with the agent subscription, so for a very large enterprise team of say… 10,000 agents, the costs might largely be absorbed by this allotment.  

 

Either way, one thing we know for sure is that the example company certainly has their hands full.  Hope this helps!

 

Brandon

1


Trying to get an understanding of resolutions, I'm looking at my bots automated resolutions and I'm seeing a few where the user just left and the conversation was clearly not resolved. Why would this count towards a resolution? 

13


I got an automated email yesterday from ZD with the following note:

Please note that pricing and requirements for the Advanced AI SKU will remain unchanged, and your Advanced AI investment will continue to provide exclusive value through AI-powered tools focused on agent and admin productivity.

What does it mean?

 

0


As long as a user leaving the conversation or not replying to the bot is being counted as a success with the AI agent, I think this pricing model has a major flaw. The assumption that a customer exiting the conversation means they no longer have an issue is a very big assumption that is not reflected by what we see.

If the way that resolutions are being counted is going to remain the same, with at least half of the “resolutions” being false resolutions, then either the allotment of AI resolutions per seat should be doubled, or the price per resolution or bundle should be halved. As long as there are more false resolutions than real resolutions, then it is an added cost without a substantial benefit. 

15


This is a huge concern for us as well! We have not made any changes to our bot with AI, so it's same bot we've been using for a few years now. The automated resolutions is now showing around 1,300 for the last 30 days, with a lot of them being false resolutions as well. We only have 14 agents, so with our allotment of 210 resolutions per month, this is a huge price increase for us as well. We will now be paying $10,000+ yearly for a service that has always been included in our subscription to Zendesk over the last few years. The AI stuff added to the bot is not enough to warrant a $10K increase to our subscription costs. 

I also agree that they should only be counted as a resolution if the customer clicks that their issue was resolved. Make it a mandatory step in each flow or at the end of a generative reply.

12


Hey Zendesk team. Having your pricing model tied to public anonymous interactions is quite risky. For eg. it would be really easy to build a script and start thousands of trivial conversations to my competition and therefore “send” them a few hundred thousands dollars in Zendesk bills to try to put them out of  business. Legal concerns apart, as shown in the previous comments, this script wouldn't even need to really say the problem was resolved, just by starting and abandoning a conversation you produce a $1.5 debt to a third party.

 

Malicious or not, public traffic based pricing tiering has it's caveats. What are your thoughts on this? 

13


They realize that this has the opposite effect of what they're aiming for... they're incentivizing the abandonment of bot usage. There are many accounts that won't be able to afford these costs and will deactivate the bots, and their next step will be to leave the brand. It should be optional; those who want it can apply it, and those who don't can continue with the previous method

11


I agree! Although the customer needs to reach the end of a flow for it to be considered a resolution, someone could significantly increase costs for any company by understanding how the Zendesk bot incurs charges. Additionally, all our users are anonymous, which complicates testing. Each interaction with the bot to test flows or generative replies could potentially add over $1.00 to our subscription costs.
 

I've also noticed that automated resolution charges apply to auto-replies. We were excited to implement advanced auto-replies now that we have intents, but we may need to reconsider due to the increased costs. This might lead us to test our ticket volume without the bot and avoid using auto-replies to evaluate the bot's impact on our ticket volume.

While generative replies are a nice feature, many of our customers who transfer to an agent complain about the bot and prefer speaking to a real person. We may have to forgo using the bot until we have the budget to hire additional support staff to offset the associated costs.

We have used the bot and messaging for a few years now, so turning it off seems counterproductive. It has always been part of our regular subscription costs, and it should continue to be integrated that way.

While I understand the need to generate more revenue for the improvements that have been made to the bot and the transition from “bot” to “AI Agent,” the communication to us as customers has been vague. I have had to search through multiple articles to understand how this will affect our company specifically. It would be helpful to have the ability to speak with someone who can clearly explain the reasons for such pricing changes, the added value they bring, and the specific improvements that will benefit our company. Detailed explanations of the increased charges for our account and the opportunity to ask questions to ensure we understand the impact and value would be greatly appreciated.

Transparency about how the additional revenue will be used to enhance the service would help maintain our trust and satisfaction, rather than making us feel "nickel and dimed."

7


image avatar

Jeff Curran

Zendesk Employee

A A 

Hey. Appreciate the feedback. 

  1. Thanks for identifying this. We’ve been taking feedback from customers during the trial period and are actively working to address cases where our AI Agent is not providing meaningful resolutions for customer inquiries. We rolled out a fix this week that addresses issues where the conversation ended with the bot not understanding the user’s request and requesting clarification. That should correct a number of these cases. We are continuing to take feedback and correct for false positives. We also have a feedback survey on the Bot Conversation transcripts page and we welcome your thoughts there as well.
  2. One of the benefits of our new pricing model is that automated resolution commitments align to contract terms. If you are an annual customer, you can commit to an annual amount of automated resolutions and will have the flexibility to use different amounts in different months. If you are a monthly customer, you have the ability to adjust your contract to fit your needs for the upcoming month. Please use the dashboard in your Admin Center to see current usage and estimate future usage. If you have questions please contact your Zendesk account manager or a Zendesk advocate.
  3. Our metric is more value-aligned than many of competitors. We only charge based on fully automated resolutions, not all conversations. We offer volume based discounts, which you can find here. If additional discounts are needed based on your use case, please reach out to connect with your Zendesk account manager or a Zendesk agent.
  4. As we developed the product, we chose to include this functionality without charging. As these bot capabilities have matured, we believe that we are well aligned to the value delivered by pricing only when an automated resolution is delivered.


Melanie Frey  

Thanks for your question Melanie! Brandon said it well. We aren’t able to calculate the cost of automated resolutions without knowing the number of agents and support plan of a customer. As you are suggesting, this is an example of a very large customer with a high number of automated resolutions. 

 

Zach Gilbert and Trudy Slaght 

Thanks for the feedback. We’ve been taking feedback from customers during the trial period and have identified issues where we are counting automated resolutions when the conversation ends in the bot not understanding the customer and edge cases where human agents are involved. We have a few fixes coming out in the next week or so that should correct these cases. There is an ability in the product to provide feedback to us and we encourage you to provide feedback there as well.

 

Viachaslau
This was part of an email sent to existing Advanced AI customers. We wanted to be sure that we are differentiating between the value provided by AI Agents, which is based around fully automating customer inquiries, and existing capabilities within the Advanced AI SKU, which is focused on agent and admin productivity.

 

Jen C and Maximiliano Cooper 

Thank you for your feedback. When it comes to other bot providers across the market, many adhere to an interaction-based model. This standard model typically incurs charges for every interaction with a bot, irrespective of the outcome or the ability of the bot to successfully resolve an inquiry. 

 

In contrast, as we were developing our product, we made a conscious decision to diverge this from the industry standard. Initially we included this functionality without charging. As our bot capabilities have matured significantly, we have aligned our pricing philosophy more closely with the tangible value our customers receive by charging only when a successful automated resolution is delivered.

 

During our trial period, we’ve been diligently collecting feedback from customers and are actively working to address cases where our AI agent is not providing meaningful resolutions for customer inquiries.  We rolled out a fix this week that addresses issues where the conversation ended with the bot not understanding the user’s request and requesting clarification. That should correct a number of these cases. We are continuing to take feedback and correct for false positives. We also have a feedback survey on the Bot Conversation transcripts page and we welcome your thoughts there as well.

 

Daniel V 

Thank you for sharing your feedback, specifically related to the optionality of the product. Our bots remain flexible and adjustable to meet the many different needs of our customers. We’ve included automated resolutions in every customer’s subscription and the ability to limit the number of automated resolutions at various tiers so customers can right size their subscription for their needs. By focusing on Automated Resolutions, our goal is to only charge customers for successful resolutions that take the load off their human agents.

 

In the spirit of providing the best possible service, we encourage you to connect with your Zendesk account manager or a Zendesk advocate who would be delighted to provide you with a detailed discussion on how we can address your unique requirements and set you up for success with the right AI agent solution.

 

 

Thank you all for sharing your feedback. We will continue to monitor this thread, and we look forward to continuing to support your business with our evolving AI agent capabilities.

0


Jeff Curran Why do you think it's okay to tell a customer that in 2 months his subscription cost will increase by 300%? 

7


Currently, I'm using the Classic Web Widget. There's no information to see where my automated resolutions are coming from to even see if this is worth. In effect, you are charging us $2 for our users to search something through our Help Centre through the web widget. 

2


We had the AI Addon before and were truly happy with the solution. However we now cancelled the Addon since the main reason for us to get it in the first place, was the generative bot. We understand that AI technology will cost everyone more, but now with the changes and the new pricing, our costs will at least increase by more than 150% and we will receive less features! You as well must see, that this is just not acceptable practise! Could you please let us know where precisely in our contract or the Main Services Agreement such a steep price increase is stipulated? We have been customers for more than 7 years now, and have been very happy with Zendesk and the development , but we simply cannot and will not accept such an increase! 

4


image avatar

Jeff Curran

Zendesk Employee

A A and Elina 

Thank you for flagging these concerns. An increase of that size would not be our intent and is not reasonable. We expect many of our customers to have all automated resolutions covered by the amount included in their seat purchase. 

 

We are looking into your accounts specifically to better understand why it would appear to be an increase of this size. We are also working with your account managers and will reach out to you directly with an account-specific plan to address. Thanks again for your feedback and partnership.

 

Jeremy Mifsud 

Thank you for your feedback and for being an early adopter of AI agents. You can see your current usage and allotment of automated resolutions in your Admin Center (details here). Feature-level tracking is being worked on right now and will be available by July 10. 

 

Hannah 

Thank you for your question. Yes, you can control the use of automated resolutions at the feature level. We are creating detailed instructions for turning off automated resolution generating features and will post it to this thread when it is complete. 

 

For conversations using generative replies, we recommend configuring answers to handle unhelpful responses to escalate to an agent if a user does not get the solution they need. This ensures that a customer’s issue is either effectively solved within the bot conversation or allows them to get the help they need from an agent. 

 

We appreciate you continuing to engage here and will monitor this thread for any updates. For any additional questions, please connect with your Zendesk account manager or Zendesk customer support

0


If an end user asks multiple questions in one conversation, each receiving an AI-generated response, will each response be considered an Automated Resolution? Or will the whole conversation be considered as one Automated Resolution?

0


image avatar

Jeff Curran

Zendesk Employee

Hello Jana 

An automated resolution covers the entire conversation with an end user within a 72-hour time period. Therefore, if an end user asks multiple questions or goes through multiple answer flows over the course of their 72-hour session with a bot and does not escalate to a human agent, that will only count as a single automated resolution. 

0


Experimenting with AI bots and email autoreplies has been an interesting experience with plenty of opportunity and plenty of risk to temper those opportunities. Based on the evidence, neither Zendesk's approach, nor generative AI as it is currently being leveraged, are ready for prime time (or prime time pricing) for this feature.

Issues with non-resolutions and hallucinations, poor or erroneous interpretations of articles, and lack of ability to work from previous conversation context still plague the product, but the reporting, which I assume is either coded by or at least vetted by humans, is impossible to understand.

Take these reports, the first two for the same 30-day period and the last from a 7-day period within that 30-day period. How can these be reconciled?

So, how many resolutions is this? I'll be charged for 2100, but maybe the actual number is either 401 or 192 * 4.3 weeks = 825? (assuming the 192 is an average 7-day count, of course). The 2100 resolution count that I'll be charged for is pure fantasy.

When I review the 192 automated resolutions, at least 40% would result in serious conversations with my staff if I was looking at an interaction with a human agent.

The only one that might be worth paying for is the 401 resolutions where the customer told us the response solved the problem. If I take my 270-resolution allotment off of that, I'm left with a reasonable upcharge of about $130/month

We have 3 vague definitions of “resolution” and no substantiation of value in alignment with cost.

Cost is dictated by the vendor, at least at first, but customers decide if they can and will pay those costs. At this time, I can't plan on paying for unreceived value.

3


+1
 

Winslow J If you read this article, you will see that basically any conversation that is not transferred to an agent is considered by Zendesk as an automated resolution. Even if a user just wrote “Hi” and then left, you will be charged $2 for it.

@Jeff Curran

It's June 24th and no one contacted us yet.

1


Issues with non-resolutions and hallucinations, poor or erroneous interpretations of articles, and lack of ability to work from previous conversation context still plague the product, but the reporting, which I assume is either coded by or at least vetted by humans, is impossible to understand.

 

I'm not sure, but I don't think anyone from Zendesk has commented on why the bot doesn't remember the context of the conversation. And will that change?

 

Regarding so-called resolutions, I check examples all the time and according to my observations, only 10% of them can be considered resolved. This is when the client has either explicitly stated that the issue has been resolved or it is highly probable from the context of a conversation.

 

We haven't had any specific conversations with our account manager about pricing either, because according to her this issue is still under discussion in Zendesk, but we are already being attacked by Zendesk's competitors offering their solution. And we have to seriously consider them in order not to get into a deadlock at the end of the current contract. 

2


While generally understanding that AI can't be completely free, I also believe that Zendesk mixes AI and normal bot features (greeting and kinda AVR menu), and is going to charge for both scenarios. We have a large number of custom flows created where we need real customer data and their transactions, and AI is only used there to understand customers' intents (but we pay for that separately as part of the Advanced AI add-on). And still, according to Zendesk, we are going to pay twice (for Advanced AI and “resolutions”). 

 

Well, even if you only take AI features, you can also see Zendesk's intention to charge the same price without looking into the details. Just because AI is ‘the new blockchain’, and some companies are ready to pay just to ‘be modern’. Re-wording a greeting or actual Generative AI replies (keeping the context of the conversation—something Zendesk's bot cannot do yet) are different features in terms of their value. As in iOS18, simple features are done on the device and are free. 

2


image avatar

Ryan McGrew

Zendesk Product Manager

Hey Winslow J 

We appreciate the feedback. We’ve been taking feedback from customers during the trial period and are actively working to address cases where our AI agent is not providing meaningful resolutions for customer inquiries. In particular, we are working to correct for conversations that end with customers replying that responses are not helpful and a set of other scenarios. We are also rolling out additional checks and validation post-conversation to correct for other scenarios where resolutions are counted for unhelpful conversations. We expect to have these changes in place before customers are asked to pay for resolutions. 

 

Looking at your screenshots, I will try to provide you with additional context:

  • Screenshot 1 - This will show automated resolutions across all channels in the last 30 days, and you can view this in more detail on the automated resolutions dashboard. This is the primary source of truth for automated resolutions on any channel.
  • Screenshot 2 - This refers to the resolve step of an answer flow. This is an optional step that customers can add to answer flows and doesn’t match the definition of an automated resolution. It is a metric you can use to track your answer flows in more detail but doesn’t capture things like Generative Replies or Article Recommendations.
  • Screenshot 3 - This is a snapshot of the last 7 days of bot conversations and allows you to then dig deeper into individual conversations. This number doesn’t account for automated resolutions from other channels like Autoreplies for email for example. 

 

We continue to work with customers and adjust based on feedback to tune this further.

0


image avatar

Ryan McGrew

Zendesk Product Manager

Hey AA 

Thank you for flagging the note about your account executive. We have raised these questions to your account team again to prioritize a response. They will follow up as soon as they are able!


As far as just saying hi to a bot and being charged, that’s not quite correct. A customer must do one of the following things within the course of a conversation:

  • Receive a Generative Reply as a response to a question sent by the user
  • Click on a recommended article link and read the article as a response to a question sent by the user
  • Navigate to the the last step of an answer flow

 

Once they’ve done at least one of the previous actions and don’t escalate further to a human agent within 72 hours, Zendesk would consider the conversation as resolved. If the customer were to say Hi and then leave the conversation, this would not be considered an automated resolution and there would be no charge.

0


image avatar

Ryan McGrew

Zendesk Product Manager

Hello Viachaslau,

We appreciate the feedback. We’ve been taking feedback from customers during the trial period and are actively working to address cases where our AI agent is not providing meaningful resolutions for customer inquiries. In particular, we are working to correct for conversations that end with customers replying that responses are not helpful. We are also rolling out additional checks and validation post-conversation to correct for other scenarios where resolutions are counted for unhelpful conversations. We expect to have these changes in place before customers are asked to pay for resolutions. 

We will also continue to improve our bot capabilities to introduce conversational context as well. We are working on timing for this and will communicate further when that is available.

 

When it comes to which features are free and which are paid, it’s always a balance. Firstly, to your point about building flows that use intent. When we launched AI Agents and outcome-based pricing at Relate we made our intent models available to all customers for use in Bots. This no longer requires the Advanced AI add-on, however the Advanced AI add-on still grants you capabilities to assist your agents and build more intelligent workflows. Additionally, if most of your flows are about capturing customer data and then handing off to an agent, these will not be tracked as an automated resolution. If instead this captured data is used to resolve a problem for the customer through an answer flow, then we would consider that to be an automated resolution.

Most bot products on the market charge based on overall conversation volume and not based on the outcome of those conversations and do not take into account the particular features a user might interact with. Our goal is to provide a value-aligned price when we solve a problem. As mentioned above, we are working diligently on addressing any misaligned scenarios. 

1


Ryan McGrew I appreciate the additional context, but I'm afraid I wasn't clear enough in my original post.

It's not that I'm unable to understand what those metrics indicate, but that the way that Zendesk is defining value is not aligned with value created for my customer or company.

Instead, the system as it is set up uses metrics that strongly favor delivering value to Zendesk with little regard to provable value for the customers. The metrics I listed were simply illustrating that point, which you have underscored in your response.

I think that we can all agree that a support interaction that doesn't deliver value to the requester or the company supporting that requester isn't worth the effort or expense.

However it's done, whether using a metric that's more aligned with actual resolutions, reducing the overall cost of these features, or some other approach, refocusing on delivering value to the customer will always be a winning strategy, especially in an extremely competitive marketplace like AI-enhanced support.

0


image avatar

Ryan McGrew

Zendesk Product Manager

Winslow J 

I appreciate the feedback and as mentioned we're continuing to tune the performance of automated resolutions as we go through this trial phase with customers. Our goal is to provide value and only charge when that is delivered. We will keep working toward that in partnership with our customers. 

We agree that this market is competitive and we believe that outcome-based pricing is how we can differentiate our product. As mentioned in previous posts, as we looked at other players on the market most bot products are monetized based on things like the volume of conversations regardless of outcome, or even pass through the cost of tokens sent to an LLM to customers. Additionally, very few of these product are self-service and often require services engagements or ongoing developer resources to build, deploy and maintain. We will continue to work with you and the rest of our customers to align pricing and value as we believe that's the best path forward for Zendesk and our customers.

0


Thanks for your answers Ryan McGrew. I always appreciate when Zendesk product managers are actively involved in discussing their products with customers.   

 

We’ve been taking feedback from customers during the trial period

Ideally, these issues should be addressed before the product becomes public. Here it looks a bit like putting out fires 😇.

 

We will also continue to improve our bot capabilities to introduce conversational context as well. We are working on timing for this and will communicate further when that is available.

 

Hopefully, this improvement will be introduced soon. Many questions remain essentially unanswered just because the bot doesn't keep the context of a conversation. And by the way, they are now counted by you as resolved.

 

When we launched AI Agents and outcome-based pricing at Relate we made our intent models available to all customers for use in Bots. This no longer requires the Advanced AI add-on, however the Advanced AI add-on still grants you capabilities to assist your agents and build more intelligent workflows. Additionally, if most of your flows are about capturing customer data and then handing off to an agent, these will not be tracked as an automated resolution. If instead this captured data is used to resolve a problem for the customer through an answer flow, then we would consider that to be an automated resolution.

This is another problem we see in Zendesk. Namely unpredictability in actions. We didn't just buy this add-on, we had multiple conversations with our account manager. And so we pay money to use intents, and in weeks the same feature becomes available to everybody, but we have to pay for resolutions that are not really resolutions because they are just our custom bot flows. We're basically paying for volume, and that's the approach of your competitors, which you challenge in the comment. 

1


Please sign in to leave a comment.

Didn't find what you're looking for?

New post