Service Incident - November 2, 2022 - ZBot - Issues reaching Zendesk Support team via bot

Summary

On November 02, 2022 from 09:42 UTC to 15:32 UTC, some Zendesk customers who attempted to contact our support team via our ZBot chat widget may have experienced errors and ticket creation failures preventing those customers from reaching our team.

Timeline

13:00 UTC | 06:00 PT
We’re currently aware of an issue with our partner that is affecting our ZBot (Advocacy support bot) and preventing ticket creation from our customers to us. We are working with them to get this fixed and have enabled our web form for your convenience.

13:16 UTC | 06:16 PT
These are issues related to our customers reaching Zendesk Support using the Zendesk ZBot Widget at support.zendesk.com and other pages. We appreciate your patience while we continue working with our partner through this disruption. [1/2]

13:17 UTC | 06:17 PT
The issue has been identified on their end and a fix is being implemented. This also means that, since tickets requests were not being created, you might need to initiate a new interaction with us. We apologise for this inconvenience. [2/2]

14:53 UTC | 07:53 PT
Our partner has released a fix for the issues affecting access to our Zendesk ZBot Widget for support (support.zendesk.com), however upon testing, this doesn’t seem to have worked. We will update you with more information in 2h or as we have them.

15:42 UTC | 08:42 PT
We are happy to report that the issues affecting our Zendesk ZBot Widget to reach our Advocacy support team have been resolved. The web form has been disabled and you can now message our support team again via ZBot after refreshing your browser. Thank you for your patience.

 

Root Cause Analysis

This incident was caused by a code change undertaken by our bot partner. The code change did not take into consideration various customizations that our partner’s customers might have implemented.

Resolution

To fix this issue, we worked with our partner to ensure the customized bot parameters were updated on their end.

Remediation Items

1. [Partner] Increased visibility into customizations deployed by Zendesk.
2. [Zendesk] Make changes to bot escalation flow to prevent similar issues in future.

 

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.