SUMMARY
Between July 30, 2024 18:40 UTC and July 31, 2024 00:30 UTC, Chat customers on Pod 15 may have experienced issues when trying to change their status, accept messages, and others may have seen error messages when engaging with the system interface.
Timeline
July 31, 2024 01:17 AM UTC | July 30, 2024 06:17 PM PT
Between July 30, 2024 18:40 UTC and July 31, 2024 00:30 UTC, Chat customers on Pod 15 may have experienced problems of being unable to change status, accept messages, or presented with error messages. From the backend work that has been done and implemented, this impact should now be fully mitigated. Thank you for your patience, and a root cause will be shared in due course.
POST-MORTEM
Root Cause Analysis
This incident was caused by an issue at our database service provider’s end, where an upstream software error caused communication issues, resulting in our Chat functionalities being disrupted.
Resolution
Our service provider identified and resolved the software defect, and we were able to confirm recovery of the Chat functions after further monitoring.
Remediation Items
- Working with our database service provider to set up alerts for similar scenarios [Scheduled]
- Adding attributes and creating new alerts based on them to address communication timeouts more quickly. [Done]
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, contact Zendesk customer support.