Service Incident - November 19, 2022 - Support & Talk | Pod 25 - Unable to merge tickets

Summary

From November 18, 2022 at 20:00 UTC until November 19, 2022 at 20:41 UTC, Zendesk Support customers on Pod 25 experienced errors that prevented ticket merging for Zendesk Talk channel tickets. 

Timeline

05:22 UTC | 21:22 PT

We’re currently investigating reports from Support customers on Pod 25 regarding issues with ticket merge, getting “There was an unexpected problem during merge” error. Next update in 2h or when we have more details.

07:08 UTC | 23:08 PT

Thank you for your patience while we continue working on fixing the ticket merge issue affecting Support customers on Pod 25. We are currently testing some changes and rollbacks. Again, we appreciate your patience. Next update in 2 hours or as soon as we have more info available.

09:03 UTC | 01:03 PT

We appreciate your patience while we work through fixing the ticket merge issue affecting Support customers on Pod 25. Our Developer Team is still working toward a full resolution. We will provide more information as it is available.

13:42 UTC | 05:42 PT

Investigation and work towards resolution of the issue with ticket merge for Support customers on Pod 25 continues. Our teams are engaged to the best of their abilities. We are unable to share an ETA at this time and truly appreciate your patience with this matter over the wknd.

20:53 UTC | 12:53 PT

We are happy to confirm that a fix has been applied to the issue affecting ticket merge for Support accounts on Pod 25, and we no longer see errors. Please clear cache and cookies and attempt to perform merges. Reach back if you need further assistance. Thank you in advance!

 

Root Cause Analysis

This incident was caused by a database and ticket model migration change that failed to accommodate a Talk ticket use case leading to those tickets failing to merge. This use case was discovered late in the incident which extended the time to resolution.

Resolution

To fix this issue, our team identified the ticket model and made the required changes.

Remediation Items

1. Additional monitoring and alerts for ticket merge functionality.
2. Review and document ticket models for future database changes.

 

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.