Summary
On May 24, 2023 from 00:50 UTC to 16:33 UTC, Support customers using Side Conversations intermittently experienced duplicate messages and notifications.
Timeline
12:41 UTC | 05:41 PT
We are aware of issues regarding Side Conversation getting duplicated messages and notification popping up intermittently in the UI, affecting multiple Pods. Investigation is underway. Next update in 1h or when we have more info.
13:41 UTC | 06:41 PT
We are continuing to investigate issues regarding Side Conversation getting duplicated messages and notifications intermittently. Currently we don’t have any further updates, but will provide another update in 1h or when new information becomes available.
14:45 UTC | 07:45 PT
We were able to narrow down the behaviour and are working on a fix for the duplicate notifications from Side Conversations. In the meantime, refreshing the browser provides a temporary workaround. We will provide more information as it is available.
16:38 UTC | 09:38 PT
We are happy to report that the problem causing duplicate notifications from Side Conversations has been resolved. Please let us know if you continue to experience any issues.
POST-MORTEM
Root Cause Analysis
This incident was caused by a bug in a routine code release for an internal service underlying the Side Conversations feature.
Resolution
To fix this issue, our engineering team rolled back the offending code release.
Remediation Items
- Add tests to ensure a message subscription item can only cause one notification action [Scheduled].
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.
1 Comments
Post-mortem published on June 6, 2023.
Article is closed for comments.