SUMMARY
On May 16, 2024 from 04:12 UTC to 04:20 UTC, Sunshine Conversation customers may have experienced increased latency, where messages may not have been delivered via Messaging or the SunCo API.
Timeline
05:30 UTC | 22:30 PT
Between 04:12 and 04:20 UTC, we detected a blip in our Sunshine Conversations service that may have resulted in some undelivered messages in Messaging/SunCo API. This issue is now resolved. Apologies for any inconvenience this caused.
POST-MORTEM
Root Cause Analysis
This incident was caused by data latency during a routine maintenance process. Specifically, the function that writes data to one of the nodes of our system took longer than usual, leading to increased overall latency.
Resolution
This issue was fixed once all nodes successfully processed the remaining write operations, and normal service latency was restored.
Remediation Items
- We have existing alerts configured to alert us in the event of a similar recurrence. There are no further remediation items at this time.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. You can follow this article to receive a notification once our post-mortem investigation is completed. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.
1 comment
Eugene Khoo
Post-mortem published June 13, 2024.
0