Service Incident - January 24, 2024 - Sunshine Conversations | All pods - Message latency and errors

Summary

On January 24, 2024, between 17:35 and 17:50 UTC, Sunshine Conversations customers across all pods may have seen a spike in latency and 5xx level errors affecting incoming and outbound messages.

Root Cause Analysis

This incident was caused by latency introduced during application server restarts during the recovery of this service issue earlier in the day.

Resolution

This issue self-resolved as our application servers completed their restarts and the associated latency recovered.

Remediation Items

1. Review recovery procedures and resiliency [In progress]
2. Increase application host sizes [Scheduled]
3. Update health checks to ensure latency is below the required threshold [Scheduled]

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.