SUMMARY
From April 3, 20:45 UTC to 18:10 UTC, Sunshine Conversations customers across all Pods using Social Messaging channels may have experienced a significant disruption in service due to an outage at Meta. This outage primarily affected WhatsApp, with substantial impacts also on Facebook Messenger and Instagram, leading to a partial interruption in outbound traffic through these channels.
Timeline
18:45 UTC | 11:45 PT
We are receiving reports of issues sending and receiving messages via Messaging through Sunshine Conversations for customers on all pods. Further updates will be posted shortly.
19:02 UTC | 12:02 PT
We have confirmed an issue preventing the sending and receiving of WhatsApp Messaging through Sunshine Conversations for customers across all pods and our team is working with our partner provider to mitigate impact. We will provide further updates in one hour or as soon as we have new information to share.
20:02 UTC | 13:02 PT
We have confirmed the issue is also impacting service across all social messaging channels powered by Meta, including Facebook, Messenger, WhatsApp, and Instagram. We will provide another update within 4 hours or sooner if additional information is available.
POST-MORTEM
Root Cause Analysis
This incident was caused by an issue with Meta's services impacting Facebook, Messenger, WhatsApp, and Instagram. The exact root cause was not disclosed by Meta, but they acknowledged the disruption and worked on restoring services. During this time, Sunshine Conversations' monitoring systems detected a high occurrence of errors when attempting to send messages through the affected channels.
Resolution
The resolution of the outage was entirely dependent on Meta's recovery efforts. Sunshine Conversations monitored the situation closely and provided updates to customers as they became available. Once Meta announced that services were restored, Sunshine Conversations verified that outbound traffic resumed to normal levels and that messages were being successfully delivered through the social messaging channels.
Remediation Items
- Improve monitoring for outbound traffic health.
- Review and update the incident response plan to include alternative communication channels with Meta in case of their primary support portal being unavailable.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.
1 comment
Jeremy R.
Post-Mortem published April 11, 2024
0