SUMMARY
On April 22, 2024 from 14:18 UTC to 18:05 UTC Talk customers across all Pods experienced issues with no sound on new calls and difficulty connecting new calls. This affected both inbound and outbound calls.
Timeline
April 22, 2024 02:56 PM UTC | April 22, 2024 07:56 AM PT
We are currently investigating an issue where Talk customers are not able to hear their users on inbound or outbound calls. We will provide another update shortly.
April 22, 2024 03:16 PM UTC | April 22, 2024 08:16 AM PT
We have confirmed an issue that is causing audio to drop from inbound and outbound calls in Talk. We are investigating and will provide further updates in the next 30 minutes.
April 22, 2024 03:50 PM UTC | April 22, 2024 08:50 AM PT
Our Talk service provider has acknowledged that there is an ongoing issue with inbound and outbound calls, including silence on calls and difficulty connecting new calls. We will provide another update in 30 mins.
April 22, 2024 04:34 PM UTC | April 22, 2024 09:34 AM PT
Our Talk service partner is continuing to investigate the issue affecting Talk connectivity and audio quality. There’s no ETA at this point but we will provide another update in 60 mins or when we have more details.
April 22, 2024 04:44 PM UTC | April 22, 2024 09:44 AM PT
Our Talk service partner has confirmed that basic call functionality has been restored. Thank you for your patience during the investigation.
POST-MORTEM
Root Cause Analysis
This incident was caused by a failed autoscaling event with our Talk provider.
Resolution
To fix the issue our provider restored capacity manually.
Remediation Items
- Increase capacity buffer [Vendor][Completed]
- Review average call duration metric and explore possible alerting [Zendesk][Scheduled]
- Keep capacity at maximum operating levels [Vendor][Completed]
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, contact Zendesk customer support.
1 comment
Charlotte Kobler
Post-mortem published 03 May, 2024
0