SUMMARY
On August 7, 2021 from 0:58 UTC to 3:53 UTC, Zendesk Talk customers on Pods 13, 15, 17, 19, 23, 25 experienced problems with inbound and outbound calls, being unable to make outbound calls or return the calls to the queue.
Timeline
05:02 UTC | 22:02 PT
We are happy to report that the issues impacting inbound and outbound calls in Talk has been resolved. Please let us know if you still experience any issues. Thank you for your patience and understanding.
04:08 UTC | 21:08 PT
Our Talk partner has reported full recovery at their end. Please refresh your browser, and let us know if you are still facing any issues.
03:43 UTC | 20:43 PT
We are investigating issues impacting inbound and outbound calls in Talk. Refreshing your browser may alleviate the issues. More details in 30 mins or when we have further updates.
POST-MORTEM
Root Cause Analysis
This incident was caused by a degraded datastore in our telephony partner’s infrastructure.
Resolution
To fix this issue, our partner performed a failover on their system, bypassing the source of the incident. Once this happened, service resumed as normal.
Remediation Items
Service Provider
- Create additional tests and alerting on failures [In Progress]
- Develop more robust failover process and general resiliency in the affected system [In Progress]
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.