On January 28, 2020 from 20:54 UTC to 21:49 UTC customers using Zendesk Talk on Pod 15 could not use voice features or the API for the duration of the incident.
22:50 UTC | 14:50 PT
Talk users on Pod 15 had experienced elevated error rates between 20:54-21:49 UTC. Service has been fully restored.
22:29 UTC | 14:29 PT
Talk errors on Pod 15 have stabilized and we are continuing to monitor performance. We’ll provide another update once we have more info.
21:59 UTC | 13:59 PT
We are currently investigating performance issues impacting Talk on Pod 15. We will follow up with another update once we learn more.
Root Cause Analysis
This incident was caused by a configuration issue within our deployment tool.
To fix this issue, we manually reverted to a previous configuration on the deployment tool to recover the services impacted.
To prevent recurrence of this issue, improvements have been made to our deployment process to ensure stability during system wide changes.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.