SUMMARY
Between November 15th, 2021 at 16:40 UTC and November 16th at 20:00 UTC, Talk customers may
have experienced call issues such as the inability to connect calls successfully, busy tone responses, andone-way audio for calls made between local Canadian numbers and a subset of Talk Canada phone numbers.
15:06 UTC | 7:06 PT
Our partner has confirmed they have resolved the cause of Inbound/Outbound call issues to/from Canada. We apologise for any inconvenience caused.
07:05 UTC | 23:05 PT
Our voice partner is now observing recovery of both Inbound/Outbound calls to/from Canada and they are monitoring for full recovery. We will provide another update once more information is available.
02:58 UTC | 18:58 PT
Our voice partner is continuing to observe recovery of Outbound calls to Canada. They continue to work with their carrier partner for Inbound call failures from Canadian Networks, affecting both Long Code and Toll Free numbers. We will provide another update once more information is available.
23:24 UTC | 15:24 PT
Our voice partner is observing recovery for outbound calls to Canada and continuing to work with their carrier partner for Inbound call failures from Canadian networks. We expect to provide another update in 4 hours or as soon as more information is available
22:16 UTC | 14:16 PT
We're aware of Talk issues impacting inbound and outbound calls for Canadian phone numbers. Our Talk partner is investigating. We'll share another update once we have more information.
POST-MORTEM
Root Cause Analysis
On November 15th, 2021 at 16:40 UTC, the upstream provider of out Talk partner's service provide experienced IP congestion and reduced processing capacity on their network which resulted in the inability to connect calls.
Resolution
The incident was resolved once the upstream provider of our Talk partner's Service Provider engaged their hardware vendor and implemented system-wide IP congestion protections.
Remediation
To avoid similar incidents in the future the upstream provider will research and implement additional options and alternatives to protect their systems and services
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.