SUMMARY
On April 21, 2022 from 10:20 UTC to 14:26 UTC, customers on Pod 17 using the Zendesk Support product were not presented with ticket counts in the Views sidebar.
Timeline
14:15 UTC | 07:15 PT
We’re currently working to fix the issue where ticket number count is not appearing in Views for Support customers in Pod 17. We appreciate your patience. Next update in 30 min.
14:45 UTC | 07:45 PT
We have identified the cause of this issue and have applied a fix. Ticket view count for Support in Pod 17 should be visible now. Our team will continue to monitor.
15:01 UTC | 08:01 PT
We are happy to report that Views ticket count for Support customers in Pod 17 is now functional. Thank you for patience.
POST-MORTEM
Root Cause Analysis
This incident was caused by a CPU spike and throttling at the backend as a result of a sudden influx of traffic, which caused views not to be able to serve up the ticket counts.
Resolution
To fix this issue, we doubled the CPU capacity, as well as disabled the count feature to reduce the count queue. Once the queue cleared up, the count feature was re-enabled, and all customers were once again able to get their view ticket counts.
Remediation Items
- Investigate the speed of draining of the count queues [Scheduled]
- Improve documentation for the on call team for such similar scenarios [Scheduled]
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.