Service Incident - September 13, 2022 - Explore | All Pods - SLA dataset not updating

SUMMARY

Between September 9, 2022 at 12:02 UTC and September 13, 2022 at 15:30 UTC, Explore customers could not see updated SLA data in the SLA dataset. 

Timeline

September 14 08:34 UTC | 01:34 PT

We are happy to report that the issue causing SLA tickets to not appear in Explore after September 9, 2022 has been resolved yesterday at around 15:00 UTC. Thank you for your patience during our investigation.

September 13 10:22 UTC | 03:22 PT

We are currently aware of an issue that is causing SLA tickets not to appear in Explore after 09/09/2022. We have deployed a fix for this and are monitoring the results. We will provide an update upon full resolution. Thank you for patience and collaboration.

 

POST-MORTEM

Root Cause Analysis

This incident was caused by an inaccurately scoped update which reduced the export limit in the Incremental API that delivers data to Explore. 

Resolution

To fix this issue, the API export limit was correctly scoped.

Remediation Items

1. Communicate any updates which affects Incremental APIs to Explore team.
2. Improve monitoring and alerting to reduce time to response and resolution.

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.