Service Incident - May 19th, 2022 - Explore | All regions - Missing data in multiple metrics

Summary

On May 19, 2022 from 16:00 UTC to 18:15 UTC, customers using Zendesk Explore experienced missing data for metrics in the Tickets, Ticket Updates, and Updates History datasets.

Timeline

17:37 UTC | 10:37 PT

We are investigating reports of some Explore metrics in the Tickets, Ticket Updates, and Updates History datasets returning 0 results. We will provide another update as soon as we can.

17:49 UTC | 10:49 PT

We have confirmed an issue in Explore causing several metrics across Tickets, Ticket Updates, and Updates History datasets to return 0 results. Our team is investigating and we will provide additional information as soon as it's available.

18:36 UTC | 11:36 PT

Our engineers have discovered a cause for the issue causing Explore metrics across several datasets to return 0 results, and a fix has been deployed. Please let us know if you continue to see any issues.

Root Cause Analysis

This incident was caused by an escaped defect in a newly deployed version of the reporting engine.

Resolution

To fix this issue, we deployed the previous working version of the service and recovery was observed soon thereafter.

Remediation Items

1. Create additional test cases covering the issues caused in this incident [Scheduled]
2. Create new monitoring and alerts based on the issue found in this incident [Scheduled]

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.