On August 23, 2021 from 5:19 UTC to 10:57 UTC, Support and Guide customers on Pod 25 may have seen “Oops” or cannot be accessed (500 errors) errors in the requests page where end-users can view tickets.
11:37 UTC | 04:37 PT
We’re happy to report that the errors affecting My Requests Guide page and classic ticketing functionality for some customers on Pod 25 has been fully restored. Apologies for any inconvenience caused.
10:48 UTC | 03:48 PT
Our teams continue to work on the fix for the error messages some Pod 25 customers are experiencing. We appreciate your ongoing patience.
10:17 UTC | 03:17 PT
We have identified the root cause for the error messages impacting some Pod 25 customers. Our teams are working towards a resolution.
09:56 UTC | 02:56 PT
We are investigating some customers receiving “Oops” errors on Pod 25. More information to follow.
Root Cause Analysis
This incident was caused by a misconfigured database shard during a scheduled database migration.
To fix this issue, the proper configuration was applied and the migration was redeployed.
- Additional validation added to database migration processes.
- Schedule database configuration check to run more frequently.
- Reduce error noise to catch similar issues more quickly.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.