On February 4, 2020 from 22:30 UTC to 02:48 UTC customers on Zendesk Guide across all Pods were unable to create or edit Guide articles. The Guide agent interface displayed 500 errors and was unable to load during this time.
03:16 UTC | 19:16 PT
We're happy to report that the issues impacting creating/editing articles in Guide have been resolved.
02:31 UTC | 18:31 PT
Our team is continuing to investigate issues with editing and creating Guide articles on Pods 13, 15, 17, 18, 23, & 25. Thanks for your continued patience
01:53 UTC | 17:53 PT
We’ve received reports of customers being unable to create/edit articles on Guide. Our team is investigating, more info to come
Root Cause Analysis
This incident was caused by an escaped defect in a diagnostic endpoint which caused it to break when a database configuration was refreshed. A service was unintentionally delisted causing the Knowledge API to be unable to connect to this dependency and present errors.
To fix this issue, the service was redeployed to bring the Knowledge API back and restore Guide functionality.
- Corrected failing diagnostic endpoint to ensure open connections to the database after database configuration changes
- Update the service to the latest version
- Reviewed and updated classification of the services affected
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.