SUMMARY
On February 28, 2025, from 10:29 UTC to 17:50 UTC, some customers on PODs 23 and 27 experienced issues with the Type and Due Date dropdowns in the Support ticketing system. They were unable to open the dropdown, and the options were not being populated correctly.
TIMELINE
February 28, 2025 05:52 PM UTC | February 28, 2025 09:52 AM PT
We have confirmed a recent update which caused the issues with the "Type" dropdown field, and have rolled back that update. After reloading your browser, please let us know if you continue to experience any issues.
February 28, 2025 05:48 PM UTC | February 28, 2025 09:48 AM PT
We have confirmed an issue affecting the "Type" dropdown field, causing the dropdown options to not show up. Our team is investigating and we will provide additional information as soon as we have it.
February 28, 2025 05:34 PM UTC | February 28, 2025 09:34 AM PT
We are receiving reports across several pods of issues selecting options from drop-down fields, including some default fields like "Type". We will provide further updates shortly.
POST-MORTEM
Root Cause Analysis
This incident was caused by a compatibility issue arising from the deployment of updates to the Type and Priority fields in the ticketing system. Specifically, when an account that had previously supported ticket forms was downgraded to a plan without this feature, the system failed to properly sync the backend queries, resulting in incomplete data being presented in the UI. This inconsistency led to the Type dropdown not populating correctly for affected customers.
Resolution
To resolve this issue, the on-call team rolled back the deployment to the previous stable version, effectively reverting the changes that caused the incident. This action restored normal functionality to the Type dropdown and addressed the related issues with the Due Date field.
Remediation Items
- Implement a check in the UI to bypass form responses when the account does not have the ticket-forms feature, ensuring all active ticket fields are loaded correctly.
- Modify the backend to associate all ticket fields with the default form when no forms are available in an account, preventing incomplete data from being presented in the UI.
- Increase test coverage for smoke tests to include scenarios where accounts are downgraded, ensuring that similar issues can be identified before deployment.
- Create additional alerts for monitoring changes to the Type and Priority fields to quickly detect any future discrepancies.
By addressing these remediation items, we aim to prevent similar incidents from occurring in the future and enhance the overall stability of our ticketing system.
FOR MORE INFORMATION
For current system status information about Zendesk and specific impacts to your account, visit our system status page. You can follow this article to be notified when our post-mortem report is published. If you have additional questions about this incident, contact Zendesk customer support.
1 comment
Bob Novak
Post-mortem published March 6, 2025
0