Summary
On October 22, 2025 between 16:54 UTC to 20:39 UTC, during a staged rollout of a new feature across multiple pods, some customers experienced delays, problems with message handling, duplicate tickets, and issues with moving tickets to the right place. These challenges affected how smoothly tickets were managed and caused some disruptions in automated and manual processes.
Timeline
October 22, 2025 17:57 UTC | October 22, 2025 10:57 AM PT
We have identified an issue with the AI Agent tickets rollout. We have had customers report issues with tickets not being escalated to Agents when they should have been. We have also received reports that AI-Agent tickets are created an AI Agent is not configured. The latter issue can be caused when basic responders are acting on pre-agent conversations. We recommend if you are seeing issues with AI Agent tickets to disable the feature following the steps here. We will provide updates as they become available.
October 22, 2025 20:00 UTC | October 22, 2025 12:00 PM PT
Our team continues to investigate the issue with AI Agent tickets escalation and routing functionality related to the AI Agent ticket rollout. Our recommendation remains to disable this functionality following the steps here if any issues are encountered. We apologize for any disruption this may cause, and thank you for your patience.
October 22, 2025 21:40 UTC | October 22, 2025 1:40 PM PT
While we have identified a small subset of customers with advanced routing configurations who experienced impact due to this incident, the majority of customers should not experience any difficulty with AI Agent ticket functionality. If you encounter any routing or escalation issues with AI Agent tickets, please reach out to our Support team for follow up and assistance in resolving these issues. Thank you for your patience during our investigation.
Root Cause Analysis
This incident was caused by a mix of software errors and gaps in the rollout process of the new pre-agent tickets feature. The main problems involved errors in how tickets were moved between stages, incorrect information from an external system, and timing issues that led to duplicate tickets. Incomplete reversal steps and setup oversights also contributed to the ongoing difficulties. Additionally, there were not enough early warnings or checks in place to catch these issues before they affected customers.
Resolution
To fix this issue, the team temporarily turned off the new feature for affected customers while working on fixes. They corrected data errors, improved how tickets were assigned and moved, and made other process improvements. The feature rollout then restarted slowly with extra precautions. For some customers, manual adjustments were made to fix ticket details and restore normal service.
Remediation Items
Standardize the information used during ticket updates to prevent errors.
Improve how ticket assignments are tracked to reduce the need for manual work.
Ensure important customer details are correctly added when tickets are passed on.
Adjust processes to avoid creating duplicate tickets due to timing issues.
Refine how new features are introduced and removed, with clear steps for turning them off safely.
Set up monitoring and alerts to identify problems quickly before they affect customers.
Improve guides and communication about the feature’s purpose and management.
Enhance testing methods, especially for safely reversing changes if needed.
Review the rollout approach to release updates in smaller, controlled stages by region or customer group to minimize impact.
FOR MORE INFORMATION
For current system status information about Zendesk and specific impacts to your account, visit our system status page. You can follow this article to be notified when our post-mortem report is published. If you have additional questions about this incident, contact Zendesk customer support.