Summary
On October 8, 2025 from 16:05 UTC to 19:29 UTC, some Zendesk Support customers on Pod 27 using Messaging and Chat may have experienced issues with omnichannel routing affecting automatic ticket assignments. Symptoms such as ticket routing delays and failures may have occurred as well as impact to Explore reports during the incident period.
Timeline
October 8, 2025 18:30 UTC | October 8, 2025 10:30 PT
We are receiving reports of Omnichannel routing issues on Pod 27, causing tickets to not route automatically. Manual routing is still functioning at this time. Our team is investigating and we will provide further updates shortly.
October 8, 2025 18:45 UTC | October 8, 2025 10:45 PT
We have confirmed an issue with Omnichannel routing in Pod 27 causing tickets to not route automatically as expected. Manual routing of these tickets is still functioning in the meantime. We will provide additional information when we have a substantive update to share.
October 8, 2025 19:51 UTC | October 8, 2025 11:51 PT
Our team continues to investigate solutions for the issue preventing tickets from assigning through Omnichannel routing on Pod 27. Manual routing of tickets can help to work around some individual cases while our team works to mitigate the impact. We will post further updates when we have new information to share.
October 8, 2025 20:36 UTC | October 8, 2025 12:36 PT
We are beginning to see improvement from the issue preventing Omnichannel ticket routing for Pod 27 customers. Our team is monitoring recovery, but please let us know if you continue to experience any issues.
October 9, 2025 01:05 UTC | October 8, 2025 18:05 PT
We’ve observed stable routing performance for the last several hours following the earlier disruption affecting Omnichannel ticket routing on Pod 27. While indicators are positive, we are continuing to monitor closely before declaring full recovery. If you’re still experiencing issues, please contact our support team. We’ll provide another update as we progress.
Root Cause Analysis
This incident was caused by a configuration issue in our messaging system that allowed a bottleneck to occur due to a large message payload in the message queue on the affected partition. This issue stalled ticket routing for a group of customers. Further, this led to a backlog in our ticket assignment system, preventing new tickets from being automatically routed to agents until the problem was addressed.
Resolution
To fix this issue, our team unblocked the message pipeline which allowed message processing to resume.
Remediation Items
- Enable timeouts for message event processing [In progress]
- Replace audit API with better performing API to reduce timeout events [Scheduled]
- Update runbooks to improve incident response time [Completed]
- Update internal escalation policy to improve incident response time [Completed]
- Improve logging for messaging service partitions [Completed]
FOR MORE INFORMATION
For current system status information about Zendesk and specific impacts to your account, visit our system status page. You can follow this article to be notified when our post-mortem report is published. If you have additional questions about this incident, contact Zendesk customer support.