On July 19, 2019, from 2:49 UTC to 10:41 UTC, a subset of Zendesk Support customers on Pod 13 and 14 experienced the inability to see if other agents were looking at a specific ticket (aka. Agent Collision).
10:45 UTC | 03:45 PT
The issue impacting Agent Collision (Play Mode) for Support customers on Pod 13 & 14 is now resolved.
10:23 UTC | 03:23 PT
Customers using our Support product on Pod 13 & 14 may experience issues with Agent Collision (Play Mode). Our teams are working to remediate this. We'll provide an update in one hour.
Root cause Analysis
The agent collision service located on unused nodes was restarted in order to stabilize depending services that got restarted after a maintenance operation.
This impacted a storage platform used by a number of other services including the agent collision service which led to customer impact in that regards.
The agent collision service got redeployed on Pod 13 and 14 which re-established the functionality for the impacted agents
- Improve Agent Collision monitoring
- Improve Agent Collision service so it does not require to redeploy after failure
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.