SUMMARY
On September 28, 2021, between 8:57 and 12:45 UTC, customers using Zendesk Chat were presented with a Widget but were unable to begin a chat session.
Timeline
16:39 UTC | 09:39 PT
We are happy to report that the issue causing chat being unavailable in the Widget is now resolved. Please let us know if you continue to experience issues.
13:01 UTC | 06:01 PT
A fix was deployed for the issue causing chat being unavailable in the Widget and we are seeing recovery. We will continue to monitor the behaviour.
12:21 UTC | 05:21 PT
Our team continues to work on a fix for the issue causing the no chat to be offered up by the Zendesk Widget.
11:59 UTC | 04:59 PT
Our team has identified the root cause of the issue causing the Zendesk Widget to not offer the Chat option. We are working on a fix.
11:49 UTC | 04:49 PT
We are investigating reports of the Zendesk Widget not offering the option for chat. We will provide further information as soon as possible.
POST-MORTEM
Root Cause Analysis
This incident was caused by the Chat environment configuration variable being overwritten during a deployment.
Resolution
To fix this issue, we reverted the deployment, and corrected the faulty configuration. Recovery was observed thereafter.
Remediation Items
- Improve deployment process on environment configuration changes.
- Investigate better error messages to present to end users.
- Review any possible improvements into better alerting mechanisms for quicker mitigations of such scenarios.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.