Summary
On July 25,2025, between 8:00 and 14:00 UTC, we received 5 reports of customers hosted on Google Cloud Platform experiencing issues with agent status values not updating correctly.
Timeline
July 25, 2025 13:28 UTC | July 25, 2025 06:28 AM PT
We have received reports of issues affecting the GCP environment and our engineers are actively investigating. We apologize for the inconvenience and appreciate your patience as we work to resolve the matter. Updates will be shared as soon as they become available.
July 25, 2025 14:38 UTC | July 25, 2025 07:38 AM PT
Our engineers have identified and resolved the issue preventing customers in GCP from seeing up-to-date agent status values. Please confirm on your end if the issue is now resolved. A prompt response would be greatly appreciated and will help us assist you further.
August 01, 2025 09:22 UTC | August 01, 2025 02:22 AM PT
We are pleased to inform you that the issue affecting customers hosted on the Google Cloud Platform, which prevented the display of up-to-date agent status values, has been resolved. Our engineering team has completed the recalculation of all impacted accounts and confirmed that the issue is fully mitigated. We appreciate your patience and understanding while we worked to address this matter.
Root Cause Analysis
This incident was caused by a problem with the agent status page for customers using Google Cloud Platform, which caused incorrect information to appear. This was due to a slow database query pulling too much data.
Resolution
To fix this issue, the team limited the data used and optimized the query. Engineering also corrected any inaccurate information from the incident as part of the fix.
Remediation Items
- Set up alerts to notify the team by email if data stops coming in or if there is a delay in the system.
- Reduce unnecessary alerts from the Kafka system to focus on the most important issues.
FOR MORE INFORMATION
For current system status information about Zendesk and specific impacts to your account, visit our system status page. You can follow this article to be notified when our post-mortem report is published. If you have additional questions about this incident, contact Zendesk customer support.