Summary
On July 9, 2026, some customers experienced issues loading the Forethought Solve widget and saw elevated errors when the widget attempted to communicate with our backend services. The issue was caused by a software change that increased backend workload and led to timeouts.
Timeline
July 09, 2026 17:34 UTC
We are currently investigating an issue where the Solve Widget is failing to load for some customers. Our engineering team has identified a potential cause and is in the process of rolling back suspected changes to restore service.
July 09, 2026 17:41 UTC
The suspected changes have been successfully rolled back. We are currently seeing traffic recovering and the Solve Widget is beginning to load successfully for affected customers.
July 09, 2026 17:58 UTC
This incident has been fully resolved. The Solve Widget is now loading normally for all customers, and our monitoring confirms that performance and traffic have returned to stable levels following the rollback.
Customer impact
Between 16:59 UTC and 17:31 UTC on July 9, 2026, customers using the Solve widget across several pods experienced one or more of the following:
- The Solve widget failed to open or load
- The Solve widget displayed frequent error messages
- Requests to our Solve API returned elevated errors
- Some customers also experienced degraded access to related dashboards and analytics
During this window, customers could not reliably use the widget.
Root Cause Analysis
A recently released change introduced a bug in a synchronization flow used by the widget. Under certain conditions, the widget and backend services retried and repeated work more than intended.
This led to:
This led to:
- A spike in “cache misses” (the system could not reuse previously prepared data)
- Increased fallback to slower backend data retrieval
- Saturation of backend capacity, causing request queues to build and requests to time out
These effects reinforced each other (retries created more load), resulting in elevated errors and widget load failures.
Resolution
We restored service by:
- Rolling back the problematic release
- Reducing background load that was competing for the same backend capacity
- Scaling up the Solve API service to clear backlogs and stabilize performance
After these mitigations, error rates returned to near zero and response times returned to normal.
Remediation Items
We’re making changes to prevent similar issues, including:
- Adding automated tests and safeguards for “non-existent conversation” scenarios
- Ensuring the widget does not continue polling/retrying in situations where it should stop
- Adding additional validation before performing expensive backend retrieval work
- Improving monitoring to detect early signals of this type of overload more quickly
FOR MORE INFORMATION
For current system status information about Zendesk and specific impacts to your account, visit our system status page. If you have additional questions about this incident, contact Zendesk customer support.