SUMMARY
On April 24, 2024 from 13:37 UTC to 16:40 UTC, mobile SDK customers may have experienced errors when initialising or connecting to the mobile SDK, resulting in the inability to receive messages from their end users.
Timeline
April 24, 2024 04:20 PM UTC | April 24, 2024 09:20 AM PT
We have confirmed an issue causing errors and inability to access messaging powered by mobile SDKs, which could cause end-users to be unable to connect. We are investigating a possible roll-back and will provide further updates shortly.
April 24, 2024 04:35 PM UTC | April 24, 2024 09:35 AM PT
Our team continues to investigate paths forward for the issue causing errors and the inability to access messaging powered by mobile SDKs. We will provide additional updates in the next 30 minutes or when we have new information to share.
April 24, 2024 04:48 PM UTC | April 24, 2024 09:48 AM PT
We are happy to report that the issue causing errors and the inability to access messaging powered by mobile SDKs has been resolved, and end-users are able to connect to send messages. Thank you for your patience during our investigation.
POST-MORTEM
Root Cause Analysis
This incident was caused by a software update release where the mobile SDK API was removed as a location from our internal routing gateway. While the initial assessment was that this location was not being used, it was later found that it was accessed by a specific routing feature in our system.
Resolution
To fix this issue, new code was deployed to re-enable the correct configuration. After continued monitoring, recovery was observed.
Remediation Items
- Review procedure for safely removing the mobile SDK API location [Scheduled]
- Remove specialised routing from any systems that have a completed rollout [Scheduled]
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, contact Zendesk customer support.