SUMMARY
On August 24, 2021 from 11:34 UTC to 12:38 UTC, Sell customers trying to call our public Sell API endpoints received 401 error responses, or in some cases errors stating they are over the API rate limit.
Timeline
12:22 UTC | 05:22 PT
Our team is investigating customers receiving 401 errors on Sell API endpoints.
12:44 UTC | 05:44 PT
Our team is seeing improvements for the 401 errors impacting customers calling Sell APIs and continue to monitor the situation.
13:04 UTC | 06:04 PT
We’re happy to report that the issues impacting Sell API endpoints have been resolved. We appreciate your patience with this matter.
POST-MORTEM
Root Cause Analysis
This incident was caused by a procedural error when updating internal application authentication services. A missed step prior to deploying an update to the service caused the API errors experienced by our customers.
Resolution
To fix this issue, we updated all necessary keys. Once this was completed, functionality returned to normal.
Remediation Items
- Our team will update their processes to make the steps leading to this issue easily avoidable.
- Our Sell engineering team is also reviewing all their other complex processes to prevent any other potential issues like this one.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.