On the 26th of February, between 19:28 and 23:33UTC, our customers on Pod 20 saw a delay in target job executions.
01:26 UTC | 17:26 PT
Today, between the hours of 19:28 and 23:33UTC, outbound target jobs on Pod20 experienced a delay. The issue was resolved, all target jobs have been successfully completed and target payloads delivered. Apologies for the inconvenience.
Root Cause Analysis
Due to a large influx of unexpected, newly scheduled jobs, our mitigation systems moved an overflow of jobs to our backup job queue. As the backlog of jobs cleared, our customers saw a delay in target jobs being executed.
After our systems ran through the queued jobs, the backlog was cleared and execution times returned to normal. To avoid delays like this in the future, our developers will work on more robust and automated processes to speed up recovery.
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.