SUMMARY
Between June 27, 2022 17:30 UTC to June 28, 2022 20:24 UTC, Zendesk customers hitting the WhatsApp API may have experienced message delivery failures when using certain attributes in templated messages.
21:16 UTC | 14:16 UTC
We are aware of an issue with delivery failures for some customers using certain attributes in templated messages through the WhatsApp API resulting in errors. If you are experiencing such errors please read more about how to resolve them here.
POST-MORTEM
Root Cause Analysis
At 2022-06-27 17:20 UTC, the SunCo team performed a batch migration of some ISV customers from using WhatsApp On-Premise to WhatsApp CAPI. The CAPI made some changes to the API to send WhatsApp message templates, namely that they will reject any requests whose payload contains extraneous properties. This caused issues for a subset of customers who were sending unnecessary fields, resulting in this incident.
Resolution
Once the issue was identified, we released a hotfix to make sure the error message was descriptive enough to act on and delayed further batch migrations to give customers time to adjust their logic.
Remediation Items
- Update WhatsApp alerting and monitoring
FOR MORE INFORMATION
For current system status information about your Zendesk, check out our system status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us via ZBot Messaging within the Widget.