How can I combat spam submitted via web service?

Return to top
Have more questions? Submit a request

99 Comments

  • Scott

    First off: ZENDESK NEEDS TO FIX THIS. PERIOD.

    It's irresponsible at best to have such an easy exploit open in every customer account by default. It's utterly absurd that customers can't kill off the "Web Service" channel with a simple setting. 

    Get your act together.

     

    Now, we need to get things working right. So, here's how I've mitigated this attack using the techniques others have discussed in prior posts. It takes two triggers to do it. (One new trigger and an edit to your automated response trigger.)

    This will work for you if you aren't using any of their Web Service stuff for tickets. 

    First Trigger:

    The first trigger I set up snags anything created via their "Web Services (API). This is a new trigger created in Zendesk as follows:

     

    This does 2 things:

    1. Marks the ticket "Solved" (I might change this to "Closed" after I'm 100% comfortable with it working as designed.)

    2. Applies a "tag" for these spammy tickets  (I used a tag I called "Instagram spam" since most of these spams seem to mention Instagram for whatever stupid reason. You can use whatever you like.)

     

    IMPORTANT NOTE: You'll want this to process first, so be sure to move this trigger to the FIRST position in your triggers so that it's the first thing Zendesk processes on all new tickets.

     

    Move a trigger to the first position by clicking the three-dot icon on the right-hand side and selecting "Move to first position".

     

    Second Trigger

    Next, you're going to need to stop Zendesk from emailing a notice to the spammer's target. That's their whole motive, so we want to make sure these spam tickets don't get that benefit!

    Do so by filtering on the new tag we just applied to the bad ticket:

     

    This has the impact of stopping Zendesk from sending the "Notify requester" email for tickets that come via the spammy channel (Web Services).

    You should repeat adding this tag-based filtering for other triggers that send out emails, like ones notifying your agents.

     

    This filter combo seems to be working OK for me. 

     

    Unfortunately, Zendesk's customer support didn't provide this detailed of a solution, which is ironic, but whatever. 

    Hopefully, these notes help someone else out there!

     

    Still, I must re-iterate: 

    ZENDESK, YOU REALLY NEED TO FIX THIS ASAP. 

    This whole fiasco completely embarrassed me as I'd just vouched for how GOOD Zendesk was with my boss and then this happens. Ugh. Not cool. Not cool at all.

     

    Thanks,

    Scott

    10
  • Patrick Townley

    Rather than these workarounds, how about an actual fix?

     

    Please add an option to disable unauthenticated/anonymous API ticket submissions only, without affecting other channels.

    10
  • Patrick

    I am glad that I left my spam catch triggers in place, since the spammers tried again early this morning. They only created 4 tickets before they presumably realized that it wasn't working.

    So Zendesk, are you ever going to plug that big hole in your API security or are we just expected to deal with random scammers generating junk tickets whenever they feel like?

    That notice email you sent last week is ridiculous. How about you make it so that the API requires an authenticated connection for tickets to be created.

    4
  • Patrick

    My company is also experiencing an ongoing attack.

    Apparently, Jim and I had a similar idea. Currently, as temporary fix, I have set up a similar trigger that looks like this;
    We have customized all of our auto-reply notifications so that they will not fire if the ticket has the "notified" tag. This trigger is in the first position and is set to apply the "notified" tag to any tickets that match the above conditions, as well as assign the ticket to my agent account.

    15 new spam tickets have come in since I set this up, and all of them have been assigned to my user agent account and prevented from sending out a notification.

    Thankfully, I mostly do internal tech stuff, so my zendesk getting filled with spam isn't an issue. After the attack we can comb through them and see if any legitimate messages got caught up in it.
    For now, I feel like adjusting your auto-reply tickets to follow a similar system is a better work around than this nonsense.

    Hopefully this will teach zendesk that they need to fix this exploit.

    4
  • Martin Cox

    Looks like the Zendesk spam filter is now classifying these bogus tickets as a "Malicious pattern" and tossing them into the Suspend queue.

    3
  • TonyLarson

    We have received roughly 50 of these messages in the last 12 hours. I would prefer a system wide solution rather than workarounds that will eventually reduce spam. 

    This wasn't an issue for us until last night. 

     

    3
  • Yu Ng

    has this been fixed yet? we just got hit with a ton of spam API tickets, asking us to remove the placeholders is not a good solution, it's been months since you guys posted this! still not fixed?! that's unacceptable!!

    3
  • J.H.

    One thing you should all do is report the domains to the domain Registrar -- for bitbiz.xyz that is Namecheap -- abuse@namecheap.com. The more reports, the faster they will respond and take down the site.

    Any time you deal with spam, the people hosting it should be notified. While I'm sure Zendesk is working on this too, it definitely helps when there are multiple reports. It will go a long long way to stop the behavior. Bitbiz is also touting instagram services, so report to instagram too.

    Also, why are you trying to circumvent this recommendation? Having a configuration that you knowingly are aware can be used as a way for malicious actors to send spam is dangerous and being irresponsible as a netizen.

    Doing the above will minimally effect you or your customers, so you should reconsider your "workarounds". Its your company domain names that are being used to send spam -- you should eliminate the possibility of it happening.

    3
  • Jonathan March
    Community Moderator

    Zendesk, could you please clarify how it's possible for a malefactor to use the API without having possession of an API Token?

    >  If you view the events of the spam ticket (see Viewing all events of a ticket) and look to the very bottom of the page, you’ll see that it was submitted via Web Service. This indicates it was created via API

    3
  • Jonathan L

    Martin sorry I misunderstood what you said, ok is this a confirmed fix across all PODS? I have shut down the support desk over the last 48hrs as it got so bad ... a target threatened me with reporting me to the ICO even though I didn’t send the message.

    I agree API changes should be planned but they have known about the issue for 12 months, no action has been taken apart from a half baked work around that doesn’t fix the problem.

    I might come across as a little stroppy but my business cannot function properly without a support desk solution that works properly. I’m not new to the platform I have been using it for 8 years solid with over 19000 tickets across 2 accounts. Spam is frustrating and distracting for engineers on a daily basis.

    Let’s face it.. life is hard enough without clearing spam all night when we should be down time.

    2
  • Uri Argaman

    Thanks Devan - Community Manager. I turned on the login and the spam stopped but I had to turn it back on as I need my clients to submit tickets without logging in.

    This is a security vulnerability that Zendesk is not addressing. Allowing an open API to do anything other than Read does not match the security standards of our time.

    2
  • Jim Stalder

    pstrauss -  Zendesk does allow the ability to create tickets via anonymous methods.    The API token and Oauth is for read, but not write access.   Specifically, https://developer.zendesk.com/rest_api/docs/support/requests#create-request     Us admins can turn this ability off via end-user settings (https://support.zendesk.com/hc/en-us/articles/203663806)     However, in turning it off, you also disable the ability to receive email from unregistered sources (e.g. new customers).   In my perfect world, Zendesk would differentiate between these two methods for anonymous requests and allow individual settings for each method.   Spammers could be doing the same thing via email, but that hasn't been an issue for me yet....

    *disclaimer.  Above are my opinions and understanding of how things work.   I could be incorrect.

    2
  • TonyLarson

    I used a combination of Scott and Jonathan's tactics above to eliminate our issues without the need to permanently modify our end user 'notify requester' trigger. Our users are accustomed to seeing their request content in the ticket receipt, so I wanted to avoid adjusting that. 

    Instead, since we do not utilize the API, I have a top line trigger that places tickets submitted via the API into a group that only I monitor, and have set the 'notify requester' trigger to ignore anything in that group. No notifications will go out to any tickets submitted via the API, so presumably the spammers are no longer interested. This eliminated the spam right away. 

    That said, this solution is problematic for anybody using the API, or if we decided to use it again. I think a cleaner solution from Zendesk that would allow us to turn off unauthenticated API submissions would be ideal. 

    2
  • Jim Stalder

    Specifically, I do this (with some additional variations) and make this my first trigger.    This way, I don't have to remove all the placeholders as Zendesk suggests above.   Of course, this only works because we don't use the API method to create tickets....

    2
  • Jason Wallis

    Mine are now going into the "suspended tickets" status and folder.... isn't that problem solved?  By Zendesk - thanks Zendesk!

    Unless I'm mistaken I don't see how they're still sending the response emails - Jonathan L are you seeing that yours are still immediately responding before they get captured as suspended?  I had to unsuspend by recovering a ticket to check and it seemed as though it only sent the auto response once I recovered it. 

    I'm assuming this fix by Zendesk

    1- doesn't send the email that the spammers want in the first place

    2- doesn't mess up my statistics for solved cases etc

    3- didn't require me writing any triggers

    4- puts them all into a nice little folder where I can see them and laugh at their feeble efforts to hijack my email

    unless I'm totally wrong about the above . . . 

     

    THANKS ZENDESK!!!

    2
  • Uri Argaman

    Having an open API with no security is a serious backdoor Zendesk is leaving open. I want clients to submit tickets from the Help Center Zendesk provides but I am flooded with spam instead. This API should be closed to anyone outside the Help Center.

    2
  • Jonathan March
    Community Moderator

    Many thanks for the info Ryan, makes sense.

    Zendesk, since we have the captcha safeguard for anonymous tickets submitted from a web form, it seems that it would be useful to support another setting to disable anonymous tickets submitted directly from the API. (I recognize that this would probably be non-trivial to implement!)

    2
  • Ben Appell

    Stephen - that's the exact problem I was having. I contacted Zendesk directly about it and they put me in some filter list on their end to push out tickets with the URLs in the requester field, but until then, I was able to find patterns of russian words in the subject line and set up a ruleset like this to sort them out of the queue:

    2
  • Sheryl T

    We have 9 more spam tickets since I last wrote.  I am opening a ticket with ZenDesk now and suggest that you do the same if you are receiving spam.  They do not respond to our messages here, but they will reply to tickets.  Thanks everyone!

    2
  • Jonathan March
    Community Moderator

    Patrick Townley

    Similar to yesterday's comment by  Sheryl T , I would recommend NOT setting to solved, but rather moving to a Spam Holding group. Then periodically select everything in that group and explicitly report it as spam. This will delete the tickets (avoiding skewing your stats) and report them to ZD as spam which at least in theory could have an impact on their spam filters at some point.

    2
  • Chris Johnson

    Please also keep me alerted to the status of this fix, we are also getting hit hard today.

    2
  • Martin Cox

    Agreed in terms of the primary issue not being solved. Disagree in terms of them using our portal to send on behalf of our name. They're stuck in the Suspend queue.

    I have to imagine that an  API change can't be taken lightly as to the overall impact of already deployed applications. One of those things that's probably easier said than done.

    2
  • Jonathan L

    That doesn’t solve the issue, it’s still sending them to the end user or the attackers intended destination as your company name ...

    2
  • Ryan
    Zendesk team member

    Hey Donato Dileo , Changing the  ticket creation trigger so it cannot be used as an Open mail relay does indeed prevent your email and account from being used as a conduit for spam. The spammers who are affecting your account will no longer have any incentive to target you.  You are correct -- It doesn't prevent the tickets from being created, but it helps to not make your account a target. This is the best solution to avoid spam.

    Jonathan L I'll be glad to look into that ticket for you, and reach out to you again. To clarify, you are more than welcome to keep your placeholders within your ticket updates (so, on a reply, they would receive the entire thread again) -- Keeping the ability to relay spam, regardless of channel, leaves you vulnerable with this configuration. Changing it as the instructions state will help, and be a minimal change. Write into support and look at the Message you receive from us -- Feel free to use that as a template.

    Jonathan March Thanks for that -- I've noted your feedback and am doing my best to relay it to that right people. Please know we're looking into a well rounded solution, but there unfortunately isn't a quick and easy answer. I will update this comment thread if I have any news -- otherwise feel free to write in and we should be able to provide some information.


    2
  • Dan Mørkbak Sørensen

    Brett - I hoped to go on vacation now.. instead i probably have to manually delete mails all weekend so my support-team does not show up on monday to 1000+ tickets - im not impressed - why is it even possible to add tickets via a "web service" without an API key. Should this not be locked down to only allow this from the help center - thus protected behind captcha? 

    I created a Ticket with "Major Impact" as level - have heard absolutely nothing back from your end. I tried opening a support-ticket via chat - no no avail - agent could not help either, just referenced the same 2 articles already seen.

    2
  • Jonathan L

    Ryan, we aren’t the target a poorly designed Zendesk api that’s left open is the problem.

    The fix needs applying to the entire ZD platform not just the odd account. This is a ZD issue not ours! Sort it out before you loose customers! If this is not resolved within the next 24 hours we will be moving to Help Scout as they will migrate our data and the system is more secure.

    2
  • Jonathan L

    3 months later the issue is still not fixed, this is not good enough! My support desk has been hammered all day!

    Instead of messing around releasing new half baked products how about you keep the core support desk solution secure!

    Remember many of your customers have been with you for many years, it appears currently the ZD management team has lost sight of the vision and mission!

    2 important products 1)Support 2)Guide

    Everything else is obviously a distraction!

    2
  • Service

    We are too have been impacted by a high volume of spam tickets via the API. Surely Zendesk should have the knowledge and tools to stop these calls to their own API. 

    1
  • Sheryl T

    I had 2 more in the middle of the night that did NOT go to Suspended but went to my custom SPAM view. I had nothing in Suspended overnight. I shut off my Notify Receiver of Received Request trigger on Saturday night.

    1
  • Ronny Hofsøy

    Unfortunately, Scott’s very clever mitigation suggestion is not possible if you`re using Essential due to the fact that Triggers on that plan is not that advanced. 

    The ZD plarform has effectively been rendered useless for both our company, but even worse, our many client that use ZD on our recommodations. Spam is 100:1 as of now, and is causing agents to a level of frustration I have never experienced before.

    ZD, please implement a fix asap.

    I wonder if this can and/or will be fixed, or if we have to shut down ZD and terminating the contract entirely.

    1

Please sign in to leave a comment.

Powered by Zendesk