Recent searches
No recent searches
Announcing enhanced Explore performance with a 37-month data retention limit
Posted Jan 08, 2025
Original article:https://support.zendesk.com/hc/en-us/articles/8356594785050-Announcing-enhanced-Explore-performance-with-a-37-month-data-retention-limit
Please give a quick overview of your product feature request or feedback and note who in your org is affected by this issue [ex. agents, admins, customers, etc.].
I was stunned when I saw this announcement.
This is not an enhancement, it's not anything, it's nothing but a downgrade.
If you make announcements like this after the fact, I would not have renewed my contract with your company.
Who is affected?
It's obvious. It affects agents, administrators, and management.
And there is something that has been bothering me for a long time.
I'm wondering why you decided to implement this measure.
The reason is not clear, and you are trying to cover it up with the vague reason of “to improve the speed of loading reports”.
Is it to reduce the load on the processing server? Is it to reduce your server costs?
We are paying you a large amount of money.
What problem do you see this solving?
As it is, I don't think your customers will have any problems unless you put restrictions on the data, right?
When was the last time you were affected by this lack of functionality, or specific tool? What happened? How often does this problem occur and how does this impact your business?
This will have an impact from now on, right?
Even if it's more than three years ago, it's still our customer's history.
So why are you restricting us from viewing that data when we're not doing anything wrong?
What would be your ideal solution to this problem? How would it work or function?
It is not to impose restrictions as they are now.
Thanks.
8
14 comments
Shawna James
Thank you for taking the time to provide us with your feedback. This has been logged for our PM team to review. For others who may be interested in this feature request, please add your support by upvoting this post and/or adding your use case to the comments below. Thank you again!
0
Jon Thorne
I completely agree.
It seems very odd to ‘sell’ this as an enhancement. Although we might see some small performance or load time improvements, really this seems to be because Zendesk haven’t got the processing power - which in this day and age just shouldn't be an issue. It seems to me that it’s a ‘Zendesk problem’ that Zendesk are making a ‘Customer problem’…
When we moved from Insights to Explore, we were expected to do huge amounts of work to rebuild all our reports and dashboards in the new system. Now we're going through another enforced migration with the new dashboard builder. These have taken huge amounts of time and resource for us as the customer.
Now, with this change, we're effectively being told that we can't go further back than 3 years with reporting on our own data (which we have to do with relative frequency in our business). Sure, we can download the data - but that's then a static download and we can't then sync it and manipulate it historically…which is surely the primary purpose of Explore in the first place.
Hugely disappointed in this and very much hoping that the decision will be reversed or that there's some kind of opt out on a per customer basis. I'd much rather have reports take a bit longer to load than to lose the capability to report retrospectively further back than three years.
12
Tim Grimshaw
Thanks Shawna, yeah, I have to admit that my initial reaction to this was that it's going to force us to move (all or some reports) to our custom reporting tool (Metabase) so that we can report on things like year-on-year trends.
I totally understand what the intent is here, and (for us at least) it will have medium negative impact - most of the reports that we run are using data from the last 3 years or so. Which also means we'd benefit from any performance increase as a result.
BUT… there's a small number of overview reports that leadership like to look at around year-on-year patterns and trends - and those few charts look at e.g. the last 7 years to see how we're changing over time. If this change goes ahead then we'd be forced to either move to Metabase completely for Zendesk reporting, or use a split system of Explore (for almost all reporting) then Metabase (for historic yearly reporting).
Is there any opportunity to instead of culling the data completely, give customers the option of an additional dataset? e.g. the default dataset would only have the last 3ish years of data, and by default reports would all use that set. We'd get the best performance for the majority of reports. BUT, if you needed to have one report showing historical data from e.g. 7 years ago, you could create a report that uses the full-historical-data dataset. Slightly slower maybe, but it can then be added to the required dashboard(s) and people can keep everything in Explore.
Thanks!
5
Stephen
Hi,
I am really disappointed with this announcement as well and definitely doesn't feel like a product enhancement. This reads as a direct loss of functionality and really put Explore in a confusing position.
This seems to actively discourage using Explore as the primary reporting repository for Zendesk data and is pushing towards data being sent to another source to report against effectively.
In addition, I don't agree that downloading data is the right path - the exec team in my company actively uses some Dashboards in Explore and the thoughts of suddenly losing data after a certain point renders the point of Explore as a reporting tool ineffective. I am fairly certain this is going to be point of contention within my organisation when I have to explain why we are suddenly losing access to report against older data in a system we are paying a lot of money for.
I really hope this decision is reversed - very disappointed all around on this one.
8
Kate
Chiming in to add that I agree that this announcement does not seem like an enhancement, but rather a degradation of features.
To take away the ability to query data older than 3 years is more impactful for some industries than others. We have regulatory bodies that we need to be able to provide holistic reporting too, and without the ability to easily query the entire history of our data, this will increase time, effort, and cost.
I agree with Tim that it seems like we should have two datasets. I similarly agree with Stephen that this seems to be actively discouraging customers from utilizing Explore as the default reporting tool for support data.
Ultimately, it seems like we will now need to budget to add a secondary reporting tool in order to meet our industry requirements. Which begs the question – why are we spending so much on Zendesk? We may as well begin looking for a cheaper support tool alternative if we have to duplicate funds for something Zendesk is no longer providing.
5
Carmen Jones
Can you please clarify that this is limited to statistics within Explore, for example, will our agents using dashboard still be able to access tickets older than the 37 months while working in the agent workspace dashboards? ie. if they want to refer to a historical email from a customer, will this no longer be available?
0
Eugene Orman
Thank you for sharing your feedback. We understand that this adjustment may raise concerns, and we want to assure you that your input is invaluable to us.
We'd like to offer an important clarification: This data retention policy will solely impact data within the Explore analytics tool. Other Zendesk features like Agent workspace, Search, Help Center and APIs will not be impacted.
The decision to implement data retention in Explore was not made lightly. It was a careful decision aimed at enhancing the overall performance and reliability of the platform. We observed up to 85% faster report execution times on the accounts with 37-month data retention. While everyone will experience improved performance and stability, accounts with higher volumes of data and longer history will likely see the most significant gains. This enhancement is crucial for providing a more efficient reporting experience for all users and enables Zendesk to continue enriching Explore with new data.
Our analysis shows that most users focus on recent data, with only 2% of the executed reports being filtered by the time ranges longer than 37 months. This supports our decision to introduce data retention for maintaining a high-quality reporting experience.
We appreciate your understanding and continued support as we strive to improve our product. Please continue to provide feedback and comments in this thread, we will respond in a timely manner. If you have technical concerns, please reach out to your Account Manager or a member of our Support team who would be glad to assist you.
Thank you for being a valued member of our community and Zendesk customer.
-8
Carmen Jones
Thank you for the clarification, we have had different responses from your support team, may want to check our support ticket https://support.zendesk.com/hc/en-us/requests/13249456
0
Jon Thorne
Hi Eugene, thanks for the response, although with respect I think the logic here is somewhat misplaced, particularly around this part:
‘Our analysis shows that most users focus on recent data, with only 2% of reports being filtered by the time ranges longer than 37 months. This supports our decision to introduce data retention for maintaining a high-quality reporting experience.’
I don't think that measurement shows the full story. We barely have any saved reports that go back longer than 37 months as a saved default; most are 12 or 24 months (so our account is probably even below 2%). However, that doesn't mean that we don't frequently use those template reports to change the duration and pull important historical data by adjusting the date filters, without saving. We need that long-term data, and we need to be able to manipulate it in Explore to give key historical insights and trends.
Even using the 2% figure at face value, 2% is not nothing. It also makes sense to me that most reports are shorter-term than that to focus on current metrics. But it doesn't mean that the longer term reports aren't extremely useful, even though there's fewer of them. I don't think it's a good way to justify the change, I'm sure others would agree, and I hope that feedback can be shared.
Please can consideration be given to an opt-out for this for customers who need a longer historical period, even if the default for the majority is to cut-off at 37 months? We don't mind a trade-off on slower speed, but it's business critical for us to be able to pull ad-hoc historical insights for longer periods.
8
Eli Towle
I would also like the ability to opt-out of this 37-month data retention limit, whether for an entire account or for individual reports. This change makes Explore largely useless for reporting on annual trends. Furthermore, I've never experienced performance issues with Explore, possibly because our ticket volumes are lower relative to larger companies. In that regard, this change is a strict (and major) downgrade for us.
3
Eugene Orman
Carmen, I checked your ticket with our Support team, and they have provided more details now. I hope this addresses your concerns.
Jon, you have a valid point. For sure, there are some useful reports that look at the time ranges longer than 37 months. However, this is the trade-off we need to make to provide good performance and reliability. Believe me, we considered and investigated many other solutions. Unfortunately, it is impossible to opt-out from this change per account basis because the data pruning mechanism is applied on the data source level that is account agnostic.
I want to clarify this data point “only 2% of reports being filtered by the time ranges longer than 37 months”. It refers to the report executions, so it means that only 2% of all report executions looked at time ranges longer than 37 months. For example, if you have a report filtered by 30 days but while using it in the dashboard, you filtered it by 4 years, this report execution is included in this 2%. I will pull the list of the reports that had a longer time range executed in your account and will send it to you via a proactive ticket.
Eli, thank you for your feedback. Your account is indeed not large enough to experience the performance issues with Explore. As mentioned above, it is impossible to opt-out from this change per account basis because of how the source data is stored.
-5
Tim Grimshaw
Is it possible to do the below? Or is the performance not based on the dataset?
3
Jon Thorne
Hi Eugene Orman, thanks for your reply.
I haven’t heard from you directly, but even if executions for 37 months+ are only 2% of all reporting, that again isn’t hugely surprising given the tendency towards recency in most daily reporting requirements. However, it’s still in my view a highly misleading stat to use, as that 2% - even though not frequently used - have outsized importance when they are for historical reporting. I just don’t think it’s a good metric to try to justify this change.
I can’t help coming back to my earlier thought on this: ‘really this seems to be because Zendesk haven’t got the processing power - which in this day and age just shouldn't be an issue. It seems to me that it’s a ‘Zendesk problem’ that Zendesk are making a ‘Customer problem’…’
Looking at the Announcement pages, the article announcing this change has more than twice as many downvotes as any other article in announcements in the past 6 months (which is as far back as I checked).
Customers clearly aren't happy about this at all. Surely there has to be a solution, or this decision has to be reconsidered? Zendesk can’t simply cut off everything more than 3 years old - it's not what anyone other than Zendesk has agreed, and fundamentally changes the utility of Explore / Insights.
7
Margarida Pereira
Hello Tim Grimshaw ,
We appreciate your interest in exploring potential solutions. Unfortunately, there is no solution without a trade-off. Splitting data into two datasets will create extra complexity but will not provide substantial performance gain because it will still be stored in the same database.
-4