5 Data Discovery Pitfalls
Page Contents
By Rado Kotorov
Data visualization is fast becoming an area of interest for organizations seeking to benefit from the wealth of structured and unstructured data in the enterprise.
Solutions are emerging to meet this need, but before companies make a purchasing decision, it’s critical that they consider the goals of the implementation and ensure they’re investing in the right technology for their specific need.
While data visualization makes information easier to interpret, understand and retain, discovery tools are not without their flaws and can be misused. Stephen Swoyer, an analyst for TDWI, warned that users of data discovery tools “will consume unvetted, inconsistent, or faulty information, or — worse yet — use this (faulty) information to generate analytic insights. The latter scenario could compromise the analytic ‘insights’ that directors, analysts, and managers use to enrich (or in some cases to drive) decision-making activities.”
Therefore, before deciding to purchase a standalone data discovery solution, take the time to thoroughly evaluate your situation and understand your need so that you don’t fall victim to one of the following data discovery pitfalls:
Poor Data Quality
Data visualization tools are only as good as the information that is inputted. If organizations lack an enterprise-wide data governance policy, they could be relying on inaccurate or incomplete information to create their charts and dashboards. Having an enterprise-wide data governance policy will help mitigate the risk of a data breach.
This includes defining rules and processes related to dashboard creation, ownership, distribution, and usage; creating restrictions on who can access what data; and ensuring that employees follow their organizations’ data usage policies. Many data discovery toolsets do not offer metadata management and data integrity solutions as part of the package. Purchasing these capabilities separately, rather than as part of a single comprehensive business intelligence (BI) solution, can cause unnecessary complications in business processes and workflows.
Advanced Analytics Limitations
Use of data visualization isn’t as widespread as traditional BI practices. However, as the technology matures there will be an increased demand for this advanced business intelligence functionality. For example, users want the ability to integrate unstructured data from social channels, or incorporate predictive analytics for improved decision making. As such, it’s critical that companies select a data discovery tool underpinned by a BI platform in order to be positioned to scale their business intelligence capabilities to address future data discovery and visualization needs
Rogue Dashboards
With every dashboard, you have to wonder: Is the data accurate? Is the analytical method correct? Most importantly, can critical business decisions be based on this information?
This is an era of data visualization “Wild West,” with rogue dashboards permeating the enterprise in a completely disjointed and uncontrolled fashion. Users modify data and change fields with no audit trail and no way to tell who changed what. This disconnect can lead to inconsistent insight and flawed decisions, drive up administration costs and inevitably create multiple versions of the truth.
Security also poses a problem with data discovery tools. IT staff typically have little or no control over these types of solutions, which means they can't protect sensitive information. This can result in unencrypted data being cached locally and viewed by or shared with unauthorized users. If the toolset doesn’t offer adequate data management and security, it’s not suitable for the enterprise market.
Hidden Costs
A common data discovery technique is to put all of the data into server RAM to take advantage of the inherent Input/Output rate improvements over disk. This technique has been very successful and spawned a trend of using in-memory analytics for increased BI performance.
Here’s the catch though: In-memory analytic solutions can struggle to maintain performance as the size of the data goes beyond the fixed amount of server RAM. For in-memory solutions, companies really need to hire someone with the right technical skills and background or purchase pre-built appliances – both are unforeseen added costs. An integrated approach as part of an existing business intelligence platform delivers a self-managing environment that is a more cost-effective option. This is of interest especially for companies that are experiencing lagging query responses due to large data volumes or a high volume of ad-hoc queries.
The Data Elite
If the goal is to maximize the ROI from enterprise information by enabling more people to benefit from it, that information must be made more accessible to a wider group. Limiting BI to a small percentage or group of workers can hinder broader adoption because it restricts understanding of data. In addition, data discovery tools alone are usually not the right approach for enterprises. By themselves, these tools can have limited usability, fail to scale easily and can’t satisfy operational demands for real-time information delivery.
Professional analysts will always want advanced tools. But the new breed of data-driven professional is also now seeking BI tools to assist with their everyday decision-making. These workers, including non-technical users, desire easy-to-use BI apps that make data visualization and other BI dashboards familiar and accessible. To make business intelligence pervasive in the enterprise, users of all types and skill levels should be able to explore and analyze data in the way they are most comfortable.
Progressive companies consider data to be a strategic asset and understand its importance to drive innovation, differentiation and growth. But leveraging data and transforming it into real business value requires a holistic approach to business intelligence and analytics. This means going beyond the scope of most data visualization tools and is dramatically different from the clunky BI platforms of years past.
Organizations need to move away from rogue, disconnected spreadsheets, dashboards and visualizations. Instead, they should move toward implementing a comprehensive BI platform to reap the benefits of state-of-the-art visualization capabilities and robust, scalable BI in a single solution.
About the author: Dr. Rado Kotorov is chief innovation officer at Information Builders. He works both with the Business Intelligence and the iWay product divisions to provide thought leadership, analyze market and technology trends, aid in the development of innovative product roadmaps, and create rich programs to drive adoption of BI, analytics, data integrity and integration technologies.
Wayne Kernochan has been an IT industry analyst and auther for over 15 years. He has been focusing on the most important information-related technologies as well as ways to measure their effectiveness over that period. He also has extensive research on the SMB, Big Data, BI, databases, development tools and data virtualization solutions. Wayne is a regular speaker at webinars and is a writer for many publications.
