Business Intelligence Challenged by Social, Mobile Data
A recent IBM survey shows that 76% of CEOs view “insight and intelligence” as key to corporate success over the next 5 years – slightly behind “getting close to the customer” and “people skills” – and that, in response, 83% of CIOs correctly perceive that leveraging business intelligence (BI) is the best way to implement these strategies.
But what's the best way to leverage BI? A recent Gartner report, flawed or not, suggests that many business intelligence projects fail, or are perceived as doing so. And the risks of failure, and consequences of success, are not negligible: another IBM study indicates that improving business agility by such strategies as “agile BI” can lead to long-term improvements in top- and bottom-line business metrics by as much as 60%.
How do successful BI implementers operate? A recent MIT study suggests that they create and maintain a “data-driven decision process.” In other words, best-in-class BI implementers consciously view their organizations as machines that scoop up information and funnel it to decision-makers, and so they design a machine that will carry out such a process as quickly as possible, with as little information loss as possible, and with the right networks to aim each piece of information at the right decision-maker. To support this machine, they buy software and hardware that will not only process fast and accurately, but also have the smarts to figure out who can use the information best, and pre-analyze for the target decision-maker.
Now, however, this beautiful model of how to operate, developed over the last 20 years and well served by existing business intelligence firms, may be breaking down. The problem is that much of the valuable information about a firm's customers, suppliers and environment exists outside the firm but is not being captured by the existing machine – and more new types of information are arriving out there, faster and faster. The Other Category of information, not handled by existing data warehouses, is now beginning to determine corporate customer relationships – social information as in Facebook, sensor information as in Google Maps, smartphone information as in cellphone videos and snapshots, or rapid-fire text conversations via tweets and texting. How should IT buyers keep the machine from breaking down as it's asked to handle information it was never designed for, or, even worse, churning away at information that is less and less relevant to the needs of the firm?
It is hard to find an answer among the clouds of cloud BI hype, the flashbulbs of BI vendor hype, and the distractions of hot-topic hype. But I believe that certain principles will serve the best-in-class BI buyer well, now and in the future.
A Modest ‘Other' Proposal for Business Intelligence
I would propose three additional criteria for the BI buyer who has exhausted the usual criteria of fit to purpose, scalability, robustness, flexibility, and conformance to government and business requirements. The first is: more truly agile is better than less truly agile. That is, a BI solution that helps the firm make itself not only faster to respond to the customer but also more effective in anticipating customer changes is to be preferred.
An example might be the whole hot topic of agile business intelligence that I have discussed in the past. As others have noted, some firms try to “do agile on the cheap” by setting loose programmers with any kind of so-called agile development process to constantly customize the vendor's existing – or latest – BI solution. Others seek to complement existing BI solutions with agile development, constantly involving customer input and vendor advice, with the responsibility shifting from the vendor to the firm as IT gains experience. Commentators have noted that this kind of approach tends to work out better, and that vendors who can support agile advice and responsibility shifting are to be preferred.
The second criterion is: third-party is better than one-stop lock-in. I don't mean that you shouldn't buy from a BI vendor that can supply everything you need in-house. No, what I'm talking about is the likely result in your architecture of adopting the BI vendor, and the attitude of the vendor. Will your choice of a vendor's solution component foreclose some other databases (say, noSQL databases), or does your vendor try to use the sale to crowd out all other vendors in a particular area, as, say, Oracle has been accused of doing in databases in the past? The ability of the vendor to handle today's Other Category data (e.g., unstructured and semi-structured Web Big Data) and integrate with the Web sources supplying that data – and the open-source tools that can also provide access to that data – is a good test of whether the vendor is open to present and future third parties.
And finally, my third criterion would be: How well does the vendor handle the “other other category”? Because the fact is that there will always be an important other category of information that you will need to seek out, no matter how much of it you cover today. Necessarily, today's business intelligence solutions, having focused on structured data, will focus only on certain parts of today's Other information – social media, Hadoop – that are easier to aggregate, and more important right now. For example, IBM's present definition of Big Data provides a separate Big Data platform, but that platform draws its information most easily from such sources as geospatial and Facebook data stores, not smartphone texting. A vendor who supports open-ended tools such as agile development environments, data integration with unstructured data support, and BI-driven CRM is more likely to allow you to evolve your BI as new types of information arise and become important.
IT Buyer Bottom Line: Let the Customer Hug You Back
The most popular current CEO strategy – “getting close to the customer” – often seems to the BI implementer to mean “hug the customer more closely”: respond more quickly to new information from the customer, analyze it better so as to understand the customer, and ensure a longer and longer relationship involving more and more “add-on sales.” But the importance of today's Other Category to business success indicates that there is another side to the customer relationship: following the customer as the customer's needs and wants change over time. That, after all, is why the Other Category arose in the first place – because the advent of the Web provided a new, faster-changing customer environment that outdated the business' data warehouse model.
Thus, the IT buyer cannot assume that simply implementing a Big Data platform, or a series of cloud BI projects, will be a long-run best practice. Instead, the IT buyer should attempt to assure, by adding my three criteria or in some other way that future Other Categories will be handled better.
The same surveys that show CEOs' ideas of successful business strategies also show that they view the environment of the business, not the business itself, as the source of the greatest business risk – because it keeps changing faster and faster. As it does, customer needs and wants will inevitably do so as well, and probably faster and faster. Your business intelligence that analyzes these needs and wants must be open to the customer's indication of those changes – which often show up as information in an Other Category. And if you want to hug the customer closer, you need to ensure that the customer's changes result in the customer finding you to be an even better fit for purpose, and thus hugging you better. To do this, pick business intelligence solutions that will continue to handle the Other Categories of the future. Your customers may well hug you for it.
Wayne Kernochan of Infostructure Associates has been an IT industry analyst focused on infrastructure software for more than 20 years.
Wayne Kernochan has been an IT industry analyst and auther for over 15 years. He has been focusing on the most important information-related technologies as well as ways to measure their effectiveness over that period. He also has extensive research on the SMB, Big Data, BI, databases, development tools and data virtualization solutions. Wayne is a regular speaker at webinars and is a writer for many publications.
