Whither Embedded Analytics?
With all the hype about Big Data analytics, a more down-in-the-details yet highly useful new application of analytics has been obscured: embedded analytics. More specifically, I am talking about middle-tier analytics, the area that embedded analytics will aim for in the next three years.
This is not massive-data-store, in-depth analytics like the data warehouse, nor is it the “smart sensor” analytics that will increasingly come to the fore with the arrival of the sensor-driven Web (e.g., analytics on your iPhone). No, I am talking about medium-sized data stores, moderately in-depth analytics and in-enterprise business intelligence applied at the level of the department, local office, loosely-coupled storage array or server network.
Unlike systems management software, embedded analytics not only monitors and “fixes” but also analyzes what is going on, and reports this analysis either to the top-tier data warehouse or a specific set of end users and/or administrators. This analytics works best when it is embedded in other software or in firmware, operates semi-automatically to pick up business-process flows and alerts the business before they get out of whack.
State of Embedded Analytics
Up to now, the fledgling beginnings of embedded analytics have begun to show up in the systems management software of vendors like CA; but they are not separable pieces usable by other distributed software. Increasingly, major vendors like IBM are now talking about taking analytic software from business intelligence and analytics software suites and applying it to organizational operations across the board.
However, these often involve databases retrofitted to business intelligence in general and decision support in particular. What embedded analytics efforts like EMC Greenplum represent is the obvious next step: applying a database designed from the ground up and optimized for querying and analytics. These will inevitably be better suited than data management approaches intended to handle updates as well as queries and result massaging.
This is not to say that an embedded analytics database is the end point of embedded analytics evolution. Because most if not all available analytics databases were designed for the top tier, they are too “heavyweight” for their intended purpose. They perform more slowly, because they are tuned for much higher data-store sizes. However, whether the next turn of the market crowns a slimmed-down top-tier database or a ground-up-designed middle-tier analytics database as the winner, either one will really do.
Over the next few years, it is reasonable for IT buyers to expect some of this technology to arrive on their doorsteps embedded in upgrades of existing solutions – but far from all of it. At some point in this period, separable analytics solutions will show up that will allow the user to go far beyond what a particular vendor is offering – if, of course, IT wants to.
Why would IT want to do this? To handle areas in which one-size-fits-all vendors are simply not moving fast enough.
Take, for example, carbon accounting. Vendors have been proactive in this area, but some of the market is moving faster still, toward monitoring that picks up on and alerts to excess emissions as they happen and connects with the carbon accounting software when necessary. Likewise, as health care providers grapple with government mandates and electronic health records, they can see coming a day in which they will need to perform damage control on breaches of privacy — but today’s tools are much slower than they could be to detect such a problem. In either case, middle-tier embedded analytics that goes beyond most likely vendor offerings is needed.
The primary organizational benefit of this technology, therefore, is deeper real-time understanding of in-enterprise problems that leads to better decision making. And this is a very cost-effective application of analytics’ general ability to improve gross margins.
Potential Uses of Embedded Analytics
It merits pointing out that vendors have relative freedom to delay delivering, say, multi-vendor or open source middle-tier analytical databases, since it’s not high on IT wish lists. It could happen next week, or it could happen three years from now. So any IT acquisition of, and use of, this kind of embedded analytics will have to wait until the vendors get around to it.
At that point, the obvious application is per-project – improving a specific business process or case-management implementation. More than other technologies, embedded analytics does not require full, integrated organizational implementation to be maximally effective. Rather, it does just fine applied to a task, a process, a function, a locality, or a local or strategic initiative. IT simply looks down the list of business-critical projects and picks one that benefits most from risk management or analytical automation.
The critical success factor in such projects is rapid implementation and upgrade, caused by automation of the implementation/upgrade process, allowing strategic projects a head start. Right now, while most vendors do well at this, high-end vendors like IBM and EMC seem to be setting the pace. And so, choosing one of these vendors’ products in all likelihood means a better chance of rapid implementation and a database better fitted to a broad range of embedded-analytics tasks – not to mention better ongoing support for tricky cases.
Bottom Line for IT Buyers
The IT buyer should view embedded analytics as a technology that may take a while to materialize. However, when it does, the new embedded analytics will turn out analytics-type benefits at least equal to the whiz-bang high-end Big Data analytics now being sold – although those benefits will arrive in smaller per-project chunks. That, in turn, means this technology is definitely worth the IT buyer’s ongoing attention.
More specifically, the IT buyer might consider a “pre-pre-short-list” approach. That would involve identifying solutions that may wind up on the embedded-analytics short list in the next two years, and steadily moving those products in the pre-pre list over to the pre-short list as their technology reaches the point where it can be applied by IT rather than being embedded in another vendor solution
If everything goes right, and your CEO hits you with a requirement that really demands embedded analytics, you will be glad you had your embedded-analytics pre-short list in your pocket. That should happen sometime in the next year or two. As I said, down-in-the-details embedded analytics isn’t at the top of the hype list; but the benefits it delivers, if you upgrade to it in a timely fashion, are nothing to sneeze at.
Wayne Kernochan is the president of Infostructure Associates, an affiliate of Valley View Ventures that aims to identify ways for businesses to leverage information for innovation and competitive advantage. Wayne has been an IT industry analyst for 22 years. During that time, he has focused on analytics, databases, development tools and middleware, and ways to measure their effectiveness, such as TCO, ROI, and agility measures. He has worked for respected firms such as Yankee Group, Aberdeen Group and Illuminata, and has helped craft marketing strategies based on competitive intelligence for vendors ranging from Progress Software to IBM.
Wayne Kernochan has been an IT industry analyst and auther for over 15 years. He has been focusing on the most important information-related technologies as well as ways to measure their effectiveness over that period. He also has extensive research on the SMB, Big Data, BI, databases, development tools and data virtualization solutions. Wayne is a regular speaker at webinars and is a writer for many publications.
