Shift from Reporting to Analysis Requires Asking the Right Questions
It used to be so much simpler to hire a data analyst. You looked for someone who knew how to take data that had been loaded into and then extracted from a data warehouse and produced reports based on that data.
But the nature of data analysis is changing rapidly with the emergence of computing frameworks like Hadoop that facilitate working with large sets of data from diverse sources, not just the structured data stored in warehouses. In a video interview with Microsoft Director of Business Intelligence Bruno Aziza, Bitcurrent analyst Alistair Croll said that with the advent of new frameworks and new technologies, companies are storing more types of data, based on the assumption they will be able to derive value from it. He told Aziza:
In the traditional business intelligence world, you had a bunch of data and you knew what you were inserting into the data warehouse before you did it. You knew it was sales by region or numbers of widgets sold. … In the Big Data world, often you're storing the data just because. You're going to build algorithms of natural language parsing to extract the value from it. You're storing it on faith that there is something valuable hidden in there.
So what kinds of data analysis skills are important now? While the whole area is still pretty nascent, it's safe to say analysts will need to focus more on building algorithms and less on building OLAP cubes.
IT Business Edge's Susan Hall wrote about this shift in data analysis skills back in March, citing Enterprise Strategy Group writer Julie Lockner's description of “Big Data ninjas” as “covert agents that specialize in unorthodox arts of data wars – waging war against the volumes of data that could not be penetrated using traditional weapons of enterprise data warehouse methods and tools.”
In Lockner's somewhat hyperbolic terms, these folks are responsible for “slicing and dicing billions of rows of data in seconds, finding anomalies, trends, or clusters that would be considered a competitive advantage, turning that knowledge into corporate victory.”
Perhaps the most important skill of all is the ability to ask the right questions. Croll said data analysts should seek inspiration from Isaac Newton . As Croll told Aziza, “Newton was not the first person to get hit in the head with an apple, but he was the first person to wonder why.”
Susan got a similar take from Jack Phillips, CEO of the research firm International Institute for Analytics, when she asked him if user-friendly business intelligence software made statistical training less important. He told her:
I would argue that it's making it more important. On the one hand, you would think it's a little like Web design, I suppose. Now you don't have to know any HTML and you can just put up a Web page, seemingly without any real effort or background or education. In analytics, I think that is not the case. The software certainly is doing the hard number crunching. It is doing the data synthesis, but it then forces the user to then be that much more creative and knowledgeable, really about the questions that need to be answered and how best to answer those questions – which data sets to bring together, which data sets to correlate. …
Ideally, Phillips told Susan, data analysts would possess both “savvy in regression analysis, econometrics, mathematics” and “a very firm sense of where analytics can really have an impact – either in increasing revenue or decreasing costs.”
This is such a tricky mix of skills that companies are turning to unorthodox ways of finding analysts that can help guide them to the kinds of a-ha moments that led Newton to create the law of universal gravitation.
Earlier this year, I wrote about an algorithm contest sponsored by California physicians group Heritage Provider Network Inc. Croll referenced the same contest, which offers a $3 million prize for producing an algorithm that will help health care providers identify people who are likely to be hospitalized. The ultimate goal, of course, is to reduce health care costs by providing preventative care to such folks and keeping them out of hospitals.
The first businesses in any industry to derive these kinds of insights from data will enjoy a “huge competitive advantage,” Croll told Aziza. But it will require a broad cultural shift in thinking, starting with senior managers. Croll said:
Leaders today are all about convincing people to do something when there is no proof. Tomorrow leaders will be about convincing people that their proof is accurate and they've asked the right questions.
Public relations, digital marketing, journalism, copywriting. I have done it all so I am able to communicate any information in a professional manner. Recent work includes creating compelling digital content, and applying SEO strategies to increase website performance. I am a skilled copy editor who can manage budgets and people.
