Data Professionals Not Going Away, at Least Not Soon

While business users can use tools for reporting and dashboards, “all the high-value stuff, especially around anything predictive or prescriptive, requires a trained professional today,” she said.

Even with dashboards, Roberts said, you need informed consumers to interpret the data. “Do you have somebody who can say, ‘And therefore this means…'? I would separate creating a really interesting dashboard with knowing what to do with that.”

What makes a great data scientist, she added, is “the ability to put together a great experiment design, domain knowledge of the business and knowing which tools and approaches to use to create the best outcome.”

Alex Langshur, co-founder and senior partner of Cardinal Path, which helps companies make the most of their digital assets by “instrumenting” their data from various sources such as point-of-sale, website, mobile apps, social, email and more, describes three roles among data professionals:

• Data collection, a very technical and challenging task at which many organizations fail. “If you get it wrong, every other piece of the chain falls apart,” he said.
• Data management, which includes skills traditionally associated with database professionals. Chances are the person performing it is familiar with Hadoop or NoSQL and can assemble data sets to really extract value from them.
• Data analysis: Data analysts are able to look at the data and understand what it means in order to derive insight from it. That takes a curious mind married to the ability to use a specific tool, along with some statistical experience and knowledge. An analyst uses tools like R or SAS and/or visualization tools like Tableau or Spotfire.

Ultimately, he said, companies must also be able to act on the insights revealed by data analysis. “If you’ve spent all this money on these other steps, but don’t act on it, it’s wasted,” he said.

Data Roles Getting More Specialized

Roberts, whose company researched the characteristics of data professionals, said preparing the data is where the bulk of time is spent. The research grouped data tasks into four roles: data preparation, programming, manager and generalist. The generalist role is disappearing as more people specialize, she said.

“Early on, there was this concept that there was this data scientist who could do everything. Now people are saying, ‘no, no, no, no.’ The person who does data cleansing and data preparation is very different from the person who does visualization and the one who presents the story back to the client, and maybe not everybody has to program,” Roberts said.

Managers running around saying, “We need more data analysts” before first identifying problems they need data analysts to solve is contributing to the outcry over a lack of data talent, Roberts added.

Machines Taking Over?

Noting that artificial intelligence software is developing at a rapid pace, Langshur said, “I think the day will come when a lot more decisions will be enhanced by machines. Right now, in the bulk of cases, software is designed to look for anomalies, then it’s up to a human to determine what those anomalies might mean.”

He pointed to programmatic buying, a new discipline in the advertising world in which an automated feedback loop is used to look for audiences marketers are trying to target and the media those audiences may frequent.

“[It] looks to place ads on those media at the lowest possible price, working through arbitrage with other people looking at the same audiences in the same media at the same time. Then looking at the click-throughs and conversions based on that ad spend, you can either dial it up or dial it down,” he explained.

Such tools will never make humans unnecessary, though, he added. They’re complicated to set up and still require human oversight to determine whether they’re working effectively.

ROI of Data Analytics

While machines will ultimately take over some of the data analysis work that humans have traditionally done, human insight will remain critical to decision-making, said Jack Phillips, CEO of the International Institute for Analytics.

As an example, he points to real-time credit scoring of a certain category of borrowers in consumer finance. While software might be able to size up a person’s credit-worthiness or give an instantaneous decision on insurance by the numbers, he sees a final 20 percent of the process remaining dependent on human judgment to consider outlying factors.

Obtaining the right data analytics skills is only a small part of the growing effort for companies to achieve ROI with their data analytics operations and make them a systemic part of their business operations, Phillips said.

“In some ways it is easier, because you’re able to point to business outcomes or clinical outcomes in healthcare. And the investment – it’s becoming easier to determine what it costs to run an analytics operation,” he said. “The challenge is to isolate the independent variable, the role analytics played in a particular business outcome. This is still an early science that will grow and cure and harden, becoming clearer over time.”

Susan Hall has been a journalist for more than 20 years at news outlets including the Seattle Post-Intelligencer, Dallas Times Herald and MSNBC.com. She writes for Dice.com and FierceHealthIT, among other publications.