Seeking Truth about Hadoop
Hadoop has been around for a decade and has been a hotly hyped technology for much of that time. Despite the hype, though – or perhaps because of it – it has been tough to get a handle on how many organizations are using it and whether or not they are deriving value from it.
Earlier this year Gartner published a study that seemed to take some wind out of Hadoop's sails, finding that slightly more than half of its respondents had no plans to invest in Hadoop in the next 24 months.
AtScale, provider of technology that makes it possible to use OLAP tools on Hadoop, did not think the study offered an accurate picture of Hadoop use, largely because Gartner based it on responses from just 125 members of its Research Circle. AtScale partnered with MapR Technologies, Tableau and Cloudera to survey their customers, polling some 2,100 C-level executives across the globe.
Not surprisingly, given the Hadoop-heavy nature of the sponsors' solutions, respondents were more bullish about the data platform than those queried by Gartner. Seventy-one percent of the organizations are using Hadoop, and more than 80 percent of them have used it for at least six months. Thirty percent of respondents have a cluster with more than 50 Hadoop nodes. Three-quarters of current Hadoop users plan to increase their use of Hadoop within three months.
“If you want to know the state of innovation with Porsche, you ask Porsche drivers; you would not talk to Fiat drivers,” said Bruno Aziza, AtScale's chief marketing officer.
How Strategic Is Hadoop?
While Hadoop was once thought of primarily as a way to cheaply store data, Aziza said, AtScale's survey shows that view is changing. Forty-seven percent of respondents said Hadoop played a strategic role for their organizations, vs. 12 percent who said their use of Hadoop was tactical.
Only 17 percent of respondents adopted Hadoop to cut costs. Thirty-seven percent cited Hadoop's ability to satisfy scale-out needs, while 19 percent plan to use it to create new applications and 14 percent mentioned its potential for revenue generation.
As a further indicator of Hadoop's ability to drive revenue, 52 percent of respondents said sales and marketing were the biggest beneficiaries of Big Data.
Aziza also noted a “reversing trend” in how organizations intend to use Hadoop. Nearly three-quarters of respondents currently use Hadoop for ETL and 65 percent use it for business intelligence. The numbers shift a bit for organizations planning to adopt Hadoop, with 51 percent mentioning ETL and 69 percent BI.
“With the platform maturing, there is less of a need to put it in front of specialists,” Aziza said. “To achieve value with a technology like Hadoop, it needs to become more mainstream. I think we are there now.”
Hadoop's evolution is similar to the evolution of business intelligence, he said. “The more business analysts you engage with it, the more demand is created for the platform, which then drives greater value.”
Aziza believes Hadoop's flexibility is a big adoption driver. “Hadoop is more flexible than a structured data warehouse,” he said. “You can throw any data at it; you do not have to spend months structuring your workflows. By the time you do that, often your business requirements have already failed.”
Getting Tangible Value from Hadoop
When asked if Hadoop has provided tangible value, respondents were fairly evenly split down the middle, with 49 percent saying their organizations had attained tangible value that could be attributed to Hadoop. Of the 51 percent that had not yet linked Hadoop to value, 6 percent said they were pessimistic about doing so.
Companies that reported achieving value shared some common characteristics, according to the study:
- Perception of value increased along with the number of Hadoop nodes. Seventy-two percent of those with more than 500 nodes said they had attained value with Hadoop, vs. 68 percent of those with 101-500 nodes, 53 percent of those with 51-100 nodes, 48 percent of those with 10-50 nodes and 33 percent of those with fewer than 10 nodes.
- Fifty-seven percent of organizations with an executive mandate for Hadoop believe they are achieving value from it, vs. 49 percent where business is the primary driver for Hadoop and 47 percent where the IT function is the primary impetus. An executive mandate is fairly rare, however, with just 12 percent of organizations using Hadoop citing it as a primary driver of adoption.
- Those that adopt Hadoop for revenue generation (56 percent) or scale-out needs (55 percent) were more likely to attain value from it than those that adopt it for new applications (47 percent) or cost savings (44 percent).
- Sixty-one percent of organizations that provide self-service access to Hadoop say they have achieved value from it, vs. 41 percent of those that do not offer self-service access. Self-service access to Hadoop rises along with length of usage. Nearly half of organizations that have used Hadoop for more than 12 months provide it, vs. 31 percent of those who have used Hadoop 6-12 months and 28 percent of those that have used it for less than six months.
Public relations, digital marketing, journalism, copywriting. I have done it all so I am able to communicate any information in a professional manner. Recent work includes creating compelling digital content, and applying SEO strategies to increase website performance. I am a skilled copy editor who can manage budgets and people.
