Big Data, Big Opportunities for DBAs
Amid all the talk about Big Data, questions keep popping up about how the database administrator (DBA) fits in. The role is changing — and it varies by company, according to Bert Scalzo, database domain expert for Dell.
“One of the biggest concerns I keep hearing is how the DBA role has either lost importance or gained importance. There’s been a change and there’s disagreement about what the change is,” he said.
DBAs More or Less Important?
Scalzo said some people feel the DBA role is becoming more important, thanks to the large amounts of data companies are accumulating – and storing, thanks to inexpensive disk space and cloud storage. That doesn't mean there are more DBAs, however.
“It used to be that you could only have so many terabytes or petabytes before it went to secondary storage. Now that’s kind of old-fashioned. Now you can keep data live until there’s absolutely no reason to have it,” he said. “In that case, the DBA is doing more. It’s not that the number of DBAs is decreasing, but the amount of data that companies are keeping is increasing exponentially with the same amount of resources.”
Those people see DBAs growing more important, but at the same time more overwhelmed, he said.
Some other companies treat data as a commodity and may think they don't even need a DBA on staff. Such companies are more likely to outsource DBA work.
“If it’s in the cloud, Amazon’s making sure it performs well, doing the tuning, optimization, backup and recovery, and all that other stuff. That’s part of the service,” he said.
Broader Definition of a DBA
Traditional DBA work still needs to be done, said Neil Raden, CEO and principal analyst at Hired Brains Research.
“If you use a broader definition of a DBA as not so much an administrator, but a data management person, those skills are going to be very valuable,” Raden said, noting that data management skills encompass a broad range of knowledge in areas such as data quality and data integration.
He said some of the most exciting job opportunities in Big Data involve statisticians and analysts who can work with predictive modeling and machine learning. Yet DBAs tend not to be mathematicians.
Raden said there are two different kinds of DBAs: designers and instance tuners. Designers “look more closely at building and modifying schema, they work with ETL people and how they get introduced to different databases,” while instance tuners “are mostly involved with writing SQL query, developing and maintaining indexes and other types of physical optimization.”
“So one is more deep down in the weeds of how the machine works, and the other is more up in the air with the data,” Raden said. “The people with the data are more likely to find, at least partially, their skills useful in this different environment. The other guys have hitched their wagon to Oracle’s [latest version] or DB2. Those databases aren’t going anywhere,” he said.
And there are plenty of job opportunities for them. The unemployment rate for database administrators stood at a mere 2.8 percent during the first quarter, according to the U.S. Bureau of Labor Statistics.
Open to a New Paradigm
Especially with cloud services, specific skills with Linux, Windows, HP-UX or AIX become less important, Scalzo said.
“The biggest thing [in making the transition] is not to presuppose or apply your Golden Rules that you’ve learned over your last 10 or 20 years,” he said. “Good database design in a relational environment might not be good design in a column-oriented data store. How you define data structures relationally might be different than how you define them in a noSQL database. It’s kind of an unlearning/learning process that you have to embrace. You still apply common sense; you just use you experience to gauge that common sense in a new paradigm.”
An open mind and inquisitive capabilities are the two most important qualities for DBAs in this new and changing data environment, he said. “You’ve got to have an open mind and be hungry to learn and see how what you used to know works in a new environment.”
‘Immense Opportunity’
In a blog post, Gwen Shapira, senior database administrator at data infrastructure management company Pythian, argues that DBAs are prime candidates for a new role called Hadoop cluster administrator. (Citing other, better technologies, Raden isn’t convinced DBAs need to run out and learn Hadoop.)
Shapira writes that any organization using a Hadoop production cluster larger than 20 to 30 nodes will require a full-time administrator. That job will be similar to traditional DBA work: responsibility for the performance and availability of the cluster, the data it contains and the jobs that run there.
While that admin will need to be able to write some small jobs in Java, for Hadoop code, alternatives such as Streaming, Hive and Pig will do. Hadoop admins will need a deep knowledge of Linux, however.
As for how traditional DBAs should be positioning their careers, Shapira said in an interview with Enterprise Apps Today that they can opt to continue what they have been doing, since relational databases are not going away anytime soon.
She believes there is “immense opportunity” for DBAs.
“In the age of Big Data, data professionals are more visible to the organization than ever, and they are seen as the critical resource they always were,” she said. “A few years ago, executives only noticed the database when it stopped working. Now they are looking at the database and they think of all the information it holds as an opportunity: How can we get more value out of the information we have here? What other information can we collect to support our decision making? How can we use data to enhance our products?”
She offered some advice for DBAs who want to use this opportunity to take their careers to the next level: Step out of the server room and into management offices.
“Talk to business unit managers to find their pain points. Which information do they need? What are the gaps in the current systems? Managers are happier to talk to technical people now than they've ever been,” she said. “Show them how your skills can help them get more value out of data, because this is what Big Data means to them.”
Susan Hall has been a journalist for more than 20 years at news outlets including the the Seattle Post-Intelligencer, Dallas Times Herald and MSNBC.com. She writes for IT Business Edge, Dice.com and FierceHealthIT.
