Meeting Challenge of Monitoring Big Data in the Cloud
Modern Big Data application architectures are all about distributed development and resources. When it comes to running apps, those distributed architectures also have multiple points of potential failure that need to be monitored for optimum efficiency and uptime.
One solution for the challenge of monitoring Big Data comes from startup Boundary, which offers a cloud service-based model for monitoring Big Data apps. Boundary emerged from stealth mode in November and is now announcing the general availability of its Big Data apps monitoring effort.
“Big Data applications are built differently than more traditional static applications,” said Boundary CEO Gary Read. “The infrastructure is very dynamic, typically built on cloud infrastructures where servers come and go on a regular basis. As a result problems occur in terms of application degradation on a distributed basis.”
Read noted that it's usually not a single root cause that causes performance issues with Big Data applications, but rather a combination of different issues that together trigger the degradation.
The Boundary system collects data from the multiple components that underpin a Big Data application deployment. That data is then streamed through the Boundary real time engine where queries are run against it. A graphical interface for end-users displays on a second-by-second basis what is happening in the environment.
Read explained that traditional monitoring solutions have employed a device-centric model where servers are the key components. In contrast, Boundary focuses on understanding the whole distributed environment.
Based on this idea of the network as the element that connects all of the application tiers, the Boundary solution looks at every packet that traverses the network, to understand what's in the packet, where it came from and where it is going.
“We say that the network is the greatest source of truth,” Read said.
Vendors like commercial Hadoop provider Cloudera currently offer monitoring and management solutions as well, though Read doesn't see them as competitors to the Boundary business.
“The key for us is to not monitor the Hadoop or Cassandra environment, for example,” he said. “The key for us is to look at all of the pieces, and that's where we see ourselves as complementary to other solutions. We see ourselves as the top layer that is looking at all of the different components and the interaction between them all.”
Sean Michael Kerner is a senior editor at InternetNews.com, the news service of the IT Business Edge Network, the network for technology professionals Follow him on Twitter @TechJournalist.
Sean Michael is a writer who focuses on innovation and how science and technology intersect with industry, technology Wordpress, VMware Salesforce, And Application tech. TechCrunch Europas shortlisted her for the best tech journalist award. She enjoys finding stories that open people's eyes. She graduated from the University of California.
