Big Data Firms Predict Mainstream Adoption In 2014

MongoDB, Datastax, Hortonworks and EMC all believe that 2014 will be the year that Big Data analytics finally goes mainstream, but have said there is still time for businesses to take the lead by adopting Big Data solutions before their competition does.

“It’s really happening. The train is leaving the station, so get on,” John Glendenning, VP and GM for EMEA at DataStax told reporters at a Rackspace-hosted roundtable in London.

The panel agreed that the Big Data market was growing at an incredible rate, but there were still barriers to adoption, such as the prevalence of data silos and issues with data governance. However, both Bernd Kaponig from EMC and Chris Harris from Hortonworks said the issue with the lack of relevant skills in the industry was overrated, and would not stand in the way of data-driven decision-making.

All aboard

According to research published by SINTEF in May 2013, 90 percent of the world’s data was generated over last two years. The variety and quantity of information will only increase, as the Internet of Things enables organisations to collect data from billions of new sources.

Businesses have started to realise that analytics can help them get to know their customers better, take the guesswork out of strategic decisions and improve the bottom line as a result. Another piece of research from 2013 suggests that the readiness of businesses to spend money on Big Data projects is increasing, but they are still examining real-world benefits of the technology.

According to Harris, technical director at Hortonworks, in order to take the plunge, a company first has to transform its data silos into a ‘data lake’. “There’s a lot of silos out there. And it’s very difficult then for an organisation as a whole to get a 360-degree view of their customer, because the customer is interacting with them in multiple ways.

“What a lot of organisations are looking to do is put together a ‘data lake’. This is not to replace all the existing systems, but to bring raw data together, and then be able to discover what value it holds by analysing it in one location, and then applying governance to it.”

“Once that’s done, you can understand who’s accessing the data, how the data is being transformed and where it is getting pushed in the organisation. The challenge with silos today is: yes, there’s governance, but it’s governance on silo level and it’s very difficult to track that data as it moves across different silos or business units.”

Governance was also on the mind of Matt Keep, principal product marketing manager at MongoDB, who called the lack of data management frameworks the biggest obstacle to adoption of NoSQL databases and technologies like Hadoop over the next two years.

“We have been dealing with very large enterprise customers who have used the technology to ‘scratch an itch’, if you like,” said Keep. “Now, we start to get into the really serious stuff, companies are expanding their use, and suddenly the corporate governance people start to get involved.

“Over the last 30-40 years, they have introduced very mature controls for governance and security. It is an area where Hadoop and NoSQL technologies clearly have been lacking because they are new, but there’s rapid evolution going on. A number of vendors including DataStax and MongoDB are investing huge amounts to make sure Big Data technologies can be integrated with existing authentication, security and governance tools.”

Skills to pay the bills

At the same time, the Big Data Breakfast panel dismissed the much-talked-about lack of skills in analytics as an overrated issue.

“What we have done at Hortonworks is actually partner with many organisations out there,” explained Harris. “As a simplified example, you can take Microsoft Excel and stick it on top of Hadoop. Majority of people out there know how to use Excel, right? So they can interact with the tools they already know.

“Try to make it as easy as possible to reuse your existing skillset.”

Meanwhile Kaponig, the solutions principal at EMC, said that since the Big Data field was relatively new, the supply simply hadn’t caught up with the demand. He added that he actually enjoyed the current elite status of data scientists, but didn’t think it was going to last.

So what is the best way to introduce Big Data in a business environment? According to Harris, patience is key: “You start off by demonstrating value, typically limited to a single business unit or application. Once you start generating enough value, you’ll get attention across the organisation, and then you grow from there. The worst thing you can do is walk into the office on day one and say ‘you have to completely restructure your organisation’. You’ve got to take them on a journey at a sensible pace, at a pace the customer is comfortable with.”

What do you know about Big Data analytics? Take our quiz!