EMC Combines Big Data, Social Nets And App Development With Pivotal Labs Buy

In a single move, EMC has combined the white-hot IT sectors of big data analytics, social networking and new-generation application development with the acquisition of Pivotal Labs, a privately held provider of software development services and tools.

EMC, one of the world’s largest data centre systems, storage and virtualisation providers, did not reveal terms of the deal except to say that it was an all-cash transaction.

Social medium for Big Data analysis

Pivotal’s self-described “agile” project management tool Pivotal Tracker is used currently by more than 240,000 developers around the world. EMC Chief Operations Officer Pat Gelsinger said at a launch event here that EMC plans to continue to invest in Pivotal Tracker to accelerate innovation in the platform and increase adoption.

“If I need to give you a clone of a 10TB database, and break down a multi-source, multi-petabyte data environment, I need to enable collaboration on top of that,” Gelsinger told a group of reporters at the Pivotal Labs headquarters in San Francisco. “That’s what our announcement today is all about. We’re creating a social environment for big data analysis.”

Pivotal Tracker has been fine-tuned to work tightly with EMC’s Greenplum database for large, unstructured workloads of varying types of data. Both of them also work natively with EMC’s own editions of the open-source Apache Hadoop and MapR for batch analytics applications.

Thanks to the acquisition, EMC, a relatively old-line IT hardware and services company, will be creating fresh relationships with new-generation Web service providers. Pivotal’s customers already include Twitter, Best Buy, Groupon, Salesforce.com, EMI, Urban Dictionary, Linden Lab, Task Rabbit, The Annie E Casey Foundation and about 200 others.

A lot has been happening at EMC in the big data area during the last few months. A few weeks ago, EMC expanded its Greenplum catalogue by launching the Greenplum Unified Analytics Platform, which provides a scale-out infrastructure for analysing both structured and unstructured data.

Firewalled social network

EMC also announced general availability of something called Greenplum Chorus. This is a Facebook-like social collaboration platform for data science teams to make sure that workloads’ datasets are correct in each workload and that useful insights are delivered to the business quickly.

The relatively new job classification of “data scientist” refers to people who combine the disciplines of data analytics and scientific research in order to fine-tune new applications in the big data sector, the fastest-growing area of IT in 2012.

Greenplum Chorus resembles Yammer or Salesforce Chatter, which are used inside an enterprise firewall or within a designated community of contractors and partners to share information and collaborate on projects. Workflows, calendars, meeting notes, documents, videos, photos – virtually any type of content can be shared, edited and commented upon by multiple people using these tools.

Inside the context of all this new software development collaboration-by-social network, how is EMC Greenplum going to manage all the intricacies of keeping some data off-limits that needs to remain off-limits?

“This is a new domain, the idea of open data,” Gelsinger told eWEEK. “This is a challenge, especially when it comes to the regulated industries (such as health care, financials, government and others). It all will depend on the use case, data set and regulations. A customer might want some large data sets to be worked on in an open manner, such as at universities and such.

“This is where we pull in the RSA [Security] guys to help solve the problem. This is what’s great about being in a company like EMC: we have people like VMware, RSA and others within the company all ready to help us out.”

All of the products and services above will be able to plug their big data workloads into EMC Greenplum’s data warehouse.

How well do you know the cloud? Take our quiz!