Microsoft Aims For Big Data With HDInsight

Microsoft has spent the past year readying Windows Azure for big data workloads on the cloud. Now, with the release of the HDInsight Service preview, Microsoft’s cloud-based distribution of Hadoop, the IT behemoth is letting the public take the reins.

Eron Kelly, general manager for SQL Server product management, announced the availability of the preview version of HDInsight Service via the Windows Azure Portal in an 18 March blog post.

Hadoop compatibility

He also assured that the service is – and will continue to be – completely compatible with Apache Hadoop, the popular open source big data processing platform.

“Windows Azure HDInsight Service offers full compatibility with Apache Hadoop, which ensures that when customers choose HDInsight they can do so with confidence, knowing that they are 100 percent Apache Hadoop compatible now and in the future,” Kelly wrote.

He indicated that setting up Hadoop clusters takes mere minutes instead of hours or days, and that they integrate with simple web-based tools and APIs for accessible management and monitoring.

Scott Guthrie, Windows Azure corporate vice president at Microsoft, affirmed that as far as creating Hadoop clusters goes, the HDInsight experience on Windows Azure is a fairly frictionless one. “A cluster will take a few minutes to create (as part of creating it will configure the necessary Virtual Machines that together make up your Hadoop cluster),” he wrote in a blog post.

Users should have little trouble managing Hadoop clusters using dashboards that mimic the style of Windows 8 and Microsoft’s recently revamped web services slate.

“Once the cluster is created, you can drill into the dashboard view to see the cluster quick glance screen. This quick glance allows you to see the basic information about your cluster and gives you a simple method to connect to the cluster,” Guthrie wrote.

Apache Hadoop, sometimes referred to as the “Swiss army knife of the 21st century”, has emerged as the standard bearer of big data technology. After six long years in development, the Apache Software Foundation finally launched Hadoop version 1.0 on 4 January, 2012.

Success of Apache Hadoop

Since its inception, Hadoop has made its way into the data centres of some big name web properties and cloud services providers, including Amazon Web Services, Apple, eBay, Facebook, Netflix, Rackspace and Twitter. It has also given rise to several startups including HortonWorks, a company that specialises in enterprise implementations of the tech.

Even chipmaker Intel is getting in on the act. On 26 February the company announced that is releasing its own Apache Hadoop distribution – technically its third – that builds on the company’s expertise in processors, on-chip security and solid-state drives (SSDs).

Microsoft, keen on establishing a strong presence on in the cloud services market, is the latest to court Hadoop shops in the hot big data space. And where there’s big data, analytics and business intelligence capabilities aren’t far behind.

“In addition, Windows Azure HDInsight Service integrates with our business intelligence tools, including Excel, PowerPivot and Power View, allowing customers to easily analyse and interpret their data to garner valuable insights for their organisation,” Kelly added.

Prices, which will double when the preview period expires, are currently 48 cents per hour per cluster for the head node (extra-large instance) and 24 cents per hour per instance (large instance).

Do you know all about IT in the movies? Take our quiz!

Originally published on eWeek.