Calxeda: ARM Beats Xeon In Efficiency Tests

Calxeda is touting a benchmark test that shows one of its ARM-based EnergyCore chips being 15 times more power-efficient than an Intel Xeon E3 server chip, the latest move in the growing competition between Intel and ARM.

The test included a server powered by a quad-core EnergyCore ECX-1000 system-on-a-chip (SoC) – based on ARM’s Cortex-A9 design – with a speed of 1.GHz and running Ubuntu Server v12.04 and Apache Server v2.4.2. The results for the Intel chip were based on published thermal design power (TDP) metrics for a quad-core 3.3GHz Xeon E3-1240, which is based on the older Sandy Bridge architecture, and estimated double data rate (DDR) memory.

ARM benefits

The results, posted to the armservers.com website, indicated that with the EnergyCore system, for 1 million requests, the server averaged 5,500 requests per second with a 9-millisecond latency and average power of 5.26 watts. In the estimated Intel configuration, the results were 6,950 requests per second, with a 7-millisecond latency and average power of 102 watts.

The results indicated that the EnergyCore SoC had a 15-times advantage over the Intel chip in the performance-per-watt category, a key consideration for ARM and its manufacturing partners.

The results, coming amid the heightening competition between Intel and ARM in such areas as mobile devices, PCs and servers, got the attention of some news sites as well as Forrester analyst Richard Fichera, who has talked about the possible benefits of the ARM architecture finding its way into data centres.

In a blog post on 20 June, Fichera noted that this was the first well-documented benchmark test between an ARM-based server chip and one from Intel, and that it is only a single test that is limited to the workload – web serving on a small server with a 1G-bit network. The results can’t be extrapolated to assume a faster network – though Fichera said he didn’t think that would make much difference – or other workloads.

Still, he said, the test did show some interesting facts, including that the Calxeda chip did demonstrate that an ARM SoC can deliver “impressive power efficiency” and that the absolute performance was better than expected.

“As a basic proof point, this benchmark succeeds as a proof of concept – ARM servers are indeed in the ballpark versus their initial promises,” Fichera wrote.

Scepticism

Even by swapping out the Sandy Bridge chip for a more power-efficient Ivy Bridge processor from Intel, the performance-per-watt advantage for the ARM chip might drop to 10 times, he said. Adding in the performance advantage of Ivy Bridge over Sandy Bridge, that might further reduce the advantage to eight times.

“Still very compelling from an overall efficiency perspective,” Fichera said.

On the website Decrypted Tech, writer Sean Kalinich said there are too many flaws in the test methodology to create accurate results. In particular, Kalinich noted that the results for the Xeon server were based on published TDP numbers for the CPU and I/O chipset, and the DDR memory estimates.

According to the results reported on the armserver.com website, “unfortunately, at the time of this blog post, we didn’t have a way to measure actual power consumption with the same level of fine detail”.

“Now that right there makes the whole claim invalid,” Kalinich wrote. “You cannot call it a comparison if you do not have both in the lab.”

He also said that unlike in the test, most IT administrators would not use an entire Xeon E3-1240-based server for a single Apache web server. Instead, they would leverage it as a host for 10 or more virtual servers, driving up the system utilisation numbers, which could offset advantages in total cost of ownership claimed by the ARM-based server.

As Intel makes an aggressive push into the mobile device market with chips aimed at smartphones, tablets and Ultrabooks, ARM is looking to move into PCs and low-power servers.

ARM ecosystem

While the company’s ARMv8 architecture – which boasts such data centre features as 64-bit capabilities, more memory and greater virtualisation support – is still at least a year away from appearing in systems, ARM is still seeing some interest among server OEMs. In May, Dell announced plans for limited distribution of its new Copper servers – powered by ARM-based chips from Marvell – to help grow the ecosystem around ARM-based servers.

That came several months after Hewlett-Packard, in November 2011, announced a deal with Calxeda to develop very low-power servers as part of HP’s larger Project Moonshot initiative. However, HP officials said on 19 June that the first Gemini servers coming out of Project Moonshot would run on Intel’s Atom-based Centerton SoC platform.

How well do you know your tablets? Take our quiz.