Here’s a Toast To Liquid Cooling!

The Green500 list ranks the world’s most energy efficient supercomputers. Released in the busy period before Christmas, it nearly slipped past me. But, as we toast the New Year, it seems appropriate to raise a glass to the winners – since they are virtually all cooled by liquid.

None of the liquid involved is remotely festive, of course. It’s been many years since Alan Turing (pretty much seriously) suggested using gin in EDSAC, an early computer at Cambridge. Gin has a mixture of alcohol and water which, Turing said, gave a “zero temperature coefficient of propagation velocity at room temperature” which meant it could be used instead of expensive mercury in EDSAC’s primitive delay line memory systems.

Liquid versus air

Liquid cooling is well established in supercomputers, mostly using oil or water, or a combination of the two.

But the Green500 list’s runner up uses air, and comes from Cambridge. The Wilkes system is named after Sir Maurice Wilkes, the head of the EDSAC project, and its use of air is sufficiently unusual to be remarked upon. At 3.6 GigaFLOPs per Watt, the machine is more efficient, by the Green500’s performance efficiency measure, than the 3.2GFLOPs/W of the Italian Eurora system which topped the list in June.

Wilkes was installed in November, using Dell and NVidia hardware, and its score is impressive, though its ranking may raise some questions about the listing itself.

The Green500 list takes the power used by the supercomputers themselves – but leaves out the infrastructure including any cooling systems. It just includes the power used by the IT equipment and is a measure of how little waste heat they produce, but doesn’t factor in how efficiently they remove that heat.

“The fundamental problem is that the Green500 list only measures IT energy and ignores the infrastructure energy cost – massively distorting the statistics,” pointed out Peter Hopton of liquid-cooled computing company Iceotope. So the world’s largest supercomputer, Milky Way 2, uses around 18MW of power, but the list ignores the 24MW of cooling it requires, he said.

The rest of the data center industry uses PUE to measure efficiency – a measure which has its own problems but does measure how much heat is wasted in the infrastructure.

Leaving out infrastructure doesn’t completely invalidate the Green500 list. A supercomputer which uses less energy will generate less heat. amd won’t make so many demands on its cooling system. But it’s pretty likely that an air-cooled system would use more energy to blow that heat away than a liquid cooled system, which can absorb the heat more passively.

Liquid is a runaway leader

And another point to make is that, even without the infrastructure, the use of liquid cooling is still clearly best in this field.

The top-ranked system, Japan’s Tsubame-KFC, uses the Green Revolution Cooling system that immerses servers in inert oil, and beats Wilkes by 0.2GFlOPs per Watt.

Green Revolution’s system is normally sold as a way to reduce the cost of cooling infrastructure. It’s clearly helped Tsubame-KFC to shine in a list that ignores infrastructure, simply because it allows the system to be built more densely without overheating. In a list which more fully covered the energy use of the supercomputers, it might have done even better.

That’s a testament to the value of liquid cooling.  Llet’s drink to it – but preferably something more palatable than inert mineral oil…

Try our super-duper supercomputer quiz!