eBay’s Data Centre Innovation Won’t Stop With Fuel Cells

eBay recently announced an extension to its data centre in Utah that will be entirely powered by fuel cells, but the data centre leader for the online marketplace told us at the Data Center Dynamics event in London that there is a lot more to the announcement than just the headline.

Data centres have used fuel cells for back-up power before, but eBay has taken a big step by using them for primary power and eliminating diesel back-up from this phase of its Utah facility, and in doing so has proved the value of on-site generation and opened up a novel source of energy reclaimed from the gas distribution system

Changing the law

Like so many other decisions in a data centre, using fuel cells was not a simple stand-alone matter, Dean Nelson, eBay’s vice president of global foundation services told TechWeek.

“For Phase Two of Utah, we wanted to have a hybrid data centre,” said Nelson, “with white space, and a spine to connect containers.”

Utah is one of three major eBay data centres, and containers got a starring role in its next stage. In 2010, eBay opened a Tier IV level facility called “Topaz”, in Salt Lake City Utah, and then built a data centre in Arizona, where it started using prefabricated containers mounted on the rooftop.

But for the containers in Salt Lake City, the company got a purpose-built ground floor housing alongside the “white space” in the form of a conventional data centre hall. This is only the equivalent of  Tier II in reliability terms, so eBay can run less critical loads there: “We can mix and match, to get the best of ecology and economy,” he said.

Nelson decided to be unconventional with power and used a bank of fuel cells which provide 6MW of electricity. He conceded that fuel cells are still more expensive in capital terms, but they had more than paid for themselves because of the other changes they facilitated.

“You can’t justify fuel cells without redesigning the core of the site,” he said, explaining that on-site generation reduces waste and that less redundancy is required.

“We can do without 2N +1 redundancy, because fuel cells have their own +1 redundancy built in. This means that 6MW of fuel cells is enough to reliably power up to 4.9MW of IT equipment, and since the site has two electricity utilities, that makes for a redundant back-up. The two sources are diverse too, since the power lines go above ground while the gas pipes are underground, and so impervious to any storms.”

Heat harvesting

Taking charge of energy generation was no mean feat. Utah’s laws would only allow firms to generate power for themselves if they did it on their own site, but eBay had bigger plans.

Following a lot of negotiation, eBay convinced Utah to change the law so firms in the state can generate their own electricity and use the utility’s infrastructure to transfer it to their own site.

Nelson then issued a blue-sky request for proposals (RFP) to see what energy firms could come up with.

“We got 30 proposals from 20 companies,” he said. “The market has really opened up in Utah.”

eBay chose a proposal that was a spin-off from getting the natural gas pumped to its Salt Lake City site and involved reclaiming heat that would otherwise be wasted at the pumping station 22 miles away. When gas is pressurised and pumped through the distribution network, a lot of heat is generated, but this is wasted.

A company called Ormat built a recovered energy generator (REG) at the pumping station, creating 5MW of extra electricity, which is then wired to eBay, which believes this could be the start of something big for Ormat.

“There’s another compression station 44 miles away, and 1200 stations across the US,” said Nelson, who added that all of them, and similar stations in Europe, are wasting heat which is “100 percent clean”, that could be harnessed.

A revolution in modules

The new extension has also allowed eBay to introduce some new ideas into containerised data centres, a mode of building that appeared to be heading for great things in 2009 and 2010, but developments have been thin in recent times.

“A lot has changed,” said Nelson, who explained that the first generation of containerised data centres used standard ISO shipping containers that weren’t the optimum size and were shipped from the manufacturer fully loaded.

This was too inflexible and for eBay, both HP and Dell created new containers which can be loaded with extra servers in the “rack and roll” model.

“Before you had to buy a full container, and depreciate it over three years,” he told us. “Now I can buy them empty, and fill them as I need them.” That means the new pods can be depreciated over nine years.

The data centre obviously doesn’t have old-fashioned chillers and instead has cooling towers which provide cold water. Nelson says the beauty of this is that it is futreproofed to be compatible with a big change that he sees coming – liquid cooling.

Liquid cooling is coming

“I believe we will go to liquid cooling in future,” he said, explaining that cooling the chip directly is vastly more logical than today’s system which surrounds the electronics with layers of boxes, and then has to find a way to get cold air into them

The chip is on a board, in a box, in a rack, in a building, he explains. Each of those adds a layer of complexity to air cooling, but liquid cooling can extract the heat direction, and allow the chip to operate at its optimum temperature.

“These layers have held back the optimisation of our infrastructure. Liquid cooling will remove the locks on chips, and take away constraints,” he said.

The next upgrade to one of eBay’s data centres could be pretty exciting.

Do you know about renewable energy? Try our quiz!