Does HPC Belong In The Cloud?

High performance computing is preparing to go mainstream, with Intel and others laying out their plans to provide exascale computing performance by the end of the decade. But the high cost of ownership and complex architecture needed for running supercomputers is deterring many organisations from buying into the technology.

This is where cloud computing comes in, according to John Hengeveld, director of marketing for Intel’s Data Center Group. The cloud can provide access to HPC for companies that lack either the money or the expertise to run their own supercomputing architecture, enabling them to process vast amounts of data as and when they need to.

“HPC in the cloud could be a service available broadly, but there are capital barriers to access – you need the money to buy a system. And then there are barriers to trial, the initial usage, that are significant as well,” said Hengeveld. “The advantage of HPC in the cloud is you can make it easier to try out HPC, and easier to run it on an infrequent basis or a scale basis, if you don’t need to use it that often.”

HPC and the ‘missing middle’

Hengeveld talks about a class of people known in the industry as the ‘missing middle’. These are either people that have knowledge of HPC but are unable to afford the technology needed to access it, or people that could benefit from investing in HPC, but do not know how to manage the complexity of running HPC systems.

Both of these groups could potentially benefit from HPC services in the cloud. ‘Renting’ such services on a part-time basis is of course much less expensive than owning the infrastructure and running it in-house. Buying HPC as a service also allows the organisation to focus on getting the most out of the application, without having to worry about the physical infrastructure.

In many cases, the social issues are just as much of a barrier to overcome as the financial ones, said Hengeveld.

“There’s a collection of people who are not HPC zealots. They don’t program their own applications, they don’t understand what an HPC software stack is, they don’t want to know about provisioning capabilities, they don’t want to know about the complete range of things you need to do in order to run an HPC system to solve a specific problem,” he said.

“That’s hard stuff, and these people don’t conceive of themselves as compute specialists, they conceive of themselves as product designers or animators or architects or bio-chemists. Whatever their personal identification is, it’s not about computation.”

Cloud: enabler or deterrent?

It is particularly important with this section of the ‘missing middle’ to set aside the barriers to HPC systematically, by creating an infrastructure that allows compute services to be delivered to the user through an application model that they are familiar with.

“Again, they don’t think of themselves as compute users, they think of themselves as technical users, they think of themselves as designers. Well the tool they use for design has got to become cloud-aware,” said Hengeveld. “There’s got to be a seamless model for computation that goes from the service providing, to the brokering of that service, to the discovery of the service, to how it’s delivered to that application.”

Continued on page 2