IBM Scientists Demo Speedy GPFS Storage System

IBM has revealed an impressive demonstration of its new storage architecture design, dubbed the General Parallel File System (GPFS).

IBM had initially revealed GPFS back in November 2010 at the Supercomputing 2010 conference in New Orleans.

At that time IBM said that its boffins had a system that could convert terabytes of pure information into actionable insights twice as fast as previously possible.

Analytics record

And now IBM researchers demonstrated its GPFS storage management technology, by successfully scanning 10 billion files on a single system in just 43 minutes. This speed shatters the previous record of a data management system by a factor of 37.

To date, the fastest time that a system was able scan a billion files was three hours. That record was also set by IBM researchers, at the Supercomputing 2007 conference in Reno, Nevada.

IBM is calling this technology the storage performance breakthrough needed for organisations hoping to move to “Big Data Applications.”

This is because most organisations today are often swamped by the sheer volume of data being generated, and are not easily able to turn this data using analytic software into actionable insights.

Instead organisations are constantly fire fighting by trying to figure out how to manage and store it all, which is in turn fuelling the ongoing storage boom.

“Growing at unprecedented scales, this advance unifies data environments on a single platform, instead of being distributed across several systems that must be separately managed. It also dramatically reduces and simplifies data management tasks, allowing more information to be stored in the same technology, rather than continuing to buy more and more storage,” said IBM.

Long time coming

Although IBM had officially revealed GPFS last November, in actual fact the technology goes back over ten years.

Indeed it was back in 1998, when IBM researchers first unveiled a highly scalable, clustered parallel file system called General Parallel File System (GPFS), which IBM then fine tuned over the next ten years to make this breakthrough possible.

According to IBM, GPFS represents a major advance of scaling for storage performance and capacity, while keeping management costs flat.

This innovation, IBM believes, could help organisations cope with the exploding growth of data, transactions and digitally-aware sensors and other devices that comprise Smarter Planet systems.

Actionable data

IBM feels it is ideally suited for applications requiring high-speed access to large volumes of data such as data mining to determine customer buying behaviours across massive data sets, seismic data processing, risk management and financial analysis, weather modelling and scientific research.

IBM revealed that it managed to achieve the 43 minute breakthrough by running GPFS on a cluster of 10 eight core systems and solid state storage.

“Today’s demonstration of GPFS scalability will pave the way for new products that address the challenges of a rapidly growing, multi-zettabyte world,” said Doug Balog, VP, storage platforms, IBM. “This has the potential to enable much larger data environments to be unified on a single platform and dramatically reduce and simplify data management tasks such as data placement, aging, backup and migration of individual files.”

“Businesses in every industry are looking to the future of storage and data management as we face a problem springing from the very core of our success – managing the massive amounts of data we create on a daily basis,” said Bruce Hillsberg, director of storage systems, IBM Research – Almaden. “From banking systems to MRIs and traffic sensors, our day-to-day lives are engulfed in data. But, it can only be useful if it is effectively stored, analysed and applied, and businesses and governments have relied on smarter technology systems as the means to manage and leverage the constant influx of data and turn it into valuable insights.”

Tom Jowitt

Tom Jowitt is a leading British tech freelancer and long standing contributor to Silicon UK. He is also a bit of a Lord of the Rings nut...

Recent Posts

Apple, Google Mobile Ecosystems Should Be Investigated, CMA Told

CMA receives 'provisional recommendation' from independent inquiry that Apple,Google mobile ecosystem needs investigation

6 hours ago

Australia Rejects Elon Musk Claim About Social Media Ban For Under-16s

Government minister flatly rejects Elon Musk's “unsurprising” allegation that Australian government seeks control of Internet…

9 hours ago

Northvolt Files For Bankruptcy Protection In US

Northvolt files for Chapter 11 bankruptcy protection in the United States, and CEO and co-founder…

10 hours ago

UK’s CMA Readies Cloud Sector “Behavioural” Remedies – Report

Targetting AWS, Microsoft? British competition regulator soon to announce “behavioural” remedies for cloud sector

1 day ago

Former Policy Boss At X, Nick Pickles, Joins Sam Altman Venture

Move to Elon Musk rival. Former senior executive at X joins Sam Altman's venture formerly…

1 day ago

Bitcoin Rises Above $96,000 Amid Trump Optimism

Bitcoin price rises towards $100,000, amid investor optimism of friendlier US regulatory landscape under Donald…

1 day ago