Ingres Promises to Bring Moore’s Law To Data Analysis

Open-source database vendor Ingres has announced a plan to use vector-processing to unlock the potential of Moore’s Law in standard chips, and offer radically cheaper data analysis.

While processors have been dramatically increasing in power, conventional databases have not been able to take advantage of their ability to handle multiple instructions at the same time. Ingres has rewritten its kernel to provide a module that will “vectorise” any queries so analysis of databases with millions of rows can take place in less than a second on cheap hardware.

“Moore’s law talks about doubling processor capacity every couple of years,” said Ketan Karia, vice president of marketing at Ingres (pictured). “We’ve never seen that translate to preformance increases in business software – in fact we’ve seen the opposite.”

Business software is blocked from the speed benefits of new chips, because it can’t feed them fast enough, he said. In fact it actually slows down, because our demands on it are increasing: “Front office systems are now dealing with twenty or thirty times more data than they were originally deigned to handle.”

The new module will deliver an immediate ten times performance boost for business applications and will continue to deliver even faster speeds as processer hardware improves he said. It will be demonstrated on Intel’s processors at the company’s IDF event in September, and will be delivered as a commercial product in 2010.

The module will be created by an Ingres-backed company called Vectorwise a new company set up to capitalise on research done at the Netherlands’ Centrum Wiskunde & Informatica (CWI) institute for fundamental scientific research.

Vectorwise is led by Peter Boncz and Marcin Zukowski, both members of the Database Architectures group at CWI who developed the MonetDB/X100 fast column store, seen here in a Google Tech Talk given by Boncz in 2008.

Specialised vector processing systems have been used in scientific research, and in database work have shown up in specialist systems from companies such as Teradata, but Ingres is now opening this up to general purpose machines, said Karia.

Although the breakthrough only applies to data analysis, not to transactions, it can speed up all oeprations, because most transactions have an analysis phase, that selects the records on which to apply a particular operation, he said.

The module will eventually available as open source code, but Ingres will have a lead on other database vendors such as Oracle, as they will not be able to countenance rewriting their software to use vector processing , said Karia:

“Open source makes our software more agile,” he said, echoing Ingres CEO Roger Burkhardt’s views. “It would be nigh on impossible for Oracle or Sybase to do this, because it means a total rewrite of the kernel.”

Ingres will be encouraging software partners – such as SAP or SAS – to use Ingres for specialised data analysis modules, opening the door to wider use of the Ingres database, he said.

Although the system has so far been implemented only on Intel processors, it could be delivered on other processors such as AMD’s said Karia.