NoSQL Database Movement Gains Ground

The buzz around the NoSQL movement in the past year has grown considerably, to the point where advocates organised a one-day conference in Boston just last week to discuss its future.

Recent announcements from Twitter and Digg.com supporting a NoSQL approach added fuel to this buzz, and while its ultimate growth among enterprises remains a subject of discussion, one thing is clear – the idea of a NoSQL database is starting to get a new level of consideration in the enterprise.

Made up of a variety of non-relational data stores, the NoSQL movement spans open source projects such as Cassandra, CouchDB and MongoDB as well as products from companies like Mark Logic, which makes a native X M L database. While the hype around the subject is relatively new, various NoSQL alternatives such as data grids, Google BigTable, Amazon Dynamo and other solutions are not, and have actually been around for years, noted Nati Shalom, CTO of GigaSpaces.

“As with many trends it’s probably a convergence of trends rather the something particular (that led to the rise of NoSQL),” Shalom said. “On the business side social networking has changed quite significantly our web experience, from read mostly websites we’re turning into heavy read/write sites like Twitter and Facebook with lots of content driven by users rather then the site provider…Under that condition, lots of common practices such as static provisioning based on peak load and read mostly clusters started to break completely and forced a completely different way of doing things.”

In addition, the data centre landscape is changing, he added, explaining that memory resources are now available at a greater capacity and at lower cost, and that network bandwidth and computing power are growing as well.

“Increasingly, the question is how to take advantage of these powerful and cost-effective resources,” Shalom told eWEEK. “The relational database, which has been the storage system of choice for several decades, was built under the assumption that memory is scarce and the network is a bottleneck – plus it doesn’t scale to the level required by today’s large-scale applications. Virtually every popular web application has found that a single relational database cannot meet its throughput requirements.”

The movement certainly has its adherents in the Web 2.0 world. Facebook open sourced the Cassandra database in 2008. Since then, Digg.com has chosen the database over MySQL, and Twitter is reportedly planning to do the same.

“Each system has its own strengths and weaknesses, but Cassandra has a good blend of everything,” Digg’s Ian Eure noted in a September blog post announcing their plan to switch to Cassandra. “It offers column-oriented data storage, so you have a bit more structure than plain key/value stores. It operates in a distributed, highly available, peer-to-peer cluster. While it’s currently lacking some core features, it gets us closer to where we want to be than the other solutions.”

But the growing uptake should not be taken as an indication that SQL is dead. For one thing, NoSQL databases lack the focus on consistency found in relational databases, trading that instead for a focus on availability and partition tolerance.

Matt Aslett, an analyst with The 451 Group, said that while distributed column stores like Cassandra have found a home in some situations, it’s the document-oriented databases like MongoDB, CouchDB and Riak that hold the most promise for enterprises in the short-term.

“While there will be isolated examples of key value and distributed column store adoption, for the most part they are projects that have been designed to fulfill the very unique requirements of their creators,” he told eWEEK in an email. “The document-oriented databases are also differentiated from the key value and distributed column stores in that they are the products of traditional start-up ventures rather than user organisations. As yet there are no commercial support providers for Cassandra, Voldemort or Tokyo Cabinet.”

By comparison, MongoDB was created by 10gen, CouchDB by Couchio and Riak by Basho Technologies, all of which are traditional venture-backed start-ups providing support, services and consulting, he said.

Rather than replace relational databases, Shalom sees a future where NoSQL and relational have a happy marriage.

“Currently there are two approaches to marry the RDBMS and NoSQL worlds,” he said. “One (is) taking a NoSQL implementation and wrapping it up with SQL façade. This is where Google as well as GigaSpaces are already providing solution that can support standard SQL semantics on top of a NoSQL storage. Two – taking an existing RDBMS and making it cloud enabled. This is where SQL Azure from Microsoft and Amazon RDS start to provide a SQL solution that has NoSQL flavor built into it.”

“I see a convergence of RDBMS and NoSQL to the point where the difference between those two models will start to blur,” he said.

To Mark Logic CEO Dave Kellogg, one of the most interesting facets of the NoSQL movement is that it shows innovations in the database space are not just coming from Oracle, IBM and Microsoft.

“I don’t think it hurt things any that the relational database market itself had become this comfortable oligopoly, where it really coalesced around the big three vendors…I think the CIO …his eyes are going to open to the realization that you don’t just have to use ( IBM ) DB2, Oracle or (Microsoft) SQL Server,” Kellogg said.