Database administrators respond by saying, "If you don't want the overhead of a database and you don't need transactional consistency, just use a file." And that's pretty much what application developers are doing and driving the NoSQL movement. The desire to store more data and inspect much larger data sets across distributed computing platforms has become a necessity for many Web-based applications. All without having to spend millions of dollars on a giant data warehouse. By reducing the complexity of the database engine, and by defining -- as they go -- how to store and reference that data, they get a lot better performance. It's a database, but it lacks most of the features and functions of relational database technology.
So what is NoSQL? That's a hard question to answer because NoSQL is more of a design concept than an actual product. Basically, NoSQL implementations are a storage engine. Most NoSQL variants strip away SQL statement parsing because NoSQL is not structured relationally. In many cases, there is no structure at all, just data with its index value. Without a relational structure, you remove the need for query execution plans, table concurrency, and locking for read consistency.
By removing all that code and complexity from the database engine, along with incredibly cheap memory from cloud service providers, most of the database code and data indices can be stored in memory. By removing tasks that require a lot of processing overhead and placing a higher percentage of code in memory, you speed operation by (nearly) an order of magnitude. Cheap and fast is a very attractive option!
I am certain there are some firms (Google, Amazon, and Facebook) and project managers out there who will be offended by my definition because BigTable, Dynamo, SimpleDB, and Cassandra are used by millions each day, whether or not they know it. But it's not off-the-shelf software. And for every relational trait a NoSQL database is supposed to forgo, you can always find one or more project or product variants that offers things like consistency, replication, or complex storage options.
Make no mistake: I am not trying to marginalize the efforts of the developers behind these projects or the need for this kind of tool. NoSQL is here and will be for a long time because it solves several fundamental problems for application developers. But don't confuse NoSQL with relational database. Think of it in terms of document storage.
Adrian Lane is an analyst/CTO with Securosis LLC, an independent security consulting practice. Special to Dark Reading.