NoSQL is a non-relational database that does not typically use Structured Query Language (SQL) to retrieve information. NoSQL databases were developed for use cases where a traditional relational database is not sufficient due to the size (volume), type (variety) or speed (velocity) of big data.
Structured Query Language (SQL) is a database management language, and currently the most popular method of accessing data from and inputting data to a relational database management system (RDBMS). SQL databases have been around for decades. First invented in 1974, SQL has continued to evolve over the years. Because it is such an important industry standard, it is controlled by the International Electrotechnical Commission (IEC). The latest version is formally defined in ISO/IEC 9075:2016.
SQL databases are comprised of records organized into tables. In a single table, data is organized into rows, labeled with a primary key, and with many columns (columnar databases) to hold values for different kinds of data – typically large data sets. These tables are related to each other through the use of foreign keys, which join together the different tables in the database.
A schema is used to define all of the tables, their composite columns and data types, and the means of joining the tables together. The structure of a relational database can be illustrated by a design referred to as an Entity Relationship Diagram (ERD).
A NoSQL database (NoSQL DB) is designed to handle massive amounts of distributed data. NoSQL database management systems store and retrieve data in a variety of ways other than the joined tabular models of relational database management systems (RDBMS). These databases can handle different types of data and can accommodate many kinds of data models including key-value stores, document, wide-column and graph formats.
The advantages of using NoSQL databases are for modern applications that need fast horizontal scaling and for those with extremely large, high velocity, distributed datasets. Fully managed NoSQL DBaaS solutions are increasing in popularity for use cases where a team of experts is needed to manage the complexity and upkeep of such databases.
The term “NoSQL” was coined by Carlo Strozzi in 1998. He had invented a form of RDBMS that used a different form of data access than SQL. Note that his database was still a relational database — tables still had joins — but the method of accessing it, using Unix pipes, did not rely on the SQL language.
Modern NoSQL databases came about a decade later with the advent of massive Internet e-commerce platforms and social media. While in the early Internet and web era many SQL databases were adapted for data applications (such as the use of open source MySQL to power WordPress sites), there were architectural limitations to SQL servers that made it too inflexible and unscalable for entire new classes of problems that needed to be solved. New data models were required. These included massive (but relatively simple) key value stores, highly distributed and highly available wide column stores, and more unique data models such as document based stores and graph databases.
Sometimes they are also referred to as “Not only SQL” or “NewSQL” to show that they may actually support Structured Query Language or can work in tandem with a SQL database. A SQL-based interface can still be useful, and these databases support hybrid operations, doing both SQL and Non-SQL related data management.
Many types of NoSQL databases are used for applications with very large datasets and some operate in real time. They are used by enterprises that need features and capabilities more flexible than a traditional database, which can accommodate many kinds of data models in a variety of formats (flexible). A NoSQL database provides the following data management features not found in relational databases:
The flexible architecture of NoSQL databases is among the top reasons why they are an excellent choice for today’s demanding application requirements. Other advantages for when to use NoSQL are as follows:
| SQL | NoSQL | |
| Orientation | Relational | Generally non-relational |
| Schema | Strict and rigid schema design and data normalization | Loose and more varied designs for unstructured and semi-structured data; data is generally denormalized |
| Language | Structured Query Language (SQL) for defining, reading and manipulating data. Supports JOIN statements to relate data across tables. | There are different languages for querying, some quite similar to SQL, such as Cassandra Query Language (CQL) for wide column databases, or others radically different, such as using object-oriented JSON for document databases. |
| Scalability | Vertically scalable. Loads on a single server can be increased with CPU, RAM or SSD. | Generally designed for horizontal scalability. Increased traffic can be handled by adding more servers in the database. This is useful for large and frequently changing datasets. |
| Structure | Table-based, which is efficient for applications using multi-row transactions or systems that were built with a relational structure. | NoSQL database structure is variable, and can be based on documents, key-value pairs, graph structures or wide-column stores. |
| Db2 | IBM’s relational database, originally released in 1983 and created to run on mainframes; over time it was also ported to Unix and Windows; now supports Linux servers. |
| MariaDB | An open source SQL server, forked from MySQL; was created in 2009 when MySQL was acquired by Oracle. |
| MemSQL | A distributed SQL server designed to run from memory (RAM) for fastest performance. |
| Microsoft SQL Server | Microsoft’s SQL server, originally designed to run on OS/2, became the mainstay of databases to run on Microsoft Operating Systems. In 2017 it was ported to run on Linux. |
| MySQL | An open source SQL server widely adopted by web and application developers. In 2008 it was acquired by Oracle; now available as both open source and proprietary enterprise editions. |
| Oracle | First released in 1979, Oracle became and remains widely adopted in enterprises over the next decades. |
| PostgreSQL | An open source SQL server that grew out of, and eventually replaced, the 1980s Ingres project at UC Berkeley. |
The NoSQL database structures include the following classifications:
— Each item is stored as a “key” (unique identifier) along with its value. This is the most simple version of a. Examples include Redis and Aerospike.
— Each key is paired with a structured data “document” (data structure). Documents can contain many different key-value pairs in a nested, hierarchical format, such as Javascript Object Notation (JSON).
— These are actually two different types of databases. In a columnar database (also called a column-oriented database), data is stored in columns instead of rows, which makes it more efficient to query data in frequently-referenced columns. For example Apache Druid or Clickhouse. Wide column stores are actually row-oriented databases, but they will have a partition key to distribute data, a clustering key to define how data is grouped, and then values across cells in multiple columns. For this reason, they may be referred to as “key-key-value” databases. Examples of wide column stores include Scylla, Apache Cassandra, and DataStax Enterprise.
— Hold social connections and other networks of data information. Examples include Neo4J and Giraph.
In general, NoSQL databases have active communities of developers who contribute to open source, and thus technically offering free NoSQL databases. Many of these also provide a commercial, enterprise NoSQL database that add additional features, as well as production support. A free NoSQL database typically only provides community support, with no guarantee that issues will be fixed or new features added in a timely fashion. Small projects can generally begin development on a free open source NoSQL database, and then upgrade to an enterprise version once the project achieves traction.
Free NoSQL is typically available to run locally, on a developer’s machine but also can be hosted and managed in the cloud. Many NoSQL database providers also provide free NoSQL hosting that runs in the cloud as database-as-a-service (NoSQL DBaaS) offerings. Developers can easily find a free NoSQL database online as well as free NoSQL database hosting from a range of providers.
Modern enterprises do not view SQL and NoSQL as an either/or proposition. A survey at the latest DeveloperWeek conference found that 44 percent of organizations use multiple databases. Among those, 75 percent use a combination of SQL and NoSQL databases.
Many organizations take advantage of the ways that these two databases can complement each other. Each brings its own strengths, making the sum greater than each part.
SQL databases handle structured data and standardize how elements relate to one another. NoSQL databases are popular when flexibility is a concern. They can create unique data structures that include documents, graphs or columns.
NoSQL database schemas are often exhibit a more flexible design without enforcing the normalized data forms and pre-defined schemas and structures common in SQL databases.
One benefit is the ability to easily transition from simple key-value stores to complicated graph databases when integrated with a graph database.
Since this is an emerging technology, solutions for NoSQL data modeling are still being developed. As more enterprises utilize NoSQL database features, the design will mature with the following benefits:
NoSQL databases such as key-value systems often have far simpler schemas, and thus, they tend to be faster than SQL systems. SQL databases tend to allow for more complex queries, exemplified by making JOINs across tables. Naturally, there is a computational cost, and time required, for such complexity. There are many additional architectural factors that go into a specific database’s performance, such as the use of in-memory tables or RAM caches, various data structures that can help efficiency (such as bloom filters) and so on. Even the programming language a database is written in can affect system performance, as it may allow for, or prohibit, low-level hardware optimization. Thus it is not possible to categorically say that NoSQL databases are always faster than SQL databases (or vice versa). Users need to create and conduct realistic tests to see how various databases perform under specific conditions, data models, and querying patterns.
They are indexed with keys that correspond with the location of an associated piece of data. The most common indexing methods are B-Tree, T-Tree, O2-Tree, as well as Log Structured Merge (LSM) tree indexing.
— Allows a variable number of child nodes, which results in more unused space and less tree balancing. The B+Tree is a popular version of B-Tree indexing where every key must reside in the leaves.
— Features three kinds of nodes: A T-Node that has a right and left child, a leaf node with no children, and a half-leaf node with only one child. Each node stores multiple tuples, or lists. The use of binary search results in more efficient storage.
— Created to enhance existing indexing methods by placing tuples within each leaf node. Advances the concept of a Binary-Search tree.
— Creates an in-memory hash table (or memtable), and periodically write these structures to disk in immutable form, known as a Sorted Strings Table (SSTable). Over time, these SSTables are merged in a process known as compaction.
The top NoSQL databases, and often most used versions are intended for high performance speed and volume. Big data and NoSQL databases are a good match because of flexible design that allows for a many kinds of datasets in many different formats.
The trade-off for speed within a large dataset is less consistency. While SQL databases provide the highest level of verification, NoSQL does not promise total data consistency.
But with companies like Amazon where each transaction requires a tremendous amount of read-write activity, a SQL database would not be able to match the amount of speed and scaling required for millions of real-time transactions. That is why NoSQL ecommerce and NoSQL systems for transactional data are vital for applications that serve high volumes of customers in real time.
The NoSQL database examples include the following:
| Database | NoSQL Data Models | Description |
| Aerospike | Key-value | A flash-optimized and in-memory open source NoSQL database |
| Amazon DynamoDB | Key-value, Document | Cloud-based database only offered on AWS |
| Apache Cassandra | Column Time series | An open source highly scalable and distributed database created at Facebook to handle massive amounts of structured data. |
| Apache CouchDB | Document | Open source and web-oriented database. |
| Apache HBase | Column | Open source and column store database developed as a part of Hadoop. |
| Google Bigtable | Column | A compressed, high performance, proprietary data storage system. |
| JanusGraph | Graph | An open source, distributed graph database under The Linux Foundation; works on top of Scylla or Apache Cassandra. |
| Microsoft Cosmos DB | Document Columnar Graph | Proprietary, schema-agnostic and horizontally scalable. |
| MongoDB | Document | Document-oriented database by MongoDB. |
| Neo4j | Graph | ACID-compliant and transactional graph database with native graph storage and processing. |
| Oracle NoSQL Database | Key-value | Scalable, distributed NoSQL database, designed to provide highly reliable, flexible and available data management. |
| Redis | Key-value Document Time series | Data structure server with keys can contain strings, hashes, lists, sets and sorted sets. |
| Scylla | Key-value Column Time series | Enterprise and open source database which is a drop-in alternative to Apache Cassandra offering higher performance, lower latency and reduced cost. |
Apache® and Apache Cassandra® are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. Amazon DynamoDB® and Dynamo Accelerator® are trademarks of Amazon.com, Inc. No endorsements by The Apache Software Foundation or Amazon.com, Inc. are implied by the use of these marks.