Scylla NoSQL Database Testing

On a 24×7 basis we hit Scylla with intensive NoSQL database testing. Many tests were adopted from Apache Cassandra and third parties, though we’ve also written some of our own. Many of the NoSQL performance tests were developed exclusively for clearing all possible database issues, from correctness, to data-loss to distributed behavior. Here are some highlights from our Scylla NoSQL testing blog series:

Apache Cassandra compatibility testing

Scylla is an Apache Cassandra compatible database. Thus, besides Scylla’s own tests, the project must pass the relevant Apache Cassandra test suite. This ensures that Scylla will handle the CQL query language exactly as Apache Cassandra does. Learn about Scylla / Apache Cassandra compatibility testing

Jepsen distributed systems testing

The latest hit on the distributed software testing scene is Kyle Kingsbury’s Jepsen. A flexible distributed systems safety research tool that causes hard-to-handle network outages, Jepsen partitions and delays for a variety of distributed database systems, including Apache Cassandra, and now Scylla. Learn more about Scylla / Jepsen testing

CharybdeFS database filesystem error testing

Testing for proper handling of filesystem errors is critical. We have developed a new emulated filesystem that can inject any error for any operation, under the control of a test script. CharybdeFS makes rare error conditions into repeatable filesystem error test cases. Learn how it works

Project Gemini data integrity testing

Distributed testing

Scylla distributed tests are an extended version of the dtest project originally developed for Apache Cassandra. Apache Cassandra dtests are functional black boxes that test Apache Cassandra cluster operation. By providing a test suite covering how a Apache Cassandra server should operate, distributed testing allows us to validate that Scylla is compatible in functionality with Apache Cassandra. Learn more about Scylla / Apache Cassandra distributed testing

Longevity testing

Longevity tests create a Scylla cluster in AWS and exercise it by running cassandra-stress and other clients specified in the test scripts. This testing finds problems in the operation of long-running clusters, as is the case with existing Scylla deployments. The system uses Boto for controlling Amazon AWS from Python. Learn more about Scylla longevity testing.