ScyllaDB Open Source Release 2.3

The ScyllaDB team is pleased to announce the release of ScyllaDB 2.3, a production-ready ScyllaDB Open Source minor release.

The ScyllaDB 2.3 release includes CQL enhancements, new troubleshooting tools, performance improvements and more. Experimental features include Materialized Views, Secondary Indexes, and Hinted Handoff (details below). Starting from ScyllaDB 2.3, packages are also available for Ubuntu 18.04 and Debian 9.

ScyllaDB is an open source, Apache Cassandra-compatible NoSQL database, with superior performance and consistently low latency. Find the ScyllaDB 2.3 repository for your Linux distribution here.

Our open source policy is to support only the current active release and its predecessor. Therefore, with the release of ScyllaDB 2.3, ScyllaDB 2.2 is still supported, but ScyllaDB 2.1 is officially retired.

Related Links     

• Get ScyllaDB 2.3
• Upgrade from ScyllaDB 2.2 to ScyllaDB 2.3
• Report a problem

New Distribution Support

ScyllaDB packages are now available for:

• Debian 9 #2541
• Ubuntu 18.04 #3598

AWS – ScyllaDB AMI

The ScyllaDB 2.3 AMI is now optimized for i3.metal as well. Read more on the performance gains of using bare-metal i3.metal compared to virtualized i3.16xlarge.

Tooling

• CQL: Identify Large Partitions. One of the common anti-patterns in ScyllaDB is large partitions. Recent releases addressed this issue and the handling of large partitions was greatly improved. However, we still had to solve issues where reading a large partition is less efficient and does not scale as well. With this release, ScyllaDB now identifies large partitions and makes the information available in a system table. This allows you to identify large partitions, giving you the ability to fix them. The threshold for large partitions can be set in scylla.yaml:

compaction_large_partition_warning_threshold_mb parameter (default 100MB)

Example:  SELECT * FROM system.large_partitions;

> More on large partitions support in scylla 2.3

• CQL Tracing: Added prepared statement parameters #1657
• ScyllaDBtop – The ScyllaDBtop tool, originally introduced with the ScyllaDB Collectd API, is now using the ScyllaDB Prometheus API, same as the ScyllaDB Monitoring Stack. Starting from ScyllaDB 2.3, Collectd is not installed by default, but the Collectd API is still supported.  #1541 #3490
• iotune v2. Iotune is a storage benchmarking tool that runs as part of the scylla_setup script. Iotune runs a short benchmark on the ScyllaDB storage and uses the results to set the ScyllaDB io_properties.yaml configuration file (formerly called io.conf). ScyllaDB uses these settings to optimize I/O performance, specifically through setting max storage bandwidth and max concurrent requests.The new iotune output matches the new IO scheduler configuration, is time-limited (2 minutes) and produces more consistent results than the previous version.
• Python re-write: All scripts, previously written in bash, such as scylla_setup, were re-written in Python. This change does not have any visible effect but will make future enhancements easier.  See Google Bash Style Guide on the same subject.

New Features in ScyllaDB 2.3

• CQL: Datetime Functions Support. ScyllaDB now includes support for the following functions:

timeuuidtodate
timestamptodate
timeuuidtotimestamp
datetotimestamp
timeuuidtounixtimestamp
timestamptounixtimestamp
datetounixtimestamp

Example: SELECT * FROM myTable WHERE date >= currentDate() - 2d
#2949

• CQL: JSON Function Support: ScyllaDB now includes support for: JSON operations, SELECT JSON, INSERT JSON, and the functions: toJson() and fromJson(). It is compatible with Apache Cassandra 2.2, with the exception of tuples and user-defined types. #3708Example:

CREATE TABLE test (
   a text PRIMARY KEY,
   b timestamp);

INSERT INTO test JSON '{  "a" : "Sprint", "b" : "2011-02-03 04:05+0000" }';

• CQL: Different timeouts for reads and range scans can now be set #3013
• CQL: TWCS support for using millisecond values in timestamps #3152 (used by KairosDB and others)
• Storage: ScyllaDB now supports Apache Cassandra 2.2 file format (la sstable format). Native support for Apache Cassandra 3.0 file format is planned for the next major ScyllaDB release.

Performance Improvements in ScyllaDB 2.3

We continue to invest in increasing ScyllaDB throughput and in reducing latency, focusing on reducing high percentile latencies.

• Dynamic controllers for additional compaction strategies. A dynamic controller for Size Tiered Compaction Strategy was introduced in ScyllaDB 2.2. In ScyllaDB 2.3, we have added controllers for Leveled Compaction Strategy and Time Window Compaction Strategy.
  
• Enhanced tagging for scheduling groups to improve performance isolation

Experimental Features

You are welcome to try the following features on a test environment. We welcome your feedback.

• Materialized Views (MV)
  With ScyllaDB 2.3, MV is feature-compatible with Cassandra 3.0, but is not considered production ready. In particular, the following are part of the release:
  • Creating a MV based on any subset of columns of the base table, including the primary key columns
  • Updating a MV for base table DELETE or UPDATE.
  • Indexing of existing data when creating an MV
  • Support for MV hinted handoff
  • Topology changes, and migration of MV token ranges
  • Repair of MV tables
  • Sync of TTL between a base table and an MV table
  • nodetool viewbuildstatus

The following MV functions are not available in ScyllaDB 2.3:

• • Backpressure – cluster may be overloaded and fail under a write workload with MV/SI – planned for the next release
  • Unselected columns keep MV row alive #3362, CASSANDRA-13826 – planned for the next release
  • MV on static and collection columns

• Secondary Indexes (SI)
  Unlike Apache Cassandra, ScyllaDB’s SI is based on MV. This means every secondary index creates a materialized view under the hood, using all the columns of the original base table’s primary key, and the required indexed columns.The following SI functions are *not* available in ScyllaDB 2.3:
  • Intersection between more than one SI and between an SI and a Partition Key – planned for the next release
  • Support for ALLOW FILTERING with a secondary index – planned for the next release
  • Paging support
  • Indexing of Static and Collection columns (same as for MV above)

• Hinted Handoff (see ScyllaDB 2.2)

Metrics Updates from ScyllaDB 2.2 to ScyllaDB 2.3

• scylla Grafana Monitoring project now includes ScyllaDB 2.3 dashboard on branch-1.0See here for all metrics changed in 2.3

Next Steps

• ScyllaDB Summit 2018 is around the corner. Register now!
• Learn more about ScyllaDB from our product page.
• See what our users are saying about ScyllaDB.
• Download ScyllaDB. Check out our download page to run ScyllaDB on AWS, install it locally in a Virtual Machine, or run it in Docker.