Sep19

Scylla Open Source Release 2.3

Subscribe to Our Blog

Scylla Release

The Scylla team is pleased to announce the release of Scylla 2.3, a production-ready Scylla Open Source minor release.

The Scylla 2.3 release includes CQL enhancements, new troubleshooting tools, performance improvements and more. Experimental features include Materialized Views, Secondary Indexes, and Hinted Handoff (details below). Starting from Scylla 2.3, packages are also available for Ubuntu 18.04 and Debian 9.

Scylla is an open source, Apache Cassandra-compatible NoSQL database, with superior performance and consistently low latency. Find the Scylla 2.3 repository for your Linux distribution here.

Our open source policy is to support only the current active release and its predecessor. Therefore, with the release of Scylla 2.3, Scylla 2.2 is still supported, but Scylla 2.1 is officially retired.

Related Links     

New Distribution Support

Scylla packages are now available for:

AWS – Scylla AMI

The Scylla 2.3 AMI is now optimized for i3.metal as well. Read more on the performance gains of using bare-metal i3.metal compared to virtualized i3.16xlarge.

Tooling

  • CQL: Identify Large Partitions. One of the common anti-patterns in Scylla is large partitions. Recent releases addressed this issue and the handling of large partitions was greatly improved. However, we still had to solve issues where reading a large partition is less efficient and does not scale as well. With this release, Scylla now identifies large partitions and makes the information available in a system table. This allows you to identify large partitions, giving you the ability to fix them. The threshold for large partitions can be set in scylla.yaml:

compaction_large_partition_warning_threshold_mb parameter (default 100MB)

Example:  SELECT * FROM system.large_partitions;

> More on large partitions support in scylla 2.3

  • CQL Tracing: Added prepared statement parameters #1657
  • Scyllatop – The Scyllatop tool, originally introduced with the Scylla Collectd API, is now using the Scylla Prometheus API, same as the Scylla Monitoring Stack. Starting from Scylla 2.3, Collectd is not installed by default, but the Collectd API is still supported.  #1541 #3490
  • iotune v2. Iotune is a storage benchmarking tool that runs as part of the scylla_setup script. Iotune runs a short benchmark on the Scylla storage and uses the results to set the Scylla io_properties.yaml configuration file (formerly called io.conf). Scylla uses these settings to optimize I/O performance, specifically through setting max storage bandwidth and max concurrent requests.The new iotune output matches the new IO scheduler configuration, is time-limited (2 minutes) and produces more consistent results than the previous version.
  • Python re-write: All scripts, previously written in bash, such as scylla_setup, were re-written in Python. This change does not have any visible effect but will make future enhancements easier.  See Google Bash Style Guide on the same subject.

New Features in Scylla 2.3

  • CQL: Datetime Functions Support. Scylla now includes support for the following functions:

timeuuidtodate
timestamptodate
timeuuidtotimestamp
datetotimestamp
timeuuidtounixtimestamp
timestamptounixtimestamp
datetounixtimestamp

Example: SELECT * FROM myTable WHERE date >= currentDate() - 2d
#2949

  • CQL: JSON Function Support: Scylla now includes support for: JSON operations, SELECT JSON, INSERT JSON, and the functions: toJson() and fromJson(). It is compatible with Apache Cassandra 2.2, with the exception of tuples and user-defined types. #3708Example:

CREATE TABLE test (
   a text PRIMARY KEY,
   b timestamp);

INSERT INTO test JSON '{  "a" : "Sprint", "b" : "2011-02-03 04:05+0000" }';

  • CQL: Different timeouts for reads and range scans can now be set #3013
  • CQL: TWCS support for using millisecond values in timestamps #3152 (used by KairosDB and others)
  • Storage: Scylla now supports Apache Cassandra 2.2 file format (la sstable format). Native support for Apache Cassandra 3.0 file format is planned for the next major Scylla release.

Performance Improvements in Scylla 2.3

We continue to invest in increasing Scylla throughput and in reducing latency, focusing on reducing high percentile latencies.

  • Dynamic controllers for additional compaction strategies. A dynamic controller for Size Tiered Compaction Strategy was introduced in Scylla 2.2. In Scylla 2.3, we have added controllers for Leveled Compaction Strategy and Time Window Compaction Strategy.
  • Enhanced tagging for scheduling groups to improve performance isolation

Experimental Features

You are welcome to try the following features on a test environment. We welcome your feedback.

  • Materialized Views (MV)
    With Scylla 2.3, MV is feature-compatible with Cassandra 3.0, but is not considered production ready. In particular, the following are part of the release:

    • Creating a MV based on any subset of columns of the base table, including the primary key columns
    • Updating a MV for base table DELETE or UPDATE.
    • Indexing of existing data when creating an MV
    • Support for MV hinted handoff
    • Topology changes, and migration of MV token ranges
    • Repair of MV tables
    • Sync of TTL between a base table and an MV table
    • nodetool viewbuildstatus

The following MV functions are not available in Scylla 2.3:

    • Backpressure – cluster may be overloaded and fail under a write workload with MV/SI – planned for the next release
    • Unselected columns keep MV row alive #3362, CASSANDRA-13826 – planned for the next release
    • MV on static and collection columns
  • Secondary Indexes (SI)
    Unlike Apache Cassandra,
    Scylla’s SI is based on MV. This means every secondary index creates a materialized view under the hood, using all the columns of the original base table’s primary key, and the required indexed columns.The following SI functions are *not* available in Scylla 2.3:

    • Intersection between more than one SI and between an SI and a Partition Key – planned for the next release
    • Support for ALLOW FILTERING with a secondary index – planned for the next release
    • Paging support
    • Indexing of Static and Collection columns (same as for MV above)

Metrics Updates from Scylla 2.2 to Scylla 2.3

Next Steps

  • Scylla Summit 2018 is around the corner. Register now!
  • Learn more about Scylla from our product page.
  • See what our users are saying about Scylla.
  • Download Scylla. Check out our download page to run Scylla on AWS, install it locally in a Virtual Machine, or run it in Docker.

Tzach LivyatanAbout Tzach Livyatan

Tzach Livyatan has a B.A. and MSc in Computer Science (Technion, Summa Cum Laude), and has had a 15 year career in development, system engineering and product management. In the past he worked in the Telecom domain, focusing on carrier grade systems, signalling, policy and charging applications.


Tags: 2.3, Apache, AWS, Cassandra, cql, iotune, JSON, large partitions, materialized views, NoSQL, open source, partitions, release, release notes, scylla, ScyllaDB, scyllatop, upgrade, yaml