The ScyllaDB team is pleased to announce the release of ScyllaDB 1.7, a production-ready ScyllaDB minor release. ScyllaDB is an open source NoSQL database compatible with Apache Cassandra, with superior performance and consistently low latency. From now on, critical bugs will be fixed in 1.7 and 1.6 release series only. If you are still using open source ScyllaDB 1.5 or earlier release – you are encouraged to upgrade. We will continue to fix bugs and add features on the master branch toward 1.8 and beyond.
Related Links
- Get started with ScyllaDB 1.7
- Upgrade from 1.6.x to 1.7.y
- Report a problem
New features in ScyllaDB 1.7
Counters (Experimental)
ScyllaDB now supports Counters as a native type. A counter column is a column whose value is a 64-bit signed integer and on which two operations are supported: incrementing and decrementing. More about Counters in CQL here[1]. Counter support is complete but will stay experimental until more testing is done and performance improves. To enable Counters add “experimental: true” to scylla.yaml
For a Counters usage example, including CREATE TABLE, UPDATE, and SELECT – read ScyllaDB engineer Paweł Dziepak’s Counters blog post.
Java tools
Java tools provided with ScyllaDB[2] are upgraded to match Cassandra 3.0. In particular:
- sstableloader supports Cassandra 3.0 sstable format, allowing easy migration from Cassandra 2.2 and 3.x clusters to ScyllaDB
- sstable2json is renamed to sstabledump
- sstablesplit, sstableupgrade, sstablescrub, json2sstable, sstablekeys and token-generator are temporarily removed (all break compatibility to current SSTable format)
CQL
Update to CQL 3.3.1, to match Cassandra 2.2 release. The changes to CQL 3.3.1 are listed here [3]. More detailed compatibility matrix between ScyllaDB 1.7 and Cassandra 2.2[5] and a list of gaps[4] More CQL updates:
- New data types: tinyint, smallint, date, and time. Example below #1284
- Binary protocol v4 (required by the new data types).
- Support UNSET in prepared statements #2039
- Advertise Cassandra 2.2.8 version from ScyllaDB so that drivers correctly detect the presence of CQL 3.3.1
- CQL aggregate functions now support floating point data types #2168
Debian Support
Experimental in 1.6, Debian Support is now ready for production. ScyllaDB 1.7 new intra-node sharding algorithm improves performance on large clusters. See below for description and impact.
- Improved performance by setting Linux scaling governor to “performance” (“powersave” on some Linux distros) #1895
System impacts – please read carefully
- New intra-node sharding algorithm
The intra-node sharding algorithm determines how data is divided among cores in a node. The change results in a better distribution of data among cores, especially in large clusters. This improves performance across the cluster. By default, the new resharding algorithm is only applied to new clusters; upgraded installations continue to use the older algorithm. If you have a large cluster and suffer from an uneven distribution of load across shards, you may want to switch to the new algorithm. To update an existing cluster (upgraded from 1.6) you should add the following to scylla.yaml
murmur3_partitioner_ignore_msb_bits: 12
and restart the scylla-server service. Please note, however, that after this configuration change the node will re-write all of the data according to the new format; this may temporarily use additional disk space and may impact performance and latency.
- Fix of Memtables flushed too early on many-core machines #2046 will increase disk usage by up to the amount of memory installed on the system. For example, if you have 128 GB RAM, commitlog will now occupy up to 128 GB of disk space. If your system storage is close to exhaustion, make sure to free some of it before the upgrade, or increase the amount of available storage. Alternatively, set the commitlog_total_space_in_mb to an amount that will leave the system enough free space to work.
New Data Type usage example
CREATE TABLE example (smallvalue SMALLINT, tinyvalue TINYINT, adate DATE, atime TIME, PRIMARY KEY(smallvalue));
INSERT INTO example (smallvalue,tinyvalue,adate,atime) VALUES (1,2,'2017-05-22','12:00:00.123');
More on CQL data types here[1]
Noteworthy new and updated metrics in ScyllaDB 1.7
There are many changes to metrics names between 1.6 to 1.7. For the full list see https://github.com/scylladb/scylla/wiki/Metrics-migration-1.6-to-1.7 If you are using scylla-grafana-monitoring, make sure to pull the latest release, and use 1.7 dashboards.
Noteworthy Bug fixes
- node_exporter, a Prometheus client reporting server metrics run as root #1968
- Writes to system keyspace tables are bypassing commitlog #1986
- Intentional short reads causing slowdown on sparsely populated tables #1995
- Wrong warning on low memory condition #2002
- System table “schema_columnfamilies” “comparator” column value incompatible with Cassandra #2019
- Installation on clean debian8 produce errors for missing lspci #2035
- Adding a new node with authentication set – causes system_auth keyspace RF to become 1 for the entire cluster. With RF=1, a node down might mean no access to the cluster #2129
- Streaming might fail when using compressed files, generating the following error message:
- stream_transfer_task: Fail to send to ….: std::out_of_range (deque::_M_range_check: __n (which is 64)>= this->size() (which is 64)) #2143
- Scanning and flush memtable may cause bad_alloc even if there is enough memory #2138
- Error during sstable flush may cause ScyllaDB to exit #2122, #2121
- Using Node-to-node SSL gave the wrong logs message. It have been updated to “Starting Encrypted Messaging Service on SSL port…” #1845
- ScyllaDB may exit during repair process #2148
- scylla_setup: setting DISCARD RAID0 is broken on CentOS #2033
- ScyllaDB housekeeping check for new version on restarts #1953
- Memtables were flushed too early on many-core machines, resulting in small sstables and high compaction load #2046 (see above for impact on disk space)
- Improved seastar performance on very large machines by reducing lock contention in the kernel during signal delivery
- ScyllaDB generated std::bad_alloc error when running commitlog replay (after unexpected restart) #2098
- ScyllaDB Docker image: Prometheus port is now open by default #1827
- ScyllaDB (and Cassandra) golang driver, gocql, may crash when Probabilistic Tracing or Slow Query Log are enabled #2179
- Commitlog replay, after a ScyllaDB restart, can fail with std::out_of_range #2173
- scylla_setup: discarding blocks on large RAID volume takes much time with no feedback to the user #1896
- nodetool cfhistograms missing information #2155
- Repair: Repair of large dataset (1TB) streams too much data than actually needed #2299
- DateTiredCompaction: fully expired sstable is unconditionally chosen for compaction which may lead to a compaction loop under certain conditions #2260
- CQL SELECT: In some cases, a query with CQL SELECT from a large partition (>10MB) may only return subset of the stored rows #2272 #2327
- CQL SELECT: on some cases, reading from a large partition using WHERE restrictions on the clustering key, can return old values since some tombstones are ignored. #2326
- Reading large partition when information about that partition being large is cached may invoke undefined behavior #2377
Known issues
- ScyllaDB AMI does not boot on c3.large (its does on other c3 instance types) #2186
References
[1] http://cassandra.apache.org/doc/latest/cql/changes.html#id9
[2] https://github.com/scylladb/scylla-tools-java
[3] https://github.com/apache/cassandra/blob/cassandra-2.2/doc/cql3/CQL.textile#changes
