fbpx

Join us at P99 CONF, the virtual event on all things performance. Oct 19 & 20. Registration is free

See all blog posts

ScyllaDB Enterprise Release 2018.1.0

release
The ScyllaDB team is pleased to announce the release of ScyllaDB Enterprise 2018.1.0, a production-ready ScyllaDB Enterprise major release.

The ScyllaDB Enterprise 2018.1 release is based on ScyllaDB Open Source 2.1 and includes new features which are only offered to our Enterprise customers, Enterprise version bug fixes, and backported bug fixes from upstream releases 1.7, 2.0, 2.1 and master.

Read more about our enterprise NoSQL features here.

Related Links

ScyllaDB Enterprise customers are encouraged to upgrade to ScyllaDB Enterprise 2018.1 and are welcome to contact our Support Team with questions.

New Features in ScyllaDB Enterprise 2018.1

Enterprise only features in 2018.1

  • Auditing
    ScyllaDB Enterprise includes an auditing capability, which allows administrators to track system and users activity for deployments with multiple users.
    ScyllaDB auditing can work in Keyspace, Table or Category granularity. Categories are:
    AUTH Logs login events
    DML Logs insert, update, delete, and other data manipulation language (DML) events
    DDL Logs object and role create, alter, drop, and other data definition language (DDL) events
    DCL Logs grant, revoke, create a role, drop a role, and list roles events
    QUERY Logs all queriesThe audit log can be output the system log, or to a ScyllaDB table. More on Auditing in ScyllaDB Enterprise: http://docs.scylladb.com/operating-scylla/auditing.
  • IBM Power support
    ScyllaDB Enterprise 2018.1 is also available on the IBM POWER8 architecture, running on CentOS 7 and RHEL.

New Features in 2018.1
(Where mentioned, issue numbers are from the upstream open source project)

  • Counters. Available in ScyllaDB open source from 1.7, Counters are now part of ScyllaDB Enterprise
    More on Counters: http://docs.scylladb.com/using-scylla/counters/
  • Time Window Compaction Strategy (TWCS). An improvement and simplification to Date Tiered Compaction Strategy (DTCS) as the go-to strategy for time series data. TWCS uses Size-Tiered Compaction Strategy (STCS) on “time windowed“ groups of SSTables, using the maximum timestamp of the SSTable to decide which time window each SSTable will be part of. #1432
  • Row Cache. In Pre 2.0 releases, ScyllaDB cached full partitions only and we chose not to cache partitions larger than 10MB. With the introduction of row cache, ScyllaDB can cache a single CQL row out of a large partition, removing this limitation. This change will significantly improve ScyllaDB performance for use cases with large partitions such as Time Series.
  • Heat Weighted Load Balancing. When a ScyllaDB node restarts for any reason (upgrade, config update, error), it starts with an empty cache. This means the node read path will always hit the disk until it’s cache is repopulated. During this time, this node will be slower, and as a result, can slow down the entire cluster. Heat Weighted Load Balancing solves this issue by optimizing the node selection in the coordinator based on the heat of each node. A hot node, with a good cache-hit-ratio, will get more reads requests than a cold node with the same data and a lower cache hit ratio. After some time, the cold node will become hot, and the load balancing will become uniform again. #1455. More on Heat Weighted Load Balancing
  • Improved Performance for CPU-bound workloads. More here.
  • DNS resolution. You can now use DNS names (FQDN), not only IPs, in scylla.yaml for seed nodes, listen_address, broadcast_address and broadcast_rpc_address #2357.
  • CQL additions:
    • Cast functions between types, for example: SELECT avg(cast(count as double)) FROM myTable.
    • Support Duration data type #2240.
    • CQL prepared statements are now properly evicted so that you’re not able to OOM a server just by preparing many unique statements. See #2474
    • Support non-reserved keywords as columns (without quotes) #2507, for example, a column named “frozen”.
    • Update to CQL 3.3.1, to align with Cassandra 2.2 release
    • New data types: tinyint, smallint, date, and time #1284
      Binary protocol v4 (required by the new data types).
    • Support UNSET in prepared statements #2039
    • CQL aggregate functions now support floating point data types #2168
  • Transitional Auth – To enable access control without downtime, use transitional modes to temporarily support applications and users that do not have accounts without interrupting services. More here
  • New REST API /storage_service/force_terminate_repair allows aborting of a running repair and all the data streams related to it. #2105.
  • Ext4 support. ScyllaDB now supports the ext4 file system in production, not just in dev mode. Using XFS is still recommended for best performance #2381.
  • CPU isolation. Improve latency using isolation between background tasks like memtable flushes and compaction and foreground tasks like CQL read/write. More here
  • Java tools. Java tools provided with ScyllaDB are upgraded to match Cassandra 3.0. In particular:
    • sstableloader supports Cassandra 3.0 sstable format, allowing easy migration from Cassandra 2.2 and 3.x clusters to ScyllaDB
    • sstable2json is renamed to sstabledump
    • sstablesplit, sstableupgrade, sstablescrub, json2sstable, sstablekeys, and token-generator are temporarily removed (all break compatibility to current SSTable format)
  • CompressionInfo.db files, part of the SSTable format, are stored more efficiently in memory, allowing higher disk: RAM ratios #1946.

System Impacts

  • Installation updates
    Starting from ScyllaDB Enterprise 2018.1, ScyllaDB packages for Ubuntu 16.04 and Debian 8 are signed. You can find the keys and instructions per Linux distribution here.
  • Upgrade procedure. You can only upgrade to ScyllaDB Enterprise 2018.1 from ScyllaDB Enterprise 2017.1.6 or later, or from ScyllaDB Open Source 2.1 or later. Upgrade from earlier ScyllaDB releases is not supported.
  • Kernel support
    ScyllaDB Enterprise 2018.1 requires Linux kernel 3.10.0-514 or later.
  • Driver support
    Relevant for upgrading from ScyllaDB Enterprise 2017.1 (ignore if you are upgrading from ScyllaDB Open Source 2.1).Certain older versions of Cassandra drivers are no longer supported. With the introduction of MV, ScyllaDB has moved to Cassandra 3.0 compatible system tables. This means that certain old Cassandra drivers, which do not work with Cassandra 3.0, will not work with ScyllaDB.To ensure your application will work properly after a ScyllaDB upgrade, we highly recommend upgrading your Cassandra driver before upgrading to ScyllaDB 2018.1. The latest drivers are backward compatible and will work with all ScyllaDB 2017.1, 1.x and 2.x releases, past, and future.At the very least, your Cassandra driver should be in the list below, using the version listed or later.
    – C++ : 2.3+
    – C#: 3.0+
    – Java: 3.0+
    – Node.js: 3.0+
    – PHP: 1.2+
    – Python: 3.5+
    – Ruby: 3.0+
    – GoCQL commit 8d27cdeae224463cadc1ee973386862baaf6662e (Mar 1, 2016) or later.
  • Counter Migration. ScyllaDB does not support Counter SSTables created by Apache Cassandra 2.0 or older. Note that if you upgraded from Cassandra 2.0 to 2.1, you may still have old Counters in the SSTables.The proper way to migrate Counters data from Cassandra to ScyllaDB is to run a major compaction on the Cassandra node, and use sstableloader to load the SSTable to ScyllaDB.
  • Deprecated partitioners. Byte-ordered and random partitioners were deprecated in ScyllaDB Open Source 2.0, and ScyllaDB Enterprise 2018.1 and will be removed in a future version. Please open a support ticket if you use or are planning to use these partitioners.
  • New intra-node sharding algorithmThe intra-node sharding algorithm determines how data is divided among cores in a node. The change results in a better distribution of data among cores, especially in large clusters. This improves performance across the cluster. When upgrading from 2017.1, please check scylla.yaml for murmur3_partitioner_ignore_msb_bits. If you do *not* have this line, please consult ScyllaDB support on how to safely upgrade.

Metrics updates from 2017.1

ScyllaDB Monitoring stack now includes ScyllaDB 2018.1 dashboard. For a full list of metrics, update see here.

Known Issues

When installing ScyllaDB Enterprise on Power Architecture, scylla_setup fail to install node_exporter. Without node_exporter OS level metric is not reported to ScyllaDB Monitoring Stack. Bypass: install node_exporter packages using the public repo.

Next Steps

  • Learn more about ScyllaDB from our product page.
  • See what our users are saying about ScyllaDB.
  • Download ScyllaDB. Check out our download page to run ScyllaDB on AWS, install it locally in a Virtual Machine, or run it in Docker.

About Tzach Livyatan

Tzach Livyatan has a B.A. and MSc in Computer Science (Technion, Summa Cum Laude), and has had a 15 year career in development, system engineering and product management. In the past he worked in the Telecom domain, focusing on carrier grade systems, signalling, policy and charging applications.