The ScyllaDB team is pleased to announce the availability of ScyllaDB Open Source 4.1.0, a production-ready release of our open source NoSQL database.
ScyllaDB is an open source, NoSQL database, with superior performance and consistently low latencies. Find the ScyllaDB Open Source 4.1 repository for your Linux distribution here. A ScyllaDB Open Source 4.1 Docker image is also available.
ScyllaDB 4.1 is focused on stability and bug fixes, as well as adding more operations to the Alternator API.
In addition, we are adding support for Red Hat Enterprise Linux 8, CentOS 8, and Debian 10.
Please note that only the latest two minor releases of ScyllaDB Open Source project are supported. Starting today, only ScyllaDB Open Source 4.1 and ScyllaDB 4.0 will be supported, and ScyllaDB 3.3 will be retired.
- Get ScyllaDB Open Source 4.1 as binary packages (RPM/DEB), AWS AMI and Docker instance
- Upgrade from ScyllaDB 4.0 to ScyllaDB 4.1
- Report a problem
New features in ScyllaDB 4.1
- ScyllaDB is now available for Red Hat Enterprise Linux 8, CentOS 8, and Debian 10
- Alternator: add mandatory configurable write isolation mode #6452
Alternator allows users to choose the isolation level per table. If they do not use one, a default isolation level is set. In ScyllaDB 4.0 the default level was “always”. In ScyllaDB 4.1, the user needs to explicitly set the default value in scylla.yaml:
forbid_rmw– Forbids write requests which require a read before a write. Will return an error on read-modify-write operations, e.g., conditional updates (
only_rmw_uses_lwt– This mode isolates only updates that require read-modify-write. Use this setting only if you do not mix read-modify-write and write-only updates to the same item, concurrently.
always– Isolate every write operation, even those that do not need a read before the write. This is the slowest choice, guaranteed to work correctly for every workload. This was the default in 4.0,
For example, the following configuration will set the default isolation mode to always, which was the default in ScyllaDB 4.0:
Note: Alternator 4.1 will not run without choosing a default value!
- Allow access to system tables from Alternator REST API. If a Query or Scan request intercepts a table name with the following pattern:
.scylla.alternator.KEYSPACE_NAME.TABLE_NAME, it will read
the data from ScyllaDB’s
Example: in order to query the contents of ScyllaDB’s system.large_rows, pass
TableName='.scylla.alternator.system.large_rows'to a Query/Scan request.
- Add support for
Alternator now support the ScanIndexForward option of a query expression
By default, query sort order is ascending. Setting
ScanIndexForwardto False parameter reverses the order.
See AWS DynamoDB Query API
- SERIAL reads and lightweight transactions which had no impact on the database (didn’t apply) are now fully linearizable. #6299
(See Apache Cassandra open issue : https://issues.apache.org/jira/browse/CASSANDRA-12126)
- Security: Support SSL Certificate Hot Reloading
ScyllaDB supports hot reloading of SSL Certificates. If SSL/TLS support is enabled, then whenever the files are changed on disk, ScyllaDB will reload them and use them for subsequent connections.
- CQL: support Filtering on counter columns #5635
- Compaction: Allow Major compactions for TWCS. Make major compaction aware of time buckets for TWCS. That means that calling a major compaction with TWCS will not bundle all SSTables together, but rather split them based on their timestamps. #1431
- Restore: ScyllaDB no longer accepts online loading of SSTable files in the main directory. New tables are only accepted in the upload directory. Till 4.1 both options worked, starting 4.1 the first will return an error.
- ScyllaDB-tools-java are now based on Apache Cassandra 3.11 tools and use Java 11.
Experimental features in ScyllaDB 4.1
- Change Data Capture (CDC). While functionally complete, we are still testing CDC to validate it is production ready towards GA in a following ScyllaDB 4.x release. No API updates are expected.
Selected updates from ScyllaDB 4.0:
- User Defined Functions #2204
ScyllaDB now has basic Lua based UDF. First released in ScyllaDB 3.3, it is still under development and will stay experimental in 4.1.
Select updates from ScyllaDB 4.0
The following commands were added: redis: setex, ttl, exists commands
ScyllaDB 4.1 dashboards are available in latest the ScyllaDB Monitoring release (3.4)
Other notable bugs fix and updates in the release
- Security: upgrade bundled gnutls library to version 3.6.14, to fix gnutls vulnerability CVE-2020-13777 #6627
- Stability: an integer overflow in the index reader, introduced in 3.1.0. The bug could be triggered on index files larger than 4GB (corresponding to very large sstables) and result in queries not seeing data in the file (data was not lost). #6040
- Stability: a rare race condition between a node failure and streaming may cause abort #6414, #6394, #6296
- Stability: When hinted handoff enabled, commitlog positions are not removed from rps_set for discarded hints #6433 #6422. This issue would occur if there are hint files stored on disk that contain a large number of hints that are no longer valid – either older than gc_grace_seconds of their destination table, or their destination table no longer exists. When ScyllaDB attempts to send hints from such a file, invalid hints will cause memory usage to increase, which may result in “Oversized allocation” warnings to appear in logs. This increase is temporary (ends when hint file processing completes), and shouldn’t be larger than the hint file itself times a small constant.
- Stability: potential use after free in storage service when using API call /storage_service/describe_ring/ #6465
- Stability: Issuing a reverse query with multiple IN restrictions on the clustering key might result in incorrect results or a crash. For example:
CREATE TABLE test (pk int, ck int, v int, PRIMARY KEY (pk, ck));
SELECT * FROM test WHERE pk = ? AND ck IN (?, ?, ?) ORDERED BY ck DESC;
- Row cache is incorrectly invalidated in incremental compaction (used in ICS and LCS) #5956 #6275
- Stability (CQL): Using prepare statements with collections of tuples can cause an exit #4209
- LWT: In some rare cases, a failure may cause two reads with no write in between that return different value which breaks linearizability #6299
- Alternator: Alternator
batch_put_itemitem with nesting greater then 16 yield
- Alternator: alternator: improve error message in case of
forbid_rmwforbidden write #6421
- Setup: scylla does save coredumps on the same partition that ScyllaDB is using #6300
- Performance: Few of the Local system tables from `system` namespace, like large_partitions do not use gc grace period to 0, which may result in millions of tombstones being needlessly
kept for these tables, which can cause read timeouts. Local system tables use LocalStrategy replication, so they do not need to be concerned about gc grace
- Stability: online loading of SSTable files in the main directory is diabled.
Till now, one cloud upload tables either in a dedicated upload directory, or in the main data directory.
The second, also used by Apache Cassandra, proved to be dangerous, and is disabled from this release forward.
- Stability: multishard_writer can deadlock when producer fails, for example when, during a repair, a node fail #6241
- Stability: fix segfault when taking a snapshot without keyspace specified
- Performance: bypass cache when generating view updates from streaming #6233
- nodetool status returns wrong IPv6 addresses #5808
- Docker: docker-entrypoint.py doesn’t gracefully shutdown scylla when the container is stopped #6150
- CQL: Allow any type to be casted to itself #5102
- CQL: ALTERing compaction settings for table also sets default_time_to_live to 0 #5048
- Stability: Crash from storage_proxy::query_partition_key_range_concurrent during shutdown #6093
- Stability: When speculative read is configured a write may fail even though enough replicas are alive #6123
- Performance: Streaming during bootstrapping is slow due to compactions #5109
- Cloud Formation: Missing i3.metal in Launch a ScyllaDB Cluster using Cloud Formation. scylla-machine-image#31
- scylla-tools-java: cassandra-stress fails to retry user profile insert and query operations #86
- scylla-tools-java: sstableloader null clustering column gives InvalidQueryException #88
- scylla-tools-java: sstableloader fails with uppercase KS trying to load into lowercase when importing few years old cassandra #126
- scylla-tools-java: nodetool: permission issues when running nodetool #147
- scylla-tools-java: via cqlsh not possible to escape special characters such as ‘ when used as FILTER for UPDATE #150
- Alternator: KeyConditions with “bytes” value doesn’t work #6495
- Alternator: alternator: wait for schema agreement after table creation #6361
- Repair based node operations is disabled by default (same as 4.0)
- Coresump setup: clean coredump files #6159
- Coresump setup: enable coredump directory mount #6566
- Stability: scylla init is stuck for more than a minute due to mlocking all memory at once #6460
- Upgrade: when required, for example, during upgrade, to use an old SSTable format, ScyllaDB will now default to LA format instead of KA format. New clusters will always use MC format. #6071