The ScyllaDB team is pleased to announce the availability of ScyllaDB Open Source 4.0.0, a production-ready release of our open source NoSQL database.
ScyllaDB 4.0 is focused on promoting experiential features Lightweight Transactions (LWT) and our Amazon DynamoDB Compatible API (Alternator) to production level.
In addition we are adding a new ScyllaDB AMI user data format and more (below).
ScyllaDB 4.0 also includes new and improved experimental features, like CDC, UDF and a major update to the underlying infrastructure, making all streaming operations based on existing repair logic (more below)
ScyllaDB is an open source, NoSQL database, with superior performance and consistently low latencies. Find the ScyllaDB Open Source 4.0 repository for your Linux distribution here. A ScyllaDB 4.0 Docker image is also available.
Please note that only the latest two minor releases of ScyllaDB Open Source project are supported. Starting today, only ScyllaDB Open Source 4.0 and ScyllaDB 3.3 will be supported, and ScyllaDB 3.2 will be retired.
Related Links
- Get ScyllaDB Open Source 4.0 as binary packages (RPM/DEB), AWS AMI and Docker instance
- Upgrade from ScyllaDB 3.3 to ScyllaDB 4.0
- Report a problem
New features in ScyllaDB 4.0
Lightweight Transactions (LWT) #1359
Lightweight transactions (LWT), also known as Compare and Set (CAS), add support for conditional INSERT and UPDATE CQL commands. ScyllaDB supports both equality and non-equality conditions for lightweight transactions (i.e., you can use <, <=, >, >=, !=,= and IN operators in an IF clause).
LWT was introduced as an experimental feature in ScyllaDB 3.2, and now, with ScyllaDB 4.0, LWT is promoted to production ready.
Selected updates from ScyllaDB 3.3:
Support LIKE operator in conditional expressions #5777
You can learn more on LWT in ScyllaDB and LWT optimizations from the latest LWT Webinar (registration required) and LWT documentation.
ScyllaDB Alternator: The Open Source DynamoDB-compatible API
Project Alternator is an open-source implementation for an Amazon DynamoDB™-compatible API. The goal of this project is to deliver an open source alternative to Amazon’s DynamoDB that can be run anywhere, whether on-premises or on your own favorite cloud provider, or in ScyllaDB Cloud.
ScyllaDB Alternator was introduced as an experimental feature in ScyllaDB 3.2, and it is now, in ScyllaDB 4.0, promoted to production.
Selected updates from ScyllaDB 3.3:
- Limit huge requests #5832
- Allow tagging tables #5066
- Support ConditionExpression #5035
- Support KeyConditionExpression #5037
- Support parallel Scan #5059
- Add support for ScanIndexForward #5153
- Hook to admission control (like CQL does) #5029
Read more here: Open Source DynamoDB-compatible API Documentation.
Deployment
- ScyllaDB EC2 AMI user data format has changed. The new format is JSON based, and while it supports all the options it is *not* backward compatible.
- Completing the new AMI format, we are releasing a Cloud Formation template, for easy launch of a ScyllaDB cluster in one region.
Experimental features in ScyllaDB 4.0
Change Data Capture (CDC)
While functionally complete, we are still testing to validate CDC is production ready, with the plan to move it to GA in a subsequent ScyllaDB 4.x release. No API updates are expected.
Selected updates from ScyllaDB 3.3:
- Support postimage #4992
- Support collections in CDC #5669
- Handle cluster topology changes in CDC #4989
- Protect CDC Log and CDC Description tables from being modified by the user #4991
- Add new CDC mertics #6002 (for a full list of new metrics in 4.0 see below)
- Include CDC in tracing info #5912
- CDC now logs Lightweight Transactions (LWT) operations correctly. #5869
- Performance improvement
- Many more bug fixes
Repair based node operations
ScyllaDB has two use cases for streaming data between nodes:
- Topology changes, like adding and removing nodes
- Repair, off line process to compare and sync data between nodes
Up to ScyllaDB 4.0, the two used different underline logic. Starting in this release, the same streaming logic used for repair is also used for topology changes, making it more robust, reliable, with better visibility into streaming progress. In particular, streaming can restart from the same point it stopped, for example in case a node,or the network, restart during streaming. This is a big time saver when adding, or removing large nodes.
In ScyllaDB 4.0.0, repair base node operation is disabled by default, you can enable by adding the following to scylla.yaml:
enable_repair_based_node_ops: true
User Defined Functions (UDFs) #2204
ScyllaDB now has basic Lua based UDF. First released in ScyllaDB 3.3, it is still under development and will stay experimental in 4.0.
Select updates from ScyllaDB 3.3
- Fix Lua: list and list were broken #5711
More on UDFA and UDA here
Other notable updates and bug fixes in the release
- Date Tiered Compaction Strategy (DTCS) is deprecated and eventually be removed. If you are still using it, please switch to Time Window Compaction Strategy (TWCS).
- API: add error injection to REST API (used for testing only)
A new API allows enable/disable error injections to different places in the code #3295 - ScyllaDB will now ignore a keyspace that is removed while being repaired, or without tables, rather than failing the repair operation. #5942 #6022
- Local secondary indexes, as well as materialized views that have the same partition key as their base table, are now updated synchronously. This improves consistency and simplifies concurrency control. #4365. Alternator was updated to take advantage of this.
- a new REST API for deleting a table from a snapshot #5805
- CQL: Fix missing aggregate functions for counters
- The long-deprecated RandomPartitioner and ByteOrderedPartitioner partitioners were removed #5636
07 May 2020
