Faster, safer node operations, strongly consistent topology updates, Webassembly-based user-defined functions and aggregates, and more
ScyllaDB Open Source 5.4.0 is now available! This new release introduces Repair Based Node Operations (RBNO) for all operations, experimental strongly consistent topology updates, experimental Object Storage backend, and numerous improvements and bug fixes. Additionally, strongly consistent schema management using Raft will be enabled automatically upon upgrade (see below for more details).
DOWNLOAD SCYLLADB OPEN SOURCE 5.4 NOW
In this blog, we’ll highlight the new capabilities that our users have been asking about most frequently. For the complete details, read the release notes.
Repair-Based Node Operations (RBNO)
RBNO provides a more robust, reliable, and safer data streaming for node operations like node-replace and node-add/remove. In particular, a failed node operation can resume from the point it stopped – without sending data that has already been synced. In addition, with RBNO enabled, you don’t need to repair before or after node operations, such as replace or removenode.
In 5.4, RBNO is enabled by default for all operations: remove node, rebuild, bootstrap, and decommission. The replace node operation was already enabled by default.
For details, see Repair Based Node Operations (RBNO) docs and the blog, Faster, Safer Node Operations with Repair vs Streaming.
UDF / UDA [Preview]
Wasm-based User Defined Functions (UDFs) and User Defined Aggregates (UDAs) were introduced as experimental in ScyllaDB 5.1; with 5.4, we are now promoting them to Preview.
Preview features aren’t production-ready, but are made available on a “preview” basis so that users can get early access and provide feedback. Unlike Experimental features, we are committed to the backward compatibility of a preview feature API.
Learn more in the UDF/UDA documentation and the blog, Wasmtime: Supporting UDFs in ScyllaDB with WebAssembly.
Strongly Consistent Topology Updates with Raft [Experimental]
This release includes experimental Strongly Consistent Topology Updates.
With Raft-managed topology enabled, all topology operations are internally sequenced in a consistent way. A centralized coordination process ensures that topology metadata is synchronized across the nodes on each step of a topology change procedure. This makes topology updates fast and safe, as the cluster administrator can trigger many topology operations concurrently, and the coordination process will safely drive all of them to completion. For example, multiple nodes can be bootstrapped concurrently, which couldn’t be done with the old gossip-based topology.
To enable it, use the new consistent-topology-changes flag. Update the following in scylla.yaml
experimental_features:
consistent-topology-changes: trueLearn more in the docs and the blog, What’s Next on ScyllaDB’s Path to Strong Consistency.
Strongly Consistent Schema Management with Raft
Strongly Consistent Schema Management with Raft became the default for new clusters in ScyllaDB 5.2. In this release, it is enabled by default when upgrading existing clusters.
Learn more in the 5.2 to 5.4 upgrade docs and the blog, ScyllaDB’s Path to Strong Consistency: A New Milestone.
Object Storage Support [Experimental]
By default, SStables of a keyspace are stored in a local directory. As an alternative, you can now configure an entire ScyllaDB keyspace to be stored on Amazon S3 or another S3-compatible object store. This is a very early step toward a production-ready S3 backend.
See the Keyspace Storage Options docs for details.
Node-Aggregated Table Level Metrics
Most ScyllaDB metrics are per-shard, per-node, but not for a specific table. We now export some per-table metrics. These are exported once per node, not per shard, to reduce the number of metrics.
Guardrails
Guardrails is a framework to protect ScyllaDB users and admins from common mistakes and pitfalls. In this release, ScyllaDB includes a new guardrail on the replication factor. It is now possible to specify the minimum replication factor for new keyspaces via a new configuration item.
New nodetool Implementation [Experimental]
The scylla executable can now act as nodetool; just execute “scylla nodetool ”. Over time, the new scylla nodetool will completely replace the legacy Java-based nodetool. The new implementation is fully backward compatible with the legacy nodetool.
Use scylla nodetool help to get the list of supported commands.
Security
Mutual TLS authentication
It is now possible to use TLS certificates to authenticate and authorize a user to ScyllaDB. The system can be configured to derive the user role from the client certificate and derive the permissions the user has from that role. Learn more in the certificate-authentication docs.
Set default superuser credentials on boot
Users may now specify the initial superuser name and a salted password hash in scylla.yaml config or command line. Note that config values become redundant as soon as auth tables are initialized.
Additional Improvements
The new release also introduces numerous improvements across:
- CQL API
- CQL Sh
- Amazon DynamoDB Compatible API (Alternator)
- Admin REST API
- Performance and stability
- Monitoring, tracing, and logging
- Operations
- Deployment and installations
For complete details, see the release notes.
