ScyllaDB is pleased to announce the availability of ScyllaDB Enterprise 2022.1, a production-ready ScyllaDB Enterprise major release. After more than 6,199 commits originating from five open source releases, we’re excited to now move forward with ScyllaDB Enterprise 2022.
ScyllaDB Enterprise builds on the proven features and capabilities of our ScyllaDB Open Source NoSQL database, providing greater reliability from additional vigorous testing. Between ScyllaDB Enterprise 2021 to this release of 2022, we’ve closed 1,565 issues. Plus, it provides a set of unique enterprise-only features such as LDAP authorization and authentication.
ScyllaDB Enterprise 2022 is immediately available for all ScyllaDB Enterprise customers, and will be deployed to ScyllaDB Cloud users as part of their fully-managed service. A 30-day trial version is available for those interested in testing its capabilities against their own use cases.
New Features in ScyllaDB Enterprise 2022
Support for AWS EC2 I4i Series Instances
ScyllaDB now supports the new AWS EC2 I4i series instances. The I4i series provides superior performance over the I3 series due to a number of factors: the 3rd generation Intel Xeon “Ice Lake” processors, the AWS Nitro System hypervisor, and low-latency Nitro NVMe SSDs. ScyllaDB can achieve 2x throughput and lower latencies on I4i instances over comparable i3 servers.
Support for Arm-based Systems
With ScyllaDB Open Source 4.6 we implemented support for Arm-based architectures, including the new AWS Im4gn and Is4gen storage-optimized instances powered by Graviton2 processors. Release 4.6 also supports the low cost T4g burstable instance for development of cloud-based applications. Since ScyllaDB is now compiled to run on any Aarch64 architecture, you can even run it on an Arm-based M1-powered Macintosh using Docker for local development.
Change Data Capture (CDC)
This feature allows you to track changes made to a base table in your database for visibility or auditing. Changes made to the base table are stored in a separate table that can be queried by standard CQL. Our CDC implementation uses a configurable Time To Live (TTL) to ensure it does not occupy an inordinate amount of your disk. Note: This was one of the few features brought in through a maintenance release (2021.1.1). It is new since the last major release of ScyllaDB Enterprise 2021.
LDAP
ScyllaDB Enterprise 2021.1.2 introduced support for LDAP Authorization and LDAP Authentication.
- LDAP Authentication allows users to manage the list of ScyllaDB Enterprise users and passwords in an external Directory Service (LDAP server), for example, MS Active Directory. ScyllaDB leverages authentication by a third-party utility named saslauthd to handle authentication, which, in turn, supports many different authentication mechanisms. Read the complete documentation on how to configure and use saslauthd and LDAP Authentication.
- LDAP Authorization allows users to manage the roles and privileges of ScyllaDB users in an external Directory Service (LDAP server), for example, MS Active Directory. To do that, one needs a role_manager entry in scylla.yaml set to com.scylladb.auth.LDAPRoleManager. When this role manager is chosen, ScyllaDB forbids GRANT and REVOKE role statements (CQL commands) as all users get their roles from the contents of the LDAP directory.When LDAP Authorization is enabled and a ScyllaDB user authenticates to ScyllaDB, a query is sent to the LDAP server, whose response sets the user’s roles for that login session. The user keeps the granted roles until logout; any subsequent changes to the LDAP directory are only effective at the user’s next login to ScyllaDB. Read the complete documentation on how to configure and use LDAP Authorization.
Note: This is one of the few features brought in through a maintenance release (2021.1.2). It is new since the last major release of ScyllaDB Enterprise 2021.
Timeout per Operation
There is now new syntax for setting timeouts for individual queries with “USING TIMEOUT”. This is particularly useful when one has queries that are known to take a long time. Till now, you could either increase the timeout value for the entire system (with request_timeout_in_ms), or keep it low and see many timeouts for the longer queries. The new Timeout per Operation allows you to define the timeout in a more granular way. Conversely, some queries might have tight latency requirements, in which case it makes sense to set their timeout to a small value. Such queries would get time out faster, which means that they won’t needlessly hold the server’s resources. You can use the new TIMEOUT parameters for both queries (SELECT) and updates (INSERT, UPDATE, DELETE). Note: This was one of the few features brought in through a maintenance release (2021.1.1). It is new since the last major release of ScyllaDB Enterprise 2021.
Improvements to Cloud Formation
A new variable VpcCidrIp allows you to set CIDR IP range for VPC. Previously, the range was hard coded to 172.31.0.0/16. Plus the Cloud Formation template was reordered for better readability. Note: This was one of the few features brought in through a maintenance release (2021.1.5). It is new since the last major release of ScyllaDB Enterprise 2021.
I/O Scheduler Improvements
A new I/O scheduler was integrated via a Seastar update. The new scheduler is better at restricting disk I/O in order to keep latency low. This implementation introduces a new dispatcher in the middle of a traditional producer-consumer model. The IO scheduler seeks to find what is known as the effective dispatch rate — the fastest rate at which the system can process data without running into internal queuing jams.
Improved Reverse Queries
Reverse queries are SELECT statements that use reverse order from the table schema. If no order was defined, the default order is ascending (ASC). For example, imagine rows in a partition sorted by time in ascending order. A reverse query would sort rows in descending order, with the newest rows first. Reverse queries were improved in ScyllaDB Open Source 4.6, and are further improved in 5.0, first, to return short pages to limit memory consumption, and secondly, for reverse queries to leverage ScyllaDB’s row-based cache (before 5.0 they bypassed the cache).
New Virtual Tables for Configuration and Nodetool Information
A new system.config virtual table allows querying and updating a subset of configuration parameters over CQL. These updates are not persistent, and will return to the scylla.yaml update after restart. Nodetool command information can also be accessed via virtual tables, including snapshots, protocol servers, runtime info, and a virtual table replacement for nodetool versions. Virtual tables allow remote access over CQL, including for ScyllaDB Cloud users.
Repair-based Node Operations (RBNO)
We leveraged our row-based repair mechanism for other repair-based node operations (RBNO), such as node bootstraps, decommissions, and removals, and allowed it to use off-strategy compaction to simplify cluster management.
SSTable Index Caching
We also added SSTable Index Caching (4.6). Up to this release, ScyllaDB only cached data from SSTables data rows (values), not the indexes themselves. As a result, if the data was not in cache readers had to touch the disk while walking the index. This was inefficient, especially for large partitions, increasing the load on the disk, and adding latency. Now, index blocks can be cached in memory, between readers, populated on access, and evicted on memory pressure – reducing the IO and decreasing latency. More info can be found in Tomasz Grabiec session in ScyllaDB Summit “SSTable Index Caching.”
SSTable Index Caching provides faster performance by avoiding
unnecessary queries to SSD.
Granular Timeout Controls
Timeouts per operation were also added, allowing more granular control over latency requirements and freeing up server resources (4.4). We expanded upon this by adding Service Level Properties, allowing you to associate attributes to rules and users, so you can have granular control over session properties, like per-service-level timeouts and workload types (4.6).
Guardrails
ScyllaDB is a very powerful tool, with many features and options. In many cases, these options, such as experimental or performance-impacting features, or a combination of them, are not recommended to run in production. Guardrails are a collection of reservations that make it harder for the user to use non-recommended options in production. A few examples:
- Prevent users from using SimpleReplicationStrategy.
- Warn or prevent usage of DateTieredCompactionStrategy, which has long since been deprecated in favor of TimeWindowCompactionStrategy.
- Disable Thrift, a legacy interface, by default
- Ensure that all nodes use the same snitch mode
ScyllaDB administrators can use our default settings or customize guardrails for their own environment and needs.
Improvements to Alternator
The Amazon DynamoDB-compatible interface has been updated to include a number of new features:
- Support for Cross-Origin Resource Sharing (CORS)
- Fully supports nested attribute paths
- Support attribute paths in ConditionExpression, FilterExpression Support, and ProjectionExpression
Related Innovations
Kubernetes Operator Improvements
Our Kubernetes story was production-ready with ScyllaDB Operator 1.0, released in January of 2020. Since that time, we have continuously improved on our capabilities and performance tuning with cloud orchestration.
Faster Shard-Aware Drivers
Also during the past two years we have learned a lot about making our shard-aware drivers even faster. Open source community contributions led to a shard-aware Python driver which you can read about here and here. Since then our developers have embraced the speed and safety of Rust, and we are now porting our database drivers to async Rust, which will provide a core of code for multiple drivers with language bindings to C/C++ and Python.
New Release Cycle for Enterprise Customers
ScyllaDB Enterprise 2022.1 is a Long Term Support release (LTS) as per our new enterprise release policy. It used to be that Enterprise users would need to wait a year between feature updates in major ScyllaDB Enterprise releases. Starting later this year, we intend to provide additional new features in a Short Term Support (STS) release, versioned as 2022.2. This new mix of LTS and STS releases will allow us to provide new features on an intrayear basis. For you, it means you don’t need to wait as long to see new innovations show up in a stable enterprise release.
For example, let’s look at this chart from the historical releases we made in the era of ScyllaDB Open Source 3.0 to 4.0, and how that tracked to ScyllaDB Enterprise 2019 to 2020.
There was a four month lag between the release of ScyllaDB Open Source 3.0 (January 2019) to the related ScyllaDB Enterprise 2019.1 release (May 2019). And there was a similar three month lag between ScyllaDB Open Source 4.0 and ScyllaDB Enterprise 2020.1.
During the interim between those major releases there were three minor releases of ScyllaDB Open Source — 3.1, 3.2 and 3.3. Enterprise users needed to wait until the next major annual release to utilize any of those innovations — apart from a point feature such as support for IPv6.
In comparison, ScyllaDB Open Source 5.0 was released on July 7th, 2022. And here, within a calendar month, we are announcing ScyllaDB Enterprise 2022.
Plus, when ScyllaDB Enterprise 2022.2 is released later this year, it will include within it additional production-ready innovations brought in from ScyllaDB Open Source without requiring users to wait until the new major release in calendar year 2023.
This narrows the delay in production-ready software features for our Enterprise users. For example, this is what the rest of 2022 could look like:
Note ScyllaDB Enterprise 2022.2 will be a Short Term Support release. Which means some Enterprise users may still decide to wait for the next ScyllaDB Enterprise LTS release in 2023. Existing ScyllaDB Enterprise customers are encouraged to discuss their upgrade policies and strategies with their support team.
Get ScyllaDB Enterprise 2022 Now
ScyllaDB Enterprise 2022 is immediately available. Existing ScyllaDB Enterprise customers can work with their support team to manage their upgrades. ScyllaDB Cloud users will see the changes made automatically to their cluster in the coming days.
For those who wish to try ScyllaDB Enterprise for the first time, there is a 30-day trial available.
You can check out the documentation and download it now from the links below. If this is your first foray into ScyllaDB, or into NoSQL in general, remember to sign up for our free online courses in ScyllaDB University.
