ScyllaDB Rust Driver was born during ScyllaDB’s internal developer hackathon. The effort did not stop after the hackathon though — the development continued and ScyllaDB Rust Driver is now released as 0.2.0, officially available on the Rust community’s package registry — crates.io. It also already accepted its first contributions from the open-source community, has a comprehensive docs page, and much more! We also ran comparative benchmarks against other drivers to confirm that our driver is (more than) satisfactory in terms of performance.
Quick Start
Our docs page contains a quick start guide designed for people who would like to start using our driver. You’re welcome to try it out!
New Features
After the hackathon, ScyllaDB Rust Driver was in a workable, but very limited state. It was capable of sending the requests to correct nodes and shards, but many features expected from a full-fledged driver were missing. The situation has changed drastically throughout the year, with the ScyllaDB Rust Driver gaining many important features:
- Authentication Support
It’s possible to connect to clusters which require username+password authentication
- TLS Support
Certificates and keys can be configured to establish secure connections with the cluster
- Configurable Load Balancing Algorithms
Multiple options for load balancing are now available, including DC-aware round robin and token-aware load balancing. Learn more. - Configurable Retry Policies
Depending on the failure type, our driver may apply various retry policies to increase the chance of success for requests. Learn more. - Speculative Execution
In certain situations it’s beneficial in terms of performance to speculatively resend a request to another node, before any reply arrives. Such behavior can now be configured per-session and comes in two flavors: based on constant delay or latency percentiles. Learn more. - Tracing Support
Query tracing is extremely useful for investigating performance issues and bottlenecks. The ScyllaDB Rust Driver supports retrieving tracing information from specific requests. Learn more. - Internal Logging
For those interested in how the driver works internally, its logs are exposed via the tracing crate. Happy debugging!
Benchmarks
The performance of our driver was tested against multiple other drivers, including:
- cpp-driver: the C++ driver (in single- and multithreaded modes)
- cassandra-cpp: a Rust driver which uses the C++ driver bindings underneath
- gocql: the Go driver
- cdrs-tokio: native Rust driver, forked from cdrs and also based on the Tokio framework
Source code of all the benchmarks can be browsed here: https://github.com/cvybhu/rust-driver-benchmarks
Testing Configuration
All tests were performed on a 3-node cluster of ScyllaDB 4.4 running on i3.4xlarge AWS instances and a powerful c5.9xlarge loader instance on which the drivers were used. That means that all drivers were given a chance to utilize 36 vCPUs and 72GiB of RAM, which allows reaching quite high concurrency.
Test Cases
Each driver was compared against all combinations of the following configurations:
- 3 workload types: read-only, write-only, mixed 50/50
- data size of 1M, 10M and 100M rows
- concurrency varying from 512 to 8192
Checking so many combinations was necessary to observe how well the drivers scale along with increasing concurrency, how well they can handle large amounts of data and whether the workload type influences their performance.
Results
We were happy to observe that our ScyllaDB Rust Driver is very competitive both in terms of performance and scalability – it was generally either just as fast or even slightly faster than the multithreaded C++ driver (cpp-multi) and left other tested drivers far behind. All the results can be browsed here, and a small sample is presented below:
Chart 1: Results for 1 million rows, mixed read/write workload and various concurrency values (lower is better)
Chart 2: Results for 100 million rows, mixed workload and concurrency of 8192 (lower is better)
Summary
ScyllaDB Rust Driver turned out to scale very well along with client’s concurrency, which is what we hoped for — one of the earliest design decisions was to make the driver fully asynchronous and thrive in high concurrency environments.
Academic Paper
The development of ScyllaDB Rust Driver was not just yet another open-source initiative – it was also an academic project co-organized with the University of Warsaw. A team of 4 talented students were engaged in implementing missing features and performing benchmarks. All details and results can be found in their thesis. Have a good read!
