fbpx

Scylla University LIVE | Free Virtual Training Event | July 28 & July 29

Scylla U Sticky Banner icon
Pick Your Timezone >
See all blog posts

Scylla Rust Driver Update and Benchmarks

Scylla Rust Driver was born during Scylla’s internal developer hackathon. The effort did not stop after the hackathon though — the development continued and Scylla Rust Driver is now released as 0.2.0, officially available on the Rust community’s package registry — crates.io. It also already accepted its first contributions from the open-source community, has a comprehensive docs page, and much more! We also ran comparative benchmarks against other drivers to confirm that our driver is (more than) satisfactory in terms of performance.

Quick Start

Our docs page contains a quick start guide designed for people who would like to start using our driver. You’re welcome to try it out!

New Features

After the hackathon, Scylla Rust Driver was in a workable, but very limited state. It was capable of sending the requests to correct nodes and shards, but many features expected from a full-fledged driver were missing. The situation has changed drastically throughout the year, with the Scylla Rust Driver gaining many important features:

  1. Authentication Support
    It’s possible to connect to clusters which require username+password authentication

  1. TLS Support
    Certificates and keys can be configured to establish secure connections with the cluster

  1. Configurable Load Balancing Algorithms
    Multiple options for load balancing are now available, including DC-aware round robin and token-aware load balancing. Learn more.
  2. Configurable Retry Policies
    Depending on the failure type, our driver may apply various retry policies to increase the chance of success for requests. Learn more.
  3. Speculative Execution
    In certain situations it’s beneficial in terms of performance to speculatively resend a request to another node, before any reply arrives. Such behavior can now be configured per-session and comes in two flavors: based on constant delay or latency percentiles. Learn more.
  4. Tracing Support
    Query tracing is extremely useful for investigating performance issues and bottlenecks. The Scylla Rust Driver supports retrieving tracing information from specific requests. Learn more.
  5. Internal Logging
    For those interested in how the driver works internally, its logs are exposed via the tracing crate. Happy debugging!

Benchmarks

The performance of our driver was tested against multiple other drivers, including:

  • cpp-driver: the C++ driver (in single- and multithreaded modes)
  • cassandra-cpp: a Rust driver which uses the C++ driver bindings underneath
  • gocql: the Go driver
  • cdrs-tokio: native Rust driver, forked from cdrs and also based on the Tokio framework

Source code of all the benchmarks can be browsed here: https://github.com/cvybhu/rust-driver-benchmarks

Testing Configuration

All tests were performed on a 3-node cluster of Scylla 4.4 running on i3.4xlarge AWS instances and a powerful c5.9xlarge loader instance on which the drivers were used. That means that all drivers were given a chance to utilize 36 vCPUs and 72GiB of RAM, which allows reaching quite high concurrency.

Test Cases

Each driver was compared against all combinations of the following configurations:

  • 3 workload types: read-only, write-only, mixed 50/50
  • data size of 1M, 10M and 100M rows
  • concurrency varying from 512 to 8192

Checking so many combinations was necessary to observe how well the drivers scale along with increasing concurrency, how well they can handle large amounts of data and whether the workload type influences their performance.

Results

We were happy to observe that our Scylla Rust Driver is very competitive both in terms of performance and scalability – it was generally either just as fast or even slightly faster than the multithreaded C++ driver (cpp-multi) and left other tested drivers far behind. All the results can be browsed here, and a small sample is presented below:

Chart 1: Results for 1 million rows, mixed read/write workload and various concurrency values (lower is better)

Chart 2: Results for 100 million rows, mixed workload and concurrency of 8192 (lower is better)

Summary

Scylla Rust Driver turned out to scale very well along with client’s concurrency, which is what we hoped for — one of the earliest design decisions was to make the driver fully asynchronous and thrive in high concurrency environments.

Academic Paper

The development of Scylla Rust Driver was not just yet another open-source initiative – it was also an academic project co-organized with the University of Warsaw. A team of 4 talented students were engaged in implementing missing features and performing benchmarks. All details and results can be found in their thesis. Have a good read!

READ THE SCYLLA RUST DRIVER THESIS

DOWNLOAD THE SCYLLA RUST DRIVER

Piotr Sarna

About Piotr Sarna

Piotr is a software engineer very keen on open-source projects and C++. He previously developed an open-source distributed file system (LizardFS) and had a brief adventure with Linux kernel during an apprenticeship at Samsung Electronics. Piotr graduated from University of Warsaw with MSc in Computer Science.