Scylla vs. Cassandra benchmark

The results presented here are out of date. Check out the new results here!

The following benchmark compares Scylla and Cassandra on a single server.

Test Bed

The test was executed on SoftLayer servers. Configuration included:

1 DB server (Scylla / Cassandra):

  • SoftLayer bare metal server
  • CPU: 2x Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz, 48 logical cores
  • RAM: 128 GB
  • Networking: 10 Gbps
  • Disk: MegaRAID SAS 9361-8i, 4x 960 GB SSD
  • OS: Fedora 22 (chrooted), Linux 3.10.0-229.11.1.el7.x86_64

7 load servers (cassandra-stress):

  • SoftLayer VM server
  • CPU: Intel(R) Xeon(R) CPU E5-2683 v3 @ 2.00GHz, 16 logical cores
  • RAM: 16G
  • Networking: 1 Gbps
  • OS: CentOS 7, Linux 3.10.0-229.7.2.el7.x86_64

All machines were located in the same data center.

Server tuning

The DB server was tuned for better performance:

  • the bond interface was dropped and bare NIC interface was used.
  • NIC rx interrupts were bound each to a separate CPU
  • NIC tx queues were bound each to a separate CPU
  • Open file limit was increased: ulimit -n 1000000
  • 128 huge pages were configured

See setup.sh

Scylla

  • version: 536f557c22e3c8932321440131ed2ba59728fcf3
  • command line:

    ./build/release/scylla --data-file-directories=data --commitlog-directory=cl -c48 -m80G --collectd=1 \
    --collectd-address=$COLLECTD_HOST:25827 --collectd-poll-period 1000 --options-file=scylla.yaml
  • scylla.yaml

Note that in this benchmark Scylla was using the default “POSIX” networking stack, not a DPDK kernel bypass.

Cassandra

The differences between the cassandra.yaml used in test and the distribution-provided one are:

  • Replaced IP addresses
  • concurrent_writes was set to 384 (8*48) as recommended

Workloads

Three workloads were tested:

  • Write only: tools/bin/cassandra-stress write duration=15min -mode native cql3 -rate threads=700 -node $SERVER
  • Read Only: tools/bin/cassandra-stress mixed 'ratio(read=1)' duration=15min -pop 'dist=gauss(1..10000000,5000000,500000)' -mode native cql3 -rate threads=700 -node $SERVER
  • Mixed: 50/50 Read/Write: tools/bin/cassandra-stress mixed 'ratio(read=1,write=1)' duration=15min -pop 'dist=gauss(1..10000000,5000000,500000)' -mode native cql3 -rate threads=700 -node $SERVER

cassandra-stress from cassandra package version 2.1.9 was used.

Each cassandra-stress process was placed on a separate loader machine. NIC interrupts were pinned to core 0 and the rest of the cores were dedicated to cassandra-stress.

Before each test server’s data/ directory was purged.

Before read and mixed workloads the database was populated with 100000000 keys:

tools/bin/cassandra-stress write n=100000000 -pop seq=1..100000000 -mode native cql3 -rate threads=700 -node $SERVER

The client processes were started simultaneously. The first 60 seconds of the results were ignored to allow for warmup. Reported throughput is the average server throughput, disregarding the warmup period.

Results

Transaction/secWriteReadWrite/Read Mix0k125k250k375k500k625k750k875k1000k1125k1250k1375k

Scylla

Cassandra 2.1.9

Average throughput. Results were rounded with accuracy of 10K.


Further Reading