VIEW VIDEO

Worry-free ingestion – flow-control of writes in Scylla – Scylla Summit 2018

Nadav Har'El, Software Engineer, ScyllaDB

26:13January 8, 2019

When ingesting large amounts of data into a Scylla cluster, we would like the ingestion to proceed as quickly as possible, but not quicker. We explain how over-eager ingestion could result in a buildup of queues of background writes, possibly to the point of depleting available memory. We then explain how Scylla avoids this risk by automatically slowing down well-behaving applications to the best possible ingestion rate (“flow control”). For applications which cannot be slowed down, Scylla still achieves the highest possible throughput by quickly rejecting excess requests (“admission control”). In this talk, we investigate the different causes of queue buildup during writes, including consistency-level lower than “ALL” and materialized views, and review the mechanisms which Scylla uses to automatically solve this problem.

Share this
Share on facebook
Share on twitter
Share on print
Share on email
Share on linkedin

VIEW SLIDES

Let’s do this

Getting started takes only a few minutes. Scylla has an installer for every major platform and is well documented. If you get stuck, we’re here to help.