For a long time, permanent storage has been the bottleneck in most computer systems. Scylla operates under that assumption and includes a fully-featured userspace disk I/O Scheduler that is used to guarantee that different tasks in the database get their fair share of the disk. The I/O Scheduler is the central component at the heart of Scylla’s workload conditioning promise: to automatically adjust the database’s distribution of requests to adapt to the incoming workload. It is capable of providing Quality-of-Service (QoS) among the various tasks in the database and isolating them from each other. Since database systems tend to be […]
Raphael S. Carvalho is a computer programmer here at ScyllaDB who loves open source software and kernel programming. He worked on the Syslinux project to bring new file system support and also worked on MultiFS to allow multiple file systems to co-exist. For his Scylla work, he has been mostly working on SSTable compaction handling and recently developed the support for the Time Window Compaction Strategy on Scylla. This strategy is a considerably better alternative to the DateTieredCompactionStrategy. Raphael has a passion for making products and solutions better with his programming experience. You can learn more about Raphael in his […]
In mid-2015, Intel and Micron jointly unveiled a new kind of non-volatile memory storage device named 3D XPoint (pronounced “cross-point”) that is 1000x faster than NAND. Now that 3D XPoint is generally available and has hit the broad market, we can start testing it. 3D XPoint uses electrical resistance and is considered to be bit addressable. It’s also worth mentioning that the endurance is much better with 3D XPoint because the stated wear leveling is 30 full drive writes per day for 5 years. 3D XPoint developers indicate that it is based on changes in resistance of the bulk material. […]
A database like Scylla can be limited by the network, disk I/O or the processor. Which one it is often dynamic and depends on both the hardware configuration and the workload. The only way of dealing with that is to attempt to achieve good throughput and low latency regardless of what is the bottleneck. There are many things that can be done in each of these cases that range from high-level changes in the algorithms to very low-level tweaks. In this post, I am going to take a closer look at fairly recent changes to Scylla which improved the performance […]
Counters are a special type of column that allows its value to only be incremented, decremented, read or deleted. Updates to counters are atomic, which makes them a perfect solution for counting—something that is otherwise difficult to do efficiently.
Introduction The most common operations with ScyllaDB are inserting, updating, and retrieving rows within a single partition: each operation specifies a single partition key, and the operation applies to that partition. While less commonly used, reads of all partitions, also known as full table scans are also useful, often in the context of data analytics. This post describes how to efficiently perform full table scans with ScyllaDB 1.6 and above.
ScyllaDB strives to offer its users predictable low latencies. However, in real life, things do not always go according to plan, and sometimes predictable low latencies become unpredictable big latencies. When that happens, it’s time to go into detective mode and go figure out what’s going on.
Scylla 1.3 has introduced better support for large partitions. It is an important feature which simplifies data modeling so that it can be more focused on the client’s needs and less on the server limitations and ways to work around them. Moreover, issues related to large partitions are not just failed requests and server crashes caused by the node running out of memory, before reaching that point cluster may experience various performance problems, something much harder to diagnose.
Block devices sometimes do bad things (or just fill up), so sometimes bad things happen to good software. CharybdeFS makes it easy to do integration testing that covers hard-to test filesystem errors. And good error handling is a sign of well-thought-out software. For example, your program will make a much better impression on users if you have it show a nice “insufficient space” message than if it just crashes for no apparent reason. The CharybdeFS filesystem lets you inject arbitrary file errors for testing. This article covers some common examples for getting started.
This is the second and last part of this article. If you haven’t read the first part, you can do it here. In this part, we will look at the design of the Seastar I/O Scheduler that Scylla uses to manage its disk I/O and discuss how it can be used to not only provide predictable latencies as we saw in our previous installment, but to guarantee fairness and proper balancing among different actors.
Apache®, Apache Cassandra®, are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. No endorsement by The Apache Software Foundation is implied by the use of these marks.