Monster Scale Summit

Summary: Asias He explains ScyllaDB’s internal data streaming, used for adding or removing nodes and migrating tablets. Traditional mutation‑based streaming reads mutations from multiple SSTables, serializes them, and writes them back on the receiving node. A new file‑based approach now streams whole SSTables directly because each table maps to a single tablet. This eliminates parsing, cuts CPU, lowers network bytes, and makes streaming up to 25 × faster and 10 × more bandwidth‑efficient.

Topics discussed

• What streaming does inside ScyllaDB to add or decommission nodes and migrate tablets
• How mutation‑based streaming pulls individual mutations from SSTables, serializes them, and rebuilds them remotely
• Why per‑mutation parsing and serialization limit throughput and raise CPU use
• How file‑based streaming ships complete SSTables thanks to tablet ownership, skipping parsing and serialization
• What performance gains appear: 25 × shorter streaming time, 10 × higher network bandwidth, roughly one‑third of the data sent, and markedly lower CPU cycles

Takeaways

• File‑based streaming turns the SSTable itself into the transfer unit, avoiding costly mutation decoding and re‑assembly. This lowers CPU and memory pressure and frees cycles for user workloads.
• Because each SSTable belongs to a single tablet, whole‑file transfers never mix unrelated data. This lets ScyllaDB stream without filtering or rewriting, simplifying the code path and reducing I/O contention.
• Field tests show streaming bandwidth improves by an order of magnitude while total bytes on the wire drop by roughly three‑fold, making cluster expansion or rebalancing far quicker even on constrained networks.
• The new method’s 25 × speed‑up shortens maintenance windows and shrinks the noisy‑neighbor effect during node operations, which directly benefits latency‑sensitive applications.

Top takeaway: File‑based streaming lets ScyllaDB move entire SSTables between nodes, cutting CPU work and network bytes and slashing streaming time by up to 25×.