PROJECT ALTERNATOR - The Open Source Amazon DynamoDB-compatible API

Share on facebook
Share on twitter
Share on print
Share on email

Project Alternator is the Amazon DynamoDB-compatible API found in our Scylla NoSQL database. With Alternator you can use Scylla as a drop-in replacement for DynamoDB, supporting the same client SDKs, data modeling and queries. However, you can deploy Scylla wherever you want: on-premises, or on any public cloud. Scylla provides lower latencies and solves DynamoDB’s high operational costs. You can deploy it however you want via Docker or Kubernetes, or use Scylla Cloud for a fully-managed NoSQL DBaaS solution.

Why Scylla is a Better Alternative to DynamoDB

Scylla Alternator provides four key benefits to DynamoDB users:

1. Lower TCO: DynamoDB can be very expensive. Companies spend a great deal of time and engineering effort in their attempts to reduce their DynamoDB costs. DynamoDB charges for read and write transactions (RRUs and WRUs). A free open source solution eliminates these costs and minimizes other operational expenses. Scylla’s design efficiency allows developers to use significantly fewer resources for the same task or workload. According to our benchmark, users can expect to save 80% – 93% overall to support the same workload (5x-14x less expensive). Because of its highly performant design Scylla also eliminates the cost of any in-memory cache (such as DAX).

2. Performance: Scylla provides far better and more consistent performance, with extremely low latencies and better handling of hot partitions, and no read or write limitations based on provisioned cost. While DynamoDB offers throughput guarantees, its latencies, especially long-tail p99 latencies, suffer in comparison to Scylla. That’s because Scylla is designed around asynchronous communications and a “shared nothing” architecture, allowing it to take full advantage of underlying modern multi-core, multi-CPU NUMA hardware. While both DynamoDB and Scylla have the capability to scale, we believe in giving you the most efficient performance possible, getting the highest utility of the underlying infrastructure you’re running on.

3. Greater Flexibility: You can now run your DynamoDB workloads on any cluster: on-premises, on your favorite public cloud, a hybrid cloud deployment, or using our Scylla Cloud Database as a Service (DBaaS). Our Alternator interface is available in both pure open source or rigorously-tested enterprise-grade releases.

4. Openness: Scylla Open Source allows users to review the source code and any known defects, and in return can add their own contributions to the project. Operationally, Scylla can run on any suitable server cluster regardless of location (on-premises, in any private or public cloud, or on our own DBaaS, Scylla Cloud) or deployment method (bare metal, containerized, virtualized, or deployed in pods via Kubernetes). This contributes to the user’s lower TCO by allowing deployment flexibility in line with their existing operations. Scylla also uses popular open source projects for its monitoring stack (Grafana, Prometheus) making it easy to use in or out of AWS, or to integrate to a 3rd party monitoring system like DataDog.

Scylla as a Platform for a DynamoDB-compatible API

Due to their different data access methods, switching between Scylla and DynamoDB had been non-trivial to users. That’s because Scylla initially used the Cassandra Query Language (CQL), which is syntactically akin to Structured Query Language (SQL), while DynamoDB data types require Javascript Object Notation (JSON) format for queries. (Learn more about the differences here.)

Based on ScyllaDB’s extensive experience in making API-compatible databases, a design principle for the Alternator project was that no DynamoDB API calls should need to be altered from the developer’s perspective. Instead, the database system needs to accept input from the client application in DynamoDB-compatible API format, and transparently translate it into appropriate calls to the underlying database. Resultant data from queries would also need to be returned to users in the DynamoDB-compatible format.

All users need to do is deploy Scylla, enable Alternator in scylla.yaml, and then point their current DynamoDB client applications to the Scylla cluster. Additionally, we offer Scylla Migrator, an Apache Spark™-based streaming solution to migrate data from existing DynamoDB instances into Scylla.

The Current Status of Alternator

Alternator is now available as part of Scylla Open Source, Scylla Enterprise, and Scylla Cloud. Our alternator.md and design doc provide detailed information of what’s supported and not yet supported today. In short, most standard applications will just work. The JSON HTTP API is mostly implemented, indexing works, multi availability zones are implemented, and many more features will work.

Alternator on Scylla Cloud — Our Managed Cloud Service

For many lean organizations, one of the most attractive features of DynamoDB is utilizing it as a completely managed Database as a Service (DBaaS). Scylla Cloud uses the robust Scylla Enterprise release. It currently runs on AWS, and you can utilize your own account to keep your existing AWS terms and discounts. Future updates to Scylla Cloud will run on Azure and GCP as well. Try it today!

Roadmap to Open Source Integration

You can run Alternator right now with Scylla Open Source on your own Amazon, Azure and Google Cloud instances, using deployment methods like Docker and Kubernetes.

We continuously harden the code, ensuring its production quality by putting it through our robust quality assurance cycles. We also continue to strive for completeness in implementing any API differences.

Scylla Open Source 4.0 also supports production-ready Lightweight Transactions LWT based on the Paxos consensus algorithm, allowing Alternator to support the equivalent of DynamoDB conditional updates.

Scylla Open Source 4.0 also provides our streaming feature: Change Data Capture (CDC), which is currently experimental. CDC will also be available in a future release of Scylla Enterprise 2020. We may add compatibility with DynamoDB streams later on.

Enterprise-Grade DynamoDB Alternative

Scylla Enterprise supports our DynamoDB-compatible interface as of release 2020.1. Scylla Enterprise is also the same production-grade code we run on our Scylla Cloud service. Like Scylla Open Source it also supports LWT; CDC will be available in an upcoming release.

Kubernetes Operator

We have a beta version of our Kubernetes operator so you can fully deploy and manage a DynamoDB-compatible database wherever you wish.

Migrating from DynamoDB to Scylla is Easy!

To make the switch as easy as possible we’ve extended our online migration tools to be relatively simple: just start streaming the changes from DynamoDB plus run a full scan. The Scylla Spark Migrator project has been enhanced to support the DynamoDB-compatible API.

Contributing to Project Alternator

Scylla and the Alternator interface are true open source projects. We encourage those looking to contribute to review the source code and any known defects, and read our guidelines on how to add their own contributions.

Developers looking to become open source contributors to the project are further encouraged to join the scylladb-dev mailing list, or to visit ScyllaDB’s Slack.

Learn More