Building a Mutant Monitoring System (MMS) Day 1

This will be a series of blog posts that provide a story arc for ScyllaDB Training.

Backstory

Mutants have emerged from the shadows and are wreaking havoc on the earth! Increased levels of mutant malicious behavior pose a threat to national security and the general public. Luckily, there are mutants who have teamed up with Government agencies to help protect people against the bad ones.

To be better protect the citizens and understand more about the mutants, the Government enacted the Mutant Registration Act. As required by the act, each mutant must wear a small device which reports on their actions every second.

Your mission, as a new member of Division 3, is to help the Government keep the Mutants under control by building a Mutant Monitoring System (MMS). The MMS will consist of a database that will contain a Mutant Catalog and Monitoring system.

Choosing a Database

A NoSQL database is preferred for MMS because it has high-availability, a scale-out data set, disaster recovery capabilities, and can accommodate IoT time-series workloads – all the requirements for the MMS database.There are many NoSQL database technologies to choose from such as Hbase, Apache Cassandra, MongoDB, Influx, and ScyllaDB.

ScyllaDB was chosen for the MMS because it leverages the best from Apache Cassandra in high availability, fault tolerance, and its rich ecosystem. ScyllaDB provides a dramatically higher-performing and resource effective NoSQL database to power modern and demanding applications with low latencies and none of the Java garbage collection woes found in other NoSQL databases.

A collection of data on each mutant needs to be imported into the database. The Mutant Catalog database will keep track of basic contact information for each mutant, including the mutant’s first name, last name, address, and photo.

To monitor the mutants, we need a separate tracking database that consists of time-series data such as the mutant’s location, speed, velocity, heat, and telepathy powers. The data will be gathered from various sources and imported into the database. After the data is gathered, we will be able to analyze the statistics.

Setting up a Cluster

Before we proceed with the ScyllaDB installation, please ensure that your environment meets the following prerequisites:

1. Docker for Mac or Windows. Please note that running in ScyllaDB in Docker is only recommended to evaluate and try ScyllaDB. For best performance, a regular OS install is recommended.
2. This Git repository.
3. 3. 3GB of RAM or greater for Docker.
4. If you are using Linux, you will need docker-compose.

Now let’s proceed and get a three node cluster up and running in Docker.

git clone https://github.com/scylladb/scylla-code-samples.git
cd scylla-code-samples/mms
docker-compose up -d

After a minute or so, the MMS should be up and running. Let’s verify that the cluster is up and running:

docker exec -it mms_scylla-node1_1 nodetool status

Here we can see that all three nodes are up and running in the cluster.

Building the Mutant Catalog

As we explained, the Mutant Catalog will consist of the name, address, and picture of each mutant. Let’s connect to a ScyllaDB node to create it. To manipulate the ScyllaDB database, we will be using the cqlsh tool. Cqlsh is a command-line tool written in Python that allows us to run Cassandra Query Language commands on ScyllaDB.

docker exec -it mms_scylla-node1_1 cqlsh

The first task is to create the keyspace for the catalog.

CREATE KEYSPACE catalog WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy','DC1' : 3};

The catalog keyspace uses the NetworkTopologyStrategy because we will need to think about replicating the data to other data centers for disaster recovery purposes later. The data replica count is 3 which means that each node in the cluster will have a copy of the data.

Now that the keyspace is created, it is time to create the table.

use catalog;

CREATE TABLE mutant_data (
 first_name text,
 last_name text,
 address text,
 picture_location text,
 PRIMARY KEY((first_name, last_name)));

The data types used in the catalog are text which is basically a string of data. The “PRIMARY KEY” section allows us to query the database by the mutant’s first and last name after we begin to gather data.

Now let’s add a few mutants to the catalog with the following statements:

insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Bob','Loblaw','1313 Mockingbird Lane', 'http://www.facebook.com/bobloblaw');

insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Bob','Zemuda','1202 Coffman Lane', 'http://www.facebook.com/bzemuda');

insert into mutant_data ("first_name","last_name","address","picture_location") VALUES ('Jim','Jeffries','1211 Hollywood Lane', 'http://www.facebook.com/jeffries');

We can see all of the data inserted with the following statement:

select * from mutant_data;

To query the database for “Bob Loblaw”:

select * from mutant_data where first_name='Bob' AND last_name='Loblaw';

Summary

We now have the ScyllaDB infrastructure in place to start building out the Mutant Monitoring System. The Mutant Catalog is the first keyspace that was created to gather basic metrics for each mutant such as name and contact information. With the latter part of the training, you learned how to add and query data from this keyspace. In the next post, we will look at building the Tracking System. The faster that we get this system up and running, the better for the fate of our country. Please remain safe out there.