What’s the problem?
Recently several NoSQL database clusters were targeted by hacker attacks. According to a report, the first known occurrences affected MongoDB, but soon after ElasticSearch clusters, Hadoop servers and CouchDB databases were affected as well.
Is it relevant for the security of ScyllaDB installations? Heck yeah! In order to prevent an attack on your ScyllaDB installation, proper configuration is critical from the get go.
In some cases the default configuration and example configurations shown in manuals are not the best ones for securing your data from unauthorized access.
What should I do?
So, what should you do to make sure your data is safe and you won’t have to pay ransom in order to recover it?
First of all, think about the security configuration in advance. The best problem is the problem that didn’t happen.
Don’t think that you are immune to a hacking attack. You will regret not taking security seriously when bad things happen and it is too late.
Now let’s review how hackers get to our precious data if we allow them:
| Connecting to the CQL/Thrift port (ports 9042 and 9160 correspondingly) | This is how the users from the report got attacked. If those ports are open and there isn’t any authentication enabled, the hacker may simply connect to your cluster with the cqlshand do whatever he or she wants – from reading all your data to deleting it. |
|
| Listening to inter-Nodes communication (port 7000 for non-encrypted communication and port 7001 for SSL-encrypted RPC) | This is a more complicated attack but it’s definitely doable. For example, if the attackers succeed in redirecting all of your inter-Nodes traffic to their own servers (man-in-the-middle-like attack), they will be able to lay their hands on all of your new data if it’s not encrypted. | |
| Connecting to your JMX port | JMX is used by the tools like nodetool and by leaving it open would let the attackers to gather the important information about your cluster, e.g. with nodetool status command. | |
| The REST API (port 10000) | REST API provides important information about your data that may be used by attackers later, for example, the names of all column families in your cluster. Therefore it is critical to close this port from unauthorized access. | |
| The Prometheus client (port 9180) | Prometheus data is your cluster’s statistics data. It is less critical compared to the data stored in your cluster, but still must be protected. | |
| The collectd statistics server | The collectd client in ScyllaDB server is a “push” client – it sends the statistics data to the collectd server. As with the Prometheus data, don’t let anybody read data from your collectd servers unless you want him or her do so. |
So, don’t be adventurous by allowing unauthorized access to these ports/data.
How to prevent unauthorized access to ports and data
- Close all the ports above in your firewall configuration on all Nodes. The ultimate configuration would be to put your cluster’s Nodes inside the VPN and deny any access to them from outside the VPN. Access your Nodes only from inside the VPN and protect the access to the VPN with the proper authentication.To check which ports are listening run:
netstat -lapn | grep scylla | grep LISTEN - Verify that your application doesn’t listen on the above ports on the public addresses unless the connection to them requires further authentication like on 7001.
- Use SSL-encryption for inter-Node RPC if it has to go outside the VPN (For example, in a multi-DC setup on AWS) and for Client to Node communication. See this manual: http://docs.scylladb.com/tls-ssl/
- Enable Clients’ authentication with the password. Some rules of thumb:
- Change the replication strategy of the system_auth keyspace to NetworkTopologyStrategy.
- Change the replication configuration of the system_auth keyspace to store all data on all Nodes in your cluster. For instance if you have data center ‘DC1’ with 3 Nodes and data center ‘DC2’ with 4 Nodes the command should be:
ALTER KEYSPACE "system_auth" WITH REPLICATION =
{ 'class' : 'NetworkTopologyStrategy', 'DC1' : 3, ‘DC2’ : 4 };This way even if all Nodes but single one goes down you will have your authentication data and will still be able to login into your cluster. - Create a different superuser with a strong password and revoke the superuser from the “cassandra” user.
- Change the password for the “cassandra” user.
- Better use a non-super-users for accessing the Nodes. Leave the superuser for administrative tasks.
- Use objects’ permissions (GRANT/REVOKE paradigm). This is going to prevent the loss/leak of all of your data in case the attacker has been able to steal the credentials of some non-superuser(s).
Do them all and your data will be reasonably protected.
