The threat of cybersecurity is real, pervasive and omnipresent. Real-time global cybersecurity attack maps, such as those maintained by Kaspersky Labs and Fortinet show there are no breaks, vacations or downtime allowance in the world of online and computer security.
The scale, variety, complexity, velocity and ferocity of attacks has compounded year-over-year. The costs associated with such attacks also continues to increase, and cybercrime is estimated to exceed $6 trillion annually by 2021. Beyond the criminal costs, there are also risks that range from national security to personal safety.
Veramine is one company tackling the national security threat. Awarded contracts from the U.S. Air Force and U.S. Department of Homeland Security to defend against cyberthreats, Veramine is also a commercially available service for enterprises.
About Veramine
Veramine provides advanced capabilities for reactive intrusion response and proactive threat detection. Using endpoint telemetry to feed a central server, Veramine scours huge amounts of network data to identify attacker activity. With its advanced detection engine, advanced rule-based and machine-learning algorithms, Veramine can identify Mimikatz-style password dumping, kernel-mode exploitation (local EoP), process injection, unauthorized lateral movement, and other attacker activity.
The Challenge
For Veramine, cybersecurity analysis begins with good data. As such, the company’s goal is to collect as much data from the enterprise network as possible, including both servers and desktops. Events collected by the platform are enriched with context information from the system. For example, every network connection that’s recorded is associated with its originating process, user, time, and associated metadata. The end result is huge and ever-growing data set.
“Even if the performance were only as good as Cassandra, and in fact it’s much better, ScyllaDB would still be a significant improvement.”
— Jonathan Ness, CEO, Veramine
According to Jonathan Ness, CEO of Veramine, “We’re trying to collect everything you would ever want to know about what goes on on a computer and centralize that data and run algorithms on it.”
All that data needs to be stored somewhere. Since the data is so sensitive, very few Veramine customers permit it to be stored in the cloud. Veramine needs to provide low-latency big data capabilities in on-premises datacenters. Given the sensitive nature of the data collected, Veramine personnel were unable to directly access databases to help with support.
Veramine began using Postgres, but quickly realized that a NoSQL database was more appropriate to their use case. They switched to Cassandra, but soon realized that it was not up to the task.
“The problem was every week it was crashing, so we created all this infrastructure just to keep Cassandra alive,” said Ness. Veramine went so far as to parse Cassandra logs in an attempt to predict when garbage collection would happen, and then apply throttling to avoid crashing the database. Without direct access to customer environments, Cassandra soon became a nightmare. The team set out to find a replacement.
The Solution
What was needed was a low-latency enterprise NoSQL database that provided extremely low administrative overhead and high stability. Initial attempts to use PostgreSQL did not meet the challenge. And while Cassandra was able to scale, it had operational management problems. That was when the Veramine team turned to ScyllaDB, the monstrously fast and scalable NoSQL database.
Veramine saw instant results from using ScyllaDB. “We started using ScyllaDB two years ago,” said Ness. “We fell in love with ScyllaDB because it doesn’t crash and we don’t have to manage it.” Since ScyllaDB is a feature-complete, drop-in replacement for Cassandra, the migration was quick and painless. “Our code didn’t change much going from Cassandra to ScyllaDB.”
According to Ness, a big benefit of ScyllaDB is developer productivity. ScyllaDB lets the team focus on business logic rather than on custom code around the datastore. Veramine’s ScyllaDB clusters that are running in production are surprisingly small compared to Cassandra.
Ness summed up Veramine’s ScyllaDB journey: “Even if the performance were only as good as Apache Cassandra, and in fact it’s much better, ScyllaDB would still be a significant improvement due to its stability and lower administrative overhead.”
