Case Study: Investing.com Delivers Real-Time Finance Data Application to 11 Million Unique Monthly Visitors

By Gabriel Mizrahi CTO, Investing.com

About Investing.com

Investing.com is a global financial portal and internet brand composed of 30 editions in 22 languages and mobile apps for Android and iOS that provide news, analysis, streaming quotes and charts, technical data and financial tools about the global financial markets. Each edition covers a broad variety of local and global financial vehicles including Stocks, Bonds, Commodities, Currencies, Interest Rates, Futures, and Options.

Founded in 2007 but called ForexPros at that time, then rebranded to Investing.com in 2012, the site has a growing readership worldwide with more than 12 million monthly uniques and close to 400,000,000 monthly page views across the network. Today Investing.com is ranked 700 on Alexa.

Source: Investing.com site

Challenges

We used Apache Cassandra 2.1 to serve real-time data and charts on our site – a critical part of our offering. We had a three data center deployment using Apache Cassandra 2.1. The main issue was unpredictable latency spikes. GC (JVM Garbage Collection) pauses up to 6 minutes cause latency issues for the site and hurt user experience.

A lot of effort was put into JVM heap tuning, but no value combination seems to solve the issue.

With 11,600,000 Monthly Uniques visitors and 690,000 Daily Mobile visitors, any delay affects many users.

Solution

“When we heard of a Cassandra drop-in-replacement with much better latency and throughput we were skeptical. But eventually, we found out that everything claimed was completely true.”

– Gabriel Mizrahi, CTO, Investing.com

When we heard of a Apache Cassandra drop-in-replacement with much better latency and throughput we were skeptical. But eventually, we found out that everything claimed was completely true.

At first, we deployed a ScyllaDB 1.2 cluster working in parallel with the Cassandra solution. All our new data was written to both platforms. After realizing ScyllaDB was stable and could handle our intense writes, we migrated our historical data using some in-house scripts and tools.

During the following couple of weeks, we slowly moved our site’s components, one by one, to work with ScyllaDB, evaluating the result at each step – performance, latency, and stability.

Since ScyllaDB and Apache Cassandra share the same drivers, the code transition was pretty smooth.

Not only were the latency and GC issues completely gone, the improved latency and better HW utilization allowed us to shrink the cluster size by half!

The only reason we are keeping the old Apache Cassandra cluster is for Counters, and once ScyllaDB introduces them (planned for March 2017) we will fade out the Apache Cassandra cluster for good.