At Palo Alto Networks, we process terabytes of events each day. One of our many challenges is to understand which of those events (which might come from various different sensors) actually describe the same story but from many different viewpoints. Traditionally, such a system would need some sort of a database to store the events, and a message queue to notify consumers about new events that arrived into the system. We wanted to mitigate the cost and operational overhead of deploying yet another stateful component to our system, and designed a solution that uses ScyllaDB as the database for the events *and* as a message queue that allows our consumers to consume the correct events each time.
Hello, everyone. My name is Daniel Belenky, and today I will be talking with you about stream processing with ScyllaDB. So my name is Daniel Belenky. I’m a Principal Software Engineer at Palo Alto Networks. My background is Kubernetes virtualization, distributed applications, big data and stream processing solutions, and let just start. So the agenda for today is, we’ll start with a brief introduction of the product and my team.