Hello again readers.
After some in-person catch up with the rest of the ScyllaDB team (yes, there are advantages to in-person events), we were up at sparrow’s fart to get ready for another big day of P99 CONF.
By this point, we all had time to reflect and read back through the many comments in chat. The organizing team were busy little beavers tweaking pre-recordings, sound levels and of course the now infamous P99 CONF playlist! This was a bit of a surprise for the team. So many people online commented on the groovy tracks picked by the conference organizer Natalie Estrada that we felt compelled to share that playlist. So here it is – the unofficial (and apparently un-shazam’able) list of tracks that kept you entertained between sessions.
With a liquid breakfast, some quick sound checks, and review of the packed schedule, we were back into day 2 with some exciting presentations to open. First up was Gwen Shapira speaking about “High Performance on a Low Budget.” This is another presentation that I would like to replay and take notes from. I loved the opening statement around setting performance expectations early. This was reassuring to me, a performance advocate, who often hits the brick wall of “No, performance is not a concern right now”. So it was great to know we’re not alone with this sentiment. There were also some quotable quotes in this presentation, like making your benchmarks “pay rent”, and if you, “have no time to optimize, at least don’t pessimize”. It was also refreshing to hear of Gwen’s experience taking lessons from a prior big organization and applying them to a relatively smaller team – not to mention cracking the whip and busting out Wireshark on unsuspecting developers 🙂
Speaking of quotable quotes, there were great tweets on social – I’m personally a fan of the “CAT” theorem, and also the two button meme, well played 😉
CAP Theorem? I thought it was CAT Theorem!
– Yogi, the distributed systems enthusiast, refreshing his Raft knowledge at #P99Conf
— Naz Ahmed 🚢 (@naz_io) October 19, 2023
True before, true this year too.
— Paul Philleo (@philpauleo) October 18, 2023
That reminds me… as a host, I was glued to stage 1 helping out and monitoring the chat. There was also a whole team of ScyllaDB staff working remotely, some all the way from Brazil including Felipe Cardeneti Mendes, who happens to be one of the authors of the new book “Database Performance at Scale.” Given I had some time to burn at the hotel the night before, I got a sneak peek at this book and can confidently say it’s a nice side companion to the art of scaling databases for high throughput, low latency applications. I encourage you all to have a read of the free book.
The next highlight of the day was hearing from Bryan Cantrill on the topic of “Corporate Open Source Anti-Patterns: A Decade Later”. Now I must admit, I’m not great with names, so it took some time for me to make the connection that Bryan developed DTrace, something I used daily way back when. But once that clicked, I once again listened in awe to an industry great, and felt privileged to hear his account of what’s changed, and where we’re at with open-source ten years down the track. Bryan showed us why he’s a renowned public speaker with his entertaining presentation, and set the scene for another live discussion with two of his mates, Adam Jacob and Ashley Williams. Essentially, the creators of DTrace and Chef and a former Rustlang team member, all in one virtual room and really sticking it to the issues, concerns, and expectations around open-source. I genuinely felt bad ending the discussion at the 20-minute mark, but the show had to go on! I feel this discussion should be given more airtime in another format.
Now that you have the luxury of watching the recordings in slow time, without being forced to choose between stage 1, 2 or 3, there were also a couple more must see presentations during the live discussion. Be sure to check out Stefan Johansson’s presentation “Reducing P99 Latencies with Generational ZGC” and also Tammy Everts on ”Performance Budgets for the Real World” which featured around the same time.
Flipping back into a technical mode, Travis Downs took us on “Adventures in Thread-per-Core Async with Redpanda and Seastar” which is the open-source C++ framework for high-performance server applications on modern hardware, built and maintained by ScyllaDB. I feel this was an excellent example of some lessons learned in the previous live panel around integrating open-source with commercial products. I’m not a C++ expert myself, but judging by the detailed conversations in chat, this presentation really hit a chord with technical experts present at the conference.
We also had a splendid presentation from Danny Kopping on “Cache Me If You Can: How Grafana Labs Scaled Up Their Memcached 42x & Cut Costs Too” which was definitely my cup of tea. Danny laid out the problem space and how they approach tuning and optimization to achieve impressive performance. It was also great to hear about the perspectives of large SaaS applications and open-source products, such as Loki referenced in this presentation, but also many of the other industry leaders in similar presentations.
Closing out the day for me, I was genuinely interested in Dmitrii Dolgov’s views on “Demanding the Impossible: Rigorous Database Benchmarking.” This presentation was the perfect balance of math, statistics and referenced studies for performance benchmarking in general, which, I felt, could be applied to wider performance topics. Definitely a track worth listening to if you want to expand your understanding of this topic. Tim Vereecke also gave a great presentation on “Noise Canceling RUM” and plenty of thinking material on the pitfalls in chasing percentiles, and how to improve that signal-to-noise ratio.
I’ve got no other secrets to share short of what was already made public by Wayne at the end of the conference – 18K participants, 4K chats and a now 30K strong P99 conference community. It was a pleasure to be part of this conference and I can’t wait to see how it turns out in future.