Apache Cassandra Apache Cassandra World Party | July 2022, 20 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Monitoring Cassandra | Apache Cassandra World Party 2022

Description

Presentation from the second Cassandra World Party on July 20, 2022. Learn more about Apache Cassandra at https://cassandra.apache.org/

Michiel Saelen of Skyline Communications presents “Monitoring Cassandra”
This lightning talk will cover how we are using Cassandra as DB for our SW. To ensure it is working well we are monitoring it with our own SW.

About the Presenter
Michiel joined Skyline Communications in Belgium just after university and has grown from System developer to Solution Architect since.

Apache Cassandra is an open source NoSQL distributed database trusted by thousands of companies for scalability and high availability without compromising performance. Linear scalability and proven fault tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data. Get started: https://cassandra.apache.org/quickstart/

The Apache Cassandra World Party 2022 was a virtual, one-day celebration on Wednesday, July 20, 2022 held in anticipation of the launch of 4.1

Join the Cassandra community: https://cassandra.apache.org/community/

#ApacheCassandra #database #NoSQL

A

Hi everyone, my name, is michiel salon. I had the privilege to be part of the skyline communications team for five years now. During that time, I started to learn about cassandra by investigating issues mainly, and from that moment almost I had a growing interest for cassandra, how it works and also how it should be used. um Then the main product of our of skyline communications is data. Miner.

A

Our ceo has a very long description for our product, uh but in short data miner monitors and control any device or system that has a way of communicating with it could be http, snmp, serial or any form of api. With data miner, you can reserve equipment in your system. uh Functions of the equipment uh through bookings run scripts that configure your complete end-to-end chain um for service delivery. You can aggregate statuses and alarms uh to the complete chain and that's just the tip of the iceberg.

A

Yeah on the previous slide, you saw some aggregated results and use cases can also be uh checked on our community platform. With the link there on data miner, we push up to 800, 000 monitoring points per seconds towards our database, and we make use of cassandra and elastic for our software to ensure that the database is not running into any trouble. We monitor the database with our own products.

A

In the background we are using the c sharp driver to connect to the cassandra system, and then we perform sql queries to get the statistical data out of it. Here you can see a overview of all the tables that are in cassandra that we retrieved through cql.

A

All this information, as I said, is retrieved through cql pushed into data miner, and here you can also see we have a column, repair checks, that's something we will come back on later um from cassandra from cassandra 4.0. We have virtual tables, for which we are very grateful to everyone that helped to make this possible prior to 400. If you want to get all of the valuable data out of it, we had to set up remote, scripting, remote jmx, which could be a very large challenge to do this in a secure, wear secure way.

A

Here we can see an example of such virtual tables, so we iterate over all of the nodes to get the data and we can join them into one big table. So it's easy to compare the different nodes and on the size column you saw there was a trend icon. So if we click on that, we can bring up also the historical data of it, and here you can see for three notes how it's progressing over time.

A

So we can easily see what happened in the past on the notes, but we can also predict a bit towards the future if you will run into uh problems and scale. The system if needed.

A

Another interesting virtual table for us is uh the settings table which allows us to easily detect anomalous between configuration and settings. So here you can see that if we search for authentication we see password authenticator is enabled on all of our nodes. So we can easily see if there would be one node that has a missing configuration on there, we're also looking yeah on the left side.

A

You can also see a lot of other pages that we cannot cover here in this lightning talk, but we're also looking to expand the driver towards the feature uh towards the future, um and one of the things that we also introduced in this driver is system checks. So all of the data that we retrieve from the system we are evaluating and then giving suggestions to the operators.

A

So here we have three uh checks that failed on the system. We have a repair check where we check if the tables were repaired within the gc correct grace seconds. That was the column that you can can disable uh for every table so that, if you have, for example, uh a gc gray seconds, which is very low for hinted, handoff that you don't get false alarms um and then immediately. You know if there is something wrong and there are also some other checks as well that we're also planning to expand here at last.

A

I also want to give you an overview of what we have done with data miners, so we have over 1500 customers, 9 000 systems and 125 countries that have data miner deployed, and some of our key customers can be found on this wall. Here, uh probably, you know a couple of them and if you have any questions uh they can be made on our community platform.

A

You can search also through our questions. uh They're already also a broad variety variety of use, cases on our community platform, or you can also send emails directly to me. If you have any questions.

A

Thank you for listening and also thank you for letting me speak on this uh lightning talk was a very inspiring.

B

That's awesome. Michelle um really amazing work, so I'm really interested in the fact that you're already taking advantage of the virtual tables, um I'm gonna, have to um catch up with you at some point to talk about maybe doing a series of blog posts on the cassandra website, because I think um is going to be awesome and in fact that goes with all the other um speakers as well. All the other presenters um yeah. So looking forward to collaborating with you and getting some content up um about data miner on the cassandra website,.