Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2019 (Barcelona), 21 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Using Jupyter Notebooks To Gain Insight Of Your Cluster - Ruben D Orduz, VMware

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Lightning Talk: Using Jupyter Notebooks To Gain Insight Of Your Cluster - Ruben D Orduz, VMware

For the last 7 years or so Jupyter Notebooks (formerly known as IPython) has revolutionized the way scholars, enthusiasts and data analysts look at, process and work with data. On the Kubernetes side, component (api-server, scheduler, controller manager, etc.) logs, events, etc. can be an indomitable fire hose of data that can rather difficult to analyze and work with in its raw form. However, as we'll demonstrate, with the right tools, you can create data pipelines so that it can observed, analyzed and visualized in an interactive way using Jupyter notebooks. Insight therein can then be used for any task from performance tuning to debugging. The outline for this lightning talk is straightforward: * Problem statement (~ 1 min.) * Solution and tooling description (~45 sec.) * How the tooling was used ( ~1 min.) * Go over the data processing pipeline (~1 min.) * Results (~45 sec.)

https://sched.co/MPgQ

A

Hey everyone, so, okay, real quick! This is me I work at VMware via hefty acquisition, my github Twitter and Kay slack is our do dev. That was the best way to reach out to me so really quick.

A

We only have five minutes so the products I mean we want to introspect and see what's going on in our cluster, and if you want to do that in a practical way or if you want to find meaningful information, you're going to be found with a bunch of depending on the logging capabilities, you might find difficult to sift through all the logs that you're gonna get. You might be able not be able to see errors, warnings and that type of information that might lead you to whatever is the information that you are actually looking for.

A

This is what you get so in the interest of time. So let's focus on three basic areas right, so we are going to need a basic log data gathering. We're also gonna have to marshal, or do some ETL with that data and eventually, when I have to visualize it in order to see to get some form inside the tools or this puzzle. For the sake of this talk is going to be Jupiter lab. This is how we're gonna use the Jupiter notebooks, the Kuban. It is Python client.

A

We also gonna need a centralized way to store data structure. Data locally Python comes bundled with sequel ID, which is great. You don't have to install any other server in it. It already comes bundled with it. Also we're gonna use, Python string, manipulation, built-ins and the pandas data manipulation library.

A

So, just to give you a real quick, so you know obviously a we're gonna look at what go through, so we're gonna simply import these. That you're, seeing here this is the environment of Jupiter notebooks, we're gonna, we're gonna import, all the libraries that we need. This is all the code that you're gonna need to instantiate a connection to a sequel light database.

A

Then you're gonna see like some basic instantiation of the Cuban IDs client and then we're gonna. In this case, for this particular example, we're gonna follow the cube, API server, the we're gonna follow the logs in a form of a stream and then we're gonna do some basic manipulation. Turning me to an array and stuff, and then how did I come up with this literally trial and error, which I started looking at the log files, their shape and I found a norm? I wait to normalize them so that way, I can store them.

A

Eventually, I'm going to turn all that eh all those Lots files and I'm gonna put them into into simple form into the database.

A

Okay, so, as we can see, you know you can have a very rudimentary. You know a structural log collection and data collection. You know, are 30 lines of code and then, of course, for that to be useful or some use via some use. We need to have some actual data to analyze.

A

We are then I.

A

Basically, I ran a bunch of different synthetic loads right, like I did empty a name, space trash and I created and deleted a bunch of them, like thousands of times, I create I, did the same with a non empty name, space and I scale up and down a deployment and number of times it I generated like one hundred and twenty two thousand lines of the ones that I care about no just total, just the ones that I care about as I already show you in the last piece of code, I'm filtering for the things I'm looking for.

A

Oops, okay, so using that code, that you show you alright. So we we already have that into the sequel light database and we can surely we could query sequel if you're a DBA and you like sequel, but you know I personally, prefer to have things in a very manipulative all in a very programmatic way, and this is where pandas comes in, like it's, a great library, very efficient and it. This is how I'm going to show real, quick, a little code of pandas, so we're gonna do basically a table data dump into pandas.

A

You generate the you know, select star from the table that we just created and then we dump it into a panda's data frame and we just give it some labels to the columns. We want I pick around at the first two rows that just so that you see that how the data will look like below that I did a pandas offers like a column filtering, and you can even do pivot things inside it without having to manipulate the underlying data, which is great.

A

If you actually need to do that and then you can also have programmatic queries in this cases, as well of all the tests, that I did. You know I group them by test and then I say like of all these things like which ones gave the most error and then, with this few lines of code, I was able to see them in this case, like the replica set thrash and the namespace thrash, where I deleted and created real quick had the most errors.

A

Almost you know in this case, over six times of the other two tests that I run.

A

Anyway, so like yeah, you show you pandas, you can do basic visualizations bars interns in charge. If you need something more advanced, you can use bokeh and just to close Python, Jupiter sequel, light, pandas and bokeh. If you need a more powerful visualization, then you can have your own full-fledged interactive communities. Analysis labs! Thank you.