Cloud Native Computing Foundation Kubernetes Community Days Bangalore 2022, 11 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: BotKube - Real-Time Kubernetes Monitoring and Alerting

Description

Developers and operators should be able to interact with Kubernetes clusters in a collaborative manner. Teams often collaborate in a chat application like Slack/Teams and this is how the idea of BotKube was born. BotKube is a bot for your Kubernetes cluster - you can talk to Kubernetes clusters using the same Kubectl commands. BotKube also comes built with plugins and extensions which inform you of critical problems in the cluster. The extensible architecture allows you to plug any external tool which can give one useful information about the cluster. This talk will demo BotKube interacting with a Kubernetes cluster and a developer trying to fix a failed deployment in the cluster using BotKube. And hopefully, the developer succeeds in his endeavor!

A

Hello, everyone welcome to kubernetes community days bangalore. My name is prasad and I'll be showcasing. What cube in this talk a brief introduction about me? My name is prasad ganga. I work as tech leader at infrablog technologies. I love working with kubernetes, co and open source. I'm also the creator and maintainer of what cube that we'll be discussing in this talk.

A

uh So this is going to be a short talk. Well, where we'll be discussing what exactly is spot cube and how it helps in different use cases um yeah. So birdcube is basically chatbot for your commodities cluster. It helps you monitoring the resource events. It could be resource lifecycle event, as well as resource error events. uh It also helps you in collaborative debugging, so uh it it. Basically, uh it can execute cube cuddle commands uh by uh right from your messaging platform window.

A

So, for example, in slack you can just uh add you can just message bot cube and it it can execute commands that you want and returns response back. So it helps you in uh collaborative debugging uh how we'll we'll be using in the demo uh yeah.

A

So what cube can be installed in popular uh messaging platforms, like slack teams matter most discord, and you will be seeing more and more coming in the next releases yeah, so whatcoup can monitor resource resource events and thanks to what keep configuration it provides you, a fine grain configuration where you can filter out the events you are really interested in. So you can sorry, you can filter the resources.

A

um If you are interested in event types and even the name spaces uh for the update events, you can even filter out uh the the uh the field in uh in which the updates you're interested in uh yeah, more information. You can, you can find on the documentation uh yeah what cube supports? um What has something called filters so on occurrence of every kubernetes event? uh The event goes through chain of what keep readers, where vodka performs a few checks and modifies the event if required.

A

For example, if you talk about inbuilt filters, uh what give us something called image: stack: checker filter where it checks uh if the event of type is of type pod, update or pod create, and if the container image has latest stack used. So we all know latest tag is not recommended. It's not best practice, so that filter adds recommendation in the message that this image stack should be avoided so like that, you can have your own custom checks which can add or remove or modify the event messages as per your need.

A

So bird cube architecture provides your way to define your own custom filter and inject that in the filter engine in which it will get executed, cool um yeah so demo time. Let's.

A

Yeah, let's uh talk about use cases now um so I'll be talking uh about slack only so what kip can be integrated with multiple uh messing platform, but for the um I will be limiting the scope of demo to slack only so most of the time I mean all the time I'll be discussing about slack, uh but same is applicable for different uh messaging platforms.

A

All right, so uh podcast installation is two-step process. Basically, you need to first install vodka app what keeps slack app to your slack workspace. So I'm having this workspace and I have installed vodka from the official slack marketplace and then for your slack app. um You have to install um install the back end for your app so um in the kubernetes cluster. You want to monitor. You have to install vodka back-end with the token that you will be getting when you install the slack app.

A

So it's like your slack app is communicating directly with your slack back end um using standard slack apis. So you don't have to worry about privacy and security all right, so I have installed botcube in such a way. uh What key backend in such a way that uh the events occurring on staging cluster uh will be forwarded to staging alerts channel and I've also. I have also allowed execution of cubicle commands now with the default permissions.

A

uh The default permissions that that's it I mean if you install bot, cube the default permission, it has are read only so you can only execute commands like uh get ports describe logs.

A

um We don't allow uh write commands by default, but if you want, you can obviously modify the cluster rules and what could config to add to support additional crypto commands all right. So, as you can see in this staging alerts channel, I have got few notifications uh regarding pod failure.

A

So it seems like the pod uh in nginx namespace in staging clusters is, uh is failing right. So this is how you can get notifications um about, create delete, and you can obviously configure what cube to uh filter out events so that you'll get only the ones you're interested in all right. uh So we'll now see how what you can be used for incident management.

A

So this is the failure. For example, we have got, and now we want to uh triage it right. So we have created a slack workflow.

A

um So if I add this image here, the message gets forwarded to the staging related stream channel. Now this this is a private channel and we have configured what cube to allow cube critical commands in this channel and uh you can obviously add the responsible forks here which are which can uh triage now right from this lag messaging window. You can now debug this this application, phillips so yeah, let's, let's, uh uh let's see how we can do that.

A

So, if I have to you, know debug the pod failures I'll follow the standard um standard steps like I'll first get status of pod right um so before that we need to, we will execute what keep ping command to just make sure that the back end is working fine. So we have uh received bomb from stating the environment.

A

uh It will list all the. So if this app is configured with multiple backend backends, uh for example, you have staging environment, div, environment production element, so you will get pong from all those clusters.

A

Now I want to execute uh what a gate command, for example, uh on staging environment cluster so I'll, instead of queue, cutter get ports, I'll execute what keep get ports uh m space engine x and then I'll mention the cluster name, because you can have multiple clusters.

A

Right uh so yeah, it's clear that it's in it's not in ready state um yeah, let's see if you know if he gets something in the dogs, so just like quick catalogs I'll, execute vodka blocks.

A

Nginx, excuse me bold ingenix, namespace again, cluster name is sdg e. I think in logging we are not seeing any error. Maybe let's describe the point and see what what what exactly we are executing uh in the readiness probe so I'll describe pod again just stop this command. We can just copy.

A

All right, um if you look at the readiness probe it's it's uh checking, it is making sure that the port is 880 and the port is. It is reading from utc config port, and if you see here, the tc config is mounted from configmap right.

A

So yeah, let's see if what what is the value we are mentioning in the config map.

A

Kick maps name is quantity and linux, cluster name, sdg e and p.

A

All right, uh yeah so.

A

Let's describe the continent and see the value.

A

Okay, so the port is, uh someone has updated the port 8081 and we are expecting 80., so yeah I mean we have found the root cause right so yeah. This is how um what can help you in collab collaborative debugging. um Now this is something bonus you can so by default. We only allow redone the commands right, but you can configure what cube to uh allow right commands as well. For example patch uh you can. You can consider bot cube and what cube cluster bindings to allow that.

A

So we have given uh bird cube access to execute few write commands like batch and then right from this slack window. We can fix this issue so I'll, execute patch command and I'll fix the port.

A

That's it and then we'll again create status of part.

A

A

uh We'll wait for a few seconds to get uh so that it will pick up the latest uh configmap changes but yeah. This is how uh you know you can uh can be used for collaborative debugging and um yeah helps uh the the responsible force to fix the issue faster.

A

A

Yeah so, let's last time get the status and it's running. We also got a notification here, all right, cool um yeah. So you can expect few interesting things in upcoming releases um yeah. So more built-in filters, like cubescan profiling, support to have fine-grained mapping like namespace to a particular channel and we're also working on a wordcube configurator which helps you uh creating what config.

A

All right, um yeah feel free to you know, explore code uh days. If you have any issues or suggestions, you can raise issues on uh infra cloud. I o bought cube, git, repo um and yeah. uh If you have any further further um doubts, you can join our slack workspace join.whatcoup.org.

A

We will be happy to help you with anything the documentation and all the installation process. Everything can be found on whatcube.org.

A

Cool thanks a lot.