keptn Community videos, 5 Jul 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Autoscaling Kubernetes Workloads Based On Metrics

Description

Title: Autoscaling Kubernetes Workloads Based On Metrics

Abstract:Tired of scaling your Kubernetes workloads manually? Tired of overprovisioning resources that lead to unnecessary costs? Tired of underprovisioning resources that lead to degraded application performance? Join Rakshit in his talk in which he will briefly explain how we can use metrics to autoscale our Kubernetes workloads with the help of HorizontalPodAutoscaler. Join him on a ride to learn about the history, the different types of scaling techniques, and finally go through a live demo of scraping metrics from Prometheus using the Keptn Lifecycle Toolkit and scaling the workloads using HorizontalPodAutoscalar. Being an active contributor for Keptn, He will briefly explain how the Keptn Metrics Operator enables us to have a single entry point for the metrics data, regardless of the source of metrics and helps us in autoscaling our workloads. Get to learn about the challenges faced while scaling workloads manually. Get to learn about the benefits of using metrics to autoscale Kubernetes workloads without having to fear unessential costs and poor application performance. Get hands-on experience using the Keptn Lifecycle Toolkit to scrape metrics from any metric provider and then use it to autoscale your Kubernetes workloads.

Speaker: Rakshit Gondwal

A

Rakshit the Bible is from Punjab college of engineering and he is going to speak on auto scaling, kubernetes workload, balance on metrics. uh In this talk he will explain the use of Matrix to Auto scale, kubernetes workload with the help of horizontal pod atoscaler.

B

So hey everyone, first of all, I'd like to thank you all for joining me here at my talk and also a big thanks to the organizers for giving me the opportunity to speak here at kcd Mumbai- and this is my first time speaking at a conference, so yeah I'm a little bit nervous, but a lot more excited.

B

So today, I'm going to speak about Auto scaling, kubernetes workloads based on metrics now, not just metrics, but I'll tell you all how you can scale your workloads based on unified observability data. So this is going to be a beginner to intermediate level. Talk I'll cover, first, the beginner level topics and then I'll move on to the intermediate things.

B

So, a little bit about myself. uh My name is rakshit gondwal I'm, a Google summer of code mentee for cncf organization working under the captain project. I am also a team. Member of we make devs community and I love contributing to open source and communities. I've contributed to a lot of CNC projects, and you can find me on Twitter at rakshit gondwal you can. If you want, you can have a look at my GitHub or you can just scan the QR code for all the links cool.

B

So the main problems that we are going to tackle today are the first. The need to manually scale in this I'll tell you about how you can Auto scale your workloads and what are the challenges you face while scaling your workloads manually, then we'll talk about observability tools sprawl.

B

This would make a much more sense once we talk about HPA, the so I'll leave this for later cool. So let me start by a scenario so suppose I am working in a company and I have a website take this container as a website. So now I have to go on a vacation this week now my manager told me that this week we are having a festival, so there would be a lot of traffic on our website, so a lot of users might be visiting our website this week.

B

So can you please take care of that, so my manager told me that, and what I do is I just you know scale my deployments, uh a website that is deployed on kubernetes so that it can handle the traffic on its own. Now my manager told me that we have released a new feature this week and there might be a lot more traffic than expected. So can you also take care of that?

B

So what I do is I manually scale my website to a lot more deployments now at the weekend, when I'm at the vacation enjoying what can there can be two scenarios, uh one three scenarios, actually one everything goes well.

B

The traffic is very well settled, but what happens if you know there is a law uh extra traffic than we expected, so there might be the website crash or things might not work properly or everything would happen, but the website working properly or there are less number of users uh visiting the website, but yeah that would lead to overcoast of your company now.

B

uh This is what I just spoke about manually scaling leads to few resources when there is a lot of traffic, so it can lead to latency outage outs, outages, then bad user experience or if there are too many resources it can lead to. You know all those resources are based, and you know a lot of Bill is coming for your company because you are, you have deployed a lot of deployments on your cluster now. This is where Auto scaling comes into the picture.

B

Now what Auto scaling do for you is you know when uh you don't need to manually scale, your workloads, you can just Define a metric or a scenario like Suppose. There are 30 users coming in, so I want the auto scaler to Auto scale itself on Fiverr on five deployments or ten deployments, or if five users are coming in I want you to scale down to one deployment or five deployment.

B

So in kubernetes there are several several Auto scalars available. The first one is cluster Auto scalar vertical power, Auto scalar and horizontal powered Auto scalar. Now HPA is the one that I am going to be more focused on, because I think it is the most powerful one. So I'll just go quickly. Go through uh the all types of cluster Auto scalers uh cluster autoscaler. It basically scales the number of nodes like if in a node your pods are not working, so it will simply scale the it will.

B

It will simply create a new node with that number of pods.

B

This might take a lot of time, because what usually happens is your cluster is deployed on a cloud, so it is dependent how much time the cloud vendor would take to create a new node. Actually so yeah and a lot of resources would be unused because some pods might not be working in the cluster in the node, so that would still exist and it's very less efficient. Actually now comes the vpa or the vertical powered Auto scaler.

B

Now let me explain like this suppose this is a pod now I want when 30 users 30 users are coming in, the vpa would scale it to this.

B

So, like the size increases of the Pod uh in, as you can see in the example before scaling there were, there were two CPUs and 2GB of memory and after VP and after the odd scaling was done, the it there are four CPUs and 4 GB of memory, and obviously you can set limits in the vpa that I only want you to scale till 20 GB of RAM and that all now comes the HPA now HPA, according to me, is the most powerful Auto scaler out there, and it is the more most efficient now.

B

What it does is like this is a pod I'll tell it to when 30 users are coming in I'll tell it to you know scale what horizontally like it will create a lot more replicas of your deployment, one two three, so it scales horizontally and the key part or the main point why HPA is useful, is it can scale on the based off custom metrics?

B

Now your custom metrics can come from various observability tools, such as Prometheus dynatories data dog Etc, and you can simply Define your auto scaling policies that when this metric hits this point you need to scale up and when you need to scale down and yeah a lot of flexibility for metric selection. This totally depends on the observability tool that you are using like some metrics are provided by a specific observability tool like data data dog provides a different, metrics and Prometheus provides different, metrics uh yeah. This is just a table.

B

I've created, you can take a screen photo of it. If you want for future reference, then this was just uh I thought this would be relevant, but here's a deployment and if we apply vpa it scales vertically and if you apply HPA, there are a lot more. You know replicas of that house, just a basic example now uh I think I've. Given you a little bit of context, what Auto scaling is and what HPA does now comes the main part of observability tools sprawl.

B

So, uh as I told you HPA uh scrapes metrics from observability providers like Prometheus dynatories datadog. Now, when you are working in organization, there can be a lot of tools which you are using to scrape metrics from, like you might be using then.

B

Is also Auto scaling tool it scales horizontally to so things can go. Often you know difficult to manage all these Integrations these plugins. So we need a standard to you know, simplify all this stuff so that it can get easier for us to maintain these Integrations and these plugins.

B

So the standard I want to introduce you all to is the Captain Matrix operator. Now what Captain Matrix operator does for you is. It will scrape the metrics for you from the provider and then it will. You know just expose these metrics via the kubernetes, API or Prometheus, and then tools like HPA cada can scrape them from the API or Prometheus now yeah. What you need to do is just Define a matrix provider.

B

It is a crd, Krypton, Matrix provider, access crd. You need to Define, as you can see, the first one is a captain metric provider. We have defined Prometheus and the target server that Prometheus is deployed on a simple namespace. Namespace should be the one where your application is running and then crd for your metric that you want to extract from the provider.

B

The namespace should also be the one where our application is running, then the name of the provider above same then a query that you want the main metric that you want to extract from the opposite value tool, a simple fetch interval seconds for you know. After how many times you want to query the tool for the latest metric now ah I think uh all are there's a lot. I just spoke a lot of theory, so we'll move to the demo.

B

Now now what I'll show you in a demo there will be a kubernetes cluster I'll have application deployed on the cluster, then I'll uh I'll have a captain. I'll have the captain Matrix operator, deployed on the cluster too, then we'll apply the provider which will be fetching which will be connected to the Prometheus, and a captain Matrix crd, that will be getting spec from uh from Metra operator, will be getting spec from the captain metric crd and we'll be writing the results into the its status.

B

If you know about you know operators, then there are two Fields, you know, crd one is spec and Status spec is the thing that we Define and status is what we get from. You know the results get written in the status of the crd, then we'll simply query the operator using a HPA and we'll see the application Auto scaled cool, so yeah.

B

Let me first show you what we have in the deployment.

B

So here's the basic application that I have it's deployed in Potato, Cube, CTL namespace, then all the basic stuff, the resources limits and I have a service deployed to uh will deploy this.

B

Yeah so I have it already deployed so that there are no, you know, difficulties for the demo, it's unchanged because I had already deployed it earlier now,.

C

C

B

You can see the Pod is running, the deployment is running and the service is running. This is default, configuration that we are providing with the basic deployment.aml.

B

Now we can see what our metrics provider look like as I just showed you in the previous slide type Prometheus Target server I have deployed Prometheus into my monitoring namespace to monitor my application, so you'll need to have a you know, observability tool connected to your application. Now. I'll also show you the metric.

B

This is a captain metric the same as I showed you in the slide, a query fetch interval seconds and the name of the provider that we have already deployed as a crd, so I'll apply this both ah sorry.

B

uh Yeah, it was previous previously deployed to it's just unchanged and.

B

Then my bad yeah, it is unchanged too so now we can get the value of our metric now, in usual scenarios. What you might do is, you know, go go to the provider and see the what the value of your metering is coming out, but you can simply see the metric using one command in the using the captain. Matrix operator, cubes utl get Captain metric and.

B

See the metric that I am getting from Prometheus is 2.84. Now you can scale your workloads based on this metric 2.84. You can say if the value is above 1, you I need you to scale up if the value is below 1 I need you to scale down now. I have already a HP file ready.

B

uh You can see the minimum number of replicas is one, and the maximum replicas is three. That means I. Want you to scale up till three and the minimum I want you to scale down is one. Then we have just defined the metric metric that we just deployed and the value at the last one. It means that the Matrix operator will is configured in such a way that HPA will scale up if the value goes above from 1 and it will scale down if the value goes below 1. so apply. F,.

C

A

B

Yep the HPA was applied so, as we saw earlier, there were only one pod. Now we can get all n.

B

You can see the HPA has been applied. Minimum number of PODS One max number of PODS three, it is still in work, I mean the HP is still uh or scaling this yeah you can see the Pod has been started. Creating there were three uh I gave the maximum replicas to be three and the value was should be above one. The target was one and the value coming out is 2.84. Now it has automatically scaled the deployment, uh the pods to three, and it should be ready now, I think yeah. The Pod is running now.

B

So that was how you can you know a basic demo of how you can utilize the captain Matrix operator now I just used a single observable tool that was Prometheus, but you can Define any number of metric providers like Prometheus dynators data dog, as number of as many as you want, and just you know, you won't have to keep track of all these plugins and Integrations. You just need one file for all your captain metric or all your metrics and observability provider.

B

uh Now that we have seen the demo, let's move on to how this Matrix operator is working, uh The Matrix operator is basically divided into two parts: the controller and the adapter yeah, there's just a very basic thing that these two do controller is used to fetch, metrics and adapter is used to uh expose the metrics so that the exposed Matrix can be used by tools like HPA or cada, and the controller fetches metrics from your observable tool.

B

Yep the same thing in the definition. Now there can be endless possibilities of this Krypton Matrix operator. You don't just need to you, know, use this for scaling or workloads, but you can use these metrics for Argo CD or your.

C

B

Deployment can I get a raise of hands. How many of you are using github's here, Arbor City here OK, so you all might be using Argo rollouts too. If you are then Argo rollouts, you know make the canary deployments Based on data if it is right or wrong, so you can use the captain Matrix operator for there too and yeah. That was all for the. For my talk, any questions.

B

A

B

I just showed you that we can configure the fetch interval seconds so there you can at the last. We have fetch interval seconds. You can Define after you know what, after these are the fetch Intel seconds in this. The operator is constantly querying the provider in between 10 seconds, so you can use this as one second two. Second, three. Second, as per your use case,.

B

A

B

Thank you, no I saw one hand there yeah. Can we get the mic over there? I am not actually able to listen.

D

Hi, um so if I I just have two questions, if to.

B

Get a better understanding am.

D

I audible, oh yeah yeah, so if I understand correct, this is uh scaling the pods horizontally right, okay, so um in HPA, by default we can also Scale based on Metric server right. So is it similar to like kind of replacing metric server? The captain metrics.

B

Not really uh the matrix, it's an operator that comes in between the your observability provider and the HPA, so the provider is a the operator is a separate thing. It will just fetch Matrix and expose its expose it via the kubernetes, API or promises. So, as you can see most of the tools they can fetch metrics for using the Prometheus format like the metrics, which are exposed by Prometheus, are a different format or uh so yeah. That's why we have mainly focused on kubernetes API and the Prometheus, so.

D

Basically, what this additionally provides is scraping the metrics from of various.

A

Observable tools.

B

Compared to Matrix yeah, so that's something additional.

D

Being provided by Captain.

B

Yeah so that you don't have to you know, take care of all those plugins, because you'll have to every Pro different provider has a different plugin for different tool. So you don't really need to take care of that. Just deploy the operator and things will work for you. Okay,.

D

And another one, um so this, as you said, or only horizontally scales up already, it doesn't add nodes right. It.

B

D

Also I mean we have seen certain issues in adding more pots, because if you are running our workloads on eks cluster right, the cluster or the ec2 instances limits the number of PODS yeah. uh So how do we, you know cater to that I mean if I give like my maximum cost to be 100, say Okay.

A

D

Ec2 types doesn't help in you know running more than certain number of ports there's a limit to it. So.

B

In ultimately, we'll.

D

Have to add more nodes.

B

That's the issue with the cloud provider that you will be using: okay.

D

I mean yeah you'll.

B

Have to you know, choose that you will have to take care of that to her on your own I. Think, because you'll have to see how many or you have to see your billing plan, how many the number of PODS your cloud provider is actually giving you and then uh use I.

D

Mean we can increase the node to increase the power, but then, ultimately that will cost right. We.

A

Cannot keep on running the same entire higher configuration yeah.

D

But right now I mean Captain doesn't uh help in in scaling the nodes right. The worker nodes.

B

No, it will only help you increasing the pods. Okay thanks. Thank you uh any more questions cool then yeah. That was all thank you for being such a great audience, and you can meet me around here. I would be happy to have a chat with you if you want to I'm actually working on this operator in my gstop project too I'm implementing new features for this operator, so yeah. If you would like to know more just you can have a chat with me and I'd love to have a chat with you. Thank you.

A

B

A

Rakshit for this wonderful workshop and congratulations to you while this was your first talk.