Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2020 - Virtual, 27 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Sponsor Demo: Datadog - Autoscaling Workloads in Kubernetes

Description

Containers and orchestrators — like Kubernetes — promise greater resource efficiency and simplified deployments. Developers provide code in the form of a container, define their required resources for operation, and the orchestrator does the rest. Sounds easy, but if your workloads are dynamic in nature, how can you ensure sufficient resources to meet the performance and availability requirements of your customers?
In this demo video we will use a demo e-commerce application to showcase how to autoscale your application workloads on Kubernetes, including vertical autoscaling, horizontal autoscaling with external metrics server and she will introduce the Watermark Pod Autoscaler, an open source project created by Datadog that extends the Horizontal Pod Autoscaler with new features.

A

Hello, welcome to this demo about auto scaling workloads in kubernetes. My name is sarah pulido, I'm a technical evangelist., I'm also certified kubernetes administrator, and this is my twitter handler my mail address in case you want to to reach out afterwards.

A

Just so, you know that a dog is a monitoring analytics platform that helps companies, improve observability of the infrastructure and applications, including, of course, kubernetes as a platform, but also the applications that you run on top of the workloads that you run on top, and this is what we are going to be talking about today. The workloads and workflows scaling. So why do we need to scale a workload? So when we scale up usually to cope with demand, we have several.

A

Suddenly we have new requests per second, we want to cope with that demand, but we also want to go back and scale down basically to save cloud resources to save money.

A

One would like to do that automatically with auto scaling technologies, basically to be able to react faster to changes. So suddenly there is a spike and you don't have to wake up in the middle of the night. In order to scale your infrastructure, so there are two ways in which we can scale our workloads in kubernetes, vertical and horizontal and vertical basically means increasing the size of our pots, and we are going to explain a little bit later.

A

What that means and horizontally, which basically means increasing the number of replicas that we have for a particular part.

A

So we are going to be using this workload for our demo. This is our application. It's a sample application, e-commerce application and most of the logic happens on the store front end. This is a huge monolithic application that has been split up into a couple of services, but most of the logic is still happening here.

A

Let's start with vertical part scaling in kubernetes.

A

So when we define our pots and the containers are running in our pots, we usually have to describe the resources request for that part, and we use mili course for cpu and me bytes for memory, and when we do that, we make that definition of resources, requests and limits for the request.

A

We establish a contract with the scheduler and basically means that the scalar is to make sure it's going to make sure that the node that your port is going to be scheduled on has the capacity for that for those requests and those are going to be just for you.

A

So what happens if we request less than we need so, for example, we requested 500 millicore and actually we're using 700 milligram most of the time. So when the scatterer needs a space for some other parts, probably your part is going to be rescheduled because it's using more than it actually requested in the first place.

A

But what about? If we do the opposite? What about? If we use more than we need, so we request 500 milligram, and actually we only need 17 millicor.

A

That means that a lot of cloud resources are going to be underutilized because again, this is a contract with a scheduler and therefore there are going to be those resources for you, even if you don't use them.

A

So the question here is: how much should I request then, and a good way to answer that question is by monitoring your system. If you have your applications all well monitored in a monitoring system, then you can find things like the number of memory and number of cpu uh to put an example. For example, here we are using the cube, dns pod deployment, and we can see for every container that we have the amount of cpu that we're using over time the amount of memory that we are using over time.

A

So by monitoring that we are able to know exactly, but there are also automated ways to do that and the vertical part of the scaler tries to solve with us the vpa or vertical world. Scaler is a component in kubernetes that basically makes recommendations for resources based on usage data. So, for example, if you have requested 500 milligram, 500 megabytes, the vpa is, is going to start gathering data and it's going to tell you things like.

A

Actually, you should be fine with 200, millicour and 50 megabytes, because that's what your pot is actually using and the vertical part of the skillet can work in three different ways in recommendation modes are no changes on changes when it reschedules uh the skeletal risk schedules for other reasons or automated, in which it actually reschedule your pot.

A

So let's see the first one, the recommendation mode. We have this front-end uh service and it's requesting 100 milligram 100 millibytes. Let's see if that is enough. Remember that our application is a monolith. Application prior needs a need more, so we are going to deploy the vpa and once we have deployed this controller and the crds that are related, then we have. We can create an object called vertical portal, scalar uh with the target of our deployment front-end and the update policy off.

A

So we are going to create that object and after a while that I've speeded up here in this demo, you get the results and you can see the cpu. Actually we need less, but actually we need a lot more memory, so probably time to change our pot definition.

A

What about horizontal scaling? So again, horizontal skilling, as we said, is the number of replicas that we have for a pot and increasing or decreasing that number of replicas. We can do that manually using the decorative way in kubernetes telling the kubernetes api what we want in results, for example, let's say I say I want three replicas. I already have two, so the controller is going to check that there are already two replicas and it's going to tell the scheduler to scale a third one, so the famous reconciliation loop in kubernetes.

A

It wouldn't be great to do this based on data, and that is exactly what the horizontal portal scaler tries to solve. I can do things like how many replicas do I need based on resource usage, and I can create an object, for example, that says as soon as my pod goes above 50 of the cpu that I requested originally please scale the deployment, but that doesn't look like the best metric to drive our scaling events.

A

We want to track our applications, we use something like the red framework or the four golden signals or any other framework out there. The red framework says that the most important metrics are requests per second or per millisecond in your application errors, the number of errors that your users are getting and latency or the time of time the amount of time that it takes for your service to do some work.

A

This seems to be metrics are a lot more useful for our users that users care about, and this is exactly what the hpa with external metric server or custom metrics tries to solve how many replicas of a pod do. I need based basically on any metric value, and the external metric server is a specification that any metric provider can implement and develdor actually implements it implements it through an agent called the data cluster agent.

A

So it gathers the metric from any metric that we have in data dog and then expose that metric uh as a sternum metric server, that the hpa controller can understand and can drive those scaling. Events. Thanks to that, with that, we can do a lot things that are a lot more useful.

A

Like give me uh drive my events, my scaling events based on latency like in this case- and this is the example that we are going to see in this case- we don't need to scale, so we have the latency of our service in seven something seconds. We are going to check check how many metrics do we have, and if we have anyone that it's uh interesting for this particular problem.

A

So we have this 99p, which is the p99 latency for our service. This is the one that we are going to be using thanks to custom, metrics and external metric server.

A

So the first thing that we are going to do is to create an object that basically says: okay with a minimum of one replica maximum three targeting the deployment front-end track this metric, the p99 latency and drive those events if it goes above seven seconds so once we apply this definition, we create the object.

A

Then we can see that the cluster agent in datadog is actually going to get that metric and we can see what value is already getting by basically asking the datadog agent the status of the cluster engine, and we can see that it's already above our limits. It's already 10 seconds. So probably we have started to see some scaling events but, let's also add more fake traffic to even scale it more.

A

So after a while, we will see all these spots being created and we can see that in this case it went up to three replicas automatically.

A

But the latency went down to three seconds, which basically was the goal, and if we describe the hp object itself, you can see all those scaling. Events that happen, because the metric went above the level that we wanted to be.

A

But uh again, this was a little bit too radical. We only have one value for a metric. It was seven seconds as soon as it goes above seven seconds. We will start those scaling events and as soon as it goes below seven seconds again, it will stop the skill event or even will go down the scaling events, so it seems like, if you're, using a metric that changes a lot of values um all the time you're going to have many scaling, events that you didn't expect and those scaling events are not for free.

A

Basically, because you need cpu to do that for that daily doc has created this open source project called watermark port scalar, which is completely open source that we want to contribute back to kubernetes that tries to solve exactly this so basically extends the hpa.

A

So, instead of just having one value, it has two values: a high water mark and a low water mark, and in between those two, there are not going to be any scaling events. It also has some other nice features like scaling velocity of the number of replicas that I want to scale at a time. Also cool down periods or the same the number of seconds that I want to wait in between scaling events- let's see our final demo um with this one.

A

So we have here our resource definition and we are going to basically do the same thing as a vpa. We are going to apply, deploy the controller, the wpa controller in this case, once we do that we have that controller up and running and we can create um an wpa object or watermark folder the scalar object. We are telling one to five replicas with a high water mark of 11, a low water mark of four.

A

So let's start doing things like applying that object. Once we apply that object, we can see that the object has been created. We don't have a value, yet it may take a couple of seconds to get that value.

A

So, after a while, we are going to get the value back from datadog and it's going to say: okay, the legendary right now is 10.5, so it's below our target.

A

So, let's apply some fake traffic once we apply that uh fake traffic afterwards, what we're going to be seeing is those scaling events right now we only have one replica, but once we apply that traffic probably is going to go up. So let's do that.

A

We have applied that traffic. We wait a little bit and after some time we are going to start seeing those replicas being created, because the latency went up.

A

Okay, some takeaways from these demos vpa can help you maximize your host utilization without getting a lot of risk at all. Pods also hpa can automatically horizontally scale your pods based on resource adopted data, but a lot more interesting is to use external metric server like datadog to get to drive those scaling. Events based on any metric and finally, the wpa or watermark portal together, which is an open source project by datadog, extends the hpa to allow for more fine-tuning your scaling events.

A

If you want to see this in action- and you want to talk to any of the representatives- please visit our booth- we will be able to help there. Thank you very much.