Cloud Native Computing Foundation Kubernetes Community Days Bangalore 2022, 11 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Understanding the Kubernetes Descheduler

Description

While all of us have used kubernetes scheduler, but very few of us must have heard of Descheduler. The descheduler is a kubernetes-sigs project which works as an add-on to an existing Kubernetes cluster and can help in overcoming the scheduling problems. The descheduler can be added to cluster in the form of a Job, Deployment or as a Helm chart.

A

Hi I'm pravar agrawal and I welcome you all to kubernetes community days bengaluru. I am very excited to be a part of this event. As a first time speaker and today I will be introducing you all to an interesting tool called be scheduling a little bit about me.

A

I work as a devops engineer at ibm, isl for ibm cooperative service and those are my different handles to reach out to me on twitter, github and slack for today's agenda. We'll see what was g-scheduler, how it works and its different components.

A

Why do you need it in your project and a little demo to show you around.

A

If you have been following kubernetes, then you surely would have heard about a core control plane component called kubernetes scheduler, while the scheduler is responsible for assigning nodes to the points when it is created, it doesn't efficiently manages the pawns based on node resources.

A

Let's take an example.

A

If you see this diagram, we have a multi-node cluster in which few of the nodes are under utilized may be less than 30 percent of its cpu and memory resources, but some of the nodes have high usage of its resources.

A

The scheduler does not automatically distributes the pod from over utilized nodes to under utilized nodes, but with the help of d scheduler, we will be able to evict some of the pods from high resource utilized nodes and place those onto underutilized nodes.

A

D scheduler is a kubernetes 6 project maintained under six scheduling. It's an open source project written in book, which runs as a kubernetes add-on.

A

It can be deployed inside a k-test cluster as a deployment gear test, job cron job or with the help of a help. Chart d scheduler also supports a cli which can be used to interact with the descheduler server running inside the cluster.

A

Its main objective is to efficiently manage the node resources by evicting the pods from over utilized nodes and moving those to underutilized nodes, and it's able to do all of this with the help of custom policies or these scheduler policies.

A

The port evictions, based on different strategies which are part of these policies, may include different strategies like high node utilization, low node utilization, pod age, failed, pod, count, etc.

A

And here's a layout of some of the main components involved in d scheduler.

A

We have a d scheduler server which runs inside a port under the cube system namespace, and we also have a d scheduler policy config map, which has the different policy that's going to be used for pod evictions.

A

Also, the server does all of its work with the help of different informers like pod, node, namespace and priority class informer.

A

These informers watch the different resources like nodes pods for resource utilization, and they also send the information back to the descheduler server to take actions based on the policy defined and current state of the cluster, and here we also have a d scheduler policy.

A

If you can see it has strategies, definitions which will include different strategy like no low node utilization, high inode utilization, pod, maximum age etc, and we need to pass some parameters like source thresholds and target thresholds which define the over utilized nodes and under utilized nodes on which we want to shift our pod after eviction.

A

The source threshold will reflect the node on which the pod will be placed after it's evicted from a highly utilized node, and the target threshold is the node once which is having the high resource utilization.

A

And now for the demo, we see how the d scheduler works, and for this I will be using a policy which is called as powered lifetime policy policy.

A

It will basically evict pods based on the maximum power lifetime uh in seconds.

A

So if we have some pods running in our cluster, which are not being cleaned up from past 30 days or 20 days or maybe over a month or let's say even a higher number of time, then we can use this policy to clean those pawns.

A

I have a kts cluster in my local and in the interest of the time I already have d scheduler running inside it.

A

I will show you the current logs of the ds, cadillac server.

A

As you can see, currently the total number of elected pods is zero, but it is still scanning all the three nodes just to find. If any of the policies, any of the criteria is being matched with respect to the policy defined.

A

We also have a config map.

A

And currently, as you can see, the strategy which is defined in the conflict map is for the pod lifetime and it takes in a parameter for maximum power lifetime. Second, where you need to put in the value in seconds, so we just replace this random value and we'll put something like 600 seconds so that all the pods, which have a time of above 10 minutes will be deleted in our cluster, will be evicted in that cluster.

A

So currently, all the parts that we have we do have few pods, which are created 31 minutes back, so we'll try to evict these pawns.

A

We'll restart the d scheduler part so that the new policy comes into effect.

A

And we'll monitor the new part for any changes.

A

Okay, so, as you can see, it has already started evicting pods, and here we can see that it has effected about nine points and the reason listed is effective part because it's exceeded its lifetime and it also specifies the pod name and the node from which it has been expected.

A

And if you quickly, search through all the pods in our cluster.

A

You will see that some of the pods have been recently restarted.

A

So one thing to note here is: it does not restart any of the cube system, part or the core or the essential part in our cluster, but we can specify a parameter for it. While we are bootstrapping our d scheduler server.

A

So we can also select any of the policy which is mentioned under the documentation policies like high node utilization, low utilization, nodes.

A

For the references we can follow, this project link the project is hosted under kubernetes on github there is a user friendly guide to follow under the docs section, which you can always use to get started. We also have few implementation examples which will be easier to read through. While you are trying to implement different strategies, then you can always reach out on six scheduling channel for any further queries under kubernetes slack.

A

I hope you would have liked this presentation. Thank you all for joining in.

A