Cloud Native Computing Foundation Kubernetes Forum Delhi 2020, 27 Feb 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How Scheduling in Kubernetes Works - Himani Agrawal & Mahendra Kariya, GoJek

Description

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects - Learn more at https://kubecon.io

How Scheduling in Kubernetes Works - Himani Agrawal & Mahendra Kariya, GoJek

A Kubernetes cluster runs pods on multiple machines. These pods may run a variety of workloads. The kind of apps run on these pods are also quite varied. It could be as simple as a webapp, or a complex machine learning model. It is super important that the pods for such workloads run on the worker nodes best suited for it. This job is taken care of by the scheduler. The scheduler takes into account individual and collective resource requirements, quality of service requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference, deadlines, and so on. A scheduler significantly impacts availability, performance, and capacity of the cluster. In this talk we see into the internals of a typical Kubernetes scheduler and features of kube-scheduler.

https://sched.co/YWJQ

A

Hey good afternoon Delhi apologize about the delay. There were some technical issues, as it always happens in conferences. So we are here to talk about how a curated, scheduler works and I am Mahindra. She is my colleague Himani, both of us work for guzik. Most of you would have heard about go-jek, but just a quick intro. We are one of the first deca corns of Indonesia and we are into ride hailing payments, food delivery and a whole bunch of utilitarian services.

A

So, rather than talking about go-jek, let's actually dig into how a Cuban and a scheduler actually works. So I'll basically be covering these five topics. At a high level, we will talk about scheduling in Cuba notice. It's use cases how it works, will give a short demo and will briefly cover what is new in scheduler.

A

So scheduling is something that's not new to the computer science community, like probably most of us, would have heard about scheduling the first time we studied computer science about the CPU scheduling in Cuba, and is it's something similar, although a bit different? So when it comes to Cuban, it is scheduling, is basically figuring out how a pod is matched to a node. For that thing, the cubelet can actually start the pod and run it on the node. So that is what we'll be focusing on for this half an hour.

A

Just high-level use case is about what we can achieve through the scheduler. So one thing is that you can ensure that the pods get the required resources. So, for example, there are certain ports, they should have memory requirements or it might need a certain volume to be mounted or you know you might have some machine learning workloads which needs GPU, for example. So you can specify all these things through scheduler. You can prioritize workloads so say, for example, your to services, a and B and your service a love, a higher priority. Now what happens?

A

If your cluster is running out of resources, the parts of service B will automatically get terminated, some of them and the pods for service a will get started, or it would auto scale based on your settings. So you can do workload prioritization. You can configure geographical distance again. Let's go back to service a and B now say, for example, these two services that talk very often and if services deployed in say us zone service B, is somewhere in you.

A

The round-trip latency would be very high, so you might want service a and B both deployed in a particular zone which are clique geographically nearer. You can do that. You can resume nodes for specific pods. So again, coming back to the machine learning example like you want only machine learning workload to run on GPU enabled nodes, so you can do all these things through scheduler and much more.

A

So what exactly does the scheduler need to run so number one? It has to know whenever a new quad has come up. Okay. Next, it has to figure out that okay, there are so many nodes out of all these nodes, which one is the best node on which my pod can run. It has to figure that thing out and finally, once it has figured it out, it has to assign the pod to the node. So we'll go into each of these things like how these things work, so starting with number 1 cube. Iratus.

A

Sorry, scheduler is nothing but a cube or a disk controller. So in the background the scheduler is constantly running and it is listening to this event. Well, pod creation. So every time a pod is created, the scheduler gets notified that hey there is a new pod, and that is all she did or gets to know that. Ok, this is something that I need to schedule on a node.

A

The next thing it has to figure out, which is the right node. So this is the two-phase process. Part one is filtering and part two is scoring. So what happens in filtering is that your pods that have certain requirements about beat memory need CPU volumes and whatnot, so the scheduler has to analyze each nodes, whether it meets the requirements or not. If it doesn't meet the requirement, the node is filtered out. If it meets the requirement it is filtered in.

A

So this is a very simple cluster that you see on the screen and one single geographical zone. There are six nodes and whenever it touch to schedule a pod that would loop through each of these things in a linear fashion. So this is the pod manifest. A new pod has been created.

A

Okay, so we look at pod number one sorry node number one if it meets the requirements, great, it's filtered in the ROI, that's filtered out now, in most cases like in most production workloads, your cluster would be across multiple geographical zones and not in the same zone.

A

So for the sake of example, let's say there are two zones. There is Winterfell in the north King's Landing somewhere in the center, and in this setting we have six nodes in Winterfell four nodes in King's Landing, so the scheduler it tries to balance out between these geographical zones. When it's doing the filtering the. So it would go one by one. First, it would look at Winterfell, one node in Winterfell, one node in King's Landing, one node in Winterfell, King's, Landing, and so on. So for a particular pod, it will look at node number one.

A

Does it match the criteria or the fit? Predicates matching yes filter in it. Moves on to load number, seven in King's Landing or the fit predicates matching no move on to node number. Two matching! Yes, move on to node number: eight matching! Yes, node number three! So it goes on and on like this. But how long will it go on like this so say?

A

For example, if you have a large cluster, if it's a thousand node cluster every time a pod has to be scheduled, the scheduler can actually go through all the nodes, but then that would be time-consuming. So there is a setting and kubera the scheduler called percentage of nodes to score now by default. This is set as 40 percent, so in our 10 node cluster.

A

That will look at four nodes and once those four nodes are filtered in it would stop the process and it would figure out which of these four nodes is the best fit for scheduling. So in our case, we have four nodes, one two three and eight the process of analyzing. The nodes will stop here. The next part is scoring.

A

So there are four nodes: all these four nodes can match. The criteria part can be scheduled on any one of these, but we have to shortlist any one particular node. So this is where the scoring comes in. It looks at a bunch of different functions and it actually score so over here you can see like node.

A

One here is the least code node node eight is the highest code node, so the pod will be scheduled on node, eight and finally, now that the scheduler has decided that okay, my new part, is supposed to go to node a it has to assign it for that.

A

That creates something called binding object, and if you look at this flow diagram, you can see that once the pod is created, the create part event Keller watches it figures out that okay, this node is there on which I have to schedule, so it creates a binding on the API server and the cubelet is actually watching. This create bind event and as soon as that event is triggered, could actually go. The cubelet would go and create the pod on the right node.

A

So, let's take off. Let's take a look at the let's take a look at few features of scheduler so over to him money.

B

So, for the sake of.

B

So, for the sake of the demo, we have created our kubernetes cluster with two notes, and the version of this cluster is version 1.16, as you can see here, and it has two notes- one named Lannister and star. So we are using her Game of Thrones analogy and going by that the nodes represent the of Thrones house and the characters were renting the pods so going forward.

B

So here first, we have a deployment called a permanent has name metadata and the match label is character. Nick the replica of this deployment would be one. Similarly, we have one more deployment called or Cersei, which has match label character, Cersei and replica one now, let's apply both of these deployment manifests and see how the pods are being deployed.

B

So here, net deployment and surfy deployment are applied on this cluster and we can see that Cersei port is currently deploy scheduled on stock node, whereas the Ned pod is currently scheduled on the Lannister node clearly saw she belongs to the Lannister house. So we would like to change this. Let's see how we can achieve this. So, in order to achieve this, what I'm going to do is I'm going to label our nodes currently, so the label that I'm adding is house and the value would be as per the house value.

B

So here the start node is how is equal to star and the label for Lannister node would be house equal to Lannister. Now, let's see how we can utilize this and enforce that Cersei pods are scheduled only on null standards, so I'm using here something called node selector and the value key value pair that we are using here is House Lannister. So whenever the schedule gets, this pot the job to schedule the spot we'll take what all nodes are present with satisfy this House Lannister and accordingly it will schedule the pods.

B

So let's apply this and see where the Cersei pods are being deployed. So here we can see the first quad of Cersei, which was deployed on stock, is currently dominating, and the second one which is 11 seconds ago is scheduled on Lannister node. So using no selector we can have a hard requirement that this requirement needs to be fulfilled. Only then the pod would be scheduled now, let's see if I create a new deployment called John where I want this node, where I want this deployment John to be deployed on House Targaryen.

B

Currently we have only two worker nodes and cluster, which is one is labeled as Stark load and the another one is labeled, as the Lannister would so on applying this John deployment. Let's see where is this going to be scheduled, so we say we see her. John is currently in pending State and, let's figure out what is the reason for it to be in pending state by doing a described on the spot. So here we see at the end it says us failed scheduling.

B

Zero out of two nodes are available, but any of the node selector didn't match. So no selector is a hard requirement if the requirement is not met. This part is not scheduled.

B

Let's see if we want to see a case where we want opposed to be preferably deployed on or preferably satisfy some requirements, but if those requirements are not satisfied, we would rather prefer it to be scheduled somewhere else unless, apart from being in pending state, so here, I am creating a new deployment called santa-san, so I'm creating two replicas, and here we are using speck affinity, and here we are defining pod affinity as preferred during scheduling and the match. Preference here would be key house and the value is Lannister.

B

So, let's apply this sand saw deployment and see where it gets deployed. So the Sansa port is current. One of it is deployed on the lannister node and one of it is deployed on this shark node. So it depends on the resource used usage and the other fit predicates which Mahindra talked about earlier, depending on that the Sansa node has being deployed. So this is provides a soft requirement, whereas the node selector was providing Hart requirements.

B

B

Let's, if I want a static, particular kind of workload to bid to be deployed closer to another set of how can I achieve that? So to achieve that, first I'm, adding a new node to a cluster called Targaryen node and I'm labeling. This Targaryen node will house equal to Targaryen. We can see how what all our labels it has added, so Lannister has House Lannister stark, has talked and Targaryen has House Targaryen value moving forward. Let's see how the I'm creating a new deployment now called Danny deployment.

B

How does this look like so here the Danny deploy has character, Danny value said, and the replicas would be too, and here we are saying no Definity here for Danny is required during scheduling, which is a hard requirement again, that it should belong, be scheduled on nodes which fulfill house or equal to Targaryen value. Let's deploy this apply. These changes, so we see Danny has been scheduled on targaryen notes.

B

Now we have Serge aura and we want him to be. He always prefers to be somewhere closer to Danny how we can do this so for this there is a new deployment with the replicas of three and here instead of using no definitely, we are using pod affinity, which basically means it would have a lightness towards the pod instead of the node, as we have seen earlier so here, the character values, character value. We have setters Danny.

B

Let's apply this manifest.

B

So here we are seeing that Serge aura our pods are deployed on the targaryen on which the Danny no Denny pods were scheduled. So using node, selected or node affinity and port affinity, we can control or we can specify how we want a pulse to be scheduled on which worker nodes moving forward. Let's say if we want us our nodes to decide what kind of workload they want to be scheduled on themselves. So for this we have something called tails and tolerance.

B

Here, I am creating a new tail on house targaryen, which says white, Walker equal to behind the wall and no schedule, so the key is white, Walker and no schedule means, if does not, have a toleration for this kind of pain. The pod would not be scheduled on this node, so here I'm creating a simple white Walker deployment, which has character white Walker replicas are three: let's apply this manifest, so here we see that the white Walker Falls are scheduled on lannister and stock notes.

B

It is not scheduled on the Targaryen notes, because it doesn't have a toleration for the pin we apply down here now. Let's say if I add a taint for execution by this I mean that currently we have seen how we can decide the schedule would be controlled, but now, let's say I want later on. When apart is already scheduled, we don't want it.

B

We still wanted to follow some sort of affinity or Labour's how we can do that so, for this I am creating a taint on House, Lannister and stock notes that is wai-kwok behind the wall. No execute, and let's see thing does so we see here a white Walker pause. There are three in terminating state which were earlier deployed and the rest of the three are in pending State. So now all of our three nodes have three things. One has no schedule and the other has no execute.

B

Now. Let's see the reason behind why a one of a white Walker quads is in pending state, so it says that the scheduling failed because three knows hard things that the pod didn't tolerate. Now, let's try add a toleration for one of these pods and see how it would behave so for doing that. I am adding toleration for a white. Walker exists, no schedule, which basically means even if a node has attained regarding this, the pod will tolerated and the node will allow the pod to be deployed.

B

Also, I am adding word, toleration for white Walker exist, no execute and let's apply this.

B

So we see here also I increase the number of replicas for the sake of demo to 5 her, and we say why cocoapods are scheduled at stock Lannister Targaryen notes. So this is the demo. I showcase how we can use pod affinity and pains and tolerance to decide where the we want the pods to be deployed.

B

Moving forward.

B

Okay, so what else is new in kubernetes scheduler or what else could poop and it is scheduler to do so? The kubernetes scheduler is quite extensible. We have a link here which gives more detail. Extensibility features that you can do use if you have more customizations and the extensibility features of cube scheduler are not sufficient. You could probably write your own cube scheduler and in the quad manifest we can specify what scheduler do we want to use for the pod to be scheduled coming further.

B

We have even fought spreading, so let's say we have two zones and we would want our pods to be deployed or scheduled on both the zones so that even if one zone goes down, the traffic is still serving.

B

Then we have a scheduling framework, which is an alpha state in v1, 15 and because skiddy is a very vast project in itself in coconut right now, six scheduling is working towards creating a scheduling framework which would provide more customizations for the users to decide how they want the scheduling to be done, and it also put the scheduling and the binding of bhindi covered now moving forward. We have bad scheduling. Currently, only one pod could be scheduled at a time, but using batch scheduling, multiple pods could be scheduled in one go.

B

These are the references that we used for this talk. If you are interested in knowing more about the scheduling and you can join a slack scheduling channel, you can check out the github repo and maybe join the mailing list. That's it guys. Thank you for listening to us.