Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2022, 12 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG-Scheduling Deep Dive - Wei Huang, Apple; Qingcan Wang, Alibaba; Kante Yin, Kensei Nakada

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

SIG-Scheduling Deep Dive - Wei Huang, Apple; Qingcan Wang, Alibaba; Kante Yin, DaoCloud; Kensei Nakada, Mercari

Speakers: Kensei Nakada, Wei Huang, Alex Wang, Kante Yin
SIG Scheduling is responsible for kube-scheduler and its related subprojects such as the scheduling-plugins and descheduler. These projects share the common goal of enabling users and developers to ensure that pods are assigned to the appropriate nodes based on various scheduling criteria. We also endeavor to ease customizing and extending scheduler for advanced usage. This talk will introduce attendees to SIG-Scheduling with an overview of the current goals and ongoing work within the SIG, as well as opportunities for new contributors to get involved.

A

Hello, everyone welcome to today's session and I hope. All of you have a great week into joy in cubecon and today we will talk a little bit on six scheduling, uh so it's a maintaining session, so I hope this session is more interactive and we can live enough time for you to ask questions, and basically this is a session that we will talk more on the latest update of the seek.

A

So today we were supposed to have four speakers here by Ching Chang from Alibaba, and the kante from Dark Cloud can now attend any person. So here is just me way from Apple and you.

B

Can say from Mel Curry yeah.

A

We will cover the topic all right, here's today's agenda. As usual, we will give a very short introduction on the scheduler itself and then give some details on recently recent developer updates on the last two or three releases and then, uh as a seek. We sponsored some sub projects that related with the scheduling. So we will also give some updates on the sub projects as well, and that's why we will have the Q a all right.

A

The first thing: what has scheduled anyways so schedulers as a core component of kubernetes. Just just one simple thing is that it assigns the nodes, according to the Clusters utilization and power distribution and other resources allocation and assign a better note to the part. So the part will have the best node to run as being obvious continuous. And if you look at the picture in the left, sorry, the the the clicker doesn't work.

A

So the part from the left, the top, is the input that usually created by the user or created by the department and the department manager, control manager, spin up The Path. So the path comes in and it enters the scheduling queue and the schedule queue will have some internal mechanisms to ensure the paths get started properly and fairly gets popped out and the ones the part pops out from the scheduling queue it enters the scheduling cycle. So the scheduling cycle is the core logic of how scheduler schedules apart.

A

So it basically leverage another internal component called scheduling cash, so certain cash will listen to all the objects. It's interesting like the no objects, running paths, information and as well as others, like storage, pvpcs, runtime cars, a lot of other information, so the core schedule cycle leverages that kind of information. So it has a up-to-date latest view of how the cluster looks like so it can then based on some predefined algorithm and policies like prefer, prefer one node over the other.

A

This kind of pre-config policies, then we'll try to choose a better note for the part to run on. So next you will check whether the part is scalable. If, yes, if it goes to another separate Implement implementation, wise is a separate go routine. So it goes to the bonding cycle. Binding cycle does nothing but bonded that node to the to the part. So then, the scheduling cycle ends with the part that carries the No, Name scheduler think is the best amount for for it, but it can be also.

A

uh The system is pretty occupied, so there's even no single node can accommodate the depart. So in that case the pot gets put back to the scanning queue and internally it will go through some internal predefined back off timer and then so that it can be fairly put back into the head of the scheduling, queue and get a try. So basically, this is the very high level structure of how scheduler works and what's the input and the output of the scheduler and a little more did zooming into the into the green yeah green rectangle.

A

So the scheduling cycle can be zoomed in into array simplifies a two-phase screening process: one is called filter or the other is called score. So filter is that we popped us apart. The first phase is to look at into each node in the cluster whether the node fits the pattern. Now, for example, if you're asking a super large CPU, maybe some you know the fit some, not not so in this case we can see uh node, 203 and Note 4 fits, but the other nose doesn't fit.

A

So in this case we call these three Nails as visible candidates for the powder to run on right. So after that, we were based on some predefined policies to see which now we should place the part two in this case we rank the three nails and give a final score based on predefined scoring policies then, finally, like in this case, we give the Note 3 with the highest score with 90. So the output of the screening cycle is that we come up with a Note 3 for the part to buy now.

A

So this is pretty simplified phase. But if you want to know more details, we do provide some fine-grained extension points that not only the internal scheduler is using, but the your customer scheduler can also use just basically, we are using the same thing, no hidden apis. Everything the internal Cube schedule uses is what you can use. So there's a another talk on Wednesday. There is a session, a section that I gave is detailing the all the design patterns and why we expose so many exchange points and what is usage of each same point?

A

So so, if you are interested in the details of the schedule for framework and each extension point, you can check it out.

A

Okay next section is about the recently development in the last two or three releases. First, one simplify the schedule config, so we provide schedule framework and we provide a bunch of extension points. But unfortunately, this is basically more developed prior oriented like you're implementing plugin, and you know exactly which exterior points you want to implement, but they can be Troublesome for the end user or for the Sie, because they just know okay, I want to enable node, Affinity or Pi Affinity plugin.

A

But what's the idea of why I should know which specific extension point I need enable or disable right so in the before? If you want to enable one particular plugin, you should know which exact extension point it implements in this case, like you have to plug in full and bar, and then you have to know full actually implements four extension points. Pre-Filter, filter preschool and the score you'll have two specifically specify them and yeah for the same case for bar, but it should be something more user friendly right.

A

So that is why we introduced the so-called multi-point extension point. So this point is not a single purpose. Someone is more compound. Extension point that in this case you just specify those plugin names under the the multiplying section that in this case you just tell scheduler that in the config you want to enable full End Bar. That's it so internally, under the hood. It will check, which is exactly extension points. It implements it's basic on some reflection based mechanisms to detect that and it will enable them under the hood.

A

So that is so that you as an end user or Sie. You don't need to worry about which exact extreme point it underneath implements, and it's available in kubernetes 123, if you're using the latest of V1, beta, 3, config and also in 120 3, if you're, using the V1 config and also we didn't abandon the old style configuration you can still use it, use the older scale style or use it along with the new style. So it's all tested, so you can give it a try.

A

The second thing I want to introduce is the working progress feature called part scheduler Readiness. So if you look at the left picture, is that once the power is created, the part will immediately enter the scheduling, queue and the waits for its turn to be to be working to be popped out. So there can be some cases, especially in some customer Solutions, like real associated with like a quarter track or some other in-house requirements that the product creation doesn't mean. The part is ready for scheduling.

A

So in this case the path is scheduled anyway, although we have back off timer inside scheduling, a path that is literally not ready. It's just waiting Cycles, especially if you're running a multi-tenant environment, scheduling the product that shouldn't be are not ready.

A

It's just postpone the uh scheduler of other paths so that you will impact the overall throughput of the scheduling as well impact the other path that should be scheduled earlier, so that will impact there end users SEO Etc, so one thing we are trying to improve is that during the skagging queue and the scheduling cycle, which is unconditional so right now we put a pink box, so we provide a knob in between so that uh before it's popped out and they've been working on a user has the choice to choose whether or not the part is literally ready for scheduling or not so the feature is being worked down is internally implemented by adding a power level, Fields called scheduling, Gates and then the control, the regular workflow can be the user created.

A

The part and the part that should carry some predefined gates and each Gates is associated with external controller and controller, should be responsible to uh remove that gate where it thinks it's ready for scheduling, so it can be different. You can Define more than one gate and internally there will be a default in-q plugin to checking the scheduling Gates and then, when all the gates get removed, it will be eligible to be popped up. So that is the design, but it's still working progress. So hopefully you will be available in the next release.

A

Kubernetes 126 yeah, that's it so next, a couple of enhancements is about participate. I will hand over to you can say yes.

B

It's a main domain in pot party spread. It's now graduated to Beta in 1.25, and it has a new field, mean domains to support, spread constraint and it defines the minimum number of the topology domains and their use case. We have it like, uh you, won't use for spreading over minimum number domains and if there aren't enough domains already present then make the crossout scale profit on them so and it can be only used with the Do Not schedule, which is a hard constraint in, but toward the spread.

B

Let's see the exam both so let's say we have a developed deployment with two replicas and using to participate constraint with Max q1 mean domain two 4dq, whose name do not schedule, and in this case one replica is now on the Node, but another one is pending. Now it's actually it's because of the mean domains. We set uh the number of domains currently one only one node, and it should be two, so the crossout skill notice, the pending board and create another node, so that this penny board can be scheduled.

B

Yeah there we go so the next one take 10 situations into consideration when calculating skill. So it now Alpha feature and we prompt you graduated to Beta in next release. So it has a new. It adds new fields, no change policy and node Affinity policy to support constraint and those options to take to specify whether take paint and operation or node Affinity into consideration. When calculating the skew. Let's see the example, so let's say we have a deployment.

B

With 40 spread constraint, Max q1, topological hostname do not schedule, which means how to constraint, and we have two nodes and node 2 has attained. So of course, Note 2 can't have any parts because it has a paint.

B

um So the first one and the first replica goes to node one.

B

Then the problem here is the second part that towards part constraint, wants to force, uh wants to schedule next part till node 2, because node one already has a pot, but there is a taint on node 2, so it will be pending, and this is the problem we want to solve with this proposal. This new feature, so we can solve this by specifying, as by setting node of sorry node change policy owner by setting node, node change, policy owner and scheduler takes things and Direction into consideration when calculating skews. So there we go.

B

The next one to participate for the spread again, it's a respect for participate after roaring upgrades, it's now Alpha since uh 1.25, so it adds a new field called much lowercase to for the spread constraint.

B

So the problem behind it was the scheduled.

B

A drawing roaring upgrade scheduler, doesn't schedule a text, both old replicas and newer applicants into consideration when calculating the skew. So the problem here is so yeah we have. Let's say we have a deployment with participate constraint and let's go rolling up parade and new replicas.

B

Now so the scheduling of new replicas are affected by existing older replicas, so.

B

After drawing upgrade, it may be imbalanced for the.

B

So we introduced much about much rubber case due to participate constraint by by setting both template hash to much other kids. Scheduler takes only Parts with the same same port temperature into consideration when calculating the skew. In other words, it s schedule or text only new replicas into consideration when calculating the skills. So we can solve the problem, nice and then not a lot. Nothing features and fixes so performance Improvement on demonstrate parts. So we updated the uh prefilters interface to return free future result.

B

Proper result is a tract of gorang, so for the future result contains the information about uh which nodes to evaluate in Downstream extensions, so by using this in node Affinity demo supports a schedule in each node by using another Affinity. So we by using a node I, know peripheral result in node operatives preferred we can improve uh performance on demonstrated parts.

B

So next one is a memory on preemption, and that was due to the harmful and fraud context and we refactored the scheduling of to prevent such a such a memorial week and in the future next fresh interval. So currently the schedule uh moves uh unschedurable for to activate you. Every certain minutes it was previously five minutes, but changed to no sorry.

B

So it was previously UH 60 60 seconds but changed to five minutes, because uh the reason behind it is we want to eventually remove this Brushing in the future, and this is the first step for that.

B

Point config, it's Debona and latency no Legacy schedule of policy config is removed to 1.23. So if you still use it, you should migrate to compound config.

B

Yes, that's it about the Upstream update from here. We are gonna talk about sub project updates.

B

First, one is schedule accumulator and the schedule schedule currently has the out of ways to expand or extend configure its like programming or extend the component coping of course, and this sumerator helps you check or evaluate the your schedule, your schedule's Behavior.

B

So it's a younger project, so it has only simple future right now. So, um basically, what's this simulator does? Is um it has its own Google API server in it, and you can do anything with the with that Google API server, and so you can create resources, employ resources, direct resources whatever, and so then, when you create Parts in the simulators, API server and this detailed scheduling result will be added to Ports annotation such as filtering result of each programming or scoring result of each plugin.

B

So usually we can see those information from the real clusters schedule, so it should be helpful and yeah. So there are some interesting discussion going on.

B

Simulator also had this web UI, so you can easily check the results from the reviewers. So.

B

Filtering result scoring result and the normalized score like this.

B

Next cue, it's also a young project and it's a job, queuing, controller design, team marriage, batch jobs as a single unit. So it has these core features: a color management, it controls who can use what and up to what's remit and next fair sharing of resources between tenants so to maximize the usage of resources.

B

Any unused caller assigned to inactive tenant should be allowed to should be a road to shared Theory between active tenants and the flexible placement of jobs across different resources, types based on availability, so you can have the different various resources, such as different architecture, GPU, CPU or different provision, name modes like sports on demand, so other than that we have different Turing strategy, a strict fasting faster. It's a first info and the best airport posting hostile. It best qualifies to infrastructure.

B

But uh if all all the work world cannot be admitted, it will not block other young younger newer. What growth that can be admitted.

B

So Q has these rpis and any other ongoing work is also interesting. So it's just a brief overview of queue, so, if you're interested in it in it, we have a official document, of course, and also we recently published a blog post about queue. So you can refer that as well.

A

So some more some projects, one is called this schedule. This schedule is a has been there for for a long year a couple of years. So basically the motivation is that the scheduling decision was made based on the moment of the casters resource utilization. But. As time goes, all the classes stays may change, so the optimal decision by the time in the past, maybe not that optimal anymore. So you may want to make the most of the clustering utilization you may want to change by moving some parts to other nails.

A

So this schedule is this kind of project for you to define a couple of strategies and the one some criteria doesn't meet and then it will uh kicks in to evict the past doesn't doesn't uh doesn't follow the strategies you are defining. So basically it's running as a controller Loop, and then you define a couple of strategies and then you will check the path, whether they fitted the strategy or not, if not the kicks in a little bit there.

A

So basically, this is the idea of this schedule and then, as time goes either before all the features are developed in this project. But as time goes a lot of demanding requirements, something and some of them are pretty I- think not that General, but it's still a requirement so to fit the diverse requirements. So the the projects is recently bringing up a proposal called this schedule framework, so that not necessarily every feature needs to be developer in this project.

A

Instead, you can develop on your own repo and dine just glue them together to come up with with a schedule. This schedule, I, would say, plus plus and then yeah to fit your in-house needs. So if you are interested, there is some new community meeting dedicated for this schedule. You're welcome to join me. This is for district schedule and next is a new project called cork.

A

So when we talk about scheduling or testing the other scalability, we are using both control plane and the data plane, but for some specific area like if you are just testing the functionality and the scalability of schedule, actually you don't quite need the data plan, because all the kinds of stuff are just API objects. I, don't really want need to speed up the real containers and back to better notes right. In that case, you can save a lot of cost and then even just run the simulation or evaluation tool in your laptop.

A

So that is why the Coke kubernetes without Cubase project comes in. So basically you can see it in the picture. It removes the needs of the keyblades and then it must be up a controller called quark and cork will watch the note and the part objects. So it well made some magic to let the node and the part behave. They are backed by cubelete like it. Well maintain the heartbeat from the note to the API server and also will make it a part of the transit from uh Penny.

A

If the podcast is a known name, right, Transit front pending to the running state, so basically it's just apparent to you.

A

It looks like you have the like five thousand nodes in your laptop, but it's packed by the Chrome, so the use case of Coke is that can do functionality in a very low cost and for some area like scheduling and also can do the scalability test for for scheduling and also you can integrate easily to your pipeline without speeding up the real runtime Cube rate to smear The, Continuous yeah, there can be other usage like.

A

If you want to test the work in progress across the auto scalar implementation, you don't need to right jump into the AWS to buy the machines, try to try it out, but you can just test test it. You should Coke bike the nose. So that is the clock project. Okay, the next one is scheduled plugins, so basically both inch both the default scheduler and maybe your customer scheduler- can follow the same pattern of use. The schedule framework to do customer to feed your customer requirement. So, but not every plugin can be entry.

A

So what what should you do so? This repo is the place like Marketplace for you to contribute the idea exercise, some Innovative starts and then work on specific area right now we have a lot of useful plugins, like code scheduling, elastic coder, topology, aware scheduling, Etc and some very Innovative and uh some Cutting Edge. Maybe a network aware, not aware scheduling.

A

So if you are interested, as happens to have the similar requirements, yeah you're welcome to check out all right, I think that's basically pretty much for today's session and the next session is for Q a any questions.

A

Yeah go ahead. Yeah.

C

Hi, thank you for the token, for all the work done on scheduler um I have a question about project Q. So what was the motivation to do it if there is already volcano, which is all same problem and part of cncf.

A

uh It's right now a kubernetes sick project, so basically it belongs to kubernetes uh I'm, not sure this is the goal to go to cncf and the same long or just stay there for as a kubernetes sub project, but that's I, don't think it does make a big difference. My.

C

Question less about cncf more about what was the motivation to create queue in the first place, if this problem is already solved by other projects in this case like is there any particular functionality that or any particular problem that you will be addressing? That is not addressed yet.

A

uh uh So the cube project, although I'm, not the Medina, so it's trying to move some the scheduling domain outside schedule itself, so it doesn't necessarily that everything needs to go to the scheduling core itself to be resolved. Instead, it's standing at the controller's perspective, like some external requirement related with, like quarter like sharing between multi Tendencies, can be the outside.

A

That is motivation for for Q, so just I think you maybe overlap with some cncf projects like volcano and some others, but they do want walking to slightly different direction like a queue as as this readme said, it will not cover the scheduling queue, but other projects, like volcano, is just putting everything together and you have just deployed all the things it provides and do that.

A

So it's basically a couple right, but the queue you can just hook up with any others kind of implementation, and then it's basically loose coupled it's just focused on one particular of the domain. It wants to resolve. Yeah I. Think that's that. Okay, thank you. Thank you.

D

Hey, um are you able to customize the scheduler when you're using a managed version of kubernetes like eks.

A

uh So it depends on I guess on the public offering kubernetes offerings, you don't have the control of the control plan right. So in that case, one thing I I usually suggest is that you run the customer logic as a secondary scheduler and you can use or disable the default scheduler that runs with the eks control plane so that the scheduling flow Works through your customer scheduler. So, basically, if you use the scheduling framework based implementation, you have 100 compatibility with the entry plugins part about spare part of any other things you can use.

A

One thing you may want to tweak is to point your workloads, scheduling, name to the customer schedule you define outside, so maybe a little hassling but yeah I think that's the way, because you have zero control to to modify the control plane of the ekis.

D

Gotcha, thank you.

E

Any additional questions- yes, this one yeah.

E

Sometimes you want to modify the Pod spec after the scheduling decision has been made.

A

Modify the scheduler spec, which spectacular the port spec so yeah.

E

Prospect has a lot of views which, let's say the container image.

A

Image so I think that so basically, it's not quite a scheduling problem uh questions so in the apis definition there are, some Fields can be mutated after the part gets created. If you checked the the website, I think image.

A

Imitation and some others can be mutated, but some others are just defined as immutable, so yeah I'm, not sure that is.

E

Can we like mutated off in the scheduling framework itself,.

A

That's a good question: the suggestion is not so. Basically schedule is not supposed to mutate, some particular Fields. It's just doing one simple job to find a note for it. So one not field one particular thing that scheduled test modify that the part is only the condition the statistical condition like if the bar is not schedulable, then it will append the condition to that. That is all schedule does and they're not well. Not all another scenario is that it's in terms of preemption, which injects a status nominate, no name to that.

A

To suggest we, okay, preempting some parts, so maybe in the later on, you can try that it's a hint a child has a non-median, no name so other than that. It's just the no name, because that is ending stage of scheduling power. So in your customs scheduling, I think it's not that recommended to mutate the parts back yeah. It's my suggestion.

A

Any questions, thank you, gentlemen, yeah and we'll still be here around if you want to have some more fun time yeah. Thank you. Thank you.