Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2018 (Copenhagen), 7 May 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Advanced Scheduling - Robert Carosi, Ad van der Veer & Boris Mattijssen

Description

Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).

Kubernetes Advanced Scheduling — For Heating Showers? - Robert Carosi, Spotify, Ad van der Veer & Boris Mattijssen, Nerdalize B.V. (Intermediate Skill Level)

In this talk, Ad van der Veer and and Boris Mattijssen, software engineers at Nerdalize, will talk about building custom schedulers. The goal is that after this talk attendees understand how to build their own a custom scheduler, and are familiar with the tools to do so. As an example they will use a heat-aware scheduler that was developed by Nerdalize to allow them to schedule compute jobs to servers where more heat is needed. This unique need arises from the situation that Nerdalize builds a distributed datacenter, where servers are placed into households, and double as water heaters. After someone takes a shower, their boiler needs to be re-heated. This scheduler allows us to move high-cpu workloads to those servers.

About Boris
Boris Mattijssen is Software Engineer at Nerdalize B.V. a Dutch company. He is also a masters’ student Computer science and the main author of the Custom “Heat aware” Scheduler. Previous speaking experience: Deltares Software Days - Delft

About Ad
Product designer by education, software Nerd by nature. Ad is a pro in designing and developing simple software products that solve complicated problems. Ad has worked with Docker since 2013 and, since then, touched almost every part of the container ecosystem while developing many backend applications in Go: his weapon of choice. At Nerdalize Ad does his part in developing our cloud products by combining new ideas with proven technologies and making sure end users have the best experience possible.

About Robert
Robert is a graduate intern at Spotify.
Working on his master thesis: 'Leveraging distributed tracing in service meshes for application profiling at scale.'
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Hey guys welcome I got some good news and some bad news. The bad news is that this is already the last day of Q cone.

A

It's already the last day of cube cone.

A

We got some good news also because you got at least one talk left, that's about. It fan, scheduling, yay,.

A

So this what we're going to talk about today, we're gonna, give a small introduction about the company in their lives. We're gonna, create a basic schedule and then move on to the more advanced stuff, I'm gonna wrap up at the end, to see what we've done.

A

My name is autumn, no fear I'm a project, engineer neuralyzed. This is a Dutch company, that's building a cloud and we're doing it a bit differently and that's what I'm gonna talk about you with you, but first this is what the data center used to be, not sure if you were there, but this is a building where people were just manually computing, difficult computations- and this is one person in the middle.

A

That's scheduling these jobs, so new competition came in and we need to find a worker, a controller that can then take over the job and calculate this single calculation.

A

Luckily, we've came a long way, so this is how modern data center looks like, as you can see, there are these huge buildings with huge amount of infrastructure, big computers and a lot of technology.

A

But this is not everything not about dead center is good, so these buildings are incredibly costly both to build, but also to maintain. You can imagine that all the power redundancy all the networking everything needs to be maintained in order to get the servants online.

A

Besides that there is a lot of energy that needs to be used and all of co2, that's wasted on to the environment. So that's where we decided to do things a bit differently. What their lies does is we take these computers that you would normally place to the data center and place them in people's homes and say well, that's insane. I can't possibly do that, but it turns out that it has three important upsides both for the owner of the home, the user of the clouds. That's you guys, and the environment.

A

I can tell you about that with we have small movie. That does a lot better job of it.

A

Streaming from the clouds.

B

Something about a startup company, maybe located in Delft, and they wanted to heat houses with the excess heat of computers, and from that first moment, I thought it was a brilliant idea. I thought it was a complete game-changer.

C

Nedda lies as a cloud provider that offers you´d computing power that is much more affordable and drastically lowers your IT carbon footprint. Instead of stacking, a lot of servers together in an expensive data center, we distribute cloud service over individual households in the Netherlands connected my fiber, with 10 gigabits to each house. Our cloud is great for all your high throughput computations, like video, transcoding life, science and engineering simulations by running computations on the neddle ice cloud.

C

You drastically lower co2 emissions, as the energy is used twice once for your calculation and once for heating houses. All savings taken into account your computer can save up to 3 tons of co2 per household per year. Nedda lies offers you a sustainable cloud solution with irresistible price and performance. You don't pay us for building and maintaining a conventional data center, simply because we don't have one compared to other cloud providers. This saves you up to 50% on your cloud bill. Nerd lyse,.

B

Has made it incredibly easy for us to give those options to the biologists I think it's going to be a no-brainer that biologists are going to be choosing the nerd lies option because they will have access to the cheapest compute that way, but also I, think there's a real draw from these customers in that they know that they're going to be getting their compute kind of the lowest carbon footprint possible.

C

So ask yourself: do you want to lower your IT carbon footprint? Do you also want to cut your cloud expenses by up to 50% experience, how much you can benefit from the affordable and sustainable Neda lies cloud visit. Neda lies, calm.

A

So, to recap: going data centerless moving the machines from the data center to a home has 3 important propositions, so the home owner can save up to 200 euros per year on his heating bill. The environment saves up to 2 tons of co2 per year and you, the cloud user, can save up to 40% on this cloud bill and in order to prove that we set up a communities cluster on our own clouds.

A

We've got one of our customers involved and we run an actual workloads and it turns out that indeed, if you compare this to other instances of an actual workload, it can save up to 40 to 50 percent of your cloud costs and I say: ok, that's all fine and good. Tell me about this technology how's this even possible, and we got many many challenges that we managed to solve in the past year and today, I'm just gonna talk about one of them. How can we schedule these jobs?

A

So could we need is pretty good at taking a job looking at CPU, see where some resources are available and then moving the job to the correct node. Just on CPU and RAM.

A

But we got a lot of problem right, so this yours, this is one of our home owners and currently he's not using any heat he's enjoying his 200 years of savings doesn't need any heat at all, but at the other end of the Netherlands we have Chantal Chantal is in need of some heat because you can see she's burn their hands on some super hot water, but luckily we got a cloud box there.

A

So this is a product we install this in your home and we got about 160 CPUs in there and a very decent network of about 10 gigabits per second with sub-millisecond latency, and the question is as follows: how can you make sure that Cuban interest schedules these jobs at Chantal and notes at yours? How can we instruct communities to do this, and luckily companies has scheduler so I'm not going to ask you to read this like bunch of texts, but I'm gonna break it down for you.

A

So there are three basic steps: one and the scheduler gets from the API server from the company's API server parts that are currently unscheduled, and then it's gonna find a note by asking the API servers. Gonna find a note that has the right amount of resources available and then it's gonna create a binding. So it's kind of create a binding that binds the part to the node, and the cool thing is that you can write this basic scheduling logic in about ten lines of bash.

A

For this example, we're gonna take a fixed node, so we're gonna, say every pulse gonna be scheduled on the single node. We're gonna start a infinite four loop. It was constantly going to ask the API server for parts that are currently unscheduled, so scheduled is set to no and scheduled name is setter, our custom scheduler and then the only thing we do is we send back a binding, so we'll use curled Senate by combining create a resource, and then say you need to schedule this part on this node. Of course, we're gonna create an echo.

A

We've got a printed out, so we're gonna make sure that we did our jobs as developer, and this is all you need. This is creating your own scheduler, a very basic one, but it neural eyes. We need something, a bit more advanced so for that I'm gonna invite Boris he's worked on his bachelor thesis to create a scheduler using queue. Benitez, that's a bit more advanced Boris takes the stage.

D

Thanks out, my name is Boris and together with Robert, who is going to present in a bit during our bachelor thesis. We worked on extending kubernetes, thanks to include our own custom scheduling logic for the scenario that that's me for the scenario that I just described so I'll talk you through how we did that and then Robert is going to give a live demonstration of that and let's pray that the internet connection is damn working. um So to show you how we did that I'll first need to talk you through some background.

D

So what you're looking at is a simulated little toy example with three homes in our neuralyzed network and then as all set each of these homes as a bunch of nerve life serves in them, but it also has a boiler or a water reservoir, and this can be used as a buffer to store the heat that's generated by the computers.

D

We constantly measure the temperature of this water with one of our devices, and then we report it back to kubernetes as a note annotation. So now we have some information about the temperatures in the system and we can make some call scheduling decisions about that.

D

So, when a home uses some warm water, the value of the boiling temperature decreases. So, for example, if yours here would take a shower, his temperature would drop. The same thing goes the other way around. When a pod is scheduled on a node, it will start making CPU cycles and warm up the computer warm up the boiler, and then the home gets warmed.

D

So, for example, Chantal here, if we schedule a pot there, then obviously the boiler starts to warm up and, as I said, this thing is constantly measured such that we're constantly aware of the current state of the cluster.

D

So let me explain the scheduling policy that we introduced, which is a very simple, yet very effective, scheduling policy if you want to reduce the heat that is wasted, because if the boiler reaches its maximum temperature, which is fifty to five degrees Celsius in our case, we cannot store the heat in the boiler anymore, so it should be so it will be vented outside. But if there's all the boilers in the system that is still or still in need of some energy, we can obviously schedule there and let it make use of the heat.

D

So this is our scheduling policy. What you're looking at is three bar graphs, each representing the boiler temperature of the home and then, if a new pop would be scheduled, we expected to go to Alex's home right because it has the lowest boiler temperature.

D

So this is indeed what happens and then, when a new pot comes in now, Chantal has the lowest boiler temperature so expected to go there, and this goes on and on and on so now that the border temperatures are saturated, we can still schedule our pots because our computers are still available, but, as I said now, the heat will be fended outside, so we consider this a waste.

D

So how do you do this in Cuban? Is? How can you add your own schedule? Logic? That's what I'll talk about now and then first I want to talk to you about how the normal scheduling is done in Cuban ITA's. So what you're looking at is keep CTR on the left and you're all familiar with this. Hopefully it's the client interface that allows you to talk to the kubernetes cluster.

D

Then you have the API server, which is the server interface which all components communicate through and then the scheduler is one of these components and a scheduler is now presented with the task of scheduling a pod with this in this case, requested seven CPUs and one gigabyte of RAM. So it looks at the notes that it has and its disposal and finds out that there's two nodes in the cluster, the first in a second that can actually take or handle this pod.

D

So because that's because they only have like two and one CPU running at the moment, able dunam do some more things like it will sort. The know it's even to see like which is the most optimal, so probably scheduled in a second, because it has the most space available.

D

So that's all really cool and well, but how do we instruct it to take our heat into account? Because this is our scenario right? We have all the CPU and RAM conditions still there, but we now also want to take the heat into account. Well, the really cool thing is that cube. Anita's allows you to deploy an extra scheduler. We call that heat scheduler in this case and, as has like the same block, so it's basically the same thing. It's just the vanilla cube scheduler.

D

It still does it's logic like before, so it still takes into account Hardware resource constraints and all that, but we can now extend it with an extender, and this is where we can define our logic. So it will talk over HTTP and present the extender with a port in a node list, and then it can do its own magic, so sort the nose.

D

However, it likes and have returned a node list and say hey this is my preference or scheduling take this into account, so I will show you in a bit more depth in code, how that could look like because it can be done in just a few lines of code. Actually.

D

This is the configuration file of the heat scheduler, so there's two containers running inside this inside this pot and the first one as you see, is just the the raw cube scheduler binary. So it still does the same thing. But now we can define a policy config map which is shown on the right, and here you have the power to give like the normal predicates that are also in the normal cube scheduler.

D

So pot fits resources, for example, takes into account CPU and RAM in in in the nodes, but we can now also supply our extender, so it's the HTTP endpoint that it will talk to which is going to be in the same port on port 8100, which is the second container here, and this is where we handle the incoming request and do our own scheduling, logic and yeah. So let me show you how that extended n works that can be described in just three basic steps. Actually, what I'm showing here is a bit of cocoa.

D

It's a bit stripped down to like get rid of all the ferbos error, handling and all that. But this is actually the thing that you could run so at first it will handle the incoming HTTP request and decode the JSON object and all that. But then in the second step, this is where we can do our thing so we'll loop and try to find the node that has the lowest annotation neuralyzed temp for temperature.

D

So now we have the minimum value boiler temperature in the system and then in the second loop we can find a corresponding node and then respond eh, oops oops. That went way.

D

Then respond with the node that has the lowest boiling temperature, so it's giving it back to the cube scheduler or he scheduler in this case, and then it will sort it or schedule it on this on this node and to see what the impact could be of such a little scheduling decision, we conducted an experiment in which we envisioned the following scenario: we simulated 50 homes, each with his own boiler, and then we assumed four people on average, four households and its two parents and two kids, and we wanted to compare the default scheduler with the Cuban or with our heat scheduler in terms of in terms of heat waste.

D

So the scenario is as follows: kids take a shower for 15 minutes, then the boiler warms up for one and a half hours and then before they go to bed, the parents also take a shower of 50 minutes and the results look like that and that's a lot so I'll talk you through it there's two graphs, each of them containing 50 lines. So that's 50 boiler temperatures for each of the households and you see the scenario on top.

D

So it's taking a shower, so boiler temperatures drop, boiler, warming up, you see the lines going up and parents taking a shower and then there's two red bars that you should note one on the top and one on the bottom and the one on the top is where it's like temperatures that the boiler can actually not reach. So this is where we would fend out eat instead of store it in a boiler.

D

But if there's other boy lives in the system that can still use the heat, obviously that's a big waste and then in the bottom bar you see. Temperatures that D like that are colder than the coal is boiler temperature. So now we need an external source to heat up the water and that's also a waste if you could distribute the heat more evenly in the system.

D

So what you see is that with he scheduler it stays good within the bounds, but with the default scheduler it just doesn't take the heat into her. Cancer jumps through the red bars and the result of that is that we can reduce co2 emissions with an additional 10%, where and also like additional to what we can already do with neuralyzed, with just a few lines of scheduling code. So that's really cool Robert is now going to give you a live demonstration of how to hold work. So, let's pray that the internet connection still works.

A

B

E

All right, so my name is Robert to show you that this all actually works prepared a little demo for you. Let's hope it works. So for this demo we have a three node kubernetes cluster, as can be seen here.

E

Yay all right, so there are three notes, and each of these notes correspond to your household now. This means that there's a server with a boiler attached and the boiler is getting heated up by the compute that is generated on the server. The boilers have a temperature which is added to the kubernetes node as annotations. So we should be able to see this if we describe the notes and we grab for the annotation.

E

So as we can see, there are three homes Chantal with a boiler temperature of 43, Yost, 37 and Alex 24.

E

So for convenience we can graph these values as bar charts, to make it similar to what you saw in the slides, so higher bars means higher boiler temperatures. So, let's see if we can schedule some compute on this cluster, so to do that we have to find a job which is essentially a finite pod in kubernetes and this job uses. The nerd lies heavy data crunching image which, as you can imagine that some serious data crunching.

E

So, let's see, if we submit this to the API server, we should watch the pods here we can see it gets assigned to Chantal household. The boiler heats up nicely. This is sped up for simulation purposes. It wouldn't go this fast and relay in reality, and we can see that at some point the boiler reaches maximum temperature and all he generated after that is essentially wasted. So we want to. We want to prevent that.

E

At the same time, we can see that Alex Alex his household his boiler is much colder, so he can make much better use of that heat. So, ideally, we would schedule it there and to do that we need to introduce the notion of boiler temperatures and instruct kubernetes scheduler how to how to deal with this. So, let's see how we can do that, so we edit the v-spec and we add the scheduler name, let's get er and we set it equal to heat scheduler he's scheduler.

E

So what this does is it tells kubernetes scheduler to leave this alone, not to schedule this part, and essentially it tells the scheduler. We have our own schedulers called Heat. Scheduler, which will take care of scheduling this pod according to our own logic.

E

So if we submit this, we should see it showing up at the bottom right and it does in fact, but it remains pending, and this is expected because our scheduler is not yet running so to actually run our scheduler. We need to do two things.

E

This is the same deployment you saw in the slides, I'm gonna run you through it again just to freshen up your memory, so it has two containers they run in the same pod. One is the heat scheduler, which is a copy of the default kubernetes scheduler. So we just get it from the mirror. Google containers repository we pass in the scheduler name, which is heat scheduler, and we pass in the config map, which is scheduler config, and this config map actually describes how this scheduler should operate. So that's very loud, so this is the conflict.

E

E

Okay, all right, so the config map describes how we configure our scheduler, so we can use the predicates that are defined in the default kubernetes scheduler, which is desirable, because that means we don't have to implement them in our own scheduler, so use the three most common ones to make sure that the pod, the node is suitable and when the scheduler narrowed down the list of all nodes to the list of suitable nodes, it hands it off to our extender and the extender runs in the same pod.

E

They communicate over localhost and our extender is responsible for choosing the node with the lowest boiler temperature. So this can be seen here. The extenders running. That is the go code that Boris showed in the slides. So if we create this the config and we create the heat scheduler itself, we expect the status here to change from pending to running and it should be scheduled on Alex's household, because that is currently the coldest one. So that is indeed the case.

E

Alex's household is heating up nicely and at some point we see that its temperature it reaches a higher level than yours is household. So that means that if we create another job, we expected to be scheduled on Josie's household, so we submitted and as expected it gets scheduled on your household. His boiler is heating up. So let's do that one more time just for the hell of it.

E

So that one get scheduled on Alex's household, which was the coldest at the time, and as a result, we can see that the heat is nicely distributed across all the households. We minimize the the waste of heat and Chantal yoson Alex can take a nice hot free shower.

E

That was a demo I'm glad it works. Thank you.

A

So let me recap those four minutes: we got a pretty awesome idea neural eyes, but then we add five lines of code to scheduler and we made the ID 10% better I. Think that's pretty amazing.

A

So what's the Baxter question, what more can we do with such a scheduler scheduler and we came up with two extra ideas. So what if we don't look at the minimum boiler temperature be instead look at the forecasted amounts or oh wait. Let's recap the boiler temperature first. So this is what we did lie right. So we looked at me in boiler temperature as scheduled, two parts, but what if you look a bit further?

A

So while we look instead of the forecasted temperature of the pot, you can imagine that if it arrives at the boiler or at a server that heats a boiler, it doesn't heat up the boiler in media. So we have to kind of forecast how much heat is gonna produce, instead of assuming that it heats up immediately.

A

So instead we had a policy that looks at the forecasted amount of heat is going to be generated. So, in this case, we've got one there, it's forecasted and then chantels forecasts heats is the second highest and so on and so on. The next one I think is even more interesting. So what if we could forecast how much heat a certain hell needs? What, if we could look at the climate, for example? Maybe Chantal lives at a place that there's a lot colder, so she needs more heat in the winter, for example.

A

So what have you taken to account the difference between the forecasted heat and the actual boiler temperature, so with scheduled pots on places where we expect the difference to be the highest? That would look as it like this, so we take a job. We take a pot and place it Chantal. First then the second biggest difference, probably Chantal and then replace the third one and Alex and the next one is yours.

A

At this point you can switch back over to the old scheduling policy, so the next pot that comes along is placed at Alex and so forth. So what can it do for you right? You're, probably not heating up the Netherlands with servers, but maybe you got another business and the cool thing is that the actual business logic is very well isolated in this extender. So it's only the second part of this minimal amount of go code that you can customize in order to get your own scheduling in there.

A

So maybe you got a bunch of SSDs that you want to use inside your cluster, or maybe you got an advanced machine learning algorithm that you want to use in order to sketch little more efficiently across your data center, and this allows you to do that in a very well-defined place.

A

So to wrap up at neuralyzed we're creating a datacenters class. It allows the homeowner safe on his heating bill. It allows co2 emissions to be reduced by about two tons and you can get a cloud that's about 40%, cheaper than the rest of the world, and what we discovered is that Q Benitez is an excellent fit not only because of the standardization that's currently occurring within our case. It could save us a considerable amount of co2 by just implementing a basic best scheduler. We saw how easy it was to customize it behavior.

A

Then we went ahead and created an extender to get our business case in there, and then we ran an experiment to show that it actually worked, and it proves that we could save up to 10% extra co2 emissions per household per year, which is a great amount if you're interested to look at the codes. If is there get up? If you got inspired and you want to work in, there lies contact us, you can help too. If you want to eat up the Netherlands with your compute jobs.

A

We just launched our company this product today, so visit us at this link. Get your pulse in there help us get through the winter. Please flies with more information, are at the entrances and join our mission check out our code, and if you have any questions and their last crews up here, and thank you for listening.

A

Any questions right now, yes,.

A

So the question was: if the graph that we showed with our custom scheduler, if there was a simulation or it was an actual run, it was a simulation.

A

Any other questions, oh.

A

Sorry Oh excellent question.

D

He saw the hoodie right in the video yeah right, cool yeah. So, as I said, we warm our boilers right and not just heat homes. So in summer you still need to take your shower and still need more water for cooking. So that's that's! Basically how that solved.

A

And once we cover the whole world with our machines, we can just schedule at the other side of the world right. Yes,.

A

Yeah good question, so the question was whether we also reschedule maybe a job takes too long or we need to place somewhere else. We did an experiment with that. Of course. In our case, the redundancy of a single home is quite low. So often the machine might shut down and then we'll reschedule the job somewhere else. We didn't calculate how much effect that had on the actual co2 savings. So yes, we did it, but you don't know what the results or what the effects of it is.

A

Any other questions cool feel free to tap us on the shoulder, and thank you very much for being here. Have a nice day.