Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2018 (Shanghai), 21 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Run Cluster Autoscaler on Your Own Cloud; or, Why Do We Need Cluster API? - Aleksandra Malinowska

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Run Cluster Autoscaler on Your Own Cloud; or, Why Do We Need Cluster API? - Aleksandra Malinowska, Google

A Kubernetes cluster consists of a set of nodes. No matter where it is running, managing the workloads within it remains the same thanks to standard APIs. But what if you want to change the size of the cluster itself? In the cloud, Cluster Autoscaler can automatically shrink or expand Kubernetes clusters. To do this, it needs to reach beyond the cluster and interact with the environment where it’s running. It has built-in support for several clouds, but what if you’re running on a different one? In this talk, we’ll walk through making cluster autoscaling work for a new cloud environment. We’ll start with the critical subset of functionality required for basic support and continue with enabling optional features. At the end, we’ll take a look at the future, and see how Cluster API will change all of this.

To learn more click here: https://sched.co/FuJU

A

Hi, my name is Aleksandra and I'm, a software engineer at Google, working at kubernetes, auto-scaling team and today I'll talk about how to add, support to a cluster autoscaler to run it on your own cloud, on an environment not yet supported.

A

So, let's start with what is cluster autoscaler? It's an open source component and I am one of the maintainer of it. It has a support in a cluster. Is there usually on the master node and it's just possibilities adding and deleting nodes to the cluster as needed by the word close deployed there, and the code can be found on github under kubernetes the slash autoscaler. There are several auto scaling components there. It should be easy to find.

A

So when we tell the story of cluster autoscaler, we usually talk about pods and nodes. We tell how we trance bin packing simulations and to ensure that it adds their correct number of nodes or that the nodes can be safely removed.

A

We focus on how it carefully drains the nodes one by one, but deletes the empty ones in bulk, but this is only half of the story. The other half is that nodes here are just cube: anita's objects. This is what you get if you run cube, CTL get nodes, but underneath they correspond to VM instances, usually, unless you run cube my curvature, cubelet or something like this.

A

So in practice, if we see an empty node and we decide to delete it, it's not enough to use queue, permittees api to try to delete node objects right then it would result in a VM still hanging there, users still paying for the VM that can turn on anything. So this is not good and we don't do that. What we do instead is we want to delete the VM and then, when it's gracefully shuts down the node will disappear as well.

A

So, similarly, when there are pending pots- and you want to add a note- we don't want to create a fake note. What you want to do is to spin up another VM instance that will register as a node in the cluster and then the boss can be scheduled there.

A

So this may all seem fairly obvious, and but it's important from cluster auto-scaling point of view to divide data. There is the cluster state, we can observe and interact via kubernetes api, and then there are VM instances that we can access only through clouds, API and well. The moment we have. We use two api's to get two data sets and we need to correlate one with the other. We start running into problems so problem. One is: how do we merge node to a VM instance? This one is easy right.

A

Note objects have provider ID, which should be set to some unique resource identifier that can be used to interact with the instance using cloud providers API, that's how it works in most cases at least so. The first thing you need to ensure when you want to run it on your own cloud, is that the cloud controller for your cluster says this provider IDs correctly, because cluster autoscaler absolutely depends on this, so we can March them. But what? If there is a node we've provided ID set, and you can't find an instance with that provider ID.

A

What? If you have an instance that looks like it should belong to the cluster, but there is no node with matching provider ID and it might be tempting to say. Okay. This is not our problem. This is just some instance. It doesn't belong to the cluster, but what if it was customer of the scaler I? Think a node created a VM instance hoping it will register us now during the cluster run pods, but it didn't. If we just ignored it, it may never be cleaned up, it might be Starks. Perhaps such instances can accumulate.

A

So, in addition to set two nodes having correct provider IDs, it's required for us to be able to list all the instances that should belong to the cluster from cloud provider API on the problem tool. So how do we actually start and now at VM that will become a node and there are many ways to bootstrapping a kubernetes cluster and in most environments there is more than one way to create a node. You can use some sort of config management tool for deploying your cluster. You can use custom scripts for it.

A

Clustering, the autoscaler doesn't know about any of that. So in order to spin up a new instance, it needs to know something and this something we'll get to. But the first assumption important here is that it makes an observation that if it there is a node a and it corresponds to a VM instance B, then perhaps if we add another instance that looks just like B, it may register in the cluster and the node will have the same scheduling properties as the original node head and by properties.

A

Here, I am in anything that affects scheduling all the resources. Panes labels, the availability zone. All of them are supposed to be identical, so this approach of using the existing nodes as blueprint helps us handle the case when there are different types of nodes in the cluster right. It works out of the box. If we want to add a bigger node, we just request the same. Is that bigger now designing her? That's fine, but then, if we want to check all the nodes in the cluster, we click, we will quickly run into scalability issues right.

A

There may be so many nodes in the cluster and it doesn't make sense to run the simulations on all of them. So there is another observation, and that is that in most clusters, especially large clusters, nodes aren't individually much difference. There are some several types of nodes and usually several thousands of nodes of every type.

A

So if we just group them into some sort of sets, we call them node groups, because we like obvious names, then we can, for example, use just one node from each node group as blueprint for adding more nodes of the type.

A

So this abstraction has no obvious benefit of optimizing. The skylab, which is one of the reasons we use it, but another benefit, perhaps kind of unexpected- is that by adding it. The core logic has a very convenient way of requesting more instances just like when you have a cuban ets deployment or replica set controlling costs, you just provide a blueprint and when you want more replicas, you increase the number of replicas set. So with respect to scalar plot groups were pretty much like this and it's very useful, but it's not a Cuban.

A

A this concept: there is no such thing as node group in a Cuban II this API. This is an extra layer of abstraction that is added in cluster of the scaler, and its purpose is not to be specific to any environment and to be as useful as possible, but it turns out it becomes very much cloud provider specific very soon, because the nodes are started differently in different environments and, for other reasons, I will get into in a second. So how does it look like there?

A

This big blob in the middle is clustered to scatter binary. It, of course, can the core Albury the logic that decides whether to other note, what type of note to add work to reduce the pulse which knows to delete it, imports schedulers code, which is important, because this is the default scheduler from core kubernetes and it doesn't make a guess: will that pod be able to shut you on this note its tracks? It's running a simulation of the shaders code, so she's and the scheduler extenders or custom scheduling solution.

A

It might not work very well with cluster autoscaler and it observes the cluster stayed by using kubernetes api, which is obviously the same in all environments, so it can be used directly, but in order to interact with the cloud API, it has also a cloud provider module which is of course different for every cloud provider, but all of the logic of interacting with that cloud provider is uncaps elated. So the main thing to do when running it on a new environment is to write the cloud provider module for that environment.

A

How to do it? Well, there are very few requirements for how this cloud provider module will interact with cloud provider API. It's super specific to environment. Every cloud works a little bit different and from cluster autoscaler point of view, it doesn't really matter how it gets done. As long as it gets done, there are only some minimal, functional requirements that must be achieved in order to make it work, but cluster autoscaler logic depends on cloud provider model behaving in a very specific way, and this is what I'm going to focus on.

A

So the entire surface is not very impressive connection. It's just to calanque interfaces connecting the main logic and the cloud provider module and the first one is primarily responsible for providing node groups, so they can be really thought of as a single thing to implement.

A

We just split into two for some syntactic sugar. So how do we implement these? No groups? We don't have kubernetes. We really need them in cluster autoscaler. Well, the good news is- and this is what actually all existing implementations, though that most cloud already supports some sort of VM groups.

A

So you can just deploy your cluster in a way that all knows we run on instances that are part of some instance groups and all nodes within that in the group are identical and then implement cloud provider module in a way that interacts with those groups requesting new nodes based on some template, which is like say, set specific to our cloud and deleting some notes. So, as usual release will but a good news. It's followed by bad news.

A

This groups work slightly differently for every cloud provider, but they rarely work out of the box with cluster autoscaler. It's mostly related to how these groups tend to limit the users ability to control their machines. So let's have a look at what is required to make the node groups abstraction work for us.

A

There are two critical requirements of an outgrowth. First. Is that all nodes within it must be identical and I mean identical with respect to scheduling properties, all resources, the labels depends the availability zone. This one is often overlooked, but if your workloads depend on it for scheduling, it must be the same for all nodes in the node group. If you want to have similar nodes across several different zones, just have several different node groups. That's fine! Okay!

A

It will work, it will save you a lot of trouble later and the second requirement no less important is that we must be able to delete a specific node cluster of the scalar coordinance drains and deletes nodes, and it wouldn't make any sense to simulate scale down if, in the end, we could just request.

A

Okay, I removed two instances: I, don't care, which we very much care, which instances are going to be removed, and so this requirements translate into some requirements for VM groups that must be fulfilled in order to make them work as node groups and I put something slightly different here than in other groups, because I believe that it is possible to use regional node groups, for example regional node groups that have nodes in different zones and artificially split them in cloud provider modules into several node groups.

A

One person, if you must use regional VM groups, but you must be able to create instance, in a specific node group and therefore in a specific zone and the requirement to delete a specific note, obviously translates directly into a requirement to delete a specific machine. If you can't do that and that node group might not be that VM group might not be very useful for you.

A

So what? If there is bad news, there is no VM groups in your cloud. Perhaps is just starting out and decided not to implement it yet or perhaps the node groups are frankly unusable for this purpose. You can delete a specific machine or you can only create a regional node group which will always have the same number of nodes in each zone. So, if you add one nose, it will actually act or free depending on the number of zones in spans.

A

So in my opinion, not all hope is lost, I haven't done, it I haven't seen anyone do it, but I don't see any issue preventing implementing, not groups directly in cloud provider, module I believe it should be possible, although of course it's going to be much more complicated because you effectively have to implement the logic of VM instance. Groups inside this cloud provider module but in principle I believe, is possible.

A

So this is enough for the basic functionality to make cluster autoscaler. Add some nulls delete some nodes more or less work for your cloud, but there are some features that it may not be obvious why? But they are optional. So, for example, the assumption that we always use existing nodes as blueprint for a group.

A

It works very well, except when you really want to delete the last note of that group. For example, it is a GPU node and it's only useful when the batch job is running or for some other reason. You have expensive nodes that you really want to scale down to zero instances in that group.

A

Escaping to zero is trivial. We can just delete the last note. The problem is without a blueprint. We will not be able to scale up again, so what you can do is provide a blueprint and we do it by implementing an optional method. In all group interface, using which cloud provider module will retain a fake note, object. How a note in that group would look like if we requested it to the core algorithm, and then we can set the minimum size of this node group to be 0 and close.

A

The autoscaler should be able to scale to and from 0 a scale from 0 might seem like pretty obvious, but sometimes it requires non-trivial logic to, for example, parse the instance template of a VM group, or something like this. It really depends on how you go with the implementation, but it's very useful and.

A

So the other option we might want to have is to have smarter selection of the node to add, because for some cases only one load group may be a suitable fit for the pods that are pending, but in other cases, for example, adding two loads to one group or adding one node to another group will both allow some of the pods to run. We call such possibilities of expanding the cluster expansion options and by default, if you do nothing personal flag, the random one will be chosen on scalable and well.

A

This is obviously not optimal, because sometimes you might clearly prefer to run your pods on 160 core machine dot on 15 one core machines.

A

So there are other options to select from that should work out of the box, for example, most pods strategy, which basically selects the option which will allow most pots to schedule least waste option which will calculate how much spare capacity will be left in the cluster. How much CPU and memory and decide that expansion option. That leaves the list under unused space is the best one. This seems like really close to what we want to achieve laughs all right.

A

We want to avoid wasting resources, but, as it turns out, the users really care about wasting one resource Martin about any other. They want to avoid burning money. If a 16 core machine is cheaper than one 8 core machine and one further machine, that's fine. They are happy with 16 core machine and burning to the 4 cores.

A

So this is super cloud provider. Specific right pricing model is very different for every cloud provider and it's hard to keep up with it and it keeps changing and there is not always a consistent API to get the prices of VMs. So how do we deal with it? Well, we introduced even more abstractions that are supposed to be called provider agnostic into our other. You can use a pricing expander, which should calculate the best utilization interest. Cost efficiency, if you provide us with a pricing model, and this pricing model needs to answer two questions.

A

First, is how much it costs to run a given note. This one is obvious right: we want to know how expensive that option will be. So this is one thing, but there is another, perhaps less intuitive, how much it costs to run a given pot, and why is that? Well in terms of cost expansion options that, for example, allow your scheduling just one note one pot on one note, but this is a very small note- will always win, even if that pot would fit with the other pots on a big machine.

A

So, instead of adding a small machine for this single pot and then adding a larger machine for the other pots anyway, we need to calculate how well utilized this will be. We need to factor in how many pots are served by this expansion option. Essentially, so this abstraction of how much it costs to run a pot is very much it's not reality. It's not how much you will pay for this pot.

A

It is based on the cost of raw resources like CPU and memory that this pot requests so that if it was scheduled in a on the perfect to utilize perfectly sized machine, it would cost at least this much. It is how much it must cost to run a despot on this cloud provider and then using this and some some math formula, which was basically invented to make sure that the option is the thing is behaving as we want it.

A

We can select them up the expansion option, that is the most cost efficient and it should be independent on cloud provider. I hope it is, but if you have like any questions or want, are interested in implementing it for your own cloud, please let us know I'd be happy to find out if it's chaos, a sweet fault.

A

So a small note on dependencies there are, you, see, multiple problems with cloud provider modules, it's hard to add, support to a new cloud and hard to maintain it in main, wrapper. There are cluster of the scalar releases and, if update to your clouds requires a release to be rolled out, then well, you might have to wait for that release and it puts an extra burden on maintenance, and all of these has to be reviewed by people who will never run your code and that's very good.

A

So this is the same thing that kubernetes cloud provider code in kubernetes faced and it has been resolved by moving or is being resolved by moving two out of three cloud providers. The other problem is that you are likely to be implementing the same thing or slightly different, but more or less the same thing over again in different places.

A

Perhaps you have some automated way of creating clusters and boot strapping node, and then you will be duplicating that logic in cloud provider there module or reverse engineering kit and third- and this is like the nightmare from nine miners point of view- is the growing number of dependencies. We simply don't have the manpower to audit all of them and we really hate increasing this binary size even more so about this dependencies. There is a fun fact: this is how some go. Apps look like think of some actual logic.

A

They use some dependency management tool, gotham's or something newer and fancier, and this dependencies are sometimes story did vendor director. So this is how cork you Bernie. This looks like it has the components which are often vendored and they are separate from the main wrapper in order so to not be to not have to vendor the whole thing. So they are separate reports.

A

Fine, we can just use them right, except they are actually developed in court kubernetes under the staging direct, which means they are sync there, but the cork you burn it is itself siblings in its vendor to staging so that the compatibility is maintained. So this works for abilities with the assumption and recommendation that culture brain it is is not intended to be rendered guess what scheduler code lives in tark or kubernetes, and this little component called cluster autoscaler depends on that.

A

So we must somehow deal with it and the way we do it is that we do some hacks around our dependencies to use the color. But it is dependencies instead of check out as fixed version and then any dependencies that we don't can take from kubernetes because it doesn't have them we put in an override. So there's a bit of nasty hug here, but it also I hope explains how.

A

Why is it so complex for us and difficult to maintain the growing number of dependencies and why we want to avoid them as much as possible so onto a better brighter future cause the API this? The idea behind this thing is very interesting like in kubernetes, you have deployments and replicas of managing pods.

A

What, if you could have machine deployments and machine set managing machines, the VM instances, an abstraction that is Kuban. It is like and independent of cloud provider. So if you want to create a node on and a cloud, you just go class there API and set one more note on that cloud.

A

Please please create a node group like this in this cloud, so this is very similar in some way to the node group abstractions we already used, although it has like it, takes it much further, obviously allowing crawling cup grades and stuff like this, and it also means that you will implement this for each specific environment.

A

Only once if you implement cluster pl-4 your cloud, then any component that wants to manipulate your cluster, the resources, the machines can just use it and doesn't need to have the specific code so maintaining that code becomes much easier because it's contained in one place and also, if you have clusters deployed on different clouds, you can manage them using a single API with many implementations each for a different route.

A

So you can find out more about it on seek cluster API, and this is just one possible future I, don't know if it's gonna happen, but if it does I I think it will help us a lot with managing dependencies and stuff. Instead of having cloud provider modules in castor autoscaler, we will have a sort of adapter module that basically implements our abstractions using the cluster API, and then it calls the cluster API for each given cloud to request. Vm delete, VMs, discover VMs, read instance, temple legs and create a node info out of it.

A

So that way we can have all the features without and a cloud provider specific code in our binary and without having, and it would basically warcraft of the box. So the obvious question to me at least, is: if we have this one took out all about implementing cloud provider modules anymore. Why should we add another one when we should be migrating to this?

A

The answer is reality cuz, our API is not yet ready by this I don't mean that it's not yet implemented everywhere, but also that the API itself is missing a critical requirement for cluster autoscaler. It's missing the ability to delete a specific machine, and if we can't do that in a transaction, reduce the size of machine deployment and delete a specific machine, then we can't migrate to it.

A

So this is like the major blocker right now, but there is ongoing work to make this migration possible, perhaps step-by-step, perhaps for some cloud provide their first and I believe this is go. I believe this would be a great thing if it happens, so that would be it. If you have any questions.

A

I believe we have a few more minutes, still feel free to ask about anything and if you would like to find out more or run into any problem or would like to ask us our skilled maintenance about anything, you can always find us on github open an issue in that report, pingas directly or on slug siegel to skiving. Thank you.

B

Let's wait for a microphone.

C

In class or scalloway, we can configure different, not group, but if we migrated to class API is there at the at the net hell ways to do the same things.

A

So the question, if I hear correctly, is that in clouds there autoscaler we can configure settings for each node group and which cluster API will not be possible. I believe so, because cluster API will provide us with abstraction, like machine deployment and machine set, and if your trusted is deployed using these abstractions, then from the user point of view, it should work more or less the same. Okay,.

C

D

Just a quick question so far I can see you are like this project, but for the node itself in the future, for example, if I have a different CNI or all these kind of thing, because so far we are not only today talking about the node itself right. What are the storage? Also the network so well that all be included in this autoscaler ap cluster API?

D

Whether was all the effort will be put in this project if we are dealing with different kind of CNI interface, for example, unlike an on-premise back and use weave, I can use some other Sdn solution can I pick from this or I. Just this party only generate nodes, so.

A

This question about custard API, how it will deal with storage and different CNI drivers and stuff like this. So the answer to is this: I honestly I'm not sure about it, because I'm not working on tosser API directly but from cluster auto-scaling perspective I, believe it should be pretty independent of that.

A

Basically, if noise in some machine deployment have storage, then we will assume that other nodes deployed there will have storage. I am NOT, fortunately not able to answer your question about cluster API plans for support of this.

A

Okay, so maybe you can pause the microphone.

E

Yeah thanks I can help a little bit because I work on the cars API, so yeah, seeing my difference. Kind of seeing I support is in there in the discussion of the house API, but it's it's obstruction and autoscaler should not care about it.

A

E

A

E

Have the question so so how you work? How do you work with acosta api? We have the cart, api weekly meeting and discussions.

E

The two project going to merge right.

A

So the question is: how do we work with toaster api and are those project going to merge or something- and the answer is I- believe? No, because we are doing different things we as kloster autoscaler, would like to use cluster api, but we don't actively work on cluster api.

A

Like I don't, of course, that's the only speaking for myself. Other people obviously do and yeah this. I don't believe these projects are going to merge because they are essentially doing different things.

E

B

Okay, so thank you very much again.