Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2019 (Barcelona), 23 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Deep Dive: Virtual Kubelet - Jeremy Rickard, Microsoft & Lei Zhang, Alibaba Cloud

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Deep Dive: Virtual Kubelet - Jeremy Rickard, Microsoft & Lei Zhang, Alibaba Cloud

Virtual Kubelet has most recently been accepted into the CNCF as a sandboxed project. As the project continues to grow in contributors and users we are always looking for ways to educate folks on how to contribute back to Virtual Kubelet. In this talk we will focus on the core interface of Virtual Kubelet and how a developer could get started with building his/her first provider! The talk will include an engineer from Alibaba who contributed the Elastic Container Instance provider and a maintainer of Virtual Kubelet, so we have a spectrum of perspectives represented.

https://sched.co/MPkI

A

B

I guess we can get started yeah all right. Everybody thanks for coming to deep dive into virtual cubelet I'm Jeremy I work at Microsoft, hi.

C

And a Jung, you can call me Hari and I work for Ali Baba Khalid, so.

B

Why are we here today? We're gonna talk about what virtual cubelet is and how you can use it, how you can build a provider, but before we do that, let's define what a cubelet is. So a cubelet is an agent that runs on every node in your cluster and it's responsible for watching the kubernetes api and figuring out what pods need to run and make sure that that state happens.

B

The way it works is kind of like this. So you've got a cubelet running on some node and it says: hey api server, I'm a node and then.

A

You, as a user, creates.

B

Some sort of pods back through a deployment or some other mechanism and the scheduler will say what nodes do I have available. Where can I assign this thing and it's gonna assign it to a node, so the pod spec will be updated and the pod spec will say: I am gonna, go to node, X or whatevs, and then meantime and meanwhile the cubelet is doing a watch specifically for speck, that node name equals node name and when it sees that it's gonna go ahead and reconcile the state to figure out.

B

Is this a new pod do I want to create? It is a existing pod? Do I need to do some sort of update like change the image and once it does, that it'll use the container runtime interface to say, create the container update again or delete the container whatever it needs to do?

B

So? What is a virtual cubelet? A virtual cubelet is a you can think of it as a controller that acts as a virtual node. There is no physical node backing it, but it appears as a node to kubernetes it'll have all the things associated with that. Like resources that are available memory CPU and you can use it to schedule mods, so when should you use it if you're going to be using something that looks traditionally like a node?

B

It's probably not the best case, because it doesn't have all of the capabilities that a regular cube would. But if you're gonna do anything else, it can be an attractive use case, so something like bursting into a cloud or container as a service provider like as your container instances or AWS Fargate bridge into other orchestrators, like maybe you want to schedule some containers on nomads, there's a provider to do that.

B

If you want to schedule to Internet of Things control planes, that's another use case, we've seen and then, at a workshop last year somebody built a provider that ordered pizza with the Domino's API. When when pods were created, it was pretty funny. So how does it work? Obviously it's magic, but it's not actually magic.

B

It's really the same thing that we just saw so virtual cubelet starts up, has a daemon process and it says: hey Cooper and his API I'm, a node and Cobra, doesn't know that. It's not a node, it just knows that somebody said hey I'm, a node, and then you create a pod spec. The scheduler says this pods been or this pod should go on this node, which is going to be virtual cubelet.

B

So then virtual couplets doing the same thing that the cubelet is it's watching the API for any pods that are scheduled to it, just like any node would and then it reconciles state just like a cubelet would but then, instead of talking to the container runtime, it's really talking to something called a provider and the provider is responsible for doing that bridge between what the pod is and where it needs to end up some external service.

B

This is what the anatomy of VK really looks like. There are really three parts: one of them is the daemon part at the top, then there's a framework and the framework has all of the stuff. That's really reusable. It's going to have the reconciliation of state, it's going to have a definition of an API and that providers have to implement and then providers implement the actual calls back to those cloud providers or whatever they're going to schedule to the framework works from a shared Informer, a the state reconciliation.

B

The cubelet has a couple of HTTP endpoints that you're gonna like exec into a container or try to get log those go through an HTTP endpoint like if you do a cube, CTL exec. So the original cubelet exposes that and figures out how to invoke the provider and the provider is responsible for translating that back down. It does things like tracing a logging, so you can get good information about what's happened.

B

So, what's a provider the provider is that mapping from the actual? You know you said you want a pod to it running somewhere in a cloud provider like KCI, it doesn't have direct access to the kubernetes api. So it's really gonna leverage virtual cubelet to get things like secrets of config maps and it's gonna leverage VK the framework to handle all the state reconciliation. All the things that are necessary, it just follows an interface and makes whatever that interface asked for happen.

B

So here's some current providers that we have there's quite a few of them. Alibaba uses it pretty extensively. Asscher uses it for a CI which is our container as a service tool. We use it for providing virtual nodes inside of aks. You can see a whole bunch of other ones like AWS and hash. Your carpet, Nomad I'll turn it over to Harry to talk about how they use virtual key would yeah.

C

Thank you, so I want reduce to very interesting user case of how Alibaba is even virtual kool-aid. So the first user case, as you can show here, is just like what Microsoft or Fargate is doing, and actually we are using word to collate to build over one of the most important product, which name is third is kubernetes. So whenever you are, you need servers kubernetes on Alibaba cloud. We actually create a control panel for you, but there's no nodes as there are notes and whenever you want to have a nose.

C

For example, you tear apart the underserviced kubernetes will be triggered to create a virtual node for you. The mercial node is actually provided by a project which name is widely a waiking project is actually a wrapper of virtual kool-aid, but in order to handle more complicated user cases, for example the coop epoxied to be honest, and to support an ingress, they use Viking to handle, restore things besides what it were to cooperate doing today.

C

So, in order to make it happen, so the server's kubernetes is basically will give you the infinite result pool and this resource pool is actually provided by the elastic container Intel instance service, including Alibaba cloud, which is basically like market or SDI from Microsoft, and the an indicia instance is actually actually works machine. But this virtual machine is not either fast and in this virtual machine.

C

First of all, we have content Iran inside and then we have a waiking engine to handle the request from or chocolate and to help and to withstand request through the monetary service load, balancer or DNS service in Alibaba cloud. So you can see here who are using our universe, crew blade and we have the Viking engine to handle rest of the request design.

C

Your pod will be delivered to this ECI instance, and that is how we create a service kubernetes for every user of Alibaba cloud, and these user do not care about which machines or hosts I do not need to care about their resource resource boundary because they're logically, they have unlimited resource poor here. So that is how big we are doing. The work, the service kubernetes stuff.

C

A second example, is how we we are using work to cooperate inside Alibaba, so a little bit very interesting because, first of all, it's offering public house public cloud product, just like I said before, and also we are supporting the whole Alibaba ecommerce platform, things that Alibaba.

C

So in order to do that, we are actually using a very interesting architecture here, because we are using a multi-tenant kubernetes cluster, but in this kubernetes cluster, which is shared by all the tenants from Alibaba, will actually create a so-called virtual cluster for every tenant that that means every tenant will have a dedicated control plan that we have their own cookie API server. They have their own Kuby control manager. But, as you can see here for this work, a cluster, how the healthier they have their nose come from.

C

Actually these nodes come from two different approaches. The first approach way is from the real cluster on the lighting cluster and actually from tenant of view, the only server so not they do not know about VAR d, its version of come from because virtual node is basically a API object, which is credit by version of the ancient or the word recuperate. So what's the difference, the difference- it is like this.

C

Whenever the 10 in the credit pod, it will actually be delivered to the real kubernetes cluster, and this real corporate cluster is running on top of the bare metal machines. So this is a most common case in a vapor cloud in Alabama in inside alibaba Club, but also we have some very interest in either cases, for example, for the Alibaba double 11. We don't we don't need to handle like 500 thousands transactions per second during the midnight of W level, that is an November 11th of every year.

C

So then we need to borrow resources from the from the from this firm from other teams cluster. We need to pour resources and how we connect to the other teams cluster. Then we will use a virtual cluster, for example, in that case, we will use worship cluster to borrow resource from, for example, the ECI service, which is not the resource pool we have, but we can borrow resource from them and also we can even use virtual couplet to borrow resource from other teams.

C

We who have who also have kubernetes runny inside Alibaba club, so we can pour resources from different teams by using word recruitment. So this is the lesson we learn of how we use or chocolate inside Alibaba, so, first of all what recuperated very interesting project which can help you, which can make you to borrow resource from unlimited resource pool. That is how we handle some bursts of the requested requests during like table xi, events or other like sales events in Alibaba cloud and also what recuperate is also a very interesting protocol.

C

You to implement a virtual cluster based multi-tenancy here, Montanans model here. So this is actually what we are using with russia club with what you're creating is an alabama club, cool.

B

Thank you. That's really interesting, so let's take a look at what it would look like if you want to build a provider so I'm gonna switch over to visual studio code and we'll walk through some code right now,.

B

Actually, let me go to the browser and we will look at the repo real.

B

Half a million tabs open, so you can find virtual cubelet at github comm, slash virtual cubelet, yeah I'm gonna share it once again.

C

B

Settings here we go mirror.

B

Okay, so let's go back to this window now: okay, so there's virtual couplet, you can find it at virtual cubelet, slash virtual cubelet and inside of here you'll find a bunch of code right now. That does a few things you can find in the CMD virtual cubelet folder. This is where the daemon is defined, so you can go in there and that's really the entry point.

B

So when you're going to run, DK you're gonna run that process and you can use that as a way to kind of dig around to the where the rest of the code is located. Inside of the providers folder, you can find implementations of all the current providers, so we can see there's Alibaba cloud AWS a couple of different azure ones, highway Nomad OpenStack and we have a REST API for scheduling things.

B

These are gonna actually move out of the tree soon we're trying to work right now towards doing a 1.0 release this week, and it's part of that one natto released.

B

We want to make things a little bit more maintainable, so we've been trying to harmonize and figure out what the exact API for the provider should look like and we've gone through and you can see like there was a commit that touched almost everything in here where we actually updated this to what we think the one dot a P I is gonna, look like and then in the V cubelet directory.

B

This is where the magic actually lives. So there's some things in here like pod, control or no-go, and these are these files- are gonna, define all the logic, that's necessary for, like faking out a node and reconciling the state inside of the pod controller, defining interfaces things like that. Okay, so let me jump back to the real studio code now and we'll actually look through that stuff in an IDE is a little bit better.

B

So the most important thing in here oops is V cubelet. So right now the server piece acts as the main lifecycle for VK. Inside of that, you can see a couple of things: there's a Kate's client and that's how it's gonna actually talk to the kubernetes api using client go, you can see things like provider, resource manager and those are things that are gonna, be what's actually talking to the provider and figuring out how to schedule those things.

B

Then, if you scroll down a little more I, just information on how to construct that thing in pod controller, the really important piece of how you would implement your own provider, 4vk starts with the pod lifecycle handler inside of that these are the methods that are actually necessary for implementing a provider so create pod, update, pod, delete, pod, get pod and so on and thereafter they're actually operating at the pod level. So not individual containers.

B

It's gonna, take the pod spec and pass that to the provider and the provider is really responsible for figuring out how to take that pod spec and turn it into the necessary calls that go along with that and then inside of API you can find server go and server go is where the HTTP handlers are defined right now. So, for example, slash exact name, space pod container that I'll end up calling back to a handler function.

B

Those things are defined right here and that's what you're going to implement inside of your provider. So as an example we'll look at the ACI provider to show you what this might look like.

D

B

Here so inside of here, you can see there's a whole bunch of methods that have collapsed because they're not super interesting for this case. But let's look at the create pod method, just right here so.

C

D

B

Let's hide this window, so you can see in here is that we get the pod spec itself, and this is just a kubernetes pod definition, just the normal object you get from the API server like that, and we start to iterate through it. So we get a list of all the containers that are in that pod. We figure out what image pool secrets are necessary for those containers. We get volumes that are attached with those from the provider itself and we start to build things up.

B

The way you would schedule something in a CI is really analogous to a pot. So we have something called the container group and in that container group you defined one two and containers. So really.

B

What we're gonna do here is turn that call in or that pods back into a call to create a container group and a CI we're gonna populate all the things that are necessary to make that happen, and then, with you know, we keep going down here and eventually we are going to call the ACI clan, so we've gone through and we've translated all of that pots back into the necessary implementation of asher, and then we call the client and the client actually makes that happen for us.

B

Then we update status and we move on so later on. If we want to update something update.

B

So back in the pod controller, this is where the reconciliation is happening. Oh, this is gonna get invoked and once it determines that something needs to be deleted or created or updated. It's going to invoke those necessary methods back in the provider, so in this case we're making a pod we're going to invoke that create, create pod method.

B

You know it's gonna come through this logic and right here, it'll say: server, create or update pod and that's what's gonna turn into the call back to the provider and invoke this method eventually, and you can see in here there's examples of how to do the same thing for deleting and getting pods all the lifecycle operations. You have to do to implement that provider yourself.

C

B

Back to the slideshow.

B

So super exciting virtual cubelet, it's gonna hit 100 this week and what that means is that the API is pretty much stable. Some things we're gonna do going forward, are again break out those providers so they're not living in the tree, and it's a little bit easier to maintain the the virtual cubelet daemon for a specific cloud provider.

B

A lot of the configuration and stuff right now is kind of split amongst all of those things, so it makes the helm chart slightly hard to maintain a little complicated, so we're moving towards stability and having a little bit better maintainability of everything. I showed you the github repo earlier. So here's a link to it again if you're interested. If you have questions you want to ask about VK, you can find us on the kubernetes slack in the virtual culet channel. We also have community meetings every other week on Tuesdays at 11:00.

B

That happens on slack our on zoom'. If we go to the website, you can find that page.

B

You can find recordings of previous ones as well continue.

B

So right here, Oh.

D

Am I show you, oh sorry,.

B

B

So you find information like how to contribute and who the maintainer czar currently for each of the providers. If you have issues what our lifecycle looks like for accepting patches all that good information you'll be able to find inside of here, we obviously followed the CN CF code of conduct and come back here.

B

Again so we have community meetings every other week on Tuesdays at 11:00 Pacific time and CN CF has accepted virtual cubelet as a sandbox project that happened right before coop con last year. So that's. Why we're here today to talk to you about it and we built a bunch of time and asked to answer questions so I'll move on to that now and if we don't have questions I'll show you some more code so open up to questions now. Anybody have questions, go for it repeat the question Torrey to or you can use mic I.

A

Get that it kind of looks, feels and smells a bit like a node, mhm but when I deploy nodes at the moment, I have to think about things like availability and spreading them across fault. Domains have to think about things like performance.

A

And virtual cubelet deal with things like that, or is that something that will come on so.

B

Particulate itself won't deal with that virtual cubelets, providing the interface for you to connect to some other thing. They will handle that for you. So in the case of like a CI, you you're gonna know when you configure virtual couplets use a CI. What regional, like what Billy's own will be in same things for Fargate. So if you want to handle like a cross fault zone, so you can scan create multiple VK instances. It would be in multiple availability zones. You can handle that by just assuming that you know this.

B

It's a cloud provider, service and there'll be some level of guaranteed SLA. Okay,.

A

Inside of it, assuming that I just say, let's balance across to fault domains, so that would just literally be two virtual cubelets one point to each I: don't have to think I need five virtual cubelets in this fault domain for a really simple use case. No yeah.

B

It's gonna depend. It will depend on the backing provider right. So if you are having a provider, that's maybe spinning up virtual machines and then scheduling containers on the virtual machines. You have to worry about where those virtual machines end up, but if it's something like VMware's are sorry as you're a sea I know, it's gonna have kind of like an endless.

B

Whatever your subscription accounts for supply of stuff right, so that one you don't necessarily need to think about how much capacity I have, because it's not going to be tied to the number of virtual cubelets. It's going to be tied to whatever the backing providers, sorts of services.

B

Well, you could ask the question and I'll just repeat it, so we go to the Mike's way, not to run back and forth.

B

Yeah you can we. We have a couple of demos where we have used things like admission controllers and an HPA to do that kind of Burstein scenario where we some one of the things that I didn't mention, is that virtual cubelet taints itself as a node so to actually schedule things by default. You have to apply that toleration so that don't get scheduled there. So you can use you know through that. You can make it kind of go that way.

B

The HPA will scale, because it's just gonna create new pods right, like when the HPA decides that it needs to make another replica. That's going to just create a new pod and BK will just pick that up, assuming that it has the Toleration applied to it. The answer question.

B

Yeah, so it would allow them there's not a way to really. You can't really force the scheduler to say like only go here. You can say tolerate that node, so it can get scheduled there. I guess.

D

E

Really depends on what.

B

The scheduler defines our sizes available, so I guess. The answer to question is no. You can't just say you only go to VK, you can do like static pods or anything where it's just gonna end up on a single node.

B

But that is the use case. We see people like using it for on like a hybrid call situation where it's on Prime and when they run out of capacity burst into the provider.

E

C

B

It was a very large number, so like 4 terabytes, so.

C

The only above a cloud we definitely have vertical to report the whole resource capacity of this cluster. It's very huge yeah.

B

Yeah, so it could be. The provider is really good to report that back so in your provider. You could basically report an infinite number. You could report it back out realistic number to whatever the real capacity is to take advantage of the scheduler when you are using something like a CI or what alibaba's doing when it appears that's an infinite. No things will just keep going to that that node over and over again.

B

So the question was: does are there any limitations when using virtual cubelet versus a regular note? Is that summary yeah? So there are virtual people, it doesn't conform to a regular node. There are things that will never really be true about something. That's not over physical node, like having attached storage, necessarily welcome map to every use case. Vk itself doesn't try to impose any of those things. They're gonna end up being limitations of the provider, so things like resource constraints. You know it's not gonna really decide.

B

Some of that is the scheduler just ayane like well. This can only get so much, but those things will be passed down to the whatever the provider does. So in the case of a CI, we do honor constraints like if you only want to run like a one gigabyte of RAM in a container.

B

Yeah, so the go back to the code, real quick, let's jump out of here so in your create pod. So this is getting the entire pot object so like when VK says: I need to go, create that pod. It's gonna pass the whole thing down and inside of here you know we're.

A

Looking at things.

B

Like what is the operating system type? What's the restart policy containers? Let's go to that so inside of here what ports are defined? It's gonna get the entire spec and figure out what it can do with that spec as its operating go. Let's go back in the blue shirt. First.

B

We haven't really built, so the question was: is there just? Is there a provider right now, that'll do like kubernetes and kubernetes, so we don't have that right now you could build something like that. We've been mostly focused on providing different kind of use. Cases then that you know connecting to external resources doing that kind of bridge. Where there's not that traditional, like there's, no cubelet of a lor, you try to bridge to another scheduler type, but it totally be possible to do that.

B

You could build the provider to do almost anything as long as it's conforming to that that air phase, okay, green shirt.

B

Do we really do we support daemon sets right now, yeah, so there's some limitations right now with you can daemon sets is one of those things where it doesn't automatically pick up and schedule, because there's not really the concept of the providers. Don't have I mean they're faking, a notes for they're, not really a node, so some things like Damon's headstone, don't get implemented yet may or may not get implemented in the future.

B

But in general it's gonna handle things as a node would send answer your question yeah the log collection. So when you are gonna so there's two two forms of logging: there's the logging 4vk itself that's available, and then inside of the interface here for the back down to server one of the things you do to get logs.

B

That comes in through the HTTP endpoint that the cubelet has so that's handled here as well, and then the providers are responsible for fetching that and streaming it back to VK for VK to report back to you get back there in the back.

B

Can I mention a virtual cubelet for Windows I can so question was, can I imagine a virtual cube for Windows and I? Don't really have to imagine that because we can do that with with a CI and with sort of service metric, so there you could write a provider to schedule anywhere right. So if you have a service that can run Windows containers like service abric, you can make a provider for that and we have some use cases where we are some some demos in software we've seen Windows containers running alongside Linux, there's say again.

B

Yet someone has tried that yeah.

B

So the question was: has some they actually tried that and the answer is yes, we actually see at Microsoft we see a lot of actually windows, container usage for kubernetes via virtual cupola and some of our backing services.

B

It is obviously there's some limitations so like with virtual nodes for aks can't do Windows nodes right now, because it doesn't support V Mets correctly for virtual nodes and I guess you have to have V Nets to have the ability to talk back and forth between pods.

D

B

Again, are there any complications? So question was: are there any complications with GPU support and it's gonna come down to the provider, so you can use like a CIO, for example, has GPU support? You could schedule things assuming the node reported back, that it supported that capability.

B

Can you replace all of the a kiss nose of original notes? Not yet I saw somebody on Twitter tweeted that they used, so they were. They were trying to run virtual cubelet as a container in ACI on a vita cluster that had a V net and then they scaled down. The node like it was a one node cluster and they scaled it down and it was kind of working, but it's not a supported thing yet yeah, it's a hacky thing but, like you, could envision like a like a node, less cluster.

B

As long as you can live with the limit. Thank you know the fact that it's not gonna have 100% compatibility with a regular cubelet, but like it unlocks a lot of cool stuff, because it's gonna give you the ability to do non-traditional things.

B

The question was what about running virtual cubelet outside of the cluster as long as the so BK has to have connectivity to the API server, and as long as it has connectivity to the API server, you could do that.

B

You would have to solve that externally. Okay, but like as long as you could make all of those complicated things happen. It would be possible.

D

Any more questions go for it.

D

Say it say one more time.

B

Does we use a CI? Does virtual qulet automatically deploy on to it, so you have to deploy in virtual people into your cluster first and configure it to point to that a CI instance. It doesn't there's not an automatic switch that just thought it comes for free with a CI all the time. It's gonna be a situation where you're going to configure your cluster to have VK as a any number of PK nodes, but something that you have to configure to to wire up.

B

Where does the first people that run by default? It's gonna run on the cluster so like if we go back here for a second one way to install it, just generically is to use the helm chart and the Helen chart is gonna, create a deployment for virtual cubelet, so just a regular kubernetes deployment. It will start up that daemon that registers itself as a node. You could run it externally from the cluster as well. Assuming you can handle all of the TLS and the authentication and connectivity between VK and the API server.

B

Any limitations over with service mesh solutions- I, don't know we haven't really tried it with a lot of mesh solution.

C

We try this on about the cloud you can actually use. East Hill on top of virtual cooperate. Best coordinates. It works very well yeah. The only issue it'll fix is about the DNS part. That's why we said we have the waiking project to handle the ingress thing. Guy stuff, that's only blocker! That's.

B

Pretty cool: let's try that.

D

Any more questions.

B

No all right go for it. We don't do static. Pods I had that question. When we were in in Seattle there's nothing that really would prevent us from supporting static pods. You could probably call VK directly and try to create something that was analogous to a static pod, but since it's really driven from the API server and the static pods, don't end up in the API server, I mean it'll, try to create like mirror pots and stuff, but you don't create them via the API server to go to the cuboid the same way.

B

So it's really driven by the API server right now, like a controller, would be from some other resource.

D

B

Well, thank you so much for coming today.