KubeVirt Talks, 22 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: KubeVirt Deep Dive: Virtualized GPU Workloads on KubeVirt - David Vossel & Vishesh Tanksale

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

KubeVirt Deep Dive: Virtualized GPU Workloads on KubeVirt - David Vossel, Red Hat & Vishesh Tanksale, NVIDIA

KubeVirt is a Kubernetes extension that supports running traditional Virtual Machine workloads side by side with containers. In this session we will explore the architecture behind KubeVirt and how NVIDIA is leveraging that architecture to power GPU workloads on Kubernetes. Using NVIDIA’s GPU workloads as a case study, we’ll provide a focused view on how host device passthrough is accomplished with KubeVirt as well as providing some performance metrics comparing KubeVirt to standalone KVM. You’ll come away with a high level understanding of what KubeVirt is capable of and the general design principles that drive the project.

https://sched.co/VnjX

A

Morning, let's go ahead and get this started so I'm David fossil from where the core contributors to the keeper project today I'm joined with vicious excel from Nvidia, and what we're gonna do is I'm going to jump into this by talking about ski pert. What it is talk about the architecture a little bit am I hand it over to vicious who's going to talk about how Nvidia is leveraging Qbert for their GPU workloads.

A

So the kick things off and others a little bit backwards. I want to talk about what Qbert isn't so inevitably, I get these questions after my presentations and I just want to go through them right here in the beginning, and maybe I'll give some one who has these questions their context of what we're doing about just knocking is out real, quick, so, first off Qbert, it's not involved with declaratively managing your AWS or GCP, or cloud virtual machines. We're talking about ringing KBM workloads on kubernetes nodes.

A

Next up key Burke's, it's not a competitor to spend like firecracker cata containers, we're not in we're not targeting this micro VM isolation, use case at all and last off. Qbert, it's not a container runtime, so we're going to work with whatever container runtime you guys are already using and that's how we play nicely with things like the network service, endpoints and persistent, buying claims and things like that, all right. So what is keep vert then this.

B

Is my definition.

A

Keeper, it's a kubernetes extension that that's gonna allow us to run traditional virtual machines, natively side by side with container workloads and what I mean by traditional virtual machines as I'm talking about the same types of use cases you might have used, AWS or GCP, or even OpenStack, for in the past, talking about taking your traditional Windows Linux virtual machines, bring them into Cooper Nettie's and managing them in a cloud native way next to your container workloads.

A

So when you're thinking about the kind of use cases you can solve, keep Bert we're in this space of solving the same kinds of things you might have done with public from this cloud provider api's in the past.

A

How would we do this? I mean we already have things like OpenStack and overt? We already have the public clouds, so why are we reinventing this virtual machine management stuff all over again so Cubert? The initial motivation for it was this idea of infrastructure convergence, so we saw, as people were transitioning to the cloud native container first way of doing things. It wasn't always a clean transition, so sometimes they were left with two stacks. They have one for the new container workloads and they had a old legacy stack for their virtual machine workloads. Operationally.

A

That's not great. So, of course, Cubert. We have one stack, so people can transition to their container workloads, bring their legacy VMs with them and they're going to then we're gonna reuse. All the same logging metrics moderating the same cluster networking storage. Things like that and this use case is proved useful for people. We see people doing The Sopranos. Now it's not it's not theoretical. At this point and as time has gone by, we've actually seen some kind of new potential emerge for the project.

A

So what's happening is the operational story around how kubernetes this deployed managed and it's scaled, especially on bare metal. It's put Qbert kind of in the crosshairs so being considered for virtual machine only use cases. So with addition of things like the cluster API and the Machine API, what we have is COO Bernays managing its own underlying infrastructure, so kubernetes, it's re super efficient advantage in your workloads, but now it's becoming a super efficient and managing the underlying infrastructure. Those workloads run on.

A

So when people are looking at places to run their virtual machines and they want to leverage that kind of infrastructure management, Cubert makes less sense for them, so it's becoming just a good place to run virtual machines stand-alone and then there's also the operational management of the virtual machines themselves so, like today, you're using a declarative approach for managing your pod workloads, we're doing the same thing in Qbert kind of make this tangible to you.

A

Here's an example everyone's familiar with of how to declare a pod you, post into the cluster, using standard, cube control, client, tooling same thing with our virtual machine API. So we one point here: is our virtual machine API? It's! It maintains the same, look and feel that you're looking for with kubernetes. The main differences, of course, is our specs gonna. Have things related virtual machines and not necessarily containers so I know you aren't posting like 10,000 pods individually to your cluster.

A

Instead, you're gonna use something like a pod and a pod template and a deployment and scale horizontally. That way, we want to maintain those same kind of patterns with key Verte, so you can take the virtual machine instance put in a virtual machine instance: replicas sent it's just another API that we support and you can scale out using similar patterns, the are with pods using that API, so big takeaway here, man sure beams like a manager pods today, all right that was a quick overview of what we're doing with key perks.

A

I want kind of look under the hood now and talk about how we're doing this so made the statement earlier. The Cubert is not a container runtime and if we play nicely with things like network services and things like persistent volume claims, how are we doing this? We're doing this because we play by all the rules, because at the end of the day, Cubert it's just running a human process and a pod? That's really just about it, a little bit of feedback going on here, all right: let's kill it good all right!

A

So it's Hakeem you process in a pod, I.

A

Don't want to trivialize the fact that there's a whole set of like api's and cluster components that are involved with everything out to the point of launching that king of processing, the pod at the end of the day, just a process in a pod, so the cadby mystify everything's involved to bring up that process. I'm gonna go through the virtual machine. Launch flow starts with buying a virtual machine instance and a manifest, and you standard keep control clients command to post that manifest to your cluster.

A

That's VM objects, gonna hit the API server, it's gonna be persisted and Etsy D, and then what we have is a keeper cluster controller called vert controller, which is watching for your VM objects. It's gonna be informed. There's new new virtual machine objects been posted to cluster and a response to that. It's gonna create a pod for that virtual machine to live in.

A

So this is the place where, if you defined, for example, the moment and virtual machine with two cores and eight gigabytes of memory that you're going to have things translated from the virtual machine instance spec to that pod definition.

A

So, in that case, we would have resource requests, one that's on the pod, reserving those resources in the same sense, if you wanted to give your virtual machine dedicated GPU, we're gonna talk a lot about here in a second, you would define that the virtual machine, which should be translated into a resource on the pod definition so for controller, it's gonna post that paw of the VM to live in something gets scheduled using your standard scheduler, it's gonna wind up on a host that has access to all the cluster resources.

A

You need it's going to start up and what we have here are two components that we need to talk about for handler. It's a bridge privilege, daemon set. That's gonna exist on all compute hosts in your cluster and vert launcher, which is it's this tiny shim process. It's only responsible for bringing up that qumu process in the pot, so bert launcher it's a dumb process has no privileges, privileges accepting facto. It needs to start that process and have no access to things like the kubernetes api server, or anything like that.

A

So the pod comes online, bert launcher it's. The entry point at that point, bert handler is going to pass over all the arguments that are needed to to start that queuing process. So bird launcher gets all the arguments it needs. It starts at that kim you process and what we have is a virtual machine running in a pod at this point, that virtual machine has actually assumed the role of that pod.

A

So if you paying that pods IP address you're actually going to get a response from the virtual machine guest and it's gonna see that it has that IP address.

A

So what do I do now is I kind of wanna zoom in on that pod and talk about how we're delivering things like storage network and access to host resources.

B

A

With storage, if you want to give persistent storage to a pod, you're gonna use a persistent buying plane. That's just how it's done so. Pod is gonna, be decoupled from the stateful storage, with the persistent volume claim we can use, since virtual machines are just processes in the pod. Can leverage these PVCs to deliver things like virtual machine images in a way that make them accessible to that key new process.

A

So, for example, if you have a virtual machine image on your laptop, you can upload that image using the containerized data importer, which is another project, we're checking out. You can upload that to a PVC and then we can attach that PVC to the virtual machines pod and make that image.

A

That was just on your laptop accessible to a kimi process and we watch IBM with it so and result here is we have virtual machines where we're delivering the virtual machine volumes using persistent volume claims and if you stop and then start your virtual machine, we've decoupled the virtual machine process from those volumes. So if you make rights or do anything to your virtual machine, those changes will persist across multiple start/stop events and with regards to networking.

A

So in that pod we have access to the the CNI allocated network in their faces, and we can use that interface to tie the virtual machine directly into the pod Network. So this is how we're playing nice with your network services and service endpoints, and things like that. In addition to this, we support the ability to time multiple CNI allocated network interfaces into your VM using Malta's.

A

So this is going to give you the flexibility to bring your VM attached into whatever network you want, or even you know, multiple networks and lastly, moving on I won't talk about hosts resources so again we're just kind of leverage, existing features and kubernetes to provide host resources to our virtual machine, CPU manager. That's how we're pinning virtual machines underlying course, alright, and then we also have topology manager, which is what we're leveraging to tie in Douma Efendi and follow this.

A

But that's a bit of a work in progress right now, both upstream and the kubernetes community and us consuming that with guests CPU and with guest memory, we're doing something similar. So we can guarantee a certain amount of memory for our virtual machines by using pod requests and limits. And, lastly, we can pass host devices into the pods environment using device plugins.

A

So this is how we're passing in things like GPUs, SR, IV into that pod, environment and, ultimately onto the virtual machine and the nice thing about Vice plugins is we can do this in a way that doesn't happen, grant that pot elevated host privileges? It also ties back into the scheduler. So when you're requesting these resources on the pod, your pods only going gets scheduled to a place that actually has these resources available. So there's some bookkeeping going on on our behalf.

A

Here all right flew through that that's peeper from like a 10,000 foot view I'm. We might have time for questions. I, don't know I'm going to the cube for know the Red Hat booth after this and I can answer it. Keep your questions or anything like that or you can catch me afterwards for now. I want to hand it off to vicious who's going to go through the GPU use case. Entity is looking at, and hopefully you don't.

C

So, thank you. David and good morning, everyone I'm vishesh I've, been working at the angriest one year, so going you know going back a bit. Why are we talking about VMs and a cube? And it is confidence.

C

Beams are critical for us at an media because there there's this big push to containerize GP workloads, but not all GPU workloads can be containerized, especially GPU workloads like gaming or professional graphics. You can't antagonize them, you still need VMs for them and that's why having containers and VMs together is kind of very critical for NVIDIA and that's why we stumbled upon Qbert I'm, sorry, which kind of helps us in that use case.

C

But Qbert lacked one of the critical features which we require, that is assigning GPUs to the VMS and that's when we decided to work on it and we have recently upstream the not change so as David mentioned, to pass on an host resource to a Qbert VM. The natural choice is to have device plugin and that's why we developed this cube or GPU device plugin, which is open source. Now you can take a look at the details of it on the link below. So how does it look?

C

Where does the GPU device plug-ins sit in the whole stack, so at the bottom you can see there are these GPUs you can either have a vfi of PCI driver attached to the GPUs. A vfi of PCI driver would allow user space process to have direct access to that GPU. Here the user space post process is a cube. Word minion.

C

If you want to split the GPU into multiple vgp used instead of a vfi or PCI driver, you can have Nvidia, vgpu manager and NVIDIA GPU manager would split your your physical GPU into multiple V GPUs and each of these V GPUs. You can then pass on to your Qbert VM and we have BGP manager, creates mediated devices, and these mediated devices are passed on to the Cupid VM. On top of these drivers sets the device plugin, which we have developed a device, plugin kind of abstracts the details of the GPUs.

C

It kind of understands the GPU topology on the host and exposes it out to cuban artists. Cuban artists and cube work together, use these devices which are exposed at the device plugin as Java kind of explained. It creates a Hubert pod that Cubert power has a new process in a tank that King new process has the GPU, and it creates me him going into bit more details of what the device plugin does so typical Cuban at his device plug-in would have these four functions. First would be discovery.

C

The device plugs and discovers about all gpgpu devices are there on the host. It then advertises it in a manner that your cubelet can understand once it does that it waits for an allocation request from the cubelet and in the meantime it is also doing a health check on the devices in which it has discovered we'll go into the details of how it does this in the later slides, I think no cube con presentation is complete without some Yama's. So here are my yam is for you.

C

So once the device plugin comes up, it kind of exposes that it kind of picks and it advertises those GPU devices to a cubelet. The node shows those GPU devices allocated by devices. You can see that on the left that there are five GPUs on that node once you have that you can schedule a virtual machine instance which is requesting a GPU and it will get automatically scheduled not automatically, but the cube word scheduler would kind of schedule it on the node, which has these device, which has these GPU devices.

C

So the the request for the GPU on the virtual machine instance kind of gets translated into and limit and request for the resources on the boat launch a pod and that word launch. A pod is scheduled on the node and then VM gets treated now going into a bit of details of how the device plugin actually does its discovery and advertisement to the cubelet. First, the device plug-in when the device patent comes up. It reads what all PCI devices are they're out of those it filters out the nvidia GPU devices.

C

Then it checks what kind of driver they have. If they have a vfi or PCI driver, it will group all the similar kind of GPUs together. It will also read if there are mediated devices created by the vgpu manager and would group all of them together.

C

Once it has identified all these devices it will, it will register itself, it will create a G RPC server and it registered with the loop accumulate on that host and it will kind of advertise that hey I have this kind of device and these many number of them once the device plug-in. Does this it kind of waits for an allocation requests from cubelet?

C

In the background, who is also doing an help check on all of the resources that it has advertised to accumulate to ensure that these devices in which it is kind of managing once it does that, then the device plugin as I mentioned, is just waiting for an allocation request. Now the allocation requests would come in form of an creation of an cubed word me and I, so David kind of explained it a bit I'm just going to close over it that you do a you request.

C

You you pass on to the API server request to create a VM I the, since it is a CRE, the word it contacts. The word controller, the word controller kind of translates that VM I expect into a pod, spec and cube node is identified and once a node is identified. That's when the device plug-in again comes into the picture now.

C

Tubular could realize that some of the it is requesting a GPU device, and it knows that the GPU devices devices being served by the device plugin, so cubelet would contact the device plugin. This is what the allocation request is so cubelet would contact the device plug in and ask for a location of the resource. Once that request comes in device, plugin will decide which of the GPUs that it is available it would allocate. It would respond with an allocation response. This allocation response would we inform a fan.

C

Pci address the device plug Intel's cubelet that this is the pci address, which I wanna locate and you have what the mechanism we have used as we set an environment variable in the corresponding pod that would be created, so the cubelet would then create this pod. This pod would have, and one variable with the PCI address once the word launcher part comes up.

C

It reads: this environment variable it gets this PCI address and when it reads the specie I address, it translates those details into a parameter into the lymphoid Dominic's, and so the it reads this PCI address translator translates it into a parameter into the Dom XML passes to the libvirt and bang labert creates a VM and the VM has the GPU, but which has this PC address. That's all that is to it. And now you have a cube, word VM, which is running on a cube and this cluster and it has GPU assigned to it.

C

You can maybe run your games on it. You can play your games on a Cuban. It is clustered if you want going ahead. This is what the GPU pass-through is all about. On Qbert I just wanted to gloss over some of the use cases or some of the features of Qbert which we are using at an media, so we found that Qbert is, is really feature-rich. On the networking side, we are highly invested in using SR IV support and the multi support that has been like a critical feature for us.

C

On the storage side, we are using the PV PVC model of storage, which kind of works really well for us, it's kind of consistent with what you give to a container, so it really works well for us. Another critical feature for us is the block PV support. We want to do desk pass-through to the cube for VMs and block P me support kind of becomes the critical feature which is there on Cuban, otters and Q. Bert also supports it, so we kind of get a get to do that pretty easily.

C

The containerized daytime porter is also really important for us. We use it heavily. It allows us to have immutable disk images. Another critical features, the copy-on-write, this support, cube Cubert, has some support for it. We planning to modify it to to be.

C

You know, be more consistent with our use case and we'll try to talk to the community, and maybe we can upstream that as well after that, some of the cuban artists kind of constructs, which are which are inconsistent with the cube, word me and make it really easy for us to have our deployment, something like adding a service account of vm or you know, having node selectors affinities, teams and toleration x', which we, which were used for making some of our scheduling decisions.

C

That really has been a godsend for us, and then there is the cloud and its support, because we're kind of trying to create a form of VM the cloud and support works. Really. Well, it's seamless, it's just the way it works in any other vn management platform. And, lastly, the the there is this on virtual on Shabbat. There is support to add a sidecar and you can tinker with the liberal parameters.

C

We we use it for some of testing purposes or for trying our few features if we want- and we are also doing some stuff on it in our production for copy-on-write this support, we also use what CTL you click about. Ctl is a Qbert utility. It gives you one of the features which we use of. It is the VNC access to the VM. You know for debugging purposes or if the VM kind of doesn't boot up or something like that. We use that a lot. And, lastly, we use the GPU support, which we have upstream.

C

So the GPU support on cuboid VMs for passing through the GPU, as well as passing through V GPUs, to Cuba themes so yeah. This is it for me. I would like to thank you about community for letting me present today and thank you.

C

Here we have time for questions yeah.

C

So scheduling and the algorithms for.

C

C

Think they won't give you my consistent, because I think this is being recorded so.

C

Go ahead, I'll, just repeat your question:.

C

So what I think what he wants to ask is: if you have different types of GPUs, how do you decide which GPU is perfect for your workload, kind of right.

D

D

C

The you can't actually do that. What will happen? Is you just there's just one GPU, just one GPU, which is exposed as a device, and you can't really it's just one. Your answer.

A

C

A

C

Take this this.

A

Is taking too long? Are there any other questions yeah right here.

A

Multiple, virtually I guess I didn't really quite understand the question.

A

ah Seriously, so the question is how to manage multiple viens as a single service. You can certainly write your own custom operator. Don't do something like that. Just like you came with the pod. The only things that come out of the box with key verts with multiple VM management would be. We have a virtual machine replica set which lets you do things at scale, but if you're wanting any sort of custom coordination between multiple virtual machines, you're gonna have to coordinate that yourself. All right, I'm sure.

E

A

C

Great, if you have not tried it out, we have just tried it with the Qbert there. Currently, the thing is: if you want to use the you have a container having a GPU, then there's a different device plugin for it. That.

E

C

Think we can, we can discuss it later on there. There is some possibility that.

A

It's possible that so the question was: if we can reuse the mechanism we're using for Qbert, expose the GPU standalone for container workloads and we'll talk about that afterwards, I, don't think that's been done yet yeah yeah, you can run Windows VM. That was the question yeah.

C

Yeah, that's the whole purpose. I think we mean Windows gaming games and they are work, so they were all fine yeah.

A

C

Linked so you're talking about understanding the topology of the so the thing is you can the whole the whole issue here is, you know we need some amount of support from from Cuba natives to kind of the the topology manager kind of needs to support that once it happens, then we can, you know kind of do that. Math. Support on on Qbert, so Cuba doesn't currently have the whole Numa support. So once we have that, then you know it can translate into the cubed means kind of understanding that and your scheduling stuff based on.

C

Their there there is some discussion going on around it.

C

So you want to say there is a application running on your cube with me, I mean you want to monitor it and check if that application is healthy or something of that sort or.

F

C

So you can obviously have some kind of a monitoring. You know, maybe a monitoring pod that runs on that same cluster, which kind of does a health check, because the VM definitely has the access to all the components of the cluster. So if you want to have something of that sort, then that's possible or the application inside the VM can also you know kind of send its status out to the outside world, where you have some application, which is monitoring it I, don't see.

A

So we have observability into the virtual machine we're exporting certain metrics using Prometheus, so you can scrape metrics that are specific to that virtual machine they're, pretty standard stuff like most CPU used and things like that. We're thinking about extending that to add guest support. So you can learn about when your virtual machine, the actual guest, has booted up completely and things like that. So I'm not sure. That's the question exactly but talk to us afterwards. All right.

A

Yeah sure we're just not targeting so the question was we're similar toccata containers. You want me to speak on that orgy had more.

A

Yeah we're using KVM just like that, but we're not targeting this micro VM use case simply because our API is and optimized for that we're creating an API. That looks and feels like something you might use for traditional virtual machines in the past, and it's not meant just to be this tiny, slimmed down thing that runs your container workloads.

A

It's meant to run full-fledged like Windows and Linux virtual machines, so I mean it does provide the same kind of isolation that you're talking about simply because they're hypervisor is involved, but we're not I mean you could use it for isolation if you want, but that's not what we're really going for.

A

Right exactly that's.

C

Because I have my new shuttle in from the back I think.

A

Oh yeah, so much all right. This guy.

A

Import input, devices and monitor pastor, so.

C

So the day you can, you know kind of, do a sidecar and do hacks around it that's possible, but there is no current support on Cuba kind of do it like from the Qbert API know kind of who you need to build that wrong.

A

All right one last question: then we gotta we gotta, go right here.

C

So there you mean kind of an uma support that you have. You want to have the V GPUs, as well as newer CPUs from one side of the numa or something of that sort.

C

So that is card that I think I answered a similar question before so that currently is not there on cube foot, we kind of need a bit of support from cube and it is topology manager, and then we kind of have to you know pass on to when once you have that support, then cuboid kind of absorbs it, and then we will be able to do it for Qbert.

C

So it's it's at random. Basically, it's yeah, there's no.

A

You're not gonna have the affinity of getting as close as possible to those PC. I address this right. Now, that's what's apology manager is trying to solve specifically for Numa. They were also we're. Looking at PCI devices like this rivo and GPUs as well or you want your CPU, so you're allocating the dedicated CPUs to be as close as possible to those devices. Just doesn't it we're working on it and it's something that kubernetes is working on and then we'll consume it when it's ready.

C

A

Yeah all right thanks.