Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2017 (Austin), 15 Dec 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kata Containers: Hypervisor-Based Container Runtime - Xu Wang, HyperHQ & Samuel Ortiz, Intel

Description

Kata Containers: Hypervisor-Based Container Runtime - Xu Wang, HyperHQ & Samuel Ortiz, Intel

Kata Containers is a merge of 2 hypervisor based container runtime efforts: Hyper's runV and Intel's Clear Containers. With Kata Containers, each container is hypervisor isolated just like an EC2 or GCE instance. It is an OCI compatible runtime and as such can seamlessly work with containerd or hyperd. Moreover it fully supports the Kubernetes CRI APIs and thus can run and manage hypervisor isolated Kubernetes pods through CRI-O, containerd-cri or frakti. Finally, Kata Containers is a multi architecture project as it supports x86, ARM, Power and s390x platforms.

During this talk we will describe the Kata Containers architecture and how it drastically reduces the virtualization overhead in order to be as fast as a namepace based container runtime while being as secure as a legacy VM. We will also run a multi tenant Kubernetes demo in order to show how Kata Containers could become the cornerstone of a secure, infrastructure free, container cloud.

About Xu Wang
Xu Wang is the CTO and Cofounder of Hyper HQ, which contributed the hypervisor based container runtime runV (secure as VM, fast as container), and provides a runV based container native Cloud. Before founded HyperHQ, Xu worked in a public cloud in China since 2011 and was working for China Mobile cloud team during 2007 to 2011. Xu had experiences on Linux Kernel, virtualization, container, and distributed storage system. And he is also an technical writer and translator on Linux, virtualization, NoSQL etc.

About Samuel Ortiz
Principal Engineer, Intel
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Grafton or everyone, let's begin the topic or how big Escada containers. Some of you may notice that in the morning's morning's keynote I might have you introduced to Qatar, and we are. We are part of the from the Canadians and I'm Siobhan from from Heber, and this is standing for me. Girl, hi,.

B

A

And so I might give the background of of Cara, and we will keep some detailed introduction of it. So this next topic we will introduce, will have three parts. First, we will give an overview of this project and then we go into some technical details. At last time.

A

We will, we will introduce some something about the protagonists self and how you'll contribute into today's project and Sammy will is my co-speaker and the first of the I will give some brief introduction and the history of the projector and the SME will give some vision and the architecture of the current Kotter project and advice. I will give some some precise details of the of Kara itself and then I will give some givataim.

A

Oh that's how the kind of California Water Rats container together work together with kubernetes and give us give a simple demo demo and at the last of you, give some contributors one for the history. The color containers come from the two two projects from hammer and inhale.

A

We both announced the project in May, 2015 and habit as I shall not. We not have have a have a diva and part of it is Ravi, that's similar to Lucy and the race is run, run container and Ravi. You run the you run, the the container inside a VM and the the in in our crew container and all the others I think. At the same week, yeah that's yeah. We have like, like a team. Two brothers doesn't.

C

A

And in the past two years we have many interaction in the between the two projects, because that's quite similar and and all the audience may know the two project where Oscar question was the difference between you and what the strands and the weak point with each other's and we want to want to eliminate the difference and make may creep some some by some means exchangeable. So people can confuse use, the module adds container technology. It settles to talk to Troy, so I was thinking which one is better. So we began. This began.

A

The merge I think it's pretty early last, not a year. Maybe that's a year. We we have proposed to to to some something made McCord more probable at first, it's, the yin VM part, the caster, the caster Eden, and then the share the same protocol between the VM and the host, and we also proto Aussie IOC s back so and in in this September. We have we have made with in Portland, and we finally decide to to accelerate the merger of the two project and and.

A

And in the in the in the in the pasta, I think three months to be working together, we working together and try to push push the protector forward, and in yesterday morning, yeah or 7:00 a.m. I think we announced the the more project and we are we to protect, come come together and it's now no lights shift the mic magnitude yeah.

C

So what do we want to do with Kara containers? There are some technical aspect of the project and there also some non-technical ones on the technical side really, and- and this is why we merge- because we we have the same vision and we're going for the same thing. What we want to do is running lights and fast, VM based containers and by VM based containers. We mean each and every one of the container is going to burn in a full virtual machine.

C

So the end goal is really to merge the two technologies, clear containers and hyper-v together in the same codebase under the same repo, we want to seamlessly integrate with kubernetes. So today we do that on take containers, and we also do that on run V. But the end goal obviously should be the same for kubernetes it's going to be multi architecture compliant. So it's not only about x86 yeah trust me so we're going to today we support x86, run, V supports more architecture and the cat.

C

A container and implementation will support more than x86, and we will also support more than k vm. We won't support, k, vm, then, and well, that's pretty much it so yeah, that's the technical vision.

C

If you look at the two containers, the two runtime sorry, they have a lot of features in common, but they also have a quite a few features that are that are specific to one or the other and really the the the objective of this is to make all those features common and merge into the same the same codebase. So things like multi architecture and multi upper visor is something that run v is very good at things like direct device assignment.

C

Sree multi-os is something that clear containers supports today, so I'm not going to go through all those features, but the idea is really to have those these join set of features merged together into one specific implementation which is Kara continues. We also have non-technical goals, it's not only about merging code and working together. It's also about being a vendor-neutral project, so clear containers obviously was Intel tainted and we don't want we we want to go further than this.

C

We want to be open, we want to be venture neutral and we want to be under neutral umbrella and that's the OpenStack foundation, so we currently manage at the OpenStack foundation. Something really important to to highlight here is that this is not an open stack software project, so it has no dependency on the OpenStack software component, so we under the OpenStack foundation and but we're not depending on the opposite act code itself.

C

So that's important to highlight and really the final goal is to have everyone that's interested in using vm based containers working under the same umbrella, which is Kara containers.

C

Ok, so let's go into slightly more technical talks. If you look at containers today and how they're run in the cloud, this is a very high level diagram, but the idea is to show that you have your containers running typically under the same virtual mode. So you spin a virtual machine and you're going to run all your containers inside this virtual machine. One really important thing to notice here is that all those containers are sharing the same kernel, so they're running under sink kernel and really the isolation between all the containers is a software construct.

C

So your containers are not reaching other containers because the kernel prevents you from doing so. So it's a soft isolation. It's not a hard isolation. What we propose with quatre containers or hypervisor based containers, is to run each container it sign inside its own virtual machine. So one big difference here is that each container runs on top of its own kernel. So you don't share the kernels across containers anymore, and you have a hardware, isolation between all containers and no longer a software isolation.

C

So it's all about security, and one thing that I want to say that doesn't show in this in this diagram. Is that in a kubernetes context, since this is a kubernetes conference, it's not the the the isolation unit is not the container it's the pod. So when you run a community spot, your party is going to run it into its own virtual machine and each container inside this party is going to run inside this virtual machine. So there's not going to be one virtual machine per container, but one per pod.

C

If you run just a docker container, it's going to be one virtual machine per container, but in the community context it's really one virtual machine per pod, and so yeah, each container or part is going to be hypervisor isolated. And at that point you have the same isolation level that you have with virtual machine again it's now: it's no longer software isolation, it's a hard one.

C

We I mean when we talk about virtual machine people, think about starting there good old legacy virtual machine in five minutes. This mega images that takes gigabytes of memory and so on. We don't want that with carry containers. Obviously we want them to boot really fast and we want them to be very small. So we're going to talk about this a bit later, but that's that's really!

C

The goal here make it extremely fast to boot and make it as small as possible, and finally, we want carrier containers to be seamlessly integrated with the rest of the container ecosystem, with kubernetes, with docker with OpenStack. Whatever your software stack is, we want them to be integrated transparently, so you won't have to change anything on your workflow and really the goal here.

C

As a matter of saying in his keynote today, you basically have to choose between speed or isolation when you have to select between a container or a virtual machine or running a container inside virtual machine, and what Kara is trying to address is to be able to run, have both speed and isolation. So we're feeling this gap here. You don't have to choose anymore, you can run cattle containers and you get the typical legacy.

C

Hardware, isolation and you get at the same speed as you as you have, as you used to you with with regular containers or native continuous, whatever you want covered so more technical details.

C

So those are all the components that make cattle containers today. So we have the runtime. As I said, it's a runtime, but we have a few other components and but before going into into this a bit deeper I really want to highlight something that Kara containers integrate at the. What what you guys are familiar with at the run sea level, so to use Kara containers, you're, not gonna, have to replace docker or you're, not gonna have to modify or modify kubernetes. You just have to specify another runtime something different than grunty.

C

Another point that I want to make clear is that it's not replacing run see, but it's living alongside 1g, so you can have run, see and color containers living together and you can run VM based container alongside with namespace containers, and they would just work together. So we're not trying to replace anything, we're not trying to replace docker or we're now trying to modify communities to use color containers. It's really important to us for us to integrate seamlessly with with all the container ecosystem.

C

So here so you have all the higher level Tucker communities and OpenStack and on one side they send or receive IO so SD in a CDR STD out, and this isn't all bye-bye shim. So all those components are actually talking to to a shim because they expect to talk to a process and not a virtual machine. So we have a shim sitting between your software components, kubernetes of docker and the actual virtual machine that holds the the container and the runtime is the one handling all the OCI command and all the OCI specification.

C

So one really important thing is that, yes, the runtime is OCI compliant, so we talked OCI and we get OCI commands and specification from the high level of the stacks.

C

Then we have a proxy here in this picture because currently we're talking to the virtual machine through a serial interface. So the red arrow here that you see is a serial link, the proxy, the Shimin, the runtime took G RPC and the proxy. Basically multiplexes and demultiplex is everything through a serial interface over a component, that's called the AMEX. Then it goes into the hypervisor.

C

Talk to the colonel and we have an agent running inside the virtual machine, that's actually responsible for spinning all the containers and all the pods inside the virtual machine, so I think as the agent as a stream down really minimal version of Francie. So we have a very, very small, R and C running inside the virtual machine and it gets OC I commanded specification and spins your containers inside the virtual machine.

C

Another version of this architecture, a simplified one, is based on vizac, which is a basically a virtual machine, providing socket semantics through the through the host, and then we don't need the proxy. So it we have the shame and the runtime talking directly to the hypervisor without the need for a proxy inside the virtual machine. Supporting this talk is easy to do, but outside we have.

C

This talk is a fairly recent addition to the kernel, so we can't just go off full V Sox, because we can't expect all the the hosts and all the distro to actually support this so today, but for those who do we're gonna, you can just keep the proxy component and have a simplified architecture.

C

Sure you wanna yeah.

A

Yeah Sam will give you the whole picture and the architecture of the of the cotta container project, components and now I will give some some detail how it how it could be run as as fast as I can. As a container.

A

You know from the kubernetes view manual, don't launch up how they it's as part of to prepare that the images the route I fast, who for the container and the other the other side, is to prepare the sandbox for the for the for the pod and for the for the traditional existing Kennedy will they will do two days serially and because the container is quite fast and that's that the main the main consumption is the route half as prepare region and for the photo VM Harvey.

A

Let's accelerate it by para, para, to put to the bozo part, as you know, and asked Suzanne to submit to some job and the lightweight VM and make it make it could be run in about maybe 100 hundred millisecond yeah and that's do consume some time and I the same. At the same time, we prepare that would have us, that's the app I.

A

Will ever ever Paris about the sandbox prepare region and the Tom Harris about the bottom hairy about the volumes volumes and ruled have has prepared, and then we used the hot rag to make to make it together and launched the start that our containers inside the inside the pod yeah. So.

C

We applied a lot of things into the.

A

C

Start really tiny virtual machine and, and, as we know more about the the pot that needs to be created stuff is a plug-in to into.

A

The ocean- yes, not only about those- would have SN the what it was about also about networking and they even bought about the memory and a cpu hot pack. So you could you could have the you co, you can you have. You could prepare the VM beef before you you, you pass all the auto specifications because you could hot pry instead of the CPU so later, and this is the how we make it faster and and make it's more. We use the minimal would have as an internal and also we have every me introduce them.

A

Bm, temporary, complete technology, that's like to be informed and make make the oh. The older vm share shared the same part of the shared binary part. That's really only read-only part, and so you don't you don't have to to pay the pay. The taxes for you to be three amps packs the power and the inhale inhale, introduce an idiom based at the ADX technology. That's that's just in place running technology.

A

You you just to do the memory map, the map and redeem device to the to the memory, and you don't need to allocate the real real memory so that could save your memory and and also we, we could use some KSM technology to do more aggressive, American memory, saving seaming seemin here and by use of these technologies or ponder how we have tests about the secure container technology. It could reach about ten plus density than the traditional VMs yeah, a still still some some consumption compared to the to the Linux container.

A

But it's very high capacity than the traditional VMs. It.

C

Just just want to highlight something here: it something we notice is that when you run really really big workloads with the case I'm enabled you actually get better density than with the typical native containers, because we we actually managed to deduplicate a lot of memory pages across virtual machines. So it's not a binary overhead. So sometime, the overhead is actually negative. Yeah.

A

If you have, if you have many CPU cost you can, you can align the details, job to the to the tutors to a specified cost and that's Chris could save the save the memory aggressively yeah and about the networking part. That's that may be well the the most of significant difference between the containers, content, containers and and VMs, and so we have. We have some different matter here and, firstly, the curio containers team have have the mikvah tap to bridge the ETH a pair to the two that have device, and also we do some.

A

Some tc2 do TC rules to do the traffic passing traffic transfer. Actually, you you can do you can do other for the photo run. We we also have a specify the interface. If you, if your CI script, no, there is no way I'm based at VM paste. The container you had directory a plug, the plug that F device to the to the VM C.

C

And I usually don't care about VM. So that's it! That's our problem. Yeah.

A

But for time being they will and the storage part. We is a piece of part of volumes either either from the broth device Oh from the life he applies to do the filesystem tell system sharing so.

C

One thing to highlight years did with vertigo block when you have a block device storage on the host. The performance is actually much better than with nine PFS. So and it's it's it's pretty close to bare metal or native performance. So it's a much better setup than not using block device storage on the host. Yes,.

A

Yes, actually for block device, even for self, like networking, proper devices, we will have some performance comparable to the truth of to the hospital to the hospital and for the for the night be advised. There is some some pity part it's it's slower than the native native file systems and they have some some bass and the on the project semantics, but we have fixed the Sun, some of them and some.

C

Of them you yeah.

A

Yes, and and for the for the for the user of the of the color containers, the color containers introduced them.

A

Some change to the to the existing house huh see the kubernetes muck scheme and the the left side is the currently we we set up kubernetes cluster for different users and they use different communities masters for their for their for their of EMS and and the right part is what can I do that? Also what what we try to do? We use the Kara containers, use the VM to isolate the part itself and use the the networking part to to give the multi tenant network to the users.

A

This just doesn't like the a Douglas fir gate. You know, make the user use the kubernetes mr. Cooper nice api's and have the native kubernetes api, but they told me too many kubernetes masters and they have their own users. They have multi tenant. You know it had in the correlated so.

C

There's this provides you with the CPU and memory multi-tenancy, but the the networking part, if you use it, just your regular C and I plug in its it's gonna, be still as soft multi-tenancy implementation. So you need to have what she was going to talk about that and.

A

Here I prepared prepared a timer for this there's another another project called step. Poopy. It's it used, yep it used. Virtualized can handle technology to provide an Audi tan, multi tendon, the Corrin, easy straw and a semi imaginated before the this car I itself did doesn't depend on any kubernetes and any I think OpenStack components, but the existing of OpenStack OpenStack the whole stack give us some some convenient convenient and when they're neutral, when they're neutral services such as neutral in the cinder and they work they working in the open.

A

You know open scheme, so they they provide a vendor-neutral interface. We could use use dance together with the character in the nurse and the kubernetes to provide multi multi tenant. Using here is a as a simple walk screaming from the from the could control you can reach the the coup. The coup Bernie's master, the API server, and then you can. You can allocate your attendance from from custom customized that result results.

A

That's based on the the opens the OpenStack Keystone and you can allocate network through the nutrient and provide the layer to isolated network for each tenant and we used the Cabrillo fractal. That's a CRI CRI CRI server to call the kurakin in there's also other rent helps to create containers and.

A

Let's do the demo itself.

A

Where's my terminal right here.

A

B

A

So the let's do the you can find I do the to the demo, just anger, Google, Cloud and.

A

And that's quite simple: let's see we have two llamo Ramirez is dismal.

A

That's great it's much much bigger than the ninth.

A

A

So that's that's very simple: we will create a user called attending a cordon Tesla and we just we just create it and I prepared a script here. So I can just place it. Yeah that's created.

A

So you could, you could find either as a last line. We have the types of and it's actually a sick sick secondary ago, and we have to read another one.

A

Yeah ii is created.

A

Yes and after we correct right, the tendons, less, I see about a network be a bit longer.

A

You can find there is a network for the for the for the namespace Estevan, and you can see that the network is the advices.

A

10.24 4.0 to 0 and for another user.

A

The sims to similar network network address area because we have the independent, independent layer, 2 network, so they can, they can have overlap, address range and.

A

Let's see, what can we do.

A

We could we could just a create, create an and part of that we run a single container for the Ubuntu and that's do something really bad yeah I think you have. You have seen this line, maybe instead overflow to do a two or four coupon, and this line is quite beautiful. So there's no no kept the letters at home.

A

You can rename the current word 2020 was to any words, but here it's let's do something like like a black geek yeah, oh yeah, that looks normal, and now you can't do anything that, because it's for it, you use the pipe to do the fork and fork and fork again now my name any the the process here and it will kill the it will killed up the part itself. But I did because it's raining in the single VM outside of VM.

A

Does everything is okay and you can do anything, for example, to kill the VM yourself, so the the VM has use the VM talked about technology. It. It means even you even some somewhere more. Some parties is hacking about by better by the others. I, don't have act on a house or affect other parts, so you can, you can simply just queue. It cue it offset and.

A

When they deleted, it will be will be reclaimed and for the next select, let's show the let's run some some hot.

B

Still here too late late it to be to be to be guarded, you collect, collected and.

A

B

Great test you.

A

And like to see the pouch.

A

Yeah, we will create a service and the crew is create another one, and then we could see them.

B

B

A

And so for for the for the test America we can see, we have we help out for the for the for the ending see, and we also could run the part to to check.

A

We could run a busy box.

A

When we launch a busy box that we will try to try to reach it reaching the different services and the because we have a multi-tenant network, it can only reach the Beco. You stop the cart.

A

It can't reach it selves, but it cannot reach the others, that's bad name, and so it's that this just usually the about it can network. You can see what what I do my operate. That's that's identical with the standard of kubernetes that what we will say, that's that's a pioneer our kubernetes wisdom, plugins and tech. They can run on top of a local, a containers and we saw the animal the vacation photo for the community itself, so I think could cover.

A

Neither you know should be feel comfortable about that and because the time is limited, so I will finish finish. It soon. I think yeah.

A

Yeah, that's that's the the standard image we does appoint from the from the.

C

You don't need to modify the image. It's just standard.

A

You don't need to do any modification for the images yeah and for the next day in the in the first half of next year, video we will have the first one point: 1.0 release and that's all the CRE integration, integration, support and and some some OS when their support and we can.

C

We take some questions if you want to get involved, there's a github repo.

B

I think it's yeah yeah.

C

Can you put the information, then we take yes, we take some questions. I think we have a few minutes left so questions on the audience. Okay, this.

B

C

C

A

So that the thymus episode we only think may be mild to castrate yeah.

C

Just a couple of questions: sorry with we have a booth, you can come and ask some more questions. Yes, so.

D

When you mentioned the 9p versus vallejo block into the VM I was wondering how how our image pulls done and is your local image cache on the hypervisor? Is that configurable or you know the choice between 9p or Bernal block based on that? No.

C

It's so really it's about the the storage overlay that you use on the host and everything is cached on a host. So we don't cache the images on the on the on the VM itself. We just exported from the host into into the VM.

D

Yeah but I'm wondering how you pull those, so you like normally do a darker poll or or yeah.

C

You still you see, look energy would.

D

Do that so do you have is one of your components.

C

D

Doing that poll no.

C

A

Poole is done from the hosts at the CRC. Rit months will put it yeah for the images for you and the prepare it, and it depends on your configuration, ekoo storage as a block device or as a system. Ok,.

D

So there's still dependency on like container D or Docherty, or something.

C

So so, if you integrate with cryo, for example, there's no dependency if you integrate.

A

With continuity's.

C

You're right, yes,.

A

Yeah for the photo continuity, it will be depend on Kennedy for this here, I. Oh, it's you CIO and for active. It was perfected. That's that's all the different different across a sec RSC RIT months. Another.

C

A

E

Not sure I'm wondering about are do give us the capability to manage the so-called lightweight it on VMs, their kernel and everything. I know that you know ROM be, and you know the hyper start will get the kernel and inner Rd from barley, something on. We have a use case where you have you know multiple kernels yeah required for different news cases they've imposed. So that's.

C

That's wrong: that's a long story, but ideally we would like OCI to to provide us with the kernel and guest OS images, and this is something we're trying to push into OCI. But.

D

C

Run v on three containers and cata containers will be able to support multiple kernels per pod, so you can run all of your parts with different kernels that you have provisions provision, internode, yeah,.

A

Because your quarter team come from the post already and a clear container what you're carrying value UK carried it in containers, and we could do it yeah. Thank you. Thank.

C

E

Thank you. We have a booth.

C

You wanna, we are both.

A

Old wishes ask fabric that yeah.