Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kata Containers Performance Evaluation and Optimization on Arm64 - Jia He, Arm

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Kata Containers Performance Evaluation and Optimization on Arm64 - Jia He, Arm

Kata Containers builds extremely lightweight virtual machines that seamlessly plug into the containers ecosystem. It is a multi-architecture project which has been supported on X86, aarch64, ppc, s390. Jia He will introduce current status of kata containers on aarch64, focusing on the performance evaluation on aarch64. Includes: 1. metrics report: boot time, memory footprint, network and storage io. 2. Real-life test case includes nginx, redis and tensorflow 3. Kata containers vs runc containers Besides, Jia He will also introduce what Arm container team has done to optimize the performance: 1. virtiofs enablement and bugfix 2. nvdimm support and bugfix 3. kvm ptp enablement 4. Misc performance optimization

https://sched.co/ekFW

A

Hello, everyone good morning, good afternoon and good evening, thanks for attending my topic today,.

A

Today, my topic is about containers, performance, evaluation and optimization of arm 64.. My name is justin kerr. I mainly focus focus on the container virtualization technology on arm 64.

A

Here is today's agenda. Firstly, I will give you a broad brief introduction about what is cut container.

A

Then I will give you the status update on arm 64.. That is what we have done so far.

A

Then I will go through the performance evaluation from several aspects. At last I will introduce to the two simple users user stories in real cases.

A

If you have used the dot container, maybe you have thought about the question: how to make the container more secure.

A

You can drop some linux capabilities such such as make you you can mount the root fence with read-only mode.

A

You can use sc, linux and app armor to protect the container and also you can use the sitcom to allow or disallow some system calls.

A

But the more you add the sick, the sick, the middle layer will be, and you will get more performance overhead.

A

So the actually the container is a combination or trade-off between the virtual machine and the container.

A

It is compatible with oci runtime spec, therefore works seamlessly with the dock engine. Besides, it also supports the kubernetes and cii through the ciio and the css container d.

A

In other words, you can choose transparently choose selected between the launchy and the cut container.

A

The this is the project, containers, architecture, design, flowchart.

A

The cutter agent is a process surrounding in the guest as a supervisor for managing the containers and the process running within those containers.

A

The cutter proxy offers access to the vm cutter agent to both custom and catalog time. Its main role is to route all the I o string, streams and signals.

A

It connects to the cutter agent on a unix domain, socket ievsoc.

A

Cutter proxy uses earmarks to multiplex gipc connections on its connection.

A

The shim process runs in the in the host environment, handling standard io and the signals on behalf of containers process, which is inside guests.

A

A

After the the cut container 2.0, some parts of this pro chart will be changed a little bit.

A

So this is what what the new items in cutter 2.0, for example, cii. I contain a dcm, v2 and it will redu reduce many uh many parts of the the architectural design flow chart and it's the roster agent to replace the golan agent and it's used. The ttr pc is a tiny jipc implementation to replace the original jipc library choose vsoc to replace the virtual I o cloud and by default the cloud hypervisor will replace the cumule hybrid also supports the guest c group, v2.

A

And what's the status of container on 64., totally speaking, it can run, can be run smoothly on 64.

A

You can you can install the cut container on arm64 by two ways. Firstly, is the snap install and, secondly, you can build it from the source code.

A

To run the container you can use the ctr command line. Here is the examples uh you can use the ctr image, pool to put the container image and run the simple application by using the ctr rom.

A

Also, you can run the container on a raspberry pi 4 platform with minor changes.

A

Here is the future comparison result between arm64 and the xcsx86. As for the hypervisor support, x86 supports cumule by cracker class, hypervisor and micro, but a chrome is not supported on arm 64..

A

The cut container will use, will let the hypervisor to create a medium device and then mount it as the guest root fs.

A

It can speed up the boot time and share the.

A

The guest root fs.

A

And the virtual io fs feature is used in the container root effects.

A

Which I, uh which I will introduce the in later slides the altruistic mark for the arm 64 kumu here is the upstream kumu had supported this feature, but we also need some time to introduce it on arm 64 for the container.

A

Next, the the vm template is a useful technology to speed up the boot time.

A

This feature is supported on both and also the raster agent is supported on both the memory hot plug. We, the upstream, had us supported on 64, but we were introduced in in the future. The vm.

A

Vcpu hot plug someone had sub uh posted a patch series to support it in in both cumule and kernel community, but I haven't got merged yet also the for the next kvm kvm feature: I'm 64 kvm.

A

The community has discussed it's one, huge one, big series and after that, after the community merge them. We can you we can introduce the into on 64 for cut container.

A

The virtual ios is a shared file system that lets virtual machine access a directory tree on the host. Unlike the existing approaches, it is designed to offer local file system semantics and performance.

A

The the low level of the virtual fs is a fuse implementation they have used. Protocol is not based on the network protocol, it means more faster, faster performance and better posix semantics capabilities, and there is a independent virtual ios demo process, more secure and needed to maintain the device mapper.

A

And uh there is a text mode you can uh with this with this text mode enabled the host and the guest can share the memory and improve the performance, and you can also bypass the guest page. Cache avoided an accessory vm exit.

A

And the the back end is in user space, which is.

A

Convenient for for for the user to further too.

A

But we once observed that the virtual frs will increase the system system level memory footprint, because you know it uses additional.

A

Shared pages which disallows the ksm to merge the pages.

A

The inner internal tests showed that it will significa significantly improve the file system performance in qatar compared with a virtual io9 p. From this chart, we use the the file read, write test. The data improves the increase about 10 to 20 times in different cases.

A

This is the the.

A

What we have done for the functional future is the development and, firstly, we enabled the runtime and the rust agent on arm 64 and we maintain the the ci tester subsystem and we enable the firecracker and even the cloud hypervisor on especially uh we cloud hypervisor is another uh individual, independent ripple.

A

We develop enable the cloud private from scratch on 64.

A

We also finished the kubernetes integration test with cutter container, there's a to-do list in the future for us to do, such as the memory and the vcpu hot block and the important next state of virtualization.

A

So to summarize, the performance comparison between different architectures.

A

We choose some important aspects, such as the boot time, the binary code, size and then memory footprint.

A

Here is the the hardware or software setup in for the host the guest, the cumule and the cutter version.

A

This is the evaluation for both.

A

I once started the run, a simple container application and you entered the hello world and exit at once for 10 100 times and then calculate the average data for both time.

A

The exit x-axis is the put point time difference to the starting point. The y-axis is the.

A

Mini-Second time unit, we observed that the most gap is between the vm started and the agent started. I the code put time and accumulable time.

A

And the boot time might be a little different between different.

A

A

The total boot time of the guest kernel is not so so so long, but we still found something to optimize.

A

We reduced the printable time from maybe 117 milliseconds to 81 milliseconds.

A

Another tunable is the systemd service, but, given that most of the district enable system d, we didn't remove the system d. As ipyle did from this chart, we concluded that a boot time gap on arm 64 compared with x86, is not the guest kernel. Boot.

A

This is the tuning items and, for example, we disable the pmu initialization.

A

If the user doesn't want to use it and by default the scarcity scale mode is synchronization mode. We can set it to now.

A

And also we, by default uh arm 64, will create about 30 32 vm uh virtual mmi, all devices, even the user doesn't use it.

A

We can disable it by default in the kernel configuration.

A

There is a one another way to speed up the boot time. It is more aggressive that is vm template.

A

A vm template is a new feature that enables the new vm creation using a cloning technique when it is enabled the new uh virtual machine.

A

Virtual machine is created by cloning from a pre-created template and they will share the same uh initial mfs kernel and uh the and the agent memory in rhythm, read-only mode. It's just like a process folk.

A

It is expected that kumu doesn't write anything to the guest ram until the virtual machine started, but it does in m64 qmil, so there's very exception on 64 when, when we enable the vm template feature, actually the room block dtp will be filled filled into ram during a room reset in in common case.

A

The room filling seems to be not required, since all the data have been stored in memory backend already, so we we bypass. We actually we bypass that process and make mediator that make the vm template enabled on arm.

A

A

This is the the binary size performance comparison.

A

Qatar will start start a guest with limited memory or cpu resource, hence reducing the binary size code. Size is a tuning aspect from this chart. We reduce final size for qatar, binary about 20 to 30 percent.

A

We also tuned the the the vmm currently only the cumule, by customizing the configuration and stripping the binary binary size. The binary size was reduced by about 20 percent. With configuration tuning, you know we can cut off all the unnecessary device creation and.

A

By by stripping, we can reduce more 60 percent for the code.

A

A

This is the memory footprint uh perform for the performance comparison.

A

You can see from this chart uh the possible reason why the virtual memory footprint of cumula on arm 64 is bigger than that on. X86 is the firmware devices creation.

A

There are two p flats devices takes about 28 128.

A

It will be created unconditionally.

A

The resident memory, uh a physical memory of arm 64., the cumulative footprint- is as much as or even better than than it on x86.

A

So this is the comparison.

A

A

This is the the resident memory summary for the for the size com comparison.

A

Sorry for the the goal agent and cutter go wrong time process.

A

The size of two architecture is closed or even the same.

A

This is the the net network throughput.

A

You know the bbr and cubic are different congestion algorithm.

A

These are two tcp.

A

uh These two different congressional congestion algorithm for the tcp internal qatar will choose vbr as the default, but from the test result shows that in the at least in the local and in the local lab environment, the bbi has lower performance than the cubic.

A

Here is the test result by changing the algorithm we we improve. We increased the throughput from almost nearly 11 gigabytes to 15 gigabytes.

A

This is the performance to the items. What we have done, enable the vm template the retirement fs decks and the persistent memory support, and they we changed. The algorithm from from bbr to.

A

A

uh These are, finally, I will introduce to two user cases. You know by two uh is a dominant chinese search engine operator in its ai cloud.

A

The by two air cloud is a complex network with huge amounts of traffic and complicate deployment scenarios.

A

The peak traffic is about 1 billion, page page views per day and 50 000 containers for a single tenant, so I do choose to use cutter containers after doing extensive research on secure container technologies and determining that cutter containers is a highly secure and practical containers technology.

A

Besides, as one of the important founders and maintainers alibaba use, uses, cut containers in its ecs experimental instance, plus kubernetes and serverless infrastructure.

A

So that's all for today's presentation is any questions.