Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2019 (Shanghai), 5 Jul 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Improve CPU Utilization by Designing a New Schedule Algorithm - Xiaoming Gao, Tencent

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Improve CPU Utilization by Designing a New Schedule Algorithm - Xiaoming Gao, Tencent

Large Internet companies usually face the problem of low CPU utilization, which can be solved by deploying both offline CPU-bound loads and online services on a single node. As a penalty, the quality of online services could not be guaranteed. We did some work on this issue, and introduce an offline scheduling class based on the Linux kernel.

https://sched.co/Nru8

A

A

B

Let's be sure about.

A

B

We've done in to the next scheduling algorithm and it's optimization.

A

B

The background.

A

B

The server intention is Melanie million logo. So it's happy.

C

B

How can we make who uses it? How can we better used make sure it's not I, don't make sure it's fully functioning.

A

B

Let's say the CPU usage ratio is only 20%. Now we try to post it to 70%.

A

A

B

Internally speaking, the server in Penson we're still trying to boost the usage ratio of CPU and GPU and actually will try to boost its.

A

B

Thus, yielding and safe cost.

A

B

Dr. with containers and other mains, we tried to boost.

A

The user ratio.

A

B

We have our internal KPM and also we can also directly run our containers.

A

B

The user issue of CPU.

A

B

We have definitely different definitions for a CPU and.

A

B

Won't go into great details.

A

B

We cannot use it for latency-sensitive part.

A

B

We can only use it for God, that's with lots of offline treads and Processing's.

A

B

Used for offline.

A

B

B

Use the KPM internally to make optimization and improvement.

C

B

Some of the solutions are trying to run the containers on raw metal machine.

A

B

Our lights are emphasized again for the online service. We are basically using it for.

C

B

Sensitive business and it's highly reliable, high reliability and.

A

B

For offline business, no latency sensitive sensitivity, and most of them are for a computing.

A

B

Impact our one-time latency is pretty minimal. Ready is part of the business.

A

B

Online business, for example, real time game- that's latency sensitive, they have really low reliance and.

A

B

Latency and if you are fighting with the other competitors, then there's very high latency, it's unbearable actually.

A

B

A flying services example if it's low, latency sensitivity.

A

Then, for example,.

B

If you want to check on the virus and back-end or sorting your photos, then these are.

A

B

Sensitivity, these are the business with low sensitivity for.

A

B

A

B

A

Offline pod, it's different.

A

B

For video games, that's not.

C

A story- that's.

B

Highly sensitive for latency.

A

B

The CPU usage ratio is all that low. It is not very high.

A

B

The mixed deployment a lead to a classical issue.

A

B

Blue part is on my series and.

A

B

A

B

A

B

Example, tasks P: we need to acquire resources from a and from C it's dependent on log and origin is CP 0 and it's fine and B is occupied by offline tasks Y and that will cause the whole sequence is chaotic and.

B

Cpu 1 to 4 to 2 it cannot be unlocked and cannot be operated.

B

For example, y&z.

A

B

They have different locks. It's very complicated landscape in this part, change go back to the original question.

A

B

Example how to solve this issue.

A

B

Different categories: different enterprises are using this platform to allocate their resources in CPU.

A

B

B

We have mixed women up online offline business.

A

B

Uncontrollable.

A

B

Is a very fair scheduling program and it would not show any preferences to any tread or.

A

A

B

Have to guarantee all the main thread can be scheduled on the timely manner. So this is shared platform, businesslike weighted somehow, and it cannot solve our problems and for quarter macadam.

A

This mechanism.

A

B

You're familiar with it see, group will provide two mechanisms setting different quarter different function, groups.

A

B

Have high frequency, timer.

B

Try to Travis the whole secret tree and it's very heavy. It means heavy operation. I need to double check on the quarter and if we set it smaller peer than we have a smaller number of offline groups,.

A

B

Who can avoid some ineffective overhead for CPU? Otherwise, it will be bad for.

A

The system this.

B

Is Sharon pure quarter starters and issues with encountered in the others includes load load, balancing, for example. In this scenario, we have 100 online limitation at 1001 thousand of line limitations.

A

B

Have two separate online offline thread and if we mix them together, then that means we add another 1000 offline shred into this 100 online thread. Then we have to do with low balancing. There will be panic and the.

A

B

Be migrated to CPU why that will cast a major impact on the.

A

B

And we'll have to change the context.

A

B

High level scenarios we have around 200 entries. If you are migrating through thread, then obviously that.

B

Means lots of consumptions.

A

B

Load balancing is not calculating these parts. We don't have a switch to control these process.

B

For example, this load balancing issue we have to strike about. You know the soffit. We have this shrug about balance in makes deployment and introduced a new categories of scheduling.

A

B

B

We can have a finally final name for it. We still discuss on that and if it's lower than the set CFS, then we have to guarantee that the online service can operate and will not be stripped off the resources by offline parts. We are differentiated threat, differentiate, the online from offline parts and the systems.

B

We have to guarantee that these two parts of online and offline are not enter affected by each other.

A

B

Is why a fine scheduling is meaningful? We have CFS, we have CPU scheduling and we have a busy schedule.

A

B

Rt scheduling? Why? Because.

A

B

That means we have to change a lot of codes and that will undermine probably the stability of the whole system.

B

So we need to stop these problems with controllable cost and the first principle is.

A

B

To prioritize Deek unlike computing, and if not.

A

B

The whole designing mechanism we'll have to guarantee the preemption of online service and.

B

We add additional support from our attack side and.

A

B

Add lightweighted low, balancing.

A

A

B

Logic behind is that.

A

B

Guarantee that the offline has their own load balancing mechanism and all my husband's own, the balancing mechanism as well- and we add two new.

A

B

One is the generic interface, they can control the overall ratio of offline service.

A

B

At the top to 20%.

A

B

Why do we set this limit? Some of the part want to do their own configuration for each set of the CPU, and we cannot allow that by setting the configuration they can control the load for each CPU and if you don't want to do that,.

A

B

Transfer the control CPU to the mechanism of load.

A

A

B

Have three categories: one load, balancing and limitation for interrupt.

A

B

Will set the priority for offline offline technologies and offline processing, turns red and.

B

So the limitations have bandwidth.

A

B

Have out live weighted statistics and accounting? We use this binary technologies, we're not doing heavy accounting like a recorder.

B

It's not necessary by our standard. For example, I didn't have to be as precise as like twenty percent twenty two percent- it's not necessary.

D

We were interrupted.

D

E

A

D

This to test the results so on this line, we have this always service a.

A

D

Four per minute.

A

D

Task, maybe we'll focus on the failure or success rate.

A

D

Is not of mixed waste? The offline is about 200 patter, with the mix of the.

A

D

Light than there's an increase of 5,000, so this task.

A

D

For the users they can not accept it and.

A

D

It's a failure rate either increased from 200 to 400, but success.

A

Rate remains the same.

D

And the we can make its own light weight, not balance into it. For the second scenario.

A

D

More about the average time delight.

A

A

D

Average honey is about a hundred and fifty minute. A second.

A

D

Adaption addition of the stickers, the failure rate is increasing, and but why user cannot accept this time is because well there's increased about 20 milliseconds. Then we can't accept it. Let's go back to this page so.

D

For each module there is an increase about 20 milliseconds, and this is quite a long time then there's a total of 120 milliseconds then go to this page. You can see that that time delay is very long.

A

D

That's why the twenties millisecond delay is has a huge impact. It's like a time increase in along.

A

The whole chain.

D

So for this, that their failure rate is not increased and it's acceptable and for the reliability is also ensured.

A

D

This is our off balancing effect.

D

So this yellow curves, we have not.

E

D

Although it does not affect the oh, my task, but it's unpredictable, it means that the offline, CPU utilization rate has not reached an acceptable level for the blue curve. It means the standard kernel. You can see that the fluctuation is not not.

F

A

E

D

The online task.

A

D

The soul for offline is not that smooth for the cray-1 after we have done a test.

E

It does not effect offline.

D

A

D

Also runs very smoothly itself, and also it has a less sharp increase.

A

D

We can see that the online can be synchronized with the offline.

A

A

D

A typical example: we, you can see an.

A

Increase of for 65% the found.

D

Just a 15% before so this is a sharp.

A

D

For the offline task, scheduling is.

C

D

To do with the characteristics of the Cask, so if it runs very smoothly, then the rate is very high, so the CPU utilization is not that high and.

A

D

Won't reach a high level.

A

D

It's with the characteristics of the online tasks.

A

E

A

D

A

D

Tencent kernel group, we are focusing only researching how to improve the integration utilization rate of the servers, SS celebrates becoming larger and larger, and it's becoming more and more competitive. So we have more requirements for the scheduling and network is also very important. Last.

A

D

Optimized a lot of us servers last year and.

A

D

We need to focus a lot of areas why the offline cannot.

A

Be realized to.

D

Acceptable level because.

A

D

Models, it emerges, it brings some problems, so PTT scheduling, algorithms, what it is about.

A

F

Need to classify.

D

F

A

D

Need to classify the processes.

A

D

Divide that into offline and offline.

A

D

A

Need to make sure that.

A

D

Make to make it a juggle: where is the different tasks? At the same time,.

A

D

To increase the utilization rate of the CPU.

A

D

Is the conclusion from our research so.

E

A

D

Sure about the crutch but I think.

A

D

Not big change to it.

E

F

D

Changed later, oh.

A

D

Are topped our thought to classify otherwise, the utilization rate cannot be improved.

A

D

Is our the other side, the travel plan and, finally, we'll finalize the whole report.

A

D

With detailed information to Linux the thousand, we have released he-she Linux and you can buy it in the crowd cloud, and this is now headquarter can't headquarter, including our open source of programs, that there is T Linux team.

D

Yes, like the hybrid that a public and the private cloud, and we also have the input in the nationalization of the device.

D

So that's the end of my presentation. So, yes, I may be running a bit faster because I don't go deep. Dive into this did.

A

D

Otherwise, it will concerned the scheduling of the colonel. If you are interested in it, you can pay attention to my slides.

A

D

A

D

Provides a news source for our offline tasks.

D

Hello, sorry, the interpreter can not hear what he's talking, because it does not turn on the microphone.

A

I'm from Alibaba.

C

We have been working on this.

F

F

C

Because hybrid.

F

C

You XI it's more about.

C

But after the mixer makes it will increase the memory.

C

F

D

The CPU will be increased.

F

E

A problem with the management of the memory and.

D

Apply for a lot of the memories, then then destroy it again and fight for it again.

E

D

F

C

To post the impact on the host OS and it.

E

D

The deadlock for the online and some pressure for online.

E

D

Deal with it yet, yes, this is part of scheduling, optimization, and we also have done on the memory.

E

D

Recycling of the page.

A

D

For each group we have this pages or psycho angel for the container.

A

E

D

It's fair ahead. We have these restrictions.

A

D

Had visited the millions of for buyers that week then, if we recycle it at a time that it will be too late. So so this presentation is about about the city, CPU utilization rate.

D

Hello, sorry, the interpreter can not hear what the others is talking about, because it doesn't use a microphone.

A

D

For the all might ask.

A

D

Have a 128 MB.

A

D

Make a hundred often online and 20.

A

Of them offline, we.

D

E

D

On the issue of the integration rate is too low.

D

Yes, we need to focus on the utilization rate of the machine.

D

So sorry, Carrie I.

D

Have a question: yes,.

A

D

The colonel about the scheduling algorithm for this colonel that changes to it. How can you differentiate between the online okay? How can you tell whether it is a fly or online.

A

D

Mark it on a page, it is a higher level scheduling.

A

D

Has this offline scheduler.

A

D

A

Provides an interface so.

D

You open an interface of the colonel and let it tell you yes.

A

D

Tasks, it is incorporated with the upper level tasks.

A

D

You just screen this simple I.

A

D

The offline scheduling is.

A

D

A

It together with, though.

E

D

I mean the upper level of the will, give you this information about how to do the scheduling.

A

D