Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2019 (Shanghai), 5 Jul 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cost-Effective Scheduling of a Massive Number of Containers in Kubernetes - Yuan Chen, JD.com

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Cost-Effective Scheduling of a Massive Number of Containers in Kubernetes - Yuan Chen, JD.com

JD runs one of the largest Kubernetes clusters in production in the world, supporting a wide range of workloads from e-commerce services to big data and machine learning jobs. The massive scale and complexity requires efficient scheduling to address the scalability and cost-effectiveness challenges. JD’s Chief Architect, Haifeng Liu, will present how JD overcomes hurdles to improve its Kubernetes Clusters’ resource utilization and cost efficiency through advanced scheduling, including fine-grained monetization and monitoring of resource usage, machine learning-driven resource allocation, co-scheduling of mixed workloads and millisecond-level elastic scaling. Specifically, Haifeng will describe Archimedes - JD's Kubernetes scheduling system, and how it handled an extreme demand with $24.7 billion of transactions on JD's Kubernetes platform during JD's June 18 anniversary sale event.

https://sched.co/Nrm3

A

I'm not sure if you have a ten my session I guess you must accompany.

A

A

We have the challenges.

A

A

Start quite early in don't actually early in 2010, we adopt a contingent mechanism and in 2016 and we have started key cabinets and for now most of you systems are continent right containerization, and we have already done this to keep that's experiments and we want to have more a reaction with a community.

B

Contribution to use Japanese community well.

A

A

For you, if you want.

B

A

Narration or I want to use concern. Oh they want to.

C

Share some experience say wrestles with you actually.

A

In 2013 we start from 0 to 1 just.

C

A

Incontinent or discontinued ization.

A

And we want all the things to.

B

Continually later.

A

In 2016 and we use an open source cabinet from Google to migrate to that into our system and to have a whole ecosystem.

D

A

Later, we transfer that into the higher cross platform to have more automation and.

B

A

We are the early adopter of the container in our skills, adapter you and the stability. The stability is quite outstanding and we have a rich experience, but now we want to do G.

D

What do we want to do this to.

A

Improve efficiency to.

D

Have the efficient.

C

A

And we want to optimize a management, and then we want to magically resolve problems that we.

A

B

Of your system.

C

According to doors,.

A

Margit, the OS, which is a shock for you, JD data center, as actually it's a customized cubanelles fighting. Don't you actually in 2016 when we are toasted accumulate, so many things were not to mature and your. We wanted them and.

D

They're difficult I'll show you we.

A

Have opted optimize, the salami.

A

Very top level.

D

We want to do a relation.

B

A

Want to measure a thing since.

B

If we want to optimize the first, we need.

A

To measure that.

A

Struggling system, we call it accuracy and Jesus is a core and we have some open source documents. Your system.

C

And we now called.

A

A the container FS, which is open source and later I, will just level it another BT and welcome. You can use that.

A

Lowest layer is about to them.

B

Might use a resource because well for resource struggling or for her brother sense management with.

C

Many challenges, accuracy.

A

How to accurately predicted amount of the reasoner's the week, sighs.

B

A

Bother the results will be released and the internal units.

B

A

The scenario will be worse, for another part, is a complicity, as most of the work are actually actively to the content of the.

A

Message system actually already in container besides it's a multi-dimensional resource, optimization with IO, been waiting and tasty and besides actually waste are some large secure projects, for example,.

D

A

And the big data migration.

D

You want to have.

A

B

A

Besides, we want to talk.

B

About the scalability fools our community.

A

The largest of you cuban, it's a community cluster, we can do the six thousand two thousand two nodes.

A

For the largest actually.

A

A

A

B

B

A

We do some customization and second, we do some optimization.

A

B

Scheduling quality.

A

Resource utilization to.

B

The maximum levels as.

A

I mentioned based on the historical data analysis and a prediction we can do. The mixed are struggling in a way that makes the workload placement that we choose to know in detail.

A

Me encanta, oh just to give you a relatively larger, secure application for that. For example, we have 1,000 in old and 25,000 apart.

A

I will say this data may be some odd fashion. I did not advocate a timely, but we can find some button. The.

B

A

B

A

Methods all the resources.

B

A

For it is, it will optimize the net. We classify that into different classes and.

A

Resource types well for the data a month, we do the.

D

Other compressing besides, we also.

A

Have done performed, the rat is catching solar for the second part, we want to have that.

B

A

B

B

A

Becomes a bottleneck.

D

A

Another sin that some configure.

B

Maps are the APS service area.

B

A

Up large amount of resources, which is.

D

Unnecessary, so we want to.

A

Change that as we go you're in this scheduling, actually it's being necessary to.

B

Specializations now it's.

A

We may take maybe several seconds or minutes, even others, localization and redistribution and the host optimization techniques. I would just assume that since it's not the main part of my presentation,.

A

And here this slide shows the results.

B

Which have been the only period for.

A

The scheduling sensor we reduce that to reduced by 90% and it is a the rate and write the rate and lightweight. It will be 70%.

B

A

B

The DVDs right.

A

B

A

About the API service most requested a micro.

B

Strain to choose configure.

A

And if not that over later, we were just delete, it is required well in the latest version ending, don't we first around the Cuban, it's 1.06 version, and for now we have the 1.2 version. Of course we have done many optimization.

D

And the customization.

C

Of this version, of course, the.

A

Api servers must be them.

B

A

We got a chip cover the fact that there's a whole cluster is quite busy so that we have implemented a savvy control and this preemption optimization the emission control.

B

A

Course, following our community motion, if it isn't busy and the scheduling will feel- and we have.

B

A

So we do get these resources into the single device. We can optimize. That's when the jobs is in the Queen Valley, especially when the rack is quite busy, so that we can have a better control of that and a better management of that well for struggling. We we really want to have appropriate control and drive out mechanism to ensure the system can have. A good performance will not be a.

A

Problem about your customized to Cuba needs and the way can optimize that I.

B

Mean the system requirements. Besides that we have another question: that.

A

B

Newly found you.

A

Have two problems for.

B

The e-commerce of online.

A

Services to a large scale of stock enlargement and analysis.

A

An application requires a large amount of data, a large amount of resources.

B

Your containers.

A

B

A

Three or five.

B

A

Instance is second item requires 100 additional familiar with the schedule for Cuban. Is, it is hot by prime schedule. It need to match up your perfect match odds. If you want to 500 in to contain this, you don't want to go. The whole process is quite time-consuming, so you want to optimize. The process have.

B

A very simple strategy.

B

This genocide in.

A

There we will select the first 100.

B

A

A distribute that if you are familiar.

B

With the stroller and.

B

A

This is a research from the.

C

Cambridge University, they come home with a full.

B

Set and Friday fine manager, it.

A

Is a workflow focus on those tools.

D

A

And if you require a large amount of container with the same size, li vision, seam will be.

D

So if you do not have.

A

High quality, you just you, do not have high requirements on the quality, but all the time you wanted a large amount in the short time you can use. Those for another cabinet has been extended. This data and Oh Elsa.

D

And this can be provisioned. This can be configured so if it is a long-running service for.

C

D

We can optimize it, maybe if it's if it is slow appointment, just 10 to 20, that's enough, so you can configure a to the parameters so for different workloads and different scenarios. We should choose different scheduling, optimization methods. So a brief summary is in other to better the performance or the scale. We need to optimize the scheduling and different strategies. We should also take those into consideration.

D

Also the access control admission control in order to ensure that when it is very busy, we can have still a stable scheduling system and it will not have the awesome.

C

D

Problems, and the second point to make is how to effectively benign.

C

D

How to realize.

C

It efficiently to.

D

Improve its efficiency and effectiveness, our mentality may be about intelligent resource management, and we can because we have a large number of variety data and the utilization, the resource utilization. So we can add some machine learning algorithms to help us to make decisions and on the one hand it is intelligent. On the other hand, it can save the cost and improve its effectiveness and present our system. It is a closed loop system for monitoring.

D

For example, we have the second level monitoring so that we can collect the statistics and to predict and then to octave some optimizing strategies and then to give the feedback. So this is just a general framework.

C

D

Let's come back to decision-making so with this framework architecture, how to optimize the scheduling strategy? First, just as I've mentioned this morning, our database is doing the CPU and memory sizing that is to give them an appropriate size of CPU and memory. For example, some oompah committing a community some data, historic resource research, statistics and future demands. Prediction and second, is to choose the appropriate server so that the whole total blaster can work more efficiently and also we can have a hybrid.

B

D

Different resources pools how to schedule so.

B

Introducing details so.

D

Detailed Lee: this is the architecture of this close to loop system. This is the container clusters and you can see from the slide the pink part and.

C

D

Emphasize a lot to the real-time monitoring in terms of second level for a large number of data, and we now have a time series database. Also quick house. It is another open source database, so object like the data we can analyze the prehistoric data and also we can have the amount prediction about his pertain stability and correlation, and then we were focused and have decision-making. So I think this are three argue for the useless management that.

C

D

First is right: signs the sizing of the container. You need to have appropriate size.

C

D

Too large, not too small and the second is right, location, the font placement in which cluster in which suffer and in which rack, even in which database data center last one is right time when to schedule it to schedule this one or another one. So the decision-making, mainly it is about this three and four right sizing I, will basically committing strategy.

D

Targeting and workloads will.

B

D

To its metadata to classify it, and then we have fun in initial setting configuration so after running for some time.

B

D

To our data analysis and prediction, we will have some adjustment.

B

D

More details this morning, I also mentioned this- that we have a mode of the right sizing of every container and.

C

Every solution we have.

D

Requested- and we have a limit so we use requested for sauce and should not be its maximum results- may be just 90 percent promise. You know.

C

D

Manta means the rest of the room, the rest of the capacity I.

C

D

Maximum should be 99%, we present I'll. Imagine.

B

D

The containers on the suffer we should give them a buffer to.

B

C

D

Realize common share and I would take this as an example to.

B

D

B

Many numbers here so.

D

Let's just assume that every server has nine containers and we will classify all those containments if they are highly correlated with each other, and that means if they will reach peak at the same time, so I will classify them within the.

C

D

D

Enough of the green part is an additional resources, its.

C

D

B

We cannot guarantee.

D

B

D

From C to C 9, this is the requested resources.

C

For schedule- and this is the.

B

Fundamental resources.

D

B

D

Do not need to and H 1 to H 9.

B

D

They aren't correlated. That means if they reach the peak. At the same time, we can add them, but if they are not, we can just choose selected some of them members, for example, from age 4 to 87 and that one's on the one hand we have requested resources with.

B

Which is not the maximum.

D

B

D

The maximum value.

B

D

I would take this as an example on the right side, this image.

B

D

And vertically it is 0 to 16 and this they're co-relation. So if the it is stuck- and it means it has high.

C

D

And, as you can see, the black line that Miss Holly related correlated with itself, you can also identify some dark blue pants and some green pants. So that means, if you just randomly distributed, they will not reach the pic at the same time and on the top of the slide. This is the maximum according to its 90, percentile and 99 years in time. So you do not need to make the requested resources.

C

100% to make it simply.

D

We can use the wise chrysalis as a guarantee and the whole server. We can leave I shared a hat room.

D

It should not be the sum of all together because they were not reached the peak. At the same time,.

C

The comparison 90% time.

D

And, as you can see on the graph and the website, a 90 percentile and the performance, the resource.

C

Allocation by if.

D

You allocate a 99% towel, there will not be no problem in performance, but we normalize the resource allocation and this is combines they requested and limit resources so that we can realize a balance to performance and there will be less complex. So the idea or the mentality instant. We can utilize better the limit their resources because we divided house.

B

D

Of the applications is memory intensive for.

C

D

Is why we also optimize.

C

Memory, and actually it is even.

D

C

B

It is compressible resources.

C

But for memory.

D

If it's active exodus than it will be out of memory and it will be killed in the memory utilization, how to.

B

D

Memory results: actually we have a prediction algorithm, but actually it is quite hard to protect it, because sometimes it will be some beautiful changes.

D

That is why we have the maximum protection memory utilization. So we observe the pink sequence.

B

D

That is why we have algorithm to protect the memory and to adjust as resource.

C

D

Also communicated with a lot of counterparts.

C

D

And they say that it.

C

Must be very accurate.

A

D

At the very beginning, we collect enough data and then.

C

D

Performance and currency, this is the memories of reliable online services. This is our prediction.

C

B

Week prediction are.

D

B

Into different schedulers.

D

B

D

Perfectly, if we do not have enough support for the workload and our prediction is not accurate, we.

B

Can just put it into matrix.

D

The targeted zoo utilization is different and.

B

This is conservative.

D

For example, when it reaches 80%, this is 85% and 90%.

B

D

And then we can put it in the middle part if it.

C

Is conservative and we.

D

C

D

Very accurate: we can risk it away to the upper layer.

D

How we do memory scheduling for online services and the Antonia in industry, and they talk about how to improve the efficiency, but why we do not apply the memory scheduling. That is because, first in memory, if you.

B

D

Pre M to a lot of batches and it will not be effective, and the second point to make is many online services. At present it is very hard for us to use batch, so that means.

C

This is totally.

D

Based on online services and I'm present, it is quite effective and it has gained good results. This is the simulation real trance driven simulation and it is 80% utilization without om, but still, if we improve it, there will be. Some thoughts are just a proportion of the scheduling, migration promotion.

D

It is of a small scale and it has been improved from 40 percent to 61 percent, so we think this results is quite ideal. This morning, I also mentioned about the hostile.

C

D

Optimization because you want to select the best host.

B

D

Kuban entus near us.

C

Approach is very key.

D

Matrix, for example,.

C

Missiles balance and the resource.

D

Availability, the more resource availability is the better it will work and the latest version is. It has very simple algorithm for the multi resource balance and we introduce similarity algorithm. The second is resource availability, the it has equal weights for CPU and memory, because.

C

D

Workload and where weights and resource usage dynamics so that we can adjust it if it is a few intensive workloads, so we can give it a higher priority. We also take.

C

D

19 took a situation like the porter affinity and the portable.

B

Correlation away so.

D

This is a comparison with the community.

B

D

In total, we have about 5000 servers and we have 225 thousand watts and within the community.

D

There will be some failures.

D

A

You can use that, according to your.

C

B

A

B

A

B

A

Years ago,.

B

A

If the developer can do some contributions, subsequently procedures consider Lots efforts. I agree with max. If the developer can have a more standard, writing Bruce.

B

A

Scheduling the developer can define.

B

The documentary when.

A

The scheduling stage- yes, of course they can offer some knowledge or contribution. It will help a lot. I will come to we work on a.

B

Marvel, a little more.

A