Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2019 (Shanghai), 5 Jul 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Some Lessons We Learned from Moving E-business Giant to Cloud Native - Lei Zhang & Siyu Wang

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Some Lessons We Learned from Moving E-business Giant

Cloud Native - Lei Zhang & Siyu Wang, Alibaba

Moving a global e-business giant like Alibaba to cloud native is never easy, it's still on going, it takes long time to complete, but it's worthy. In this talk, we would like to share some lessons we learned from these efforts we made during last year from both technical and community aspects, specifically: 1. What is the main blocker for tech giant to move to cloud native? 2. What are main tech debts for giant like Alibaba? How we tried to fix them (partially)? Does it work? 3. What should we do if application management is totally different from Kubernetes in your org? Tips: CRD & controllers 4. Why predictability is critical for e-business? Is predictability out-of-box in Kubernetes? If not, why? How to fix it (maybe not)? 5. How to verify scalability issues in cluster of thousands of nodes? 6. Is it possible for an "elephant" team to dance with upstream community? How?

https://sched.co/Nrms

A

B

Thank you for coming for attending. This is a small topic. Actually today we're talking funny stuff, so the topic is eternal, so the story is simple: in a story about, in the past two years, transport from in-house container to the cloud native architecture, which is in Cuba Cuba Nets, large-scale migration issues.

B

Started to deploy long.

C

B

As early as that, 20 a container so.

A

B

Are no talker technologies available at that time,.

A

B

Something based on TOC now many companies, and by do they have a matrix like thanks to or trace a similar problem. So this is the container enough layers.

C

B

Of the containers, models are virtual, so in one part, is only one container and several sessions in one volume and after 2018.

A

B

Young Cuban antics members joined the team, so we were promoting a people.

C

B

System, so this is so. The starting point is a cloud native progress. 2019. We started an experiment cloud.

A

A

B

Account to a user ally cloud.

A

B

To deploying data class actually our slogan for a cloud negative.

B

A

B

A

We support multiple, continuous music one.

B

And after all, these reforms, ELISA ELISA clouds.

A

Is based on kubernetes.

B

I'll show you a chart based on some of the mainstream open source projects, for example, kubernetes apprentice framework community si. All these things are part of the framework. Even our monitoring system are quite pervasive, a standard thing. We have a container to provide a very.

A

High isolation and.

B

The cluster we are using is actually I know any I can our framework from ACK. We use a cuban attic to deploy it TC, and this model is quite a standard industry I, deploy a metal coulis and then from there to deploy multiple clusters facing the users and this cuban. That is, we can use this chart to illustrate our team as infrastructure developer. We serve two clients are the group second client.

B

So we are facing a wide portfolio of all kinds, including the past. Om systems devil tools, as well as some of the existing.

B

A

And some of alibaba's businesses.

B

Also, businesses should all be done so ever like a ten about ten large cast clusters. So that means that you know over 10,000 units and over 1 million containers for some a company. They only paid tens of thousands.

B

But we have some QB Sabah, we were not modified. We will use the upper strength because the maintenance of the server is.

C

B

If you want to balance it will.

C

Be difficult for.

B

For us to maintain ourselves.

A

B

It provides a standard entry point is the very so an idea of our communities. So if we don't want to, you know.

B

Policy must be very diversified and the archival stream cannot fully cater to the needs. We have also optimize the performances. At the very least one Ali employee said when choosing the mall, you will only take a hundred samples, even if there are 10,000 available.

A

B

A sample chastity ginger- and this is a standard practice using some strategies to reduce quantity and later on, we added a surface feature. It uses a horrible change.

B

The content of the idea.

A

In a faster 10,000 sampling.

B

Is required but now not.

C

B

It can process patch, as well as our businesses,.

C

B

A mixed business on.

C

B

In terms of controller manager, we use the upper stream system. We have an in-class highlighter in place.

B

Change this is not conducive and deadly for our business and we also do that a mixed multi-tenancy. So it's a it's a result of.

B

Distributive resources, this schedule is facing tremendous pressure. Indeed, so you know to maintain a foreseeable and reasonable, and also we every step must be precisely controlled by operator. I will talk about it later. We have separated.

A

C

A

Separated as a independent.

B

C

A

C

D

Under control that is suitable to our usage and also in need to deal with some climates on depth and.

C

Also, the other queries and.

A

In every bubble and.

D

We have some projects.

A

D

We use the logic.

A

D

Based on its functions and for this one, for this logic is a special instance and in this in this class logic class, we are going.

A

D

Do not use the heart because you can test the.

C

D

Of toca, because it's much.

C

Slower, it has used a lot of extra things to it, so.

D

It's meaningless to us and before we use punch container, but now we have improve it and as for drought, we use a very hight ACK, si si si and I and to make it into a cloud provider and also for some of the big products in not the elastic and also sometimes it should be flexible, because this is physic host. Who is not easy to scale it's fixed, so we needed you able to borrow some resources from others.

D

So we connected with the elastic resource-poor and we could ec high and it's elastic resource pool so that it can.

A

It's a place where.

D

We can borrow some.

A

Of your sources to pin the scenarios so.

D

It has built up a new system before we do not use it like that. We can, and also we.

C

Have a special metal instance.

D

You can see there lot of tendons.

C

D

Our word systems- and this is not what your composter.

A

What your composter and.

D

In this resource cost, then users are not going to use it. We.

A

D

Some plugins here so that the component is able to show and expose that ap as.

A

D

A

The we have maybe a.

D

Service to all of this, it.

C

A

Ap objects, it's approaching objects, and this.

D

Were true the virtual things it's a smooth.

D

There was no, because all the business will be located into into.

A

D

A

D

Of them are sharing the same pose and each of them will have an independent API. So this is a washer cursor, and this is what we are working on and in future.

C

D

Be an open source- this is resource, I will tell resource model now, and it's for some of the technologies here. I could use to it study from 2018. We are doing take container things.

D

What is the sub container I think of the reach container? They coded the system container this system container, apart from the competition of the business, they also have some.

A

Artists, like the.

D

A

D

And we reach container of animal.

C

D

Kind of resource we have.

D

All kinds of things that you put in it, including some one of the boot scripts and etc. You.

A

Don't even know who have put those things in it.

D

To set container- and they.

C

D

Put inside and they are going to start.

A

Their script because.

D

At the time they want to connect with business insides much easier. Why we want to eliminate reach-in container, because it has got away a bat mode for operations.

C

D

Randomly write those codes in containers, any kind of.

A

So we want to eliminate the rich containers because.

D

There was, we don't think.

A

D

Container, if you are able to containerize the operations and Rickon today, ok, but in our conditions now, with this rich container.

C

The operations of any purpose model has some problems so.

D

Starting from 2018, we start we decided to eliminate the rich container. Now we have a lot of improvement. All of our containers containers nowadays I actually like weighted.

A

D

So that we can make it into the ports with several thoughts because of upstream.

A

D

A

Some standardized.

D

Work our users, for example, the cloud native one of the ports, will.

A

Include these containers for support.

D

Is business container? It's also. We have some not like.

C

A grading site sidecar.

D

For example, I have an application, wanted upgraded and then it's going to load a cast in the sidecar into it, and then you don't have to redo it again they've all. Maybe you need to cooperate to to the.

A

Rich container, no, you don't have to.

D

Do that? Because we have the sidecar, we.

C

Have make a psycho psycho just one.

A

Thing now you want to agree: okay, so sidecar.

D

The input of the psycho is a great content, so it's going to copy this psycho concept to this because they can.

B

D

The volume inside, and also we can sure they can share this lock as well. So in this part, we're.

C

Able to enabled.

D

Upgrading as well, next, we also have the assistant ng also help us to fix some small bugs as well, so it will include different kind of poison. All of those rich containers procedures has become.

A

One of the input here inside the ports, but there were some problems here.

C

And it's actually rely on reading the.

D

Documents, the files.

C

Like the secure.

D

And other values as well.

A

D

Need to find any way to address.

C

These problems already known it's a rich container.

A

And it's working on to BM if.

D

You want to convert it into the possibility.

C

To so, first was what.

D

Is the booting sequence and we.

C

Need to use a concept called.

D

Nick container.

C

D

The containers may be early.

C

And others the latter butit will.

D

Would need to set someone off to pre pre requisite conditions so.

C

D

C

Define discussions both restart and post art.

A

D

A

Which one should start.

C

First, I give you an example here.

D

An altar clubs should.

A

First, define old life cycle so.

D

That you can so that it's able to run properly because we are trying to evolve to the cloud native. This is very good path forward and we have utilized a lot of things in competitive night cycle continue life cycle controlling. Of course, you also have lived here defined my place.

A

D

What way that you can consider a container has been started.

D

After we have eliminated the rich container, then you will find that originally I utilize.

C

The common active which will cause different.

D

Impact one is automation why we will find an optimization.

C

Because each container was.

D

Order application of cycles connections, they don't have any coupling.

C

Or any relation between them.

D

However, the life cycle, it's a life with life cycle.

C

D

You are going to identify those systems, then the systems will be better able to run. This runs over.

D

So the human said you are able.

C

D

Build up the morning very fastly.

A

D

Able to focus on their functions like monitoring, you just need to focusing on monitoring work and later we are going to have sign-in power to operators. Do.

C

Import, this icon.

D

Inside the containers, you don't have to do with it yourself, but we have a colleague strange.

C

Music, it can automatically put aside heart in it and, accordingly, upgrading.

D

And emitting, and it would.

C

D

Automated way in an.

C

Automated way, everybody.

D

Were able to focus on their functions, so we are going to find a lot of benefit to eliminating the reach containers and they. So we are able.

A

To utilize it to work on all things.

D

First, it's a resource utilization and also it's something.

D

Some of the life cycles are similar to the containers life cycles, so we are going to have.

C

Application management.

D

To manage cycle and then we are able to put.

C

The data into volume, the.

D

C

A

Do I don't have to find.

D

A

To write this so.

D

In the future, it's going to upgrade space on the community so.

C

When we first built the 1.1 own later, we have set to release and then.

D

Upgraded 1.2 and now we have a to 0.14 so.

C

D

C

D

See we are not going to 1990 release of the communities because the kubernetes its able to for kubernetes you can control the minimum.

A

Degree and changes, and next.

D

C

Native abdomens right, we.

A

D

Whole set kubernetes and we.

A

Can't do some after.

D

A

D

Need to find different cutting management and also relationship, and also how to deal with multiple.

C

A

And interaction between them, so here.

D

Is application of an operation of it.

A

D

Of all is operations necessary application operations and we need to communicate with different teams and they need to follow the kubernetes concept in its development.

A

D

We also need to explore what kind of different nation of the definition of it.

D

Now we are using help to define the applications. However, some of the resources is not applicable, so for.

C

Example, how can we define.

D

The native resources on health and this kind of resources are not able to define by the documents is not easy to define.

C

So we're still working with it and.

D

Communicating in the community that whether we are able to build up the application definition that is facing the cloud it's based on clouds later we are going to.

C

A

Its problem we have and.

D

As for management, we are doing.we.

D

Thinking because.

C

We are utilizing the definition from so we.

D

Need to build up a half and epilogue, but the official app hub is.

C

Not a big in China also.

D

There different kind of challenges like the GCR 0.00 and we need to do some documentation based on this house and we have putted in the.

C

Le cloud and provide some free services.

D

You can utilize.

C

This we can automatically synchronize, we don't show help and.

D

Also, you can utilize them.

A

Up to be on there as well, because.

D

A

D

Github is also completely open.

D

A

Tools, it's made me yond.

D

Documents and how can we allocate the documents and also, how can we configure those parameters and we utilized the help to change, is changing some of the definitions and also parallel parameters, and we have two ties.

A

D

To work on some changes.

C

And in each change use the status to do to.

D

Do the D control incus, the old Jing, is you need to start it from the beginning to the end, to put it into the KBS.

D

D

And to automatically modify.

C

Digitization States application.

D

And need to scale.

A

D

Instances is that they need to modify the replicas and they need to change the number of the replicas to ten, so that.

A

D

To change upwards, no.

A

Matter where you have put your.

D

Documents right, your.

A

D

C

A

Are right in there.

D

For example, I do not allow you to store two formats, then my a hvh.

C

Jpa are not able to change.

D

I have the documents. However, it's going to be modified by the therapist. It also able to modify the Kuban edges. If you need the kubernetes to change it, you should have the open protocol.

C

That could be understood by the guests so that you.

D

Can operate this process so I think that CI didn't appear able to do this communication. However, sorry.

C

From the path from the source code and then.

D

Build up different to go through different process and improve these documents into resources and all those generated documents to be put into deep.

D

Into gate because it has been put at put a delegate so.

C

We are going to have we.

A

D

We need to go ever further.

D

After we put it into the ePub, we are going.

A

To manage this process.

B

Talk about how our.

A

B

Be very hungry so keep.

A

B

Speaking of Ali, the first thing Levin's.

A

A

B

We have nearly a million containers cloud distributed in over a hundred server rooms with.

C

Application, so this.

B

Quantity infrastructure moving the to cloud, we are facing the stability and other risk factors. We are also assisting the past platforms and the upper stream to be migrated to migrate to the to the cloud. Instead of you.

A

A

B

Unchanged otherwise he cannot cater.

C

A

B

Our upper applications have done 411 for that team or share this, discloses information about our infrastructure or be tripping edges.

A

B

It prepare for the video.

A

About the 11th.

B

Composer, the 11th, the traffic like n times daily average, even.

A

Over a hundred times.

B

We far more resources that what we need daily basis so before the first.

A

B

To establish station.

A

B

Scalable resources.

A

B

We should upscale the resources before November the 11th on one hand, to improve resource utilization right. One.

C

A

B

A

B

Pod into 10,000 units.

A

B

We rely on changing the quality of replicas.

A

B

A

B

They're scheduled at all itself, let's.

C

B

What publications and.

B

It can only check the current status.

C

Quo since whether it.

B

C

B

C

B

Were to scale out hundreds of thousands of hearts if we rely on this strategy like this individual scheduling their strategy, the outcome.

A

B

And definitely not caters to some of the resource overlap.

B

The variance would not have met when establish.

A

A

B

Use our old flying scheduler to understand the status quo, no resources and then match me. Hundreds of thousands of parts to.

A

Enable centralized distribution.

B

Now, tasks through batch placement.

A

A

B

To all the the three parts we.

A

B

Placement to enforce.

B

Internally, we use the to zeratul to deliver the function. One is called a batch allocation plan. The other is called about adoption allocation plan by its.

A

C

B

A it creates powerful parts of its kind, that is to say in this flexible resource-poor. If we were to.

C

B

Hundreds of thousands of parts we cannot guarantee the first distribution.

C

B

Line with our expectation, so the batch allocation planner allows the creation of a popper.

A

B

Distribute the resources they're not reasonable, or when this when.

A

B

Our expectation we will reinforce.

B

In this way, the overall resource distribution means expectation.

A

B

You're familiar, that is the default release. Strategy is to.

A

C

A

B

A

Problem is, after.

A

B

There are hundreds of thousands of parts and finally expectation and when one application released it then the all other parts will might it's away. So it would.

A

B

Meet our expectations before November 11th I will go through several.

A

B

A

A

A

B

During the testing, even if it upon.

B

Release we cannot guarantee the actual outcome can meet our expectation.

A

B

Is these release strategy from.

A

B

Some really strategies.

A

B

Matter what strategy we use.

A

A

B

Breaks our expectation to the stability over another part, so we.

A

Developed this.

B

It's called a in-place update, so.

A

When releasing.

B

Great the container, on the very same spot.

A

B

Of recreating a part so for our strange business II don't need to know.

B

Of it, so the within D advanced us type for said, if we set up the partition to be one under this condition,.

A

B

A

B

Only operate upgrade container the corresponding container that you intend.

C

To modify given.

B

X will identify the containers, a hatch has changed.

A

B

Will use new image mirror image to create to recreate the file in.

C

B

Process the celica containers in a pot is in operation. That means it's and effective on the other way around. If we need to upgrade the satakam container.

A

B

Will not affect the application container, and this is what the default model cannot support in a template if there are multiple containers in the part.

B

Your components and container is to operate a version, the mirror image, because it uses a real part, recreation strategy. It will cause the deletion of the out part and to create a new part, and this does not meet our expectations. We won the upgrade of the component not affecting the business going online because it effects a stability for comparison.

B

Is simply the default model and.

A

B

Rolling update.

A

B

Place update a better cater to our needs, for example, to your sure system. We know that docker and containers of mirror image if it's upgraded on the very same spot. Some of the layers can be continued to be used, so the true efficiency, even if it falls the powerful resource demand and it's another issue, kubernetes. The default strategy for recreating it is typical for tally to execute, for one is because of the uncertainty and also alleys share value.

B

Every day, our online ad release reaches seven six thousand even ten thousand times, and each just like a John has said there are, you know over ten thousand under each. If we were to recreate it in order to operate it, then it's a huge pressure upon the scheduler, so every release into a distributor tens.

C

Of thousands.

B

Of pots and for the supply chain for the service registration will cause a tremendous pressure through in place at that we can.

A

B

The part lightweight apart without modifying you know without a re shelling.

A

B

Like to show you a few things about another CA than.

A

B

A

Site site carbo.

B

This first took a take a look at that the problems of the traditional beaten you define as step said whether the template.

B

Sidecar container your OEM components, your agents and other authorized components, and this components were all defined in this block for some small-scale operation, its.

A

B

Or large-scale operation, the first thing we encounter is is for the operators, the managers, you know that's, and over a hundred thousand applications and the operator of the application. They don't know that which sidecar size is required.

B

They don't want to pay attention to such details, related to it om and the operators business operator. They only care about the stability of the system upgrade the system, they don't care the side, components or. Second, you should be nuts when we define a sidecar in working environment. The question was how to operate it. We know that.

A

B

Workload, you know, hundreds of thousands of unit, oh and a component operator is, is expected to manager hundreds of thousands of black side. It is painstaking.

A

B

The application is the definition there, everything the same so when the upgrades of the container and components are in confliction are conflicting with each other. It will also affect the container stability fix up all upgrading the application and in maintainer or further.

C

B

I identify the same problem and he's modifying the SATA current version of the stateful. Then.

A

B

Two actions will conflict with each other to cater to this problem. We developed a lightweight development development tool.

A

B

Does a very simple thing: it separates.

C

B

And other components up great so with the site.

B

B

It only needs to define and maintain the mirror image of the application.

A

B

Application development of see how does not have to care about deciding us all the circuit components are all defined within the sidecar sense.

A

B

We have dedicated a sidecar maintenance teams to say to the maintenance and soon answer that the part scale is.

A

Scaling up every.

B

Time the parties is filling up, the.

A

B

Said admission will put to intercept this action and whether the pot level is aligned with the selection criteria. If.

A

B

Will inject into the corresponding part the sidecar as needed.

D

For example, on Ali we have me a new world 101.

C

Medium hot and.

A

How can we manage those points.

D

A

D

Let it define psychotic and we.

A

D

Input the definition of the psych arts, what are all those pots online and this psych heart are going to a jet the containers we need in order a portable.

D

So in this way we combined a sidecar combined with separate from.

C

The business first.

D

We have do to us as developer. Developer, do not have to pay attention to cyclonic container and management.

C

And also the miniature not decide heart.

D

Can a city use this icon set to manage all those psychotic, continuous online, wicked or greedy, so that it will not affect the original business.

C

D

Curious a map I've introduced once we have defined to sign cards, sign cards the.

C

Rd and operators behaviors like that, it's going.

D

To engender side cards into orders, the ports that fit the selected conditions, the upgrades and psychic will just begin its.

A

Mechanisms of working these cycles.

D

For example, when the operator want to manage it, it just need to match, with the mirror.

A

D

The psyche art going to utilize years upgrade strategy.

A

To every other cycle.

D

A

D

Don't have to the managers did not have to utilize the the other resources to do that, so it would be a much easier way to couple.

A

D

We know that the psychotic and is so injured, have some similarities, but there was a difference. Is.

A

D

Based on the selected conditions and ejected into the containers and have a specific location of cava, jetting and psychotics provides a richer set of functions. For example, dough is designed maps for the for.

A

The locks and they.

D

And this components need to read.

A

D

From me, so Anita cycles to share that with the containers on data. So this time the cycle has become a.

D

Data mounting data warning wanting new tools that able to share the data volumes waiver with the container and also we are able to in this way we are able to gradually upgrade the online sidecar containers without awareness from from the business people, and also the managers and the users of these containers that actually do not have to acquire these upgrades, and we also have the local upgrade functions.

A

D

You will not feel about it. These few functions, however. We have provided to open source but identities, open crew, open crews,.

C

D

You can visit.

C

This- and this is the output.

D

And all of the output functions has been proved internally.

C

In Alibaba.

A

C

A

D

Demand scenarios.

A

D

We are not going to 100% output. Deliver all those functions, we're going to gradually push some of the gen generic functions into the community so that we can build up the these functions together. I would introduce the.

C

D

Functions and it's the crust is the core function of open crews, projects and also it has providing some of the basic and ad functions. One is for Orchestra and one is the cyber set psycho set. Maybe you haven't heard about broadcaster before we also have some of those straps. It's.

A

D

To combination.

A

D

Between the two regional functions so once the job runs, then it will finish: it's going to dispatch different jobs to different different places across different notes in the clusters. Okay,.

A

D

To generally talk about advanced Mustaine Fassett, we provide two function. One is Maksim edible.

A

For the default stable.

D

Set if their Pro C has choose parallel, however, when is releasing is going to release poets one by one. If we have 500 pots, even though you.

A

D

The partition exactly it into ten patches, but this 50 pots in each batch will still be released one by one, however, the most available able to do the parallel, or at least of these pots together, for example, I know all the temperatures in each batches. The 50 pots will be released her in a parallel way, so it will be 50 times a cell related or during this process.

C

D

Of updating, yes, is in place update, maybe.

B

You have been familiar with it because.

A

D

Since I cassettes, when we're going to inject the side cat into the container, that has other pots that has been selected based on some conditions and become by combining with some containers. They are going to be run on each of the notes across some pieces.

A

D

Going to stop here for today.

A

D

Can contact me on WeChat and also welcome all of you.