Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2021, 29 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Fluid - Build Data Orchestration in Kubernetes - Yang Che, Alibaba & Yuandong Xie, Tencent

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Fluid - Build Data Orchestration in Kubernetes - Yang Che, Alibaba & Yuandong Xie, Tencent

In the cloud-native field, data-intensive tasks such as big data and AI will face many problems in the context of the separation of storage and computing architecture. For example, network IO bottlenecks lead to reduced computing efficiency, and the underlying storage system is under huge overhead pressure. On the other hand, the management of multi-source data is very complicated, which is a challenge for algorithm scientists. In this talk, we introduce an efficient and convenient data abstraction, which abstracts data from storage, provides data affinity scheduling, distributed cache engine acceleration, and multi-source data integration data lake through Fluid. In Alibaba Cloud and Tencent Cloud, a large number of Big data and AI workloads are accelerated through Fluid’s data scheduling.

A

All right, hello, everyone welcome to join our session. This session is about cognitive native data observation framework flow. Fruit is an open source project which is started from september last year. The.

B

A

The maintainers are from alibaba cloud and university a lecture. They have a lot of experiences in kubernetes in big data and areas.

A

First, let me introduce the presenters today. My name is young chung, I'm from alibaba cloud communities team and I'm the founder of the I'm, the co-founder of the through the community. Another speaker is, he is the key contributor of flute project and he is working at times in the club. Now both of us contributed to fluid projects from very beginning.

A

In this session first, I will talk about food projects itself, including the history of the project, why we want to start creating such a project and what problems we'd like to solve and- and you know, we'll go through the overall architecture as well- some major design thinking behind the architecture, and he will also give a demo to show how full speed up the development applications.

A

In the end, we also will discuss the roadmap of the fluid project, we'd like to invite all of you. If you are in turn interested in this project,.

A

B

A

It in this session in in the recent years, we noticed that there is a big trend in big data and air areas. According to partners prediction, 70 percent of the ai applications will run on containers on the kubernetes platform by 2020.

A

In addition, the new stack technology reported that google will replace scheduling, spark applications from ya to kubernetes.

A

By reviewing the the changes in data driven application companies, we found we found that the changes took place to places in architectural level from three dimensions: first, the location relationship between the computing resources and the storage service; second, the diversity of the storage and workload types.

A

Third: the deployment mode of computing is changing from fixed mode to dynamic mode.

A

Let's, let's take a look in 2010 mapreduce tasks run on, hdfs is mainstream and they are coupled time. Tightening in 2015 task tabs are gradually extended, extended from hadoop to spark flink and test flow. The storage tabs were also enriched to various tabs, such as hdfs and saf.

A

Computing and storage were separated, but at this time the ability, the application is still on fixed mode deployment, without ina flexibility and containerization capabilities.

A

Now in 2020 we can see, there are also a lot of changes, tasks and storage tabs will become more diverse, more diverse, this aggregated computing and storage is the mainstream and the environment. It will also become hybrid cloud and coordinates.

A

One of the most typical features is flexibility. Computing is no longer fixed on certain nodes. This architecture can bring many benefits like on-demand creation and automatic elastic scaling. The computing resources.

A

As this data application deployed into cognitive cluster, it improves the developers, productivity and sales cost, but it also brought out many new type of nickel challenges such as is not good at handling history, genius, data sources and it brings data. I o latency, due to the disaggregated computing and storage architecture.

A

Furthermore, it lacks the capability of being aware of data affinity while doing the workload scheduling these challenges slow down. The promotion of data driven applications running on the cloud native platform like kubernetes.

A

Okay: let's go to details the first. Let's take a look at the first challenge: matpot is storage service in hybrid cloud.

A

Data driven applications need to exercise the underlying history, genius, storage architecture, different than service and various low-level data access apis like posix and hdfs, but with the development of the business new ai models to build requires exercising data across different storage systems from various business departments.

A

Together, for example, some data is from site, something that is from hdfs how to use them together in the same application. Does the developer have to write different data of size api in the same application?

A

So we introduce the site a high level abstraction of the data hiding the details of different data sources, implementation.

A

Second, slow data io: this is because the cloud platform uses a disaggregated computing and storage architecture to introduce io or hud, which will make the application very slow under the computing resources like cpu gpu idle in, in addition running on elastic cloud instance, which is creating and destroy on demand, also lead to the failure of the data locating and this.

A

This behavior results the remote data exercise. Even we already have cash in the last run. Moreover, the bad advice, but online of the storage services, such as hdfs saf, also affects the data access pro performance.

A

To solve this problems, we need to speed up the date of science through united states inelastic, distributed. Caching.

A

The last inefficient workload schedule it's because the current community's scheduler is not aware of the data location, so will not reuse the existing cache information in the classroom in the cluster level and is also not able to prefetch the data intelligently for the workload before the run, starting in order to solve this problem. We designed this aware scheduling, strategy on communities, coordinate to coordinate the applications with the cache runtime automatically.

A

Now, let's see what's fruit, indeed, first we provide the decided abstraction that allows the pores to excise. Multiple independent storage systems through assist in the same process and hd advanced apis interface, rather than communicating to each individual storage systems. The port can delegates can delegate this responsibility by connecting through flow, which will handle this underlying storage systems.

A

Second, under the hood of the data set abstraction, we are able to make the cache worker auto skill and portable.

A

In the end, the information of the data site can also be used by cognitive scheduler to improve the data affinity, scheduling. So fruit is an elastic data, abstraction and the acceleration platform in cloud native environment and the fluid is also a sincere sandbox project and it's also uh open source community.

A

There are many contributors from different companies and they are working together to discuss the features and contribute code in the next part, juendo from tencent cloud will introduce the fluid architecture and how to use it and the roadmap of the community community. Thank you.

C

Hi everybody I'm yuan, dongshi from tencent cloud thanks to youngs for explaining what the fluid project can solve. Now, let me introduce the architecture of fluid from top to bottom. First, the user need to define two custom resources. One is this site and another is runtime. The relationship between this site and the runtime is one on one. So far we spoke to three kinds of runtime location, time, gender and the good fs runtime and inside the fluid operator there are two major components: the one on the left is through the controller manager.

C

It is responsible for scheduling this site included. We convert the scheduling problem of this site into the scattering problem of data acceleration engine. It contains three components. The fourth is this side controller.

C

It manages the lifecycle of this site and including creation, bounding and unbounding region, time and deletion.

C

The second one is runtime controller. It manages lifecycle of runtime, including creation, scaling and the scale out catch privilege and clean. The luxury runtime is one of the runtime implement the third one is volume controller. It is responsible for the creation and the deletion of data volumes related to the data site on the right is the scheduler.

C

It is responsible for assing the pose to appropriately node with the information of the data cache. There are two components: catch call, locality, plug-in and the prefetch plug-in catch co-locality plug-in will schedule the job to catch. The node intelligently, no need the user to specify it, and the prefetch plugin will notify the runtime to profit data to a specified node which the job will consume it.

C

Next, let's quickly introduce the use of fluid the co component in fluid. Is this site, which is essentially a crd and provides abstraction of data? The operation of creating at this site is very simple data cells from idc and the cloud can be accessed through unified interfaces and through the crd definition.

C

At the same time, users can defend the scheduling information of data. The following example comes from a real customer ai training case. Its training data is relatively large and the data didn't say, tradition is completed.

C

It is placed on cloud object, storage like s3, but the validation date is sensitive and cannot be placed on a cloud it needs to be placed on on the idc storage.

C

Like save on one hand, fluid crt provides a unified wheel and, on the other hand, it provides the ability to accelerate the distributed catch so know the affinity you can dynamically move data save data to gpus instance on the cloud at the time of training and accelerate data access, when the training is not required, data can be migrated to low cost cpu nodes avoid the use of gpus and network dedicated land.

C

The method of creating this site is very simple: the user only need to defend the yammer field, as shown in the top right of the finger on the right edge once created.

C

You can see the total amount of data in this site and how much data can be catched in total and the cash ratio, so data information can be used to trigger elastic scaling later under this side can also cite these cells from oi size calls. R3 hdfis, safe all, can be accelerated through distributed catch and can also accelerate exciting pvc means persistent volume claim.

C

In this example, we use luxuries runtime to accelerate the deep site. If you need to deploy an application that uses this site, you only need to need to specify the name of the site at pvc and the scheduler will slide a property node based on the data.

C

Catch information, select node with cache capabilities first and then select the nodes with more cache data, and this is transparent to any user.

C

At the same time, we also spelled the automatic and the elastic expansion of this cider cache volume.

C

In this example, we also use a luxury cache engine as a default runtime default when loaded current studio reaches a certain percentage. Data eviction will be triggered which will affect the throughput of data access.

C

Therefore, we provide an automatic expansion mechanism when voted cash reaches a certain percentage. We need to expand the distributed cash flow to provide great catching capability.

C

In this demo, we will show how to use fluid accessory to the machine, learning training job and we will compare the train job, speed and the final accuracy result of the task, and we will also put this demo on our github for your reference.

C

First, let's take a look at the factor of directly reading 20 data through object, storage for training.

C

Here you need to define the pv and the pvc of object, storage in advance, and the sensor driver of pv needs to be designated as object, storage.

C

After creating pv and pvc, we can view the data site through the cable flow arena commanding line.

C

Next, we use arena to submit deep learning tasks here we specify the data parameter as the data side name.

C

We will check the log of training tasks. We can see the imagine loading speed. Drn training is about 638 images per second.

C

Under the final training, accuracy of top file is 92.26 percentage.

C

The enter task runs for total of one hour and four machine, and every machine has eight. We 100 gpu card.

C

Let's take a look at the training effect after cache acceleration through fluid. Here we take the luxury cache engine as an example.

C

First, we defend the date site, specify the source address of the training data storage in the cloud object, storage,.

C

After creating the data site, you can also view it through arena command, lab and create deep learning tasks in the same way.

C

We can see that we can design. Imagine loading speed during training is about 2800 images per second and the final training accuracy of top file is 92.5 percentage.

C

Comparing the two experiments, it is fantasized without affecting the final accuracy of training and the youth habits. Your fluid can comprise the ring time of the task from one hour to 24 minutes under training speed it increased by 4.5 times.

C

Up to now, we have released reaching 0.6.0.

C

Which means satisfied the use cases of the infrastructure engine who can simply under efficiency, effectively use catch to accelerate the air application.

C

B

C

1.0 is released. We hope that the infrastructure engine can achieve performance, acceleration of the big data and ai workloads without having to care about data management.

C

In order to achieve this goal, we need to implement the following features before reaching one 1.00, the first story to further accelerate the big data application on kubernetes through food and on the basis of already spotting ai applications.

C

The second is improve the locality of data through scheduling optimization, thereby improving the efficiency of data access ministry, scheduler, comprehensive scheduling of data under workload.

C

The third is to ensure the ability to manage lifecycle of data site in large-scale application scenarios through performance and steady state probability optimization in the community environment.

C

After introducing the 1.0 release, the specific plan can be divided into the following parts: the first part automatic operation, matinees and observability in order to ensure the stable operation of the system through component self recover, multimaster sport and the small upgrade to achieve runtime power, availability, surveillance, declarative crt such as metadata backup and the recover.

C

The cash migration did pre-room ad capture to achieve the purpose of automatic operation and maintenance. By talking with promisius, it automatically collects data site indicators to ensure the theater, minitory and learning capabilities.

C

The second part, a chest ratio should decide scheduling strategies such as affinity or tolerance schedule the to the other node.

C

In addition, the catch location information is reported to guide the workload scheduling to nodes, further speed up the data access part three most runtimes boat concurrently, the community has spotted a luxury jinder wifis and the good stuff catch engines, part 4, fluid agent, which can obtain node cache information, so agent pool or push module can also implement catch, clean and other operations through the agent at catalog part.

C

Five elastic scaling through the definition of elastic scaling strategy lies the horizon and the radical expansion and contraction of the catching stance to adapt to the scenarios of workload electrically part 6 application access model pass, voting fields, means field system in user, space interfaces and.

B

C

Means high double compatible field system interfaces flows can play a role in air and the biggest data scenarios.

C

The community is currently developing steadily and there are greatly more than 18 contributors from different companies and organizations.

C

If you are interested in participating and have questions, you can conquer us through cncfs like and search through the channel or you can concur us to github, we will hold the community meeting every two weeks on wednesday in beijing time. We will discuss the progress of some features and share proposals at cadre.

C

At the same time, you can visit our website for more information. Welcome, join us at brand. Many companies have have already used food in production environments, for example, voice air company unison, autonomous driver air company commodity, ai social network company vivo, and the information service operator channel telecom are all using fluid to accelerate air cloud and security related internet company 361 is using float to accelerate big data applications at the same time, alibaba cloud and the tencent cloud also integrates fluid acceleration service on a container platform based on companies.

C

I would like to thank you for your time and open up for question and answer. If you have any question here, thank you for joining.