Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Intro to KubeEdge: Kubernetes Native Edge Computing Framework - Kevin (Zefeng) Wang, Huawei

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Intro to KubeEdge: Kubernetes Native Edge Computing Framework - Kevin (Zefeng) Wang, Huawei Technologies Co., Ltd.

KubeEdge is an open source edge computing framework that extends the power of kubernetes from central cloud to edge. In the year 2020, KubeEdge made big progress on user adoption, community development, cross-community collaborations and successfully moved to incubation level. In this talk, Kevin will review KubeEdge motivation, architecture; then go through latest updates on new features, user adoptions, SIG updates and new subprojects. After that Kevin will introduce where the project is heading to, updated project roadmap and how new contributors to get involved. There will be an open Q&A for attendees to ask questions.

A

Hello, I'm kevin wang, currently working on qbh project. Today I will introduce qh the kubernetes native age computing framework to you guys, so the talk basically come up with four parts. First of all the cubase brief, then the community updates and after that we will go through the latest, the use case and also the uh virtual plan.

A

So the kubej project was started in 2018 targeting on providing the extending the kubernetes and the cloud native technologies to the aeg computing cases.

A

The project was donated to cncf in 2019 and last september we moved to the incubation level.

A

Now the pro the community have around uh four thousand github stars and one github folks, since joining cncf, we received over 600 contributors from all over the world and including around 20 200 codes of meters from over 60 organizations.

A

Since we last met, the community now have four cigs and working groups, including the aic, the device rdc, the mecc and the working the wireless working group so from the list of the community contributors and the adopters. Actually, you can see there's a very good diversity over the industry.

A

We have the uh the contributors and adopters from the hardware iot companies, and also the uh there are teleco operators and the iit service provided providers working together in the community and also there are a lot of cloud service providers serving uh providing the cubah services in the uh in their products and also we have a lot of uh academic organizations uh doing research around the cuba for the age, computing platforms and applications.

A

From the overall architecture, uh cubbage is actually targeting on providing the functionality of of managing both the nodes in the cloud and the nodes on the age within the same kubernetes api cluster.

A

So in the cloud we have the cloud core, which is the component to deal with the shadow management for the board, the applications and the resources on the age on the edge. We have the all-in-one component, the h-core, which provides the uh node level uh autonomy uh functionality and also uh uh it's actually uh optimized for the uh low resource uh hardwares and to better integrate with the uh the iot devices and the uh the different protocols used in the industry.

A

Courage provide the extent extensibility framework, the device mapper to simplify the integration with uh different protocols, including mqtt, modbus, bluetooth, obviously, uh and a lot of other uh protocols.

A

Users are able to use the cubah crds to define their own device protocols, the uh device types and also easy to uh integrate with the producers by develop developing their own device. Members for the data plan to simplify the the service communication between the applications located in a in different networking on age qh provides the edge mesh framework to uh to serve the online network agnostic layer for the applications to discover each other and easily communicate just as the same experience when they are running in the finance cluster in the cloud.

A

So for the key features. uh One thing we added in the recent uh release is that cuba hd now support the uh native api endpoint excess on edge so by the autonom autonomic uh cuba, api and the point on the age.

A

uh The applications, the operators or the other uh plugins that relies on the api or even the crd mechanisms, are now able to uh run very well on the edge. Even when the nodes are disconnected, the api is still uh accessible from the node. So that means your application will not be disrupted.

A

Besides that, cuba hr uh good at providing the seamless cloud and age of coordination mechanism by providing the bi-directional communication terminal to uh make it able for the system to manage the nodes, even when the nodes are located in private southern subnet or the even the node. The network between the cloud and age are very limited. For example, there are very high latency, the package loss are very high and to implement the node autonomy are actually qh persists. The bit of data on every node.

A

That means even the nodes are disconnected from the cloud for a very long time. It can still uh recover or get ready very quickly since there's no uh any uh requests that needed from the uh the responsible, depending on the response of from the master on the cloud and the cube age optimized the oem1 component in the cloud which takes around only 70 megabytes.

A

It also supports the uci conformant runtimes, including container d cio for the less run time overhead cases and also support like cutter container or vetlet.

A

Architecture for the security concern use cases and to simplify the uh the industry, the iot uh devices integration. Cuba provides the uh the the device uh api and also the the underlying device protocol extensibility uh framework to uh make users able to define their own device portal or integrated with their own device types.

A

And another thing is that the cube edge provides the arm v7 and um v8, as well as x86 as the native uh hardware architecture uh in the overall project. The development cycle, including the code code, development, shipping, building testing and also the releasing so how it works. So basically, for example, when deploying a container to the age node. So what kubeh did is actually it hooks the uh the vanilla pod spinning up uh process.

A

So after the the scheduler made, the decision, the cloud core from the cube edge will got the uh first, the notification and forward the part information to the corresponding nodes located on the age and then, after edge record. The uh updates, the meter manager will persist. The uh the part information on the node and then forward it to the uh hd or cubelet and the rest of the steps are just as the uh the same as the uh the two net running in the cloud.

A

So what are inside the cloud core? So basically, uh the most important thing is cloud hub. It's a uh symmetric uh component with another another one, the hub in the in the age. So basically, these two components are providing the uh the messaging over a websocket or over a quick to deal with.

A

The bi-directional communication over the different quality of network, or even the ensure the uh data uh synchronization are, can be successfully uh received and processed.

A

The age controller are uh providing is providing the shutter management for the core communist core apis, including nodes, pods, config maps, etc on the edge the device api and the device controllers are for the the iot and agent device modeling and the integration.

A

It also provides the shutter management for the devices on edge. The sync controller are reasonable, responsible for inconsistency detection and triggering the reconcilement.

A

The cuba gcsi driver is actually a plugin to hook the storage, provisioning, etc, uh requests to the age to make it easy to integrate with the existing cs. Csi drivers, csr, battens, that are running on age.

A

The admission hooks are responsible for the uh the extent extended uh api validation and provide the best practice, including um if best practice enforcement, including the um the uh uh automatic uh part autonomy configuration.

A

So what are inside the agent core agent core is actually uh building together with hub the meter, manager, uh hd and investing and event bus, so uh media manager are dealing with is dealing with the node level on metadata persistence and also it now serves the uh autonomic cube apis endpoint on edge to support uh the api access request from the uh the plugins and the operators running on age.

A

The hdd is optimized, the lightweight cubelet and device screen deal with device status synchronization between the cloud and the age and also provides the direct access for the uh applications from the uh on the edge nodes to the devices.

A

So what are new in the community?

A

uh We now have a special inter interest group focusing on the uh ai for clothes running better around the underage, and also uh researching on a better pattern of the workloads running both on the hr and in the cloud, including the, uh for example, the joint influencing the incremental learning or federated learning stuff.

A

So, in this segment we will uh first of all verify the integration with the mainstream ai frameworks to make sure they are able to, uh and around uh run uh appropriately on different type of age, devices, agents, age servers and also we will keep researching on the uh synergy mechanism for the uh workloads and also uh we will define the benchmarking uh stuff for the uh relevant uh research.

A

So the sedna project is actually one of the uh toolkit uh working on focusing on this set of work.

A

So in the center project, we're trying to provide the uh h cloud, senator ai framework by uh providing the data set and model management across the cloud nh, and we also tried uh trying to provide the uh h cloud synergy, training and the inference uh frameworks to support, including the joint influencing uh incremental training and the federated learning.

A

So uh in the uh uh architecture you can find out that actually we have a centralized global manager in the cloud to deal with and coordinate with all the components on different edge nodes and on each stage. Node will have the local controller to support the uh the ai workload synergy pattern, including joint inference, increment learning and federation learning.

A

So for the uh application developers they actually uh are able to use uh stereo able to use like the uh tensorflow, pytorch or the other uh ai frameworks. They just need to import and uh helper library to uh to expose some of the measurement data to the uh sedna project and then the setting up project the framework will automatically automatically uh hook the, uh for example, the inference function to make it able for another try when the local influence on age fails.

A

So the device iotc is focusing on providing the better extensive extensible framework to easily integrate with the different devices and the protocols.

A

So the errors or focuses are including uh the api api obstruction and extensible the us community communication framework and also including the data management workflow mechanism, and also to simplify the mapper development for the mec sig. We actually are focusing on doing the research to design and implement reference architecture that cubbage is used to empower the mecr platforms.

A

uh The re the works uh falls into uh this thing are uh including the services, for example, the service discovery and the communication between different network and also the underlying network functionality, better integration with the and better explosion to the container network or to the uh higher level layers, and also uh we will verify and do more research on the uh I'm using infrastructure layer, including underwear virus hardware, acceleration and also the particle analysis. Cluster multimedia management.

A

The wireless working groups are really uh targeting us uh optimizing cubbage to better running on the uh different wireless infra environment, including the low quality.

A

Wireless network, and also uh including the cases that the each nodes are frequently changing the location and also the age nodes may randomly get offline very often, and also we will think about and do more research to find out how to optimize the uh the uh quality requirement to the underlying network.

A

So new use cases first, one is that uh they're managing the overdr monitor devices and the sensors sensors on the world's longest, uh the cross c bridge. So this is actually a bridge tunnel system uh uh over the sea, which is around um 30 kilometers long and there are. The network, are mainly uh wireless network with very limited bandwidth and with very high latency, and also there are a lot of uh sensors and monitors that provide by different providers, and you use this different protocols.

A

So kubernetes is very valuable in this case, because they are able to uh it's able to easily integrate with different uh device types and the different device protocols, and it uh cuban provides the functionality to able to manage the containers on the agendas. Even the even the network, between cloud and hr are very uh limited.

A

Another use case is the uh the by the china mobile they're, actually uh using the cloud age sender, uh synergy uh platform to manage over 100 subsidiary corporation from the central uh cloud. The network environment, is that all the subsidiary corporation needed the net to the uh central cloud. That means the center all the uh components in the central cloud they can have floating ip, while the components on the edge are not so.

A

Cubbage is very uh useful in these cases to provide the bi-directional terminal to simplify the remote publication, monitoring management and debugging from the central cloud, and also the applications located in different network are easy to can easily talk to each other without worrying about too much about the learning online network, environment.

A

All right for the uh longer term this year, we basically want to uh provide the data plan. uh Communication functionality across different uh networking environment for the applications located in different subnets were located in both cloud and age. We will also verify the integration with the existing third-party c9 plugins to help users easy easier to integrate with their existing software stack, and we will also implement improve the device management extensibility and simplify the uh customized customization uh work uh when uh people adopting cube h, uh especially the iot or manufacturing or in other industry cases.

A

For the platform itself, we are working on to support multi-active, a train mode for cloud core to support a larger uh scale of a single cluster, and also we are working on to develop managing the clusters management on age.

A

For the community perspective, uh we will uh we were talking there. We are targeting uh improving the contributor experience and also hold more contributor events to welcome the new developers to become contributors and the contributors to grow in the community.

A

We are also expecting to have more cross-community collaboration with the other open source project and organizations, including the aerophage, the korino, the uh uh uh ethps foundation, et cetera, and here are the useful resources of qh community. So we basically uh hold uh week limiting, uh alternatively, uh between the pacific friendly time and europe-friendly time. So, if you have any questions or any ideas to discuss, please feel free to find us on github in slack or on the community meeting.

A

Okay, that's all about the uh talk today. Hope you enjoy. The show see you.