Cloud Native Computing Foundation Kubernetes on Edge Day EU 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: KubeEdge and Kubernetes help manage all the monitoring devices on the world's longest cr... Huan Wei

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

KubeEdge and Kubernetes help manage all the monitoring devices on the world's longest cross-sea bridge - Huan Wei, HarmonyCloud

With a total length of 55 km, the HongKong-Zhuhai-Macao Bridge is the longest cross-sea bridge in the world. In order to watch and maintain the safety of the bridge, a large number of sensors are deployed on the bridge, such as temperature sensor, humidity sensor, PM 2.5 sensor, vibration sensor, noise sensor, etc. Then each sensor continuously collects data, processes data locally, and reports the abnormal information to the cloud center for alarm through real-time inference of AI model. In this presentation, Huan will share his experience on: 1) An investigation on common cloud native edge computing frameworks, and the reason why he chose KubeEdge. 2) Introducing how to mapper every kind of devices on the bridge as CRD in Kubernetes. 3) Practice on how to manage applications deployed on edge nodes. 4) Lessons learned and further planning.

A

Okay, let's start um today, my topic is uh cooper edge and kubernetes help manage all the monitoring devices on the world's longest rc.

A

Bridge, okay, uh first, let me introduce myself. uh My name is I'm from a chinese company called harmony cloud, I'm the chief of dutch and I'm responsible for for ed computing. I contribute to projects like ks and kobach, and this is my email. If you have any questions about computing or you have any scenarios about any computing that you want to discuss with me, please write to me.

A

Okay. Today, the uh the content of my speech includes the following following expats the first. I will um give a brief introduction about the crash, including my uh investigation of the current mainstream cloud native and computing frameworks, and why we choose kubernetes, and then I will focus on how we apply the the kuber edge to the uh hong kong, draw high micro bridge project, including how to define the devices on the bridge, how to associate each device with the cid in ks and how to manage and operate the applications deployed on edge nodes.

A

And, finally, I will briefly share some of our best practices on health computing.

A

uh Okay, this is a uh overview of kubat. Kuberas is the first cncf incubating edge computing project. It starts from november and 2018, and now it has more than 900 force and almost 400 stars um and and also some key points from the perspective of the architecture.

A

First, it's very compatible with the equip api and it's very stable and because it has a very reliable message, push from cloud to edge and it, and it can be very lightweight uh we can. We can tailor the the led and also is, is tailored from uh converted and it can support um some very wide area, no success and uh sports, very large skill devices access and- and finally, it has the uh the edge autonomy.

A

And uh also, I investigated some cognitive open source on edge computer frameworks like k3s, on yachts and super edge, and of this.

A

These frameworks are uh become popular yeah in in china in china, but um while we choose uh cooperage and from my point of view- and I think we have the some reasons first from my point of view- k3s is a solution to manage engine nodes at local and the k3s does not provide a solution like what earlier node access and uh both open arch and super edge do not provide the lightweight solution for node access and device management, and uh both open yacht and super edge may have the stability risk in the case of cloud edge network fluctuations.

A

We know we all know that um it's caused by the lead risk problem of informer.

A

Okay, so uh so what's this fast about the long, the world's longest privacy bridge with total length of the 55 kilometers, so hong kong, through high micro bridge, is the longest across the bridge in the world and in order to monitor the safety of the bridge. At times said a lot of number of sensors are deployed on the bridge and each sensor will generates regenerates large amounts of data. All the data need to be created and processed in real time, and any abnormal information from on the bridge need to be alert immediately.

A

But there is some some problems in context. It says large amounts of monetary equipments and.

A

Say large, very large number amount of data and has very low value density. That means- and there are lots of invalid and duplicated data that we don't have and uh the bridge is very long. It has a long instance under the very difficult construct, construction and also, if we want to position the the problem of the pitch, we will need. Obviously we will need to see 5g transmission.

A

And so uh here's our solution, we use the 5g communication.

A

Plus beta position and plus edge computing and and and we use kubernetes and in so in this way, uh that all the data created from the sensor, you protest will process locally at edge and any abnormal information uh could be found through real-time ar influence.

A

And this ai information program also runs at edge and all the app applications are managed at the cloud and the arc key can all be. It can be distributed by clouds and supports the dynamic operation and the maintenance at the edge.

A

As you can see, we will have many add box deployed alongside the bridge these edge bars. It also uh means edge node in the group edge framework and also we use. We have the 5g antenna and many sensors like here is. The sensor is a shutter sensor, and here you can see uh how the audi capacity is on the bridge on the on the left. It's the edge box.

A

It is composed by a cardboard and the base plates and on the right uh here you can see the many many other fences like the snow sensor, microphone and the environment, aware integrated structure and also the imu totally we can.

A

You can measure for 14 types of sense, sensor, data.

A

And here is a picture of the structure. This structure can crash light intensity, carbon pressure, noise, temperature, humidity, yeah and also, um but how to manage this structure as a medicine structure, so in in equipment in the in the cooper edge um we can.

A

We can define this device, uh we can define the this kind of shutter device uh called device model and we can define each device instance use this model.

A

Here's here is the the yaml file of the device model. Here we can see we can define many properties regarding this, this device model you can have the property name and the property description, property type and the method of property collection yeah. Currently, here you can see read-only mode and also here you can see they are different and we can define many different dress model properties.

A

And also we can have an yaml file to define the the device here. We we define the the shutter difference instance, and here we can in the specification we can define the reference device model name and the uh protocol and the uh we used. Yes, here's the mode box and also the slave id, and also um the uh like the serial port boundary data bits priority bits. These are, these properties are all uh special. They are specific, easy see, structure yeah here also we can see there's a segment regarding the data.

A

The data means it set. The data field for the third part data push. This means the data created from this shutter can be pushed to the cloud, but it also can be pushed to some third-party data storage.

A

uh Just like, uh like nifa db, and here also, we can also see you can see the status. This segment is used to define the data reported to the cloud that is, and also means the drastically addressed screen.

A

Here we, the many types of the trains like uh yeah there, you can see the last yeah, the carbon, the pressure and the humidity and um and we can also um deploy and manage uh the different applications to the edge from the clouds and currently yeah.

A

Basically speaking um and the targets that the tabs deployed to the edge can be can be summarized as three types when use the business applications like engineers, pump, cache, etc, and we and all the device member program that we that we we write to class to the clutch data and push your data to a cloud see all um they are deployed as container handlers, and this consists containers are also divided- are also deployed to the to the edge and also there's a type with another problem.

A

Like the ai influence programs said in order to do the inference work immediately at edge.

A

And this is um a picture of the whole uh whole whole architecture, from the clouds to the edge and to the sensors yeah. We can see the clutch clutch, we have the kubernetes and the uh the claw core.

A

The claw cord is the the keyboard cover buttons, of course edge and at the edge you have, the uh the edge called edge called as coil is the key component, the key component of cobalt at edge, and also here we can see how mqtt will occur, and here we also, we have many components in other, should do the uh data, collect data collection, data processing and the data uh conversation, and here the cloud is responsible for connecting to the bridge health, the bridge health monitoring system and managing the edge nodes and add applications and the edge knowledge is responsible for uh for.

A

For uh for connect various sensors of the bridge and uh push the other data to the cloud, and here you can see- we have already connected many many sensors at the edge.

A

And also, finally, we also, I have also have some best practices.

A

First about the data collection frequency, the data question frequency of the mapper actually before will directly affects the cpu and the memory resources and occupied by the mapper and and generally speaking, the system. Resource consumption is proportional to the collection frequency of the mapper and when the edge resources are limited and the correction frequency should be strictly controlled and, uh generally speaking, there are two types to push later uh that correct uh from mapper.

A

First, uh you push to cloud call. uh As we all know, the sql writes installed in the edge of the edge. You only store the latest piece of data from the current device and the edge cloud does not provide storage for a large amount of data creation and the uh second.

A

Second, we can push the data to the database, and that means, and after the mapper class data it does not uh push it to mqtt at local, uh but uh can be directly pushed later to the third party developers like input db and also I have some best searches um regarding the data reporting frequency and, generally speaking, the data reporting frequency of mapper is also uh proportional to the h-cord resource usage, and so, if either source is limited, the data reporting frequency should also be controlled and about the catch.

A

um That means when the cloud and edge node is in a weak network, all intermediately disconnected. We hope the business data at any node will not be lost, but the storage space of the edge node is also limited, and we hope that the edge catch could be designed as lightweight solution. That means, once network is good. The ads can directly push data to the cloud.

A

When the network between cloud and edge is disconnected and then the edge nodes will catch later and when the network is restart again, the data will be synced to the cloud using uh like a scheduled task and about the ar model warm up at edge. That means- um and this, uh I think, is very useful.

A

You just can't use like uh with the edge ai. We need to update and add an ar model in seconds, but sometimes the image size of their model may be very large, and the network of edge nodes may be very poor in this case. If we don't adopt um any image warm-up function, then the ai influencer work view likely to be interrupted during the image updates.

A

uh This is what we don't want to want wanted to happen. I think, and also uh we need some massive anti-node access.

A

For example, if we have more than 100 thousands nodes need to connect in this value of the massive node in excess, we all know we can guess that egcd api server and the cloud call are unlikely to become the performance bottleneck.

A

Generally speaking, there are two ways to resolve the police problem. First, by simplifying the node status object report status, object the traffic in patch on the car call caused by the large scale edge called reporting, information can be reduced and the second use the dynamic I use some an operation like the dynamic effect of organization.

A

We can avoid the same, simultaneous excess of a large number of action nodes. Yes, this is also very useful and finally, uh in the case of my of the massive node excess, we also should adopt some scheme like as long as the edge node is powered on and the connects to the internet.

A

Then the cloud site should automatically discover the edge node and automatically install any edge components yeah. I think uh this is all my my lesson learned and the best best practices yeah from the for the coverage.

A

Thank you. Thank you.