Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Akri: Making IoT Devices Accessible to Your Edge Kubernetes Clusters - Kate Goldenring & Jiří Appl

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Akri: Making IoT Devices Accessible to Your Edge Kubernetes Clusters - Kate Goldenring, Microsoft & Jiří Appl, Microsoft

As Kubernetes solutions move to the edge, nodes are taken out of the homogeneity of the cloud and put in environments surrounded by an array of IoT devices such as sensors and cameras. These devices are often too small to run Kubernetes, so how can they be dynamically leveraged by Kubernetes workloads? Akri is an extensible open-source Kubernetes-native project that aims to provide a solution. Akri provides an abstraction layer, removing the work of finding, utilizing, and monitoring the availability of IoT devices. We’ll dive into how Akri can discover IoT devices, advertise them as resources in your cluster, and then automatically deploy workloads to utilize those devices. Akri handles the dynamic appearance and disappearance of devices and automatically creates services, removing the need for an application to track the state of devices, pods, or nodes. Come learn about Akri and how it can be leveraged to build a more connected edge with Kubernetes!

A

Hi everyone, my name, is kate goldenring.

B

My name is yugi apple.

A

And we're both software engineers at microsoft, working on an open source project called ocry, and today we're going to talk about how you can use ocari to make iot devices accessible to your edge kubernetes clusters.

A

So we'll first start off by talking about the motivation behind ocari and then we'll look at an iot edge scenario and how it can be simplified. Using ocry, then we'll dive into awk's architecture and look at one of awk's latest enhancements that make it more easily extensible and finally, we'll talk about how you can get involved.

B

Okay, so a bit of motivation, a typical cloud: kubernetes application has a fixed set of dependencies. These are other cloud services that the application depends on for their functionality, and these services are generally highly available. On the other hand, an edge application depends on variable number of data sources, especially in each application that does any data processing.

B

Now these data sources, they might not be highly available because they might be represented by a simple devices such as ip cameras, temperature sensors. Your edge application can also do control such as control of robotic arms and again, these are not highly available sources.

B

So then, the question is: how can your application discover these uh then, when there is a variable number of them?

B

We we cannot put these devices to be part of a kubernetes cluster, because many times there are single purpose devices they have a small footprint or they have not been certified to run kubernetes, and so how can your application take a dependency on them?

B

When we looked at this, when we look at this, we couldn't find a standardized way to achieve this, and that's where we came up with agree. Akri exposes the individual iot devices into kubernetes clusters extended resources.

B

This then allows your application to take a dependency on them. Similarly, like you would do on any gpu, where you can ex, where you can request uh access to the extended resource acry itself is, the name is an acronym. It stands for a kubernetes resource interface for the edge and it also stands for edge in greek.

B

Now akri works in two phases. First, it discovers and advertises the iot devices to the cluster. It then monitors the availability of those devices to make sure if a device disappears. Maybe it's disconnected or starts malfunctioning. It stops advertising it to the cluster. Second, this is an optional piece.

B

You can request accrete to schedule a workload when a new device is discovered. This allows you to dynamically in a plug-and-play fashion. Schedule your workloads if and only if the devices are available now to achieve this akri has an extensibility point called discovery handlers.

B

We don't think that any project can handle discovery of all iot devices.

B

There is just so many protocols out there, and so the discovery is offloaded to the discovery handler and, uh and you can take a dependency on several discovery- handlers that we have. We have already available with acry, so we support onvif for ip cameras, udef for locally attached devices, an opc ua for industrial settings, but you can also bring your own or take dependence uh dependency on any other discovery handler coming from a community.

B

Acre itself is purely open source project. There is no microsoft hidden source in it. It is released as part of their slabs side by side with other days. Labs projects such as helm, akri has been built uh in rust, but it doesn't mean that the device discovery handlers needs to be written in the same language. You can pick any language of your choosing.

B

So now, let's look at a typical iot scenario and let's see how a creek can help say: you're building a smart farm, you will you'll care for your crops and for your animals for your crops. You might want to deploy some soil, moisture and ph sensors some temperature sensors for your animals.

B

You might want to deploy some ip cameras and microphones to ensure there are no predators attacking your animals, so so, in order, in order for you to take advantage of all these sensors, you need to have some application doing the data processing, and so so, let's zoom in specifically on the ip camera scenario and see uh how you could design your application.

B

First, we'll go over the scenario without agreeing to see to understand better the complexity involved. So say you start with. You know your ip cameras deployed and you need to deploy the application doing the processing we'll take a simple micro, services-based architecture, where we have a frame server, port running for every camera.

B

It's processing the rtsp stream extracting individual frames and passing it on to the inferencing, which takes advantage of gpu and ml model to discover any predator and when a predator is found, it sends an alert, it's all great, except for the fact that we are running on a farm not in a data center, and so we might not have the connectivity and the bandwidth necessary to push a full video streams or multiple video streams into the cloud. And so, instead uh the data attracts compute data gravity.

B

So so we can deploy the kubernetes cluster onto the edge. It's a simple kubernetes cluster. To begin with, uh with a single gpu, we have our alerting port running in the clusters and that's okay, because that produces a low bandwidth stream of data that we can push onto cloud for additional processing uh for fun. We'll also keep track of how many steps an operator needs to do to provision all the pcs uh into the edge cluster for your application.

B

So we have the ip cameras and in the micro service version, we have deployed a frame server port per camera, and we have to manually provision the configuration.

B

The connection string so that the frame server can start pulling frames from its assigned camera once we have that we need to provision uh an inferencing uh port with a ml model and attach it to a gpu so that we can start doing processing this inferencing port needs to understand what are all the frame server port assigned to it.

B

So it can pull the frames and start processing them and then, when, when the processing is complete uh and the predator is discovered, we send an alert the same way as we did, but in the cloud time now, what happens if a po? If a camera goes down so maybe say it goes disconnected or it's malfunctioning.

B

Well now you have a frame server, pod sitting in your cluster and either idling or it just keeps restarting, that's not great, just consuming resources, so you might need to send an operator to go and dis, remove it from from your cluster next say you want to scale well now you need to again go schedule. Multiple frames reports provision them individually with the connection string to the cameras so that they can pull on the individual frames uh say you want to scale your cluster okay, you deploy a new node.

B

You need to again go and manually, deploy the inferencing port to take advantage of the other gpu. uh What if any node goes down?

B

Well, maybe you re-partitioned, maybe you re-partitioned, uh you will need to re-partition which frame server port goes to our process on which gpu, and so there is a lot of manual steps you kind of have to go along the way in order to create an application that can react to changes in the physical environment surrounding the cluster, and so, let's see how we can simplify this flow with acry, we'll start with the same set of cameras, we'll deploy the same kubernetes cluster, but but we're going to deploy acry onto this cluster. That's the green box.

B

Kate will later show us what are the uh pieces of akri, but for now, let's focus on high level, so with acre in place. Instead of manually deploying the frame server, pods we'll deploy accrete configuration. This is a custom resource which allows you to declaratively state. What are the devices you want to expose into your cluster?

B

It's composed of two main pieces. So first we request a discovery handler. We are using here onwave to discover ip cameras in practice. There will be additional configurations such as filters, and then second, you specify in for the plug-and-play part you specify the broker you want to deploy.

B

uh That will take an advent that will expose the device into the cluster. Now, in our case, that's the frame server pod. This broker will be automatically provisioned with the configuration to access the individual cameras. In our case, it's a rtsp url.

B

Furthermore, akri also deploys a service, that's the blue box sitting behind the frame server pods, and this aggregates and load balances across all broker pods that are deployed on behalf of single accurate configuration, and this allows us to access all the frames and reports in a uniform fashion.

B

Next, we need to take advantage of the gpu. We need to do the inferencing, so similarly, we can use acry, in this case our udev discovery handler to detect the gpu and when a gpu is detected, deploy the inferencing port.

B

Now, in this case, say again. If any camera goes down, that's okay. Acrylic will discover that the device is no longer available and it will automatically evict the frames report. If we want to scale all we need to do. Is we just connect? The ip cameras acre will automatically discover them provision the frame, server, pods configure them correctly, and our inferencing solution didn't have our application didn't have to be changed in any way on from the operator site.

B

If you want to scale the cluster we add, a new node acrylic will automatically detect it because we have provided the gpu configuration, deploy, new inferencing, both and again application scaled in completely kubernetes native way, without the operator having to go and make changes based on the changes that happen in the physical environment surrounding the cluster.

B

So with that I'll transition over to kate who's, going to walk us through the details of the green box,.

A

Awesome yeah: let's dive into this green box, that's awkwary's architecture, so ocari is kubernetes native. It's made completely of kubernetes components, the first of which we saw earlier is awkry's configuration and that's where you tell ocry what you want to find and you do that by specifying what discovery handler you want to use to discover devices.

A

So this could be onvif like we saw earlier for ip cameras, udev for local devices such as gpu or it could be opcwa for industrial machinery or it could be your own custom discovery handler and we'll talk a little bit more about the development lifecycle of creating your own discovery handler later.

A

Also in this configuration you specify what custom workload, if any you want deployed automatically to discovered devices. We call this our broker.

A

So once you apply your configuration to the cluster, the hawker agent will see it and it will tell that discovery handler to start looking across the network or locally on the node for those devices for every device that discovery handler discovers.

A

The agent will expose it to the cluster using the device plug-in framework and also create our second custom resource. The opry instance to represent it and its usage.

A

The controller will then see this instance and if a broker pod was specified, it will automatically deploy this broker to the discover device and inject in it as environment variables, all the information it needs to connect to that specific device. So, for example, if these were ip cameras were discovering, maybe that would be an rtsp url for the specific device.

A

Let's look at the flow of how you could use ocry and in this case say we have some custom application that wants to utilize the frames from ip cameras that are around the cluster.

A

So the first thing you need is a kubernetes cluster, so here we have one with a control, plane and two worker nodes, and it has all the kubernetes components already a part of it. So at this point uh we can go ahead and deploy our custom application. That's going to benefit from the work that ocari is going to do and we'll say that this custom application has been pre-configured to point to a service that will automatically be created by ocari.

A

Now we're ready to install ocry an ocry can be installed using its helm, chart it's just a simple helm installation.

A

So once you do that, you see all of the aquarius components that we saw previously on our architecture slide pop up, so we have our controller running in the control plane and our onvif discovery handler and agents running on each worker node.

A

At this point, we can tell ocry to start discovery by applying our configuration to the cluster in this case, because we want to discover ip cameras, we're going to specify that we want to use the onvif discovery handler and since we do want workloads automatically deployed to the discovered devices, we're going to specify the image for our workload- and in this case we'll say it's a frame server like we saw in our previous scenarios, so this upload will connect to a camera, grab frames from it and serve those frames over some grpc interface.

A

The last thing to know about this configuration is this capacity setting. That's where you tell ocarie how many nodes can utilize a device at once. So here we're going to say only one can deploy a broker to use this device at a single time.

A

So once you've applied your configuration to the cluster, the ocry agent will tell the onvif discovery handler. Hey start looking across the network for these ip cameras, we don't see any ip cameras right here on this uh slide, so let's go ahead and connect a camera, so this could have been done before deploying ocry. But let's say that you do it now, so the discovery handler will see this camera and tell the agent and the agents will create an instance to represent this camera.

A

The controller then sees that instance and automatically deploys our broker pod to one of the nodes that could see this camera. It also automatically creates a service for all of the frames from all the cameras and that's the one that the custom application was pre-configured to point to. So at this point the application application's doing the work it was intended to do it's getting those camera frames and doing whatever it wanted to do. But what happens if this node were to go down?

A

Well, since the other node said that it could also see this camera, the controller will notice that the node went down and automatically rescheduled the broker so that once again that camera's being utilized, however, there was a little bit of downtime.

A

So in order to create a more highly available scenario, you could have set that capacity number to two when deploying the configuration and then from the start, the controller would have deployed a broker pod to each one of these worker nodes so that if the node went down there was no lag and down time for this custom application.

A

All of that flow that we just saw could have been initiated with one helm, installation so you'll see here. um You can install just helms controller agent and discovery handlers and separately. You can uh create your configuration, yaml and use cubectl to apply it to your cluster, or you can do that all in our using our helm, template and helm installation, let's look at one of ocari's latest enhancements and that is ocry's discovery handler model.

A

So our discovery handlers do discovery, so they're they're the ones who that discover all the devices that are either embedded in your nodes of your cluster attached to it or around the nodes, and these can be protocol implementations like onvif or opcwa, and it could even be your own proprietary protocol that discovers your own unique devices.

A

Originally, these discovery handlers were embedded in the awkward agent, which meant that if you wanted to extend ocarina to discover even more devices, you had to fork the agent and work within its code, which is all rust to include your discovery handler. This was also this was clearly inhibiting and we heard that feedback from the community and now our discovery, handlers live behind a grpc interface and we'll look a little bit more at that in detail in a second.

A

uh What this means now is that discovery handlers are deployed their own pods and they can be implemented in any language and there they can easily be deployed using our helm chart. So you can just specify the image for your discovery handler and they'll automatically be deployed in a demon set fashion like our agents.

A

So for the developers that may be watching this right now, let's go ahead and look at this discovery handler um interface.

A

So it consists of two services, the first of which is this registration service and that's hosted by the ocari agent, and that is the service where, when a discovery handler comes online, it will register with the agent saying hey. This is my name. This is where I live, and I discover shared devices such as ip cameras that are visible to multiple nodes or unshared devices such as local usb devices and the name that a discovery handler registers with is the name that you specify in your configuration.

A

So when you specify you use that discovery handler when you apply the configuration the agent will say. Oh that's the discovery handler that I need to ask to do discovery and it will pass the discovery details that are also set in that configuration to that discovery handler when it calls discover on it. We previously abstracted away these details in our previous slides but discovery details are how you tell a discovery handler what subset of devices you want to discover.

A

So for udev, we don't want to discover all the devices in the linux device file system, so the uw discovery handler allows you to specify a uw rule for filtering. In this case, we want to discover only video devices, let's review um all of awkry's functionality that we've talked about so far.

A

So awkari is a kubernetes resource interface and just like container network interface, abstracts away the networking details from a kubernetes operator ocry aims to abstract away all the details of finding and utilizing iot devices and aims to be a standard way to connect these iot devices to your kubernetes clusters. It can handle devices coming and going so when devices appear it'll create resources to represent these devices when they go away.

A

It'll remove that kubernetes resource that represents that device, and it can enable a plug-and-play scenario so that when devices are discovered, ocry can automatically deploy workloads to utilize them, and it also makes it so that multiple nodes can utilize the same device at once.

A

Allowing these devices to be shared using that capacity setting that we discussed earlier it can automatically deploy services for you, that point to specific devices or all devices of the same type or configuration, and we currently support several discovery, handlers that we've discussed so far and we have community contributions underway for um zero comp discovery for mdns based devices and co-op discovery for devices in constrained environments.

A

But, as we saw earlier, we really are trying to make our discovery handler model as extensible as possible, so that community members can easily add more and we can continue to discover even more devices.

A

Another way that ocari is extensible is via its broker pods, so you can specify any image for the workload that you want to automatically be deployed to discover devices and one way that we see broker pods being used is as protocol translation gateways. So maybe you have awkry discover some usb video cameras and your broker pod will automatically translate that usb camera into being an ip camera.

A

So what's next for ocry, more discovery, handlers, the more that ocry can discover the more powerful a solution. Kubernetes will be on the edge. We also want uh to support more deployment strategies. So currently, ocari has a microservice approach to utilizing these iot devices it discovers and we'd like to have multiple approaches there and, finally, and most importantly, we want to grow our community um awkry open sourced only in october of last year, we're still very young, and we're really excited to look to the community for direction and see where ocarie is most powerful.

B

If you find acrylic interesting, you can come and deploy our end-to-end demo for yourself and try it out. You can also learn more about aquari by visiting our documentation.

B

Everything we talked about there today and much more is described in our docs. We are available, you know, if you, if you encounter any issues, please let us know on github or you can talk to us on kubernetes, like accurate channel. We are always available there for any thoughts feedback or help with any issues.

B

We are always open to proposals as well. So if you find something that you think akri should do for everybody or even just for yourself, please let us know send us a proposal. uh We are open to contributions. If you want to contribute that would be great. Otherwise, you can also come meet us at our community meetings. We have a once a month community meetings on zoom, uh where we can discuss other ideas that you might have.

B

Thank you all for your time. uh We will be here for a little while longer for those of you watching live to answer any questions. Otherwise, like I said, feel free to reach out to us on any other channels, and I hope talk to you soon.

B