Kubernetes IoT Edge Working Group, 3 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes WG IoT Edge 20200603

Description

June 3 2020 meeting of the Kubernetes IoT Edge Working Group - KubeEdge 1.3 update, device update and Hono discussion, adhoc home automation on Kubernetes demo

A

Okay, recording started.

B

Okay, so this is the the um monthly meeting on the apec cycle for the kubernetes iot edge working group and, as usual, our meetings abide by the kubernetes code of conduct, which inherits from the cncf code and the linux foundation code. The meetings as always, are recorded and published. So if that bothers anybody drop off now, we've got one item officially on the agenda and then anybody here is free to add more as time allows, and that was a project update on cube edge.

B

So yen is here to do that.

C

Sure yeah, because my the other maintainer is not here, so I will talk about that. First, let me share.

C

Oh, please give me the co-host, so I can.

C

B

Yeah and you'll have to do it because I don't think I have.

A

Yeah, can you try now I'll yeah.

A

B

Okay, we can see it.

C

Yeah, so so I because I think this material is uh prepared for the seek review and also all the people are not very familiar with cooperage, because I think all of you guys at least uh for us in the uh in the meeting, I'm pretty familiar with this background. So I will.

B

Why don't you pretend we're not because I think we get an audience on youtube, so it would be better if we do this for even.

C

Yeah, I mean yes, please chime in because we only have a small group, so you can come in to ask questions anytime, but if you think that's not need to go very in detail, we can skip or talk quickly, so kubej. So uh when we design which we try to build on kubernetes, so basically we provide the fundamental or core infrastructure support for network application, occupation deployment and also the metadata synchronized between the cloud and edge.

C

So the challenge we met is three major major parts. The first is network reliability and also the bandwidth. So uh first is not, may not be reliable and through the isp internet it could be done at any time and the latency is unpredictable.

C

The second is the resource screen on the edge, especially in the iot edge. So when the mat is a the raspberry pi actually is very generous already. You have a red pair of size. Three. You have two gig memory. uh Four is four gig memory, but in some iot gateways it's maybe reduced to 128 megabytes or 64 megabytes and the arm cpu running at the five or six hundred megahertz, so very limited resource.

C

On the edge, the third is a highly distributed and heterogeneous device management for the iot cases, yeah, that's all brand and all the iot protocols involved, so the cool badge. So our implementation try to do. This. First is a similar cloud edge communication.

C

So uh I will talk about the architecture later, so we communicate in the menu plan. We sync both metadata and the data and edge autonomy. Basically, we autonomous operation on the edge, so even the edge is disconnected from the cloud because the internet connection is broken, but the edge can still run independently and manage the resource on edge and the low resource are ready. So we can have a minimum footprint on the on the edge, so we save another resource to for the application.

C

So the sum we in a small print we can less than 600 megabytes, even 400, sorry 16 megabytes or you even reach a full 40 megabytes deployment on a small on a small edge.

C

But you may compromise some some functions. If you reduce to the very very limit the minimum deployment, then we have a simplified device communication so from the company application does for the application device for iot and industrial iot cases. We support now it's for amiquidity protocol, but in the future we want to integrate other lte protocols.

C

Architecture, so we call it a cloud core and the edge core. That's the main part running main program running on the cloud side and the edge side does start a back socket channel between the cloud and map and the edge, because normally there's sometimes and maybe a lot of times the edge running behind either your home firewall or industry firewall right. So they don't have a public ip. You cannot access directly from cloud to the edge in under the control we set up this channel.

C

So I initial the websocket from azure cloud. Then through this channel we can manage the edge from cloud side. Then we support I'm going to give a more update for the oneplus 3 release. Initially, we only support docker when we support the crio container d and the kata container for security uh reason that that's already supported in the 1.3 release it's released in may.

C

I will go over the uh release, notes for 1.3 release after this presentation and we work with the c9. We work with uh flannel net network and also clarico, that's all free tested and other network.

C

Also, we work with the csi open csi and we are looking forward to more integration with the soda it basically still the open, sds foundation. They rename it and through mosquito we uh support amerity protocols and for now and we have the divide device twin device shadowing.

C

um That's basically the architecture. Yeah just ask me any questions if you have uh coverage um so kubernetes. As I said at the beginning, is uh it's an extension of kubernetes? We build on and leverage current kubernetes infrastructure and the code pays provided for deployment. When, when you deploy first uh you deploy kubernetes, then you deploy the cool cobatch component, so we're using the crd to deploy here.

C

So on the edge side you can see, we have the component basically uh edge controller device, controller, synchronization, controller synchronization controller is for the uh many data synchronized between the cloud and the edge edge controller is for the node management device. Controller is device shadowing to uh to manage and control device that connected to the edge node, the css driver for the uh the maintenance broke.

C

That's for the api validation and the best practice enforcement. The cloud hub there is the connection open for the connection between the edge and the cloud. This is the cloud side in the edge side you can see in the next one we have an edge hub to basically does provide a websocket connection between the edge and the cloud.

C

So we have the event bus, that's for the mnqt client and then we connect to mqtt broker to connect to the iot devices.

C

The hd is more like a uh more like a simplified cubelet, so we still report the resource back to the master. However, that's the reduced amount and frequency, and we we can uh in this, I think yeah this chart may be a little bit uh early version now. We also support a cutter container, so we also support a color container from 1.3 release.

B

Would you prefer questions now or should I wait till the end.

C

Yeah, you can ask now.

B

Okay, so um at the edge, even though you you said that this can go to edge nodes as small as 16 megabyte, but.

C

No, we use a 16 megabytes, I mean. Oh, you use okay.

B

Okay, okay: I was a little confused then, by how you even had room to run things in docker containers.

C

Yeah we use about uh 60 megabytes. We leave about 60 megabytes, so compare when the first developer died in 2018, so green grass use about 80 to 90 megabytes so be significantly smaller than green grass.

B

Okay, so you appear to be unopinionated on the container runtime, where you can pick and choose from uh many um is that based on users actually asking for all these different container runtimes? I'm just curious as to.

C

Yeah I said, while you're holding the wall, uh we asked we got asked from the uh customers, they said. Okay, uh we use the container idea. We can we just right. Do you support it? We said okay and the qatar is we 4c people have a security reason so running on. They said: okay, let's pick the cutter container, actually uh yeah.

B

Because I think some of the other um distributions of kubernetes, designed for small footprint and resource like you might find at edge, are a little more opinionated in terms of which container run times. They're willing to support and have a more restrictive list than this.

C

uh Yeah, that's all from the customer and the community participates, and they ask for that and they have a smart city. They said. Okay, we have that company. Can you support or crio? I said, okay, how.

B

About the os that it runs on, are you opinionated on that and in terms of what the you know, I assume that you start with a linux of some sort, yeah.

C

B

Cube edge have an opinion on what that should be or provided, or are you open on that.

C

Currently, we only use the on the edge side, we only run on linux right and uh we uh already test the ubuntu, the centos, that's all fully tested, and also the raspberry pi and okay, the raspbian yep they're all fully tested, and we have uh our community members that are running in their either test environment or production environment right. So.

B

I take it, then it's tested on both arm and x86 too. That's.

C

B

And with um storage drivers and cloud providers for the edge it would seem like, maybe you don't use a cloud provider at all and remove it or.

C

No, we don't have a cloud provider on the edge side, so we only have uh for the cloud. We only support the veneta version of kubernetes, okay, so actually there's a members running on the aks and the eks. It doesn't work because we take over some node control and the conflict with the uh with their management plane. They refuse, I mean so. Okay.

B

I understand not only that one thing that might help you is that the kubernetes trend is to go to these out of tree cloud providers and remove it out of the base kubernetes anyway. So uh there there is a mechanism to externally apply them. So maybe that would work out if somebody has a use case for doing it and then for csi storage.

B

Do you keep the entry storage plug-ins in for now or is it strictly csi? Only.

C

Csi only for now, okay and uh actually the soda foundation, it's basically the the older open sds they they said they are going to actually they offer they build the old csi drivers. I don't know if you are familiar with their proposal. Basically, they said okay in kubernetes, so each storage they need to develop their own csi plugin right.

B

C

So but their proposals say: okay, they provide one unified uh interface, all the css plugin they are going to evaporate. So in the theory, if we use their interface, they are going to provide the necessary plugin, basically another layer below us, so connect to the older storage. They even have the plugin for azure storage, the google story, even that s3 they said this- should be transparent to us right. Okay,.

B

Well, that's interesting too, because it uh it contrasts to some of the other uh low resource, kubernetes distros, where I think some of them restrict some of the entries because they take space, uh but I think a lot of them leave in kind of generic ones like nfs that are have common utility, so they don't necessarily script them all and uh I'm not sure what the story is on their csi support.

B

In the longer run you probably have to go to csi, just because kubernetes itself is headed that way, but right now you could. Can you have the option of doing both.

C

Yeah, I think so because if you are loading using the restricted, the local storage, that's another story right. If you use a csi plugin connect to a you can attach to a network story, so it doesn't matter, it seems to me it will be a very large volume of storage you can attach to it right at.

B

The edge you would think that, maybe you know if you get beyond local and you need some storage, that kind.

C

B

Generic things like nfs and iscsi might be commonly found that that be my conjecture anyway, yeah.

C

I think that will be commonly found and also uh I'm thinking, that's only for iot cades right. If you are going to mec or large edge cases that will be much popular right.

B

C

They'll probably have an object.

B

Store and a lot more.

A

There's no telling what they might.

B

Have in that environment for the cnn, the cni, the network plug-in? uh I gather that that's open-ended too, that if kubernetes supports it, that cube edge does or is it more restrictive than that.

C

uh We only test a few, actually the flannel, the cletical, and I could not remember the other one, but it's only the popular ones already tested it's working, so we installed a flannel plug-in and all this, however, we're still trying to do more uh edge-to-edge collaboration, because uh sometimes, if two edge nodes within a very close in within uh the subnet or something we want to set up a p2p connections, we may looking for more complicated cases. That's in our roadmap, but now we only have a simple plugin to support network.

B

So when you do have this list, is there an installer for the user, then, where you would fire up an installer and then fill out what your choices of cni and the installer puts it in for you, or is it more manual than that.

C

It's a kind of a manual, it's a more like, follow the the kubernetes. So basically it it doesn't.

C

You can see our dock basically say: okay, go to the flannel, cni or clinical cni download any store, and then that's it. It's basically manual it's more like a manual steps.

B

Okay and if somebody in terms of community for adding to some of these, um you know, let's suppose, hypothetically, that I had my own cni implementation and I want to fit it into cube edge. Is it so open that I wouldn't have to do anything other than write a document on how I did it manually or is there some structure there where it would be better? If I go to the cube edge project and put some kind of a wrapper on my cni too, I I don't think.

C

We haven't thought about that, but the the current situation we don't need to. uh We don't require you to do some code change. Let's just write down the the menu steps because it replaced the uh flannel c9 with your scene. It's supposed to be working.

B

Okay, the reason I ask that is, I've gone through micro cades and they have built in they. They have an installer or a configuration associated with their snap, where you designate, which one you want. But in order to support a new one, you'd have to you, know you'd best go into their project and enable that, as one of the add-ons.

C

No, we don't have that and we have our installer. Actually we have a debian package, so you can use appgas or snap in store, but it's like you said we don't. We don't have a built-in cni plug-in or something you just open say and if you buy uh by buy the cni policy, it's just install another plugin. It should work.

B

Okay thanks those were those were the questions I had.

C

Okay, so that's our last year's projection. It's a contributors! Now it's a more than 300 and from more than 25 companies and actually the actual checking code checking the code is uh it's a more than 100 people checking the code to our reports and keep a steady trend.

C

Does the contribution, the major contribution companies, and also we have uh english chinese and also korean website.

C

So if you go to kuba at kubernetes, io, there's a community member just offer to build a korean version because they are using that and also we just ask our another committee member to set up a japanese version of our website because we have the committee member over there. The sony guys just come just came last week to know two weeks ago to introduce uh yes actually last week introduced their plan, how to integrate with kubernetes.

C

There's some user cases, that's the etc system is basically the the toe system. Is it's a you?

C

You use the the video to capture the plate number and to charge back to charge to their account so and.

D

C

A production use yeah. This production is a hybrid it's now. The uh open source version is the the huawei cloud. They take the virgin their version, but it's based on open source two is uh they have add-ons for multi-tenant and also the im integration all this stuff. Is uh we open? So you can actually you can plug in any imam system. However, this is a community version already deployed on the harvey cloud, so it's called intelligent ad services.

C

I I ief. Oh, it's intelligent edge fabric. It's a the same code base with the uh enterprise add-on with iam multi-tenant.

C

That's the the two major feature that uh the other one is the, how they uh add a more automatic deployment, cicd integration or something like that, but they come with the the home, the same code base and also they also contribute back to the community.

B

Those contributor stats and things look pretty good. Does it look like you? Are you expecting this to possibly move out of the sandbox and graduate to the.

C

Higher tier project- yes, we talked to chris uh and the the cto, and also we talked a few tt numbers. They. uh They think this looks good, so we are going to start our.

C

We talked to amy to start our due diligence review and also we are going to have the the seek review, because you know now they are required, uh seek to recommend the product first, so we are going to go to the the sig meeting to do the presentation it uh this uh two days later thursday morning, then after that they're going to send a recommendation to the dc for review, and then we uh talk about the original tcr, sponsor they're, already waiting to help us to get a t3 review and also we talked to chris chris, a he yeah and the check that he can look at all the numbers, and I don't think he'd go to the technic detail.

C

He could only look at the contributors and the oldest yeah.

B

I think that's why they elected to go to that process of having the sigs look it over, because there's a lot there's a lot of deep technical expertise in within those sig, so you get developer eyes looking at it. In addition to kind of management, tier of the foundation eyeballs.

C

Yeah the foundation only like oh, how many contributors, how many uh adopters uh and, uh what's your chart, what's steady or it's just the uh we say you see, we have the continuous, steady, the commit and from that.

B

So yeah yeah and they're looking at how many different organizations so it isn't dominated, I think they have something they call an elephant factor and yes,.

C

Yeah, it's kind of interesting yeah this in our project about the 40 to 49 percent from huawei and all others from other communities. I think armed contribute about 17 and samsung or yeah, and I think that china unicom have another about 20.

C

Just we have about three or four big contributor organizations, and then we have a few active individual contributors from university from london and a couple guys from german.

B

When you deploy outed edge, do you have any kind of support? You know an interesting thing when kubernetes goes on-prem out of a public cloud and something the cloud provider optionally can do is provide for a load balancer so that you can get a load balancer ingress from your cluster and have you had anybody wanting to expose something publicly directly from an edge location as opposed to only send it up to the cloud.

C

No, we haven't, I think, in the mec case. We may have this load balancer requirement, but we don't have that part yet, and we were thinking uh for the for the from the application side. You can always get a load and balance array uh because we just use the standard coordinate. U uh the service api to deploy a new service.

C

Then you have a service ip. Then you have the load balancer to load balancing across your.

C

Across your the instance, your application, your pause right, so that one, if it is within a small data center, or I mean within a node or within a class one node within the same area- that's fine, but I we don't think you should uh load balancing across the whole cloud I mean because in in edge cloud computing, we were thinking why we need that because we want to make the computation close to the location the data generated.

C

But if you I mean if the data location is in point a and you want to relocate it to a far side of a point b, it doesn't make sense. Why? If that case, why not yeah.

B

Go back directly, I agree that you don't want these edges to use a load. Balancer, that's up at the public cloud tier, but.

C

Yeah then you just.

B

Go back to the cloud.

C

It's faster than another edge location.

B

But let's say the use case is something like a retail edge like a sandwich shop. There might be some utility in deploying some containerized apps at an a sandwich, shop, location and then exposing them to users. You know the public uh and you don't want them connecting directly to those cluster nodes. You might want to put a load balancer in front just to predict. Protect them from you know, hits from denial of service and things and have kubernetes still have kubernetes, manage the configuration of it uh and maybe put in place security policies.

B

um So it'd be an interesting concept. It's something that I, I don't think it kubernetes. I think, may assume that the load balancer is up at the that top level, rather than many distributed ones out at edge locations. But the concept of having this thing handle those use. Cases of uh exposing things behind a facade safely would be interesting.

B

C

In our case, we actually have that one, it's more like a uh community surveillance camera things, so basically that location, because they have a big uh server, actually two big servers.

C

We deploy multiple instance of one application running within that big server. That server, I believe, have a 24 cpus and a bunch of gp, uh maybe two gpu uh a few gig memories. So basically we deploy uh the application there multiple instance, but the load balancer is there because you know we just deploy a service running in within that edge node. I think that'll achieve your goal. It's just a if you have a power edge, so you can, I mean, deploy. I mean load balancer like services right.

C

We just use a built-in combination services. So you build you deploy a uh a deployment with a three or four instance, but then you deploy a service and you expose that service ip, but that's only available within that lane.

C

So I think that's that's the doesn't mean working as a retail edge. The similar thing right. If you have a big server, then you deploy a multiple instance. You have a built-in service and it was exposed service ip. Instead of expose the individual port ip, okay, well I'll, let you go on, I think I've monopolized all the questions.

B

C

I think that's good discussion. We may need to think yeah, but we want to not only technical driven for our committee, but also the customer driven. We want to get the real real world use case to drive our uh project to get more adoptions.

C

So that's the the toll system basically is more like a like a I 90 92 bridge 92 the automatic charging system. They have the video camera to capture the the place number and to go back to the cloud to to uh process. That's the they said. The number is from 15 second per car, improve to 2 seconds per car.

C

This is the smart cameras. That's that's interesting cases, that's a from a high school or a private high school.

C

So what they do is they. They have the face recognition at the gate. So if, if a student entered at the school gate, they take the picture and actually attacks to their parents, say: okay, they safely arrive at school so and also uh they also on a playground.

C

When the people have a uh recess, they take a the happy picture sent to their parents and say: okay, you're still really happy in our school. I mean they said. That's really uh that's fake recognition. uh Algorithm the model must be accurate. You don't want to send one student picture to another parent. I mean.

B

Well, does it all also filter it out so that if there are unhappy moments, it all means yeah.

C

But the the main is that they they want to make sure that the parents get because their parents are busied. I think that they're high on the private school they they said they they use their. I don't think their parents drive their kids to the school, but maybe uh the butlers or house helpers, who do that they are not 100 to trust them or that's why they do this say: okay, your kids arrive school safely, or something like that and in this case, uh why they need that they they claim if they upload.

C

This all has a resolution picture up or stream video up to the cloud. They only have a limited number of bandwidth they using for other stuff. That's why they constantly update their model from the cloud, because that's connect to another ai services to update the model, to do the training to give them more updates.

C

So that's our release. Schedule is the last year march we enter the sandbox and june. We have 1.0 release 1.1 1.3 one point: now we are on the three-month 3d cycle, we're following the uh the upstream kubernetes release, so we gave about a month after the combination official release we gave a month to uh do the compatible testing everything.

C

Then we release our uh our release. So it's a moving train. If you miss this really it's just three months later, another feature just catch on the next train.

C

It says this, the that's the future plans, so uh the community. We want the more adopters everything and we want to actually uh to have to focus our uh group. One is iot, the other one is more like mec, it's a multi-excess edge, that's more like a technical cases. It have a relatively much larger resource, so a lot of resource constraints will be gone but they're facing other problems, for example the global dns.

C

You need to route your dns, I mean draw your service.

B

C

Easily, that's this should be uh the strong part of kubernetes, but how we uh integrate this and make it working in the mec cases. That's a challenge. So we have a couple of china unicom china mobile and we are looking for other technical providers to contribute together.

C

We want to set up a c game within our group to have two focus: iot and mec um yup. That's about the presentation and I'm going to give you a preview of the not previous.

C

Go where let's go over the.

C

C

C

So uh here you can see. Actually we have the korean chinese. That's all volunteer from a committee member to build the other version of this.

B

C

So that's the uh 1.3 at least, so we have a major maintenance improvement, so we can unlock collection from so before this release, all the log you need to log into the edge node to collect the uh the log. So all the power lock. Now you can use the kuber control locks to collect all the logs from agino back to the cloud, so you don't need to log into individual edge, and then we improve the monitoring.

C

We have a more the cloud component. You see that the cloud core we don't rely uh on the combination to restart it because it may um have issues it's it's taken longer than expected. It take a 10 second or 30 seconds to come back. It's uh we got complain. We have a more ha deployment method to have a husband buy and it's automatic uh before that, because you want to do the deployment you need to manually set up the certificates. We now do the automatic generation we from this release.

C

We support the crio and the cutter container.

C

So that's more details. It's a part. We collect pod of logs from cloud, so you don't need to log into the edge. We improve the model monitoring.

C

So we can aggregate the information into the cloud and instead only depends on kubernetes to restart our cloud core part. We have a hot standback, so we don't have a service interruption on the cloud side and all the additional that we don't need to manually copy the certificates. Again, we do the automatic, automated certificate issuing and we have more runtime, crio and kata container and other kind of a fix.

C

I think that's about it. Thank you.

D

Yeah thanks very much for this deep dive.

C

Yep, thank you, I think any more questions, or we can down with this topic. I guess even more questions.

A

Just from these, your muted.

B

Yeah, I was just saying: no, I don't have any. I think I hit you with a lot of them, as you were going along.

A

Cool just one about device twins, so is device twin uh the edge only concept, or do you have something that that maps that to the cloud or.

C

uh That we from the cloud side, you also have that metadata. So when you change it from the cloud side, we do the synchronization. We synchronize this device train the metadata from the cloud to the edge. Then they add another.

C

Service broker, I mean not it's even the bus and to then you can actually uh operate the uh device. It's a we. uh We not only support, I'm good, he actually the gpi or jrpo gpio. You can operate that on that too. That's uh the general purpose io. This is easily working.

A

C

So uh I see you as you and the kitten to submit a topic on. Well, I don't know who, on the uh eclipse con about this uh example running running with hono and io fog right. So, oh.

A

C

Can you give introduction that one the case preview.

A

Oh yeah, I think it might be a little bit. I don't have anything here at the moment, but yeah we are discussing it uh extensively at the uh yeah.

C

You guys meeting two hours three hours, yeah.

A

Yeah yeah yeah, so yeah we're basically trying trying to to create a better integration and see to bring more more closer to the two projects and and enhance all the benefits from from both of them to to be able to use them in the in the in in the in the more scenarios. For example, if you take a look at the honor, it all started as a like a iot cloud gateway, so basically enabling different different kind of devices to to scalably connect to to the cloud.

A

But we now more think about it as a as like a device connection api, where you know it doesn't matter if the device connects to the cloud or or or the edge. So so we try to re-architect some things that this protocol adapter thing can be at least deployed on the edge and and we are using io fog as a as as a as a as a perfect uh edge platform that we can work with within the eclipse community.

A

But that was on only only the the first first and an easier part of integration like running uh honor, as in so to say application on top of the ifoc. I think at this second phase we are talking about and kill, feel free to to jump in yeah.

D

You're doing great so far so I haven't, I haven't, wanted to say anything yet but sure yeah. I.

C

Will so so we covered the firmware update? I remembered somebody mentioned that during a meeting say firmware update.

D

Yeah usually firmware and other artifacts like that are usually delivered via eclipse hawk bit, which is in the same ecosystem. Here, um yeah.

B

D

Also talked about having so something with hawk bit has to implement the actual firmware update process.

D

um But um if you use hockbit uh and and have implementations for different hardware um available at the edge or you can start them manually or sorry automatically and remotely, it becomes really kind of possible to deliver firmware updates for a device. Maybe that just popped on the network right and as long as you have a microservice that can do. The the firmware update process then hawk bit is the way that you would manage that campaign right to like say, flash the firmware on a bluetooth device or something so there's a lot of potential here.

D

But that means there's a lot of touch points and the work through the mall is what we're going to do with the um the european way and take our time and get it planned right and then we're going to now we're going to knock it out of the park. Oh, so you don't do more.

C

Like a waterfall, instead of a scrub.

D

Well, no, I would say I what I mean is just that we like to talk we're going to talk through all the different ways that we could figure the implementations and integrations and and then um yeah, but uh no, I think uh everywhere in the world is agile. At this point. So as uh dan was saying, we did a first pass.

D

That was interesting and we showed that at bosch connected world back in the spring right before all the conferences went virtual and- um and that was just a way to get familiar with what integration might look like and now we're actually flipping it and it seems more like we might even want to carry the io fog traffic over hana, so that it is a basically a registered type of traffic. That's coming through with all of the the gateway management that uh hano provides. So it's been interesting so far,.

A

Yes, so, basically, instead of having honor as an as a add-on service on top of the io folk, basically trying trying to yeah, as skelton said, to do the other way around and and base the base, the io fog, control, plane, communication or over over the whole stack right and and and then you can, you can see the the edge nodes basically be controlled by by the by the rest of the stack.

A

That's one one of the one of the reasons I ask for for the you know, device uh between in in in cuba, and we I talked with cindy early on when she she was uh on cuba and- and you know, trying to see you know, there's a lot of uh device twin projects there and, and things like that, so so it might be. You know beneficial at some point to talk about how you can open up that for for the wider ecosystem. So maybe maybe we can do something there.

B

So the state now I mean I, I thought I heard that cube edge had something like device twin where they're representing devices but they're, not using the eclipse project. To do it is that the summary of the state of it.

A

Yeah, okay: this is how I get it. Yeah.

C

So for the firm update I mean, how do we handle the rollback, and I mean that everything like that? I mean that looks so different.

A

That's the job of of the whole bit so basically, hog bit is a roll out manager. So you can do campaigns. You can do different kind of things uh there, but the hobbit cockpit is only a trigger to do something and then it's a it's. It's a device job to implement a proper agent that will actually uh do the update.

C

Right yeah, if I remember I mean the other os I'm touching, is they they make actually make another fully copied space. So if this one fails, they actually have two active system running.

D

That's a that's, a very good, very good technique, yeah and you'll even get that with like some um devices that only only receive their over-the-air updates or like truly over-the-air, like via wireless they'll, have enough memory for storing the prior version of the firmware so that if they fail to restart and establish a connection, they'll revert back to the old firmware, um allowing you to kind of have a lower risk aversion of pushing down updates to things that might be our you know, in a mostly automated warehouse or some place, that's very difficult to go back and fix all of it, but that comes down to the device manufacturer, just as it comes down to like an iot gateway right.

D

If you don't implement this separate space for storing another copy of the os or the firmware or whatever, um there's not much, you can do to change that, except maybe implement it in the software layer. That's delivering it so right where you would have that hawk bit.

D

Client that would be um talking to like the device over wireless and pushing at the firmware and stuff um might be the place where also if it doesn't have an extra copy, you could maybe detect that it doesn't come back online and then it still has to have some kind of an emergency back door right for for fixing it.

D

If it won't reconnect so I'd, say in the end it comes down to whoever made the device needs to have the foresight to make it robust for firmware updates, and I had consulted a wearable company um some years ago and um they had just before they realized they needed better firmware um rollback roll forward capability.

D

They had already shipped a bunch of units that actually had a firmware problem that that related to their connectivity. So basically um you know to fix it. You would need to get your hands on the physical device and that gets very expensive. So they were basically saying if we were just one release ahead, we would have been able to fix this but um yeah, so they hadn't thought of that ahead.

C

Of time, which is when you need to think probably just shift a new word and say, give you a free new version instead of the old one, it's cheaper, much cheaper.

D

Yeah, it's usually a lot cheaper. In the end, I think um we tried to. We try to find some way to hack it so that we could maybe push down an app that that you could use. That would talk to the device, but um there wasn't really a great way to do it. So, in the end, I think it was like you could download software for windows that would allow you to talk to it and fix it. Otherwise, you needed to ship in your device.

C

Yeah, it's it's in this internet. So when you lose the connection, basically you can nothing to do from external.

C

Cool, that's the interesting topic. I mean I in a lot of iot cases. I got a request from customer say how do you support a firmware update? I think hobbit is really a good solution. It's let's see. Maybe we can integrate it with that too.

D

That's good stuff.

A

Yep definitely.

D

uh Hey day on real quick question, there was I I saw in the uh in this working groups, uh slack channel some discussion of there's an event coming up um and we are you were being asked if you could present something. um Did I miss that uh when is that event.

B

Well, the third are.

D

B

About the virtual cube cons or the.

A

No, not it wasn't, it wasn't kubecon. It was just someone. Oh the.

D

Eclipse so.

A

No, so it's a it's a openstack project project team gatherings, uh so they have like a three days of uh team gatherings between different working groups uh within the openstack and there's the edge working group there.

A

So we were asked to join uh just to see they were they wanted to talk about the collaboration between the the kubernetes and openstack working groups and it was monday, tuesday, wednesday, uh basically uh a mid-european time zone for two hours. So I was there on monday, but there was no talk about any anything about further integrations. They were or all internal, and yesterday I I joined for an hour there's also today, uh one session for for two hours. I can send you. Let me try to send you a link.

C

But I think that flag already closed at the uh edge group of for a couple years, the openstack they they stopped to have stopped their edge group discussion for about a couple years. I looked there they're talking about the wiki. They already say that what group already terminated really wouldn't that isn't that the? How the home, the major home of the starling x project right, which is always different, I the starting axis- I remember in 2017 in san francisco- they had a few discussion, but but then they they are gone.

C

I think maybe I was wrong.

A

They have an update on starting on monday yeah.

D

A

I think they're different well yeah, so here you, you can see. There's a meeting today, starting and at 1 pm utc.

A

And uh yeah the the guy who originally invited us yesterday asked for if somebody from the cuban team could join.

C

Yeah kevin: can you john.

C

Yes, I I can join so anything. I I need to do to join that.

A

No there's you need to register, but it's free and then you'll get uh the zoom password.

A

But if you know, if, if you can't join ping me and and you know or or uh gurgley and and we can figure out I'll I'll- probably join for it at least a bit, therefore, because it's a it's a two-hour status stretch, I don't think we will need to be there the whole time.

A

Let me also try to find the notes from the first two days.

A

Okay, let me let me paste it in the chat as well.

D

B

Thank you, speaking of uh other conferences um at the end of june, the open source summit virtual event is on and they have um they had enough edge-centric sessions there that interested me and I registered so I'll, put a link to it there and then you can look over the schedule.

C

Cool, so this one is called embedded linux right, it's not edge, you mean the one I just linked to yeah it's used. I remember it's youtube called open source summit and embedded linux right well,.

B

It it depends, they put these in various points around the world and sometimes the name changes slightly. So it often has the linux people co-located that are in yaakov and things like this. uh The japan version of this conference combines it with automotive linux and it kind of changes flavor just a little bit, but for linux foundation, this probably has more edge-related content than a kubecon would.

C

Yeah, I remember because I I remember the title was open-source summit with embedded linux because they have they merged the edge with the uh networking they called open source network, open, networking and edge. That's why I think, maybe that one have more.

B

Yeah in the us, I think they fork it into more flavors or.

C

I guess technically it's north america and then.

B

C

B

Open network summit too, I think uh that got cancelled this year, but they had it no.

C

I don't think that cancer is still moving to a networking, a virtual event. Oh, it's.

B

Going to be a virtual event, okay, yeah, but it's going to be in the fall it's pretty far out. I know I was registered for it in maybe april or something and it didn't happen, but I didn't know they had rescheduled it yeah. It's it's pushed out pretty far, but my experience is.

D

When they go to other venues,.

B

They cram together a bunch of these linux foundation, things into one conference yeah and.

D

You seem to be right because the the tracks for the open source summit, steve the link that you just sent it looks like it looks like the embedded linux conference- is a is a small track within it, not necessarily small, but there's a track within it, so that the embedded linux conference being virtual is maybe maybe they had a reduced amount of presentations or something yeah.

C

The probably they reduced the the name to attract, and also I post, that the actual open networking edge does have more edge topics over there.

B

Anyway, I I looked through the schedule on this and I found enough to find it worth paying the 50 bucks to register, so just throwing in that out there just in case others are interested in this kind of material. You might find something of interest there, yeah very cool.

B

I'm kind of interested in seeing one of these virtual conferences, uh because I'm slated to be a speaker at some of these virtual events later in the year myself- and I know they've got to be different from doing a physical one, but I want to see you know, I figure that I would like to see some examples of it done well before I go up there and try and do it myself.

D

Yeah um I last week I did uh I presented at an eclipse event, which was the um uh the iot and edge day, and um it was. It was different because I needed to get into the green room. You know, which is virtual, green room. We all know how to get into the you know the speaker's room before we go on stage and prepare, but to do it to do it virtually. um You know from essentially what is my like my small home office area.

D

It was very strange, but I would say it was executed really really well, because, although I wasn't sure where I was all of a sudden going to end up, I ended up all of a sudden being able to. You know, share my screen and stuff right at the right time, and I just trusted that everyone could see what they were supposed to see, and everyone said it turned out good.

D

So but it was it's different because you just don't have the normal cues to even understand that there's an audience out there and all of a sudden, I guess it was a good turnout, was 100 plus people from my session, and I just have no idea they're there versus standing on stage and seeing that you know people are interested, and I you just don't get the visual feedback either of seeing that they know what you're saying or they can't understand you. You just don't know.

B

Yeah I've seen a few other examples with multiple speakers where you know all it takes is somebody to have bandwidth difficulties and it kind of messes up the whole flow of the thing.

D

Yeah, I can see that happening too. Yeah.

B

Okay, well we're five minutes after the hour. Maybe last call if somebody has any other topics or the other thing I'm always interested in is suggestions for what to cover at the next meeting. You can pick either the next meeting on the north american cycle or on this one, but um you know I.

A

B

A

I I would vote for your home automation.

E

Experience yeah.

A

Maybe it will be polished.

C

E

It'd be good, just you know what what you tried and then what you I'll.

B

Give you a quick tease right now. If, for those I mean, if somebody can't stay, feel free to drop but um sure yeah, I I literally had this running an hour ago, so this thing is like. Can you.

A

A

Yeah, your security center.

B

Okay, um so what I've got here? This is actually running literally in my house, but I have something maybe a little unnatural in that I have a vsphere hypervisor. I guess it timed out and I logged out, but I'm running kubernetes uh on top of an esx hypervisor. So yeah, it's a plain.

C

Vanilla kubernetes.

B

What's that it is single box, it's a single box, but it's a fairly not that I needed that the the resources here are not that great here. Let me log in and uh I'm going to drag this off just while I log in so I've I'm running kubernetes on this, and I don't mean to sell a product, but this obviously I work for vmware. So it is the vmware tanzu, but what I've got is a kubernetes control.

B

Plane is on a couple vms and my runtime is actually just a single node uh worker node for the kubernetes cluster. So uh let's see it's this one. These are actually these cluster nodes are actually fairly small.

B

Yeah, those are the management ones, so I've got the cluster node itself. uh Let me see if I stretch I think I can show you what the vm looks like if I make a little room.

B

So it's two cpu cores four gig of ram, so in terms of a kubernetes cluster note, this isn't that big, uh two cores and four gig of ram. So in theory, you could run this on bare metal of this size. Just you know something like a a low end, x86 uh with four gig of ram, uh because that's all I've given for this vm and this particular uh distribution of kubernetes is deploying a load balancer, which is h a proxy running in a vm and that's kind of what my vm looks like now.

B

When I deployed this, I used what's called cube, apps and that's a vmware wrapper that came from v bitnami that uses a helm chart.

B

So I went and deployed home assistant out of the catalog and what it pulled was the helm this helm chart is published in the helm, chart repository for the app version 108.7 and the experience of doing that. um I don't know if somebody really cares I'll save this for next time, but it took about 10 minutes for this thing to be deployed out of helm, chart and actually be running, um and uh I can go to the url here. For the management interface, so I think if I click this, it will open it up.

B

So this is the home assistant and right now this is it it. Actually, when I installed you go to configuration and integrations, and I installed mqtt and I've got a raspberry pi with um a z-wave usb flash device. It's uh by aeontech um called the z-stick and.

C

It is that oven more expensive than you are a silver. Oh, no, I think those z-sticks are thirty-nine.

B

C

Rather, pie and all these devices, it's more like that.

B

C

B

I maybe they are certainly with the switches so anyway, when I installed this mqtt integration, it it asked for the. um Let me just see if I can get to it.

B

I I'm kind of new to this thing, so there was a configuration step here on this, but I and I filled it out to give my mosquito broker credentials. Well maybe this is it.

B

I don't know, I don't know how to get back to the configuration, but I did it once and it auto discovered my lights and uh I'm sitting in my living room. Now. Let me.

B

Let me just see if this is working. I think it is yeah, so I'm gonna turn out the lights here at my desk, so it gets dark, I'm in a corner of this room and I've got a couple of lights on, but I'll turn them all off, so that light on the other side of the room is going to be where most of this is coming from and I'll turn a couple of these off and.

B

I don't know if you can tell if it's if it seems to be getting dark on my face here or not, I think, what's happening is the light from my monitor is enough to keep my face lit up, even though the room is getting light and dark, but these things are like working and the helm chart deploy, took about 10 minutes, and I had a working z-wave configuration that I've had in my house for a few years, but the home assistant seemed to have found all that stuff, and I was really surprised at um how easy to use the user experience is, and I poked around on this and logged in and their architecture is that they are running this in a vm or in a container they're running it in a container under kubernetes, and that.

B

That kubernetes container.

B

Is able to host uh or the pod is able to host multiple instances to do these plugins, so their architecture is that when you load one of these integrations, I think what it's doing is essentially bringing alongside cars that run the integration like mqtt in a sidecar container, along with the home assistant itself, and it's an interesting architecture that seems to have a really small footprint, where you can run it on a pie with just two gig of ram. uh Yet I'll show you the available uh integrations. There's appears to be a pretty huge list.

B

So they've got a lot of home automation uh and entertainment devices in here security devices. I think I saw a modbus in here. So a little bit of industrial iot devices are mixed in here as well, and it also appears that it will control some network switches, switching and routing equipment. That's popular for home use like a micro, tick and a ubiquity is in here.

B

uh Obviously you see the mqtt, so you'd have real potential to cross this over into kind of a you know, kind of a disruptive form of industrial automation. I don't know if this is up to it. The network ups tools are something that would get signals out of common ups's.

B

um I think open therm is a temperature controller, there's kind of an interesting mix. I think rain machine is an irrigation controller and a lot of consumer electronics. I think I saw samsung tvs, panasonic tvs.

B

Sonos audio systems and it's it has feeds from a number of weather services from around the world.

B

There's the unify for kind of the ubiquity network things I don't know you can see the list but, um like I say, I've just been playing around this with this for a few hours and it looks interesting and you can run it on kubernetes, so that's cool. I think this qualifies as an edge application for kubernetes. If you chose to host it on kubernetes instead of pi bare metal.

B

So anyway, at a future meeting after I've had a few weeks to play around. Maybe I can do a more organized presentation than what you what you're getting here, which is like really ad hoc.

B

I don't know even what these other things are. I think matt.