Kubernetes IoT Edge Working Group, 12 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Edge-Native Application Principles: Taking Your App Beyond the Cloud - Kate Goldenring, Amar Kapadia

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Edge-Native Application Principles: Taking Your App Beyond the Cloud - Kate Goldenring, Fermyon; Amar Kapadia, Aarna Networks

Speakers: Amar Kapadia, Kate Goldenring
Over the past few years, edge computing has been thoroughly discussed, with different technology fields having varying views on what defines edge computing. But in the end, despite the varied use cases of telco, retail, industrial, etc., it's clear that edge is seen as an extension to the cloud and that everyone is interested in bringing their cloud-native infrastructure and applications to the edge. In this session we will focus on this migration. Can we identify common characteristics of edge native applications? How do we tackle some of the common issues like hardware diversity, resource constraints, network availability and more? Basic cloud-native principles are well known and represent a valuable resource for developers. They serve as a check-list for developers, instructing what to keep in mind in order to successfully develop and operate their cloud applications, such as considerations for scalability, observability, ease of deployment on the edge. In the same way we see edge computing as a superset of cloud computing, we can consider edge-native principles a superset of cloud-native principles. So instead of starting from zero, let's focus on additional considerations of the edge. Let's see what additional principles we need to consider for edge-native applications in order to enable successful projects. The CNCF IoT Edge working group has started working on a white paper that aims to define these principles. In this session we will present our progress so far and give a call to action to join the effort.

A

Hi everyone uh thank you for being here today. My name is Kate Golden Ring I'm, a software engineer at fermion and a co-chair of the cncf iot edge working group.

B

Hi and I'm Amar kapadi I'm co-founder at a startup called RNA networks and.

A

We're here to talk about Edge native application principles, so the cncf iot edge working group wanted to kind of work together to find what does this word? Edge native mean? We know what cloud native means, but what does Edge native mean and so we're going to take walk through the process that we went through as a working group that culminated in nine Edge native application principles that we put out a white paper around.

A

So we start off by defining what is Edge and we came up with our own definition for that and then uh we went through what does cloud native versus Edge native mean? What are the differences, the similarities, and this led us into our nine Edge native application principles and then we're going to talk about some case studies, so some applications where you can see these Edge native application principles being used and finally, we'll talk about the working group.

A

So we really see this white paper as a draft, and so we expect future revisions future breakout papers. So, if any of this interests, you, if you have feedback, agree, disagree, we'd, love to hear, come, join the working group and tell us how we can make the next revision even better.

B

All right, so, if you're at this event, you know what the edge is. You've heard enough definitions, but we are going to give you our definition just for context settings. So you know where we are coming from.

B

So we are going to Define Edge Computing as a paradigm which brings compute and processing data processing closer to the source or the user, for example a robot control in a factory, and there are four key benefits to Edge Computing, which again I'm sure all of you know so I'll just quickly repeat: reduced latency, for example, if you're doing virtual reality or robot or drone control bandwidth management.

B

If you have hundreds of cameras and instead of pushing all that data to the public Cloud, you want to process that at the edge increase privacy, depending on your local regulations and uninterrupted operations, if you have poor, Network kind of connectivity or even connectivity is absent in an air-gapped environment. So, if you're on a cruise ship, if you're in a remote mining site Etc, so that's how we Define the edge and next we are going to classify the edge and there are different ways to classify what Edge means we are going to use geography-based classifications.

B

So how far is The Edge site from the data source or the user? We are going to actually reuse a definition by Linux Foundation Edge. They put out a white paper where they described it and the broadly split up the edge into two service provider Edge, which is a shared Edge, and then a user Edge, which is dedicated for a given user or an organization.

B

The service provider Edge, is further broken into two. The regional Edge and the regional Edge looks very similar to the cloud. Actually, it's multiple racks of equipment and you can use a lot of the cloud-like methodologies access Edge and then, when you get into the user Edge you have the on-prem data center edge, which again has some similarities. It's traditional industry, standard, server, storage, networking, but then you start getting into the smart device Edge and the constrained device Edge, and this is where it gets.

B

The form factor gets really tiny and your that's sort of the the farthest end of the edge. So with that, so.

A

Now that we know what the edge is taking a step back, what is the definition of cloud native, and a lot of us may know this. The cncf has their own definition, but Cloud native is really that movement away from your monolithic tightly tight architecture and more into this microservice approach, where you're able to run these highly scalable applications.

A

It's Loosely coupled it's resilient because of that and a big part of that is that it's all very observable and you're able able to take management actions based on what You observe and the result of this is that you can make these high impact changes frequently.

A

So if we transition to what is Edge native and- and this definition was put out by the open glossary of Edge- and one thing to point out here- is that edge native principles build off of what cloud native is so we're not talking about an on-prem data center here we're talking about something that ultimately connects up to the cloud and still adopts that Loosely coupled architecture.

A

But it's also really taking into account those unique characteristics of the edge, so resource constraints, different security, uh different parts of security that you need to take into account latency and autonomy. A lot of your applications are going to be a little bit more tightly tied to data.

A

um And so then, if we take a look at what is the similarities between Cloud native and Edge native, like I mentioned, and both of them, you want this portability of your apps and services.

A

So you should be able to deploy your same application from one Edge site over into another Edge site, just as you can horizontally scale in the cloud observability, you should still whether you're deploying your application on the cloud or the edge be able to observe this all in one platform and see where your application is living, see its Health, Data, Telemetry, metrics Etc should all be still a part of your Edge native applications manageability based on all that data that you're.

A

Seeing all of your observations, you should be able to take those management applications so, whether you're doing that in the cloud or the edge, those actions may look a little different. But the idea there is certainly the same and then finally, one of the things about Cloud native is the idea that you're not doing these monolithic applications. Where everything is in the same language. You can have different services using different languages, and that certainly does not change whether you're running your application. Only in a cloud or it's spanning to the edge.

B

So now we get into the interesting stuff. If you are a cloud native developer and you've gotten comfortable writing Cloud native applications. This is the slide. That's really important in terms of what are the differences between Cloud native and engine native, so you can start writing Edge native applications that are that are more suitable for the edge environment. The first set of differences can be classified as State Management. They deal with how the state of the application is managed. The first one is application models in Cloud native.

B

We have gotten comfortable with stateless microservices that scale horizontally, and you have a centralized data store and in the um Edge environment you can't always assume that. So, if you are in the service provider Edge, it may look similar to to the cloud, but as you get to the more and more constrained Edge, your applications may start to get monolithic and in either case, whether it's service provider, Edge or the user Edge. You are probably going to have your data co-located with your application. So that's one big difference between the two.

B

The second is in the data model, so we just talked about it in the cloud you're going to have a centralized data store and which will be backing up a bunch of stateless microservices in the edge you're going to have a lot more diversity of data models. You may have that exact data model that I mentioned, but you might have caching. So if you are in a poor connectivity or an air gapped environment, you might have streaming.

B

If it's a lot of video or content, it might be distributed where your data might be shorted between Ed sites or Edge and Cloud. So expect a lot more diversity of data models elasticity in the cloud you assume: infinite elasticity. You just start scaling horizontally and use as many resources as you want on the edge.

B

That's not always the case, because the edge is constrained, so you're going to be elastic only to a certain limit and, as you get more and more to the user, Edge, which is even more constrained, you'll have extremely limited to no elasticity and in in fact you may want to scale vertically by and by that. What I mean is that you might want to scale to the public Cloud. So you might have a piece of your application at the edge and a piece in the public Cloud. So elasticity has to be thought through differently.

B

Resilience in the public. Cloud resilience has outsourced to the cloud provider right. You assume, if I go across a certain number of nodes and if I need even more resilience, I can go across sites and I got my resilience in the edge again. You can't assume that and in the edge we are going to go backwards in time now your application may need to start thinking about. Do I need dual redundant power supplies. Do I, need dual running and fans.

B

How do I recover if there's a failure, so you have to think of these things and ultimately, on the edge you might have to accept less resilience just by the nature of the edge.

B

The next set of differences deal with orchestration and management, and these all stem from scale. The scale for a cloud is, you might have a few public Cloud sites and a few applications in the edge the scale dramatically changes. Now you have tens of thousands of sites, and two years ago Walmart said they had 10 000 sites. These are not big numbers and you might have hundreds of thousands of devices that are being managed by those ad sets. So the scale is, you know, orders of magnitude higher at the edge.

B

So what that means from an orchestration point of view is again when you're in the cloud you are assuming centralized orchestration into a few sites and you're doing it: sort of for efficiency and you're going to scale horizontally right. That's what we just discussed in the edge you're going to have distributed orchestration, so you might be taking the same application and putting it across a large number of sites. You might be taking the same application and splitting it across Edge and cloud or multiple edges, and your orchestration may become location specific.

B

So in the cloud we generally don't worry about, the location in the edge location is very important. I want to be five milliseconds from Plano right, so so on and so forth from a management both are centralized. So in that sense they're similar.

B

But the big difference in the edge is that you might need Central management and Remote Management. If there are certain actions that need to be taken that are time sensitive, that need to be taken very quickly, then you might need a sort of a distributed management framework.

B

You have no staff on the edge or even, if you do have some staff they're generally not going to be trained to the same level as the as a public cloud or the data center. So you have to have zero Touch provisioning of hardware and software. If you are upgrading devices, those upgrades need to be robust or sort of brick resistant upgrades.

B

So those are some differences and I'll hand it over to Kate for some more.

A

So next we can think about networking. So in the cloud you have very strong, highly available networks that you're guaranteed in these Big Data Centers. However, on the edge, that's not always guaranteed and oftentimes not expected, and so this is when you take into account some of those data models. So maybe you do more of store and forwarding storing when the Network's, not there forwarding when it is and then another part of networking is the domain of the network.

A

So in the cloud you have a public network on the edge, your whole site may be isolated in a private Network and there are security benefits to that on the third part of networking. You can think about networking, Beyond IP, so the other protocols that your Edge servers may be using to interact with the local devices that are generating all that data on the edge.

A

So protocols such as OPC way for industrial Machinery on Viv for IP cameras, mqtt more and more, and you need to be familiar with all these different protocols and have to think about which one fits your needs on the edge of the best.

A

On the security side, it's quite related, so on the edge you have all these devices. So even though you may have a private Network, your your physical site is way more accessible than these lockdown data centers.

A

So someone could come in and put a device on your network that should not be there, and so part of this is knowing what should exist on your network and what shouldn't and keeping an inventory of that so that you're only using data from devices that you trust and so that xero, that zero trust of everything on the edge is really important and continuing this discussion of devices as I was mentioning you're, going to expect a lot more of external devices on the edge the cloud very homogeneous racks of servers, all the same static system, Hardware Edge, the environment is way more varied.

A

That's when we're talking about this user edge of that spectrum, and so that's those are the devices that you want to be leveraging. That's the reason why we have that compute so close to them, and so a huge part of an edge native application is thinking about those devices and using them as we move into the servers themselves. The hardware is also more varied on the edge.

A

So while in the cloud you kind of have these standard flavors of architectures and os's on the edge, you may have more specialized Hardware with specialized Hardware interfaces that you want to use and access, or you just need to be aware of that also comes along with Hardware awareness in the sense of what restraints do you have on your Hardware, you need to have a bit a better sense of how much CPU and memory do you have, because you can't easily just add another VM or add another server.

A

So, like I was saying at the beginning as a working group, we went through this discussion and this led us to our nine Edge native application principles, and this is going to be very based off what we just talked about. um So the and then we took the nine and broke them down into five smaller categories, and so the first larger category transitions from what we were just talking about. It's that being aware of what resources and devices that you have is really important on the edge so with Hardware.

A

You need to be aware of what you have on your servers and try and use any tooling that you can to make that awareness as simple as possible. So, for example, if you're using kubernetes, you can use the device plugin interface to understand what physical Hardware is available for you and make that known to kubernetes. So then you can make a scheduling, decision and kind of make that less of a decision you have to make as an operator and put that on your tooling for external device.

A

Connectivity you're going to want to use tools like device Registries like I, was mentioning certificate, rotation and think about how you can choose the right set of tooling uh to make that as seamless of a process as possible, because you're going to need to make sure your applications know the access endpoints for each of those devices and ideally that same application can be deployed from one Factory site to another or one Edge site to another.

A

Regardless of the number and the different um addresses of the devices there and finally, being aware of variable connectivity is, of course important, and that goes back to how are you going to design your application for that? So is it going to be purely air gapped? What is what gets forward up to the cloud? What alerts are important to move forward in that chain?

A

The next larger category is at scale management. So, as we've been discussing, Edge native uses the idea of cloud native, and so this starts with the infrastructure. How do you make that zero touch onboarding experience of your physical infrastructure, so you could use something like Fido device on board, which makes it so that you can literally just turn on your server and it'll.

A

Do a remote call to flash your firmware on and then your platform and your application and go from there and actually, if you're, more interested in FDO, our working group talk last year, Stephen Wong talked a lot about FDO. If you want to learn about how to do that, to get your infrastructure up and running the next level above, that is application management. So what is your platform that is going on top of it?

A

That's orchestrating your workloads to the appropriate sites and to use the appropriate devices and so you're going to want to use some sort of orchestrator, just as you would in the cloud and then. Finally, all of this should be centrally observable and you may be using the same tooling that what you would be using for your Cloud, so Prometheus for monitoring and so on, and then the last three principles stand on their own. So uh the first one is a caveat of sorts like we mentioned it's portable and reusable with limits.

A

So as you move more and more to the user Edge to more of the constrained side of things, your application is getting more tightly tied to us in its environment. It's portability and reusability goes down. As you move more to the service provider Edge, it gets more and more portable. The environment is more predictable um and, and the community in general is building out tooling for the user Edge to try and make that um as easy and reusable and portable, but we're still working on that more one that we may have not touched on.

A

As much is resource usage. Optimization we've talked about how you can easily, in the cloud, add a server and on the edge, that's not always possible, and so you need to be aware of. How can you design your application to use as little resources as possible and then share that with other applications? So that could be time based sharing, for example, and then finally spanning so deciding what stays where it is and what moves vertically up to towards the cloud?

A

So do you have an edge Hub and what is going there and what is further going to the cloud and what management actions are coming down from the cloud to your Edge.

B

So far what we showed you was perhaps a little abstract. So we're going to give you two examples, so I'm going to give you one example and then Kate will give another one, so one this example is for 5G.

B

So how are 5G applications using these Edge native principles? Now, first of all, many of you may not know that 5G networks are actually going to be software. Driven previous networks used to be Hardware, you would buy appliances and hook them together. 5G is all software driven so other than the radio. Everything is virtualized and now moving to containerized workloads orchestrated by kubernetes.

B

So the diagram which hopefully you can see the left hand side- is the Enterprise Edge or the at the tower of the the cell tower and you're running, something called the ram the radio area network. So that's the piece working running there. You might also be running some Edge Computing applications, that's possible, then the next you get to the Telco or the cloud Edge.

B

This could be the Telco Edge or a Colo or Data Center, and there you're running something called the 5G core and you might be running more Edge Computing applications depending on the characteristics and then you're in the public Cloud, where you might be running voice, Services collaboration services- and this is just representative- some people. You know people move things around, so you might be running. For example, some people might run the core in the in the public cloud.

B

So anyway, with that architecture, let's see how it maps to the five group principles so resource and device aware so the ram software is interfacing with a physical radio. So that's where it actually has to talk to a device and is also aware of acceleration. The Rand software typically needs to Hardware acceleration. There's like Digital Signal processing, that's going on, so you need things like GPU dpu or an fpga at scale management.

B

Tens of thousands of brand sites are not unusual and they all have to be managed centrally because, as you know, there is no staff member at a tower right spanning we just saw the ran core and voice Services span. All the way from the user Edge or the tower Edge to the public cloud resource usage optimization. This is very important in a public 5G Network, the Rand can make up for 65 percent of the cost.

B

If we can make that efficient, that directly affects the the deployment cost of a 5G Network. So for that reason, resource usage, optimization is critical, portable and reusable within limits.

B

The company that is creating the ran software or the core software or the edge Computing applications. They have no idea whether it will be run on a public Cloud, a hyperscaler type, Edge or it'll be run on a private cloud and if so, who's the kubernetes the cash vendor they have no idea. So all of these have to be portable and reusable so that maximum number of the the company's customers can use it without any changes.

B

What would you for the second one.

A

So I'll do the same and set the scene and then talk about how it maps to our principles but more for the user Edge side of things so say you have a factory floor, an assembly line with a lot of robots, maybe some cameras who are doing some visual inferencing of the robot so that maybe you're doing some predictive maintenance. And then maybe you have some sensors that are doing some health monitoring in the area. You could collect data from all these devices from the cloud, but it would be leaving your private Network.

A

You may be having security concerns there. There's latency concerns as well. So maybe you follow the laws of data gravity and process that compute closer to where it exists and you so you set up some servers right in your factory setting and you can do some clustering.

A

Maybe you use Cube Edge, which has an edge Hub, so you can cache data there before you forward it to the cloud hub, whatever orchestrator you're, using you're, able to process the data from these devices and then only send up through a Gateway what you need to the cloud, maybe that final Health alert you can see in the cloud you can send a technician down, but only those final actions are are going up there.

A

So if we once again go through our principles being a resource and deviceware is clearly very important here, you have a variety of devices having that inventory once again of all of them and being sure to handle the credentials there. Also in the scene that we just painted there.

A

We have multiple protocols going on so making sure that your applications are configured to know which devices do I need to be talking to and how do I communicate over that said, protocol at scale management you need to remotely manage and configure these services so deploying them to each of these sites.

A

This is just one picture of one Factory site, but you may have multiple so being able to remotely deploy, deploy to multiple of those sites and configure your applications to use them once again, so that term management when we get to the user Edge goes beyond just your application to also managing these devices themselves, so making sure they're getting updated appropriately and one way you could do that, for example, is with digital twins.

A

So that's a common use case for this user Edge Where in the cloud you're representing in some textual way, the ideal state for your device and then on that and then, when you do a change to it, you have something on the something on the edge. That's watching that and will reflect the change there. So, for example, say in your assembly line. You want to speed up how fast the robots are moving.

A

You may change that parameter in your digital twin in the cloud and see that be reflected by your Edge applications, which are managing those devices and spanning once again, a very clear visually on this one. So you're going all the way from maybe your private Network, where you're processing data fast and making these real-time decisions all the way to the cloud where you're, making more General decisions um based on what was processed resource usage, optimization once again expands beyond the server in this setting.

A

So doing things like predictive maintenance to be able to make sure you're using your resources outside of your servers as optimally as possible, reducing the amount of downtime of those and then also being sure to collect that that data, so that you're able to make those decisions and then finally, portable and reusable with limits. So this is where we're getting to the part where the portability is harder, but you should still be able to import the same applications uh from one setting to the other, uh and there are certain protocols that help with this.

A

So, for example, with mqtt. Instead of having to directly figure out how to connect to every device, you can connect to one broker and thereby get all the data from all the devices. So there's ways to generalize how do I interact with all these devices on the edge.

B

All right, so to summarize, we if you're a developer and you're planning to develop Edge native applications. We told you the similarities between engine native and Cloud native. We showed you the differences.

B

We gave you a set of nine principles or which can be grouped into five, and we gave you two concrete examples, so hopefully you're you're well equipped now now, as you can imagine, there are a lot of cncf projects that help with Edge native and there are non-cncf as well. Linux Foundation, networking, non-linux Foundation networking.

B

So we have started to create a spreadsheet and the spreadsheet has some seed, some initial projects that we have put in, but we need your help. So if you have the time and interest, please go through that spreadsheet. Add any open source projects that you think are relevant and Mark Which principle. That project applies to.

A

Yeah- and this was in part and uh kind of inspired by the cncf landscape- it would be great to have some sort of resource where you could decide. What principle is your pain, point and kind of look at a document and see what projects or tools could help you with that? So yeah help us build up this resource and beyond that, the working group in general.

A

If you want to read the full textual paper of this, um there's a link to our white paper there we fall under the cncf runtime tag, so anything about us will be found on that GitHub page, including the white paper draft. We also have a slack Channel wgiot Edge. So if you want to chat at any time a lot of people just pose their questions there. So if you have any questions about Edge people can respond to them.

A

If you're more of an in person or I guess a visually in person person, we have meetings every two weeks at 9am and that's on the cncf community calendar and more QR codes. So um if you want to leave feedback for this session, we would greatly appreciate that and then, if you want to leave feedback for the working group, we'd also appreciate that so right now, we've been focusing on the white paper in the past.

A

We've done uh presentations for people from people in the space where you can learn from them, and so we're really looking for what of these Edge areas are interesting to you. Would you be interested in presenting and more so um yeah happy to receive any feedback on either of those areas and with that I think we have about 10 minutes for any questions or maybe five. uh If people have any and then Steve, one of our other co-chairs will be passing around the mic.

C

Hi I got a security question so.

C

Go into some stuff, I wrote down here, uh I I think I'm, going to make some assumptions here. So initially you opened up saying that there's actually some benefits to the edge in terms of security and I'm.

C

Assuming that's because you can go off of killing data at the edge right like you, don't store anything there and just keep moving it on, but in order to have an OP uninterrupted operations, how do you make sure that you don't store any customer data there and I'll just add one other piece there, a lot of them examples I hear about are kind of um you know, even though you have remote devices that people aren't getting to if it's in a factory it's still sort of under the protection of that um yeah versus something that is actually sitting you know, maybe in a CDN or a radio tower right, uh you know you could have somebody go up there and steal a whole bunch of cell phone data right.

C

So those are my questions. How do you uh balance? What are some things that you've heard about about? Getting that uninterrupted operations, as well as making sure your customer data is not at risk.

A

Great thanks: well, I can start with the data part, uh so I wouldn't say on the edge. You generate data and kill it. uh That's that's not why it's secure it's not the fact that you're getting rid of that data as quickly as possible. Rather uh the thing that I was emphasizing there is that it's a private Network, so the data is staying within a private Network.

A

Well, if you're, using Cloud, if you're sending data across a public network up to the public, Cloud, there's a there's, a security risk in that, um and so it's actually really important. The data processing is the most rich and important thing about the edge and a lot of times when we're talking about that. We're also talking about data pipelines, so you may have data stored in one spot that another microservice on the edge is using in order to add data from their devices to this pipeline.

A

That creates this really rich experience of making enlightened choices on the edge. um The other question was about.

B

A

Okay, so and I forgot to rephrase the first question, so the the second follow-up question was: how do you make sure someone doesn't lock in and take your server and use your data was that was that the question.

A

Yeah, so so multiple things listen in and then physically access data I mean these are the key security questions is how do I make someone not take my machine and use it? um I I, don't know the answer behind.

A

How do I make someone not access my secure, like my physical Zone and take my devices, but uh I would certainly hope that you have some sort of hardening of your of your devices there so that, even if you do have access to it, you need some sort of credentials to go in and get access to the data.

A

But I don't know beyond that, and then listening on the private network uh yeah, you need to tighten your private networks and once again, if you're spoofing devices having a way of detecting that Omar, would you add anything to that I.

B

Think it's really Case by case and you are right. So, if you're in a hospital- and you have a hospital Edge, there is going to be personal data um in general, you have data at rest and data at in-flight encryption techniques. So if you use those best practices, whatever you store, you store encrypted and then you have secure Key, Management, so I think those methods exist. The zero touch you just have to make sure those are applied correctly.

B

I would say.

D

Hi question around the design principles that you talked about in The, Edge, space kind of the differences between Cloud native and Edge native. Have you seen I mean there's a lot of desired states there? Have you seen a pattern where this could be useful to be applied back into Cloud native, and how does that look like in your mind.

A

I think that's a really good question. So the question was: we've talked a lot about how Cloud native influence Edge native, but is there a way that edge native should influence Cloud native things that we can bring from Edge native to Cloud native I'm, more on the small device side of things. So my first instinct was um I. I've worked a lot with the device plugin interface and the fact that it needed to be created was because we had this varied, more varied way that we have a static physical Hardware.

A

So we needed to create more resources that we can advertise on the Node level and so I think there are ways that we're bringing this idea of more varied Hardware back into the cloud so that clouds now can have more specialized Hardware such as servers with gpus Etc and that could have been inspired by more specialized Hardware on the edge.

B

And I think it's a really good idea, um but I think it's very early I'm sure it will spill over. So a lot of the management and orchestration techniques for the edge will be a lot more sophisticated, so that sophistication could back could go back to Cloud native or things like multi-cloud. So I I agree with that. Yeah.

E

Okay, this is better so so, if I take similar use cases and apply it to energy utility sector, there's a lot of cloud Edge there. However, there's a nerve, zip compliance rules that don't allow OT networks, data ever to come to the IIT or to public Cloud. So they have a clear separation when I know of companies who deliver services in in a public Cloud, but then the way they do it is they actually take a physical, hard drive from the OT Network plug it into the server.

E

That's part of the IIT Network that lives that has a connection to the cloud to do stuff. So is there any work, that's being done or or how do you? You know? How do you bridge that Gap when it comes to electricity or existing and and the ability to use public cloud in OT, Network.

A

So the question: if I get it right right, they're unique need for OT L want that data leaving its private Network. So are there strategies that exist today for just having the data we want to leave it and go to the public. Cloud was that the general question.

A

And and the follow-up question there is: do we have any standards? Are we creating standards to be able to enable there to be some interface? That's allowed for sharing that uh I do not know the answer to that.

B

I think a lot of that I think the point that is the point of edge Computing I, think Edge Computing can help with that by terminating the OT traffic on premise and sending only, for example, the use cases I've seen and sending only metadata or or things that are acceptable, but I think that again becomes Case by case and I. Don't know if you can come up with a generalized sort of a mechanism it it I, think the burden, Goes On, The, Edge Computing application, and then they would have to get certified or meet compliance.

A

But this might be a good transition because I think we're at time.

B

A

You want a group to start thinking about that to join the working group. If you have thoughts on that, you could present kind of a way for us to wrap our heads around that and see if we can kind of build resources around that together.

B

A

Right. Thank you thanks. Everyone.