Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2019 (Barcelona), 24 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Managing Edge Computing with Serverless - Lev Radomislensky, Iguazio

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Managing Edge Computing with Serverless - Lev Radomislensky, Iguazio

Lev Radomislensky will talk about a Kubernetes-based edge solution for retail analytics based on spinning Kubernetes clusters at the edge. The solution relies on a combination of an MQTT broker such as Google IoT Core together with an edge deployed registry, database and serverless functions. In this retail solution, all edge devices are managed in the cloud, while edge computing is managed through serverless functions. Edge storage is managed with a multi model database which stores time series and machine learning models, building applications on top of Kubernetes to benefit from a cloud-native architecture. Lev will demonstrate the update of functions through the MQTT broker, managing revisions and monitoring status. He will also demo machine learning models for image recognition and predictive analytics with serverless.

https://sched.co/MPeg

A

Hello, welcome to my talk about managing several is computing. Yes, sorry managing edge computing with serverless. My name is Leo I work at the Guardia was developing a platform as a service for data science and machine learning, workflows to be deployed in the cloud at the edge or on Prem part of that frame. Part of that platform we are developing nuclear nuclear is an open-source project.

A

It's a service framework written for a kubernetes.

A

So what are we going to talk about today? We're going to talk about current approaches about managing edge devices? We propose some other way to do things. It will include kubernetes. It also includes several is a nuclear know, do some short demo and we'll talk about use cases.

A

So, let's talk first about data data is everywhere: it's not only users typing in passwords or viewing shopping list. Data also comes from various sources like smart light bulbs, smart kitchen, appliance or even cameras.

A

So let's take an example, something we can talk about during this talk and reference to imagine you have a company and this company wants to help retool changed retail stores, various suppliers to maximize their profits well as as the well. So we need data to do that. We need lots of data. We want to have things like cameras at the store, which will give us great view about products, product placement, about consumers in the store, but cameras will producers with lots of large volumes of data to to ingest.

A

So what are the current approaches to do that? Well, you can send all this data all the all the raw data to the cloud and process it there, but with things like cameras and images, you will hit bandwidth limitations pretty fast. If you have remote source in remote locations that are not near big data centers, you will have latency issues.

A

You also cannot rely on internet connection too much so light as it comes in play also when if we want to inject work on like local data, let's say we want to open another cache registry. When we see an overflow of consumers and something we can do locally without sending it to the cloud, it's pointless to send it to the cloud.

A

So another way to do it is to run it all locally right and when I do it by ourselves, we can take something like in the Boonta box. Maybe put some docker containers on top of it and ship it to the stores, but we'll come to into other issues will require local IT staff to install those boxes and all the network permissions and such things it will be hard to manage the infrastructure.

A

Software will need to write lots of custom code to do that, and we don't try to want to write complex code if it doesn't involves managing our data or our workflows.

A

When we will want to combine data from various stores aggregated, it will be much more complex, again more complex software needed to be written.

A

So we need some other approach. We want to take the best from local compute and to take the best from the cloud we want to use the tools we are familiar with. We don't want to learn new tools and we want to have good perform.

A

So what does it take to rip? Apart from our clouds, our event-driven actions, our logic, things like processing image and move it to the edge well kubernetes provides us a great solution for that I saw in the boots. Many companies proposing kubernetes at the edge kubernetes is really great for that it kubernetes abstracts away our infrastructure.

A

It handles complex projects and does does a lot of work for us.

A

So let's take another expect from the cloud something we're all familiar with, and it's several s communities does a great job, but not quite enough. We still won't want to write as less code as possible to reduce our DevOps.

A

So how do we do it using server list? We can have and think in the atomic way of a function. One unit of deployment, one atomic unit of deployment, which is the function, reduces the amount of code we need. We need to write to handle of our logic using several lists. We can take sensors ingest data from the cameras locally at the store act upon it using some complex logic, some custom logic and send only the important bits to the cloud saving us, the bandwidth and latency limitations.

A

We can do aggregation at at a local level, at a local store and and out sizing lots lots lots of uh needed code. We shown right, but several today is not a perfect fit for the edge. It's not a perfect fit, especially because it has slow performance and it limits us most frameworks limit us to one worker per container. Either it's a thread or a process. It limits us to one worker.

A

So we are wasting valuable resources, something we cannot waste at the edge at the edge. We don't always have the scale up and scale down of the cloud, and we want to preserve as many resources as possible.

A

Most thorough swimmers don't come with GPS reports. We so machine learning inference edit edge proves to be difficult. A service volume by almost its definition is stateless. So when we want to have multiple functions, working on the same data, it's also becomes a challenge will require masters service permits. Frameworks will require us to copy data and not only pass reference. Several s is also pretty known for her to debug and diagnose.

A

That's the reason why we in the guado build nuclear nuclear comes to address all these challenges. Nucleo is fully is an open-source project. Several its framework written in go entitled, has three thousand stars at github, a great community, a slack channel. It's entirely open source. You can use it, as is at your kubernetes cluster, and it it. It was built for performance.

A

So, let's dive in what makes nucleo so special first it tackles the issue of one worker per container. Nucleo can have multiple workers either fred's or processes multiple workers per container. So we can have data processing happening in real time with great parallelism.

A

Nuclear also counts with GPU support using the kubernetes api.

A

Nucleo, it also is not limited to HTTP most frameworks. They are limited only to HTTP. If you want to take data at the edge in, we are in what some costume logic of consuming it, you will probably need to convert it to HTTP. If you want to use other service remotes and nuclear, you can you. It first comes pre-built with many event sources as well.

A

It's open source and it was built for two extensions and it's really easy to write new event. Sources. Nucleus also has the notion of Stateville, so nuclear. You can use kubernetes api shared volume to pass data reference. Not copy data around nuclear workers have a notion of context cache as well, so you can have data inside the container passing workers passing between workers no copy around, not not doing any complex project to know. If your database connection is open or not, it can be easily achieved with nuclear right, so we have nuclear yeah.

A

We have kubernetes and we have the cloud. Let's combine them free. The kubernetes gives us cloud api's.

A

It's works on the edge as well as the cloud it has the same tooling for the edge on the cloud it abstracts away. Many many logic with shouldn't write for our for our software. It's one platform both for your several as functions and your document aeneas your custom logic.

A

Nucleo obstructs away more things. Nucleo does to kubernetes what communities does to the hardware so using kubernetes using new Clio sorry, we cannot automate our DevOps, deploy with one-click from development to production and maximize your performance and resource efficiency nucleus also built-in integrated into kubernetes. So everything is a community's resource, so the same tooling you're familiar with from kubernetes, you can use it.

A

You at nuclear-nuclear supports the coop cutter and all other tools can export Yama's and import Yama's deploy Yama's and such we also have the cloud in the cloud we want to do our maths work. We want to train our machine learning models, so we can use nucleon kubernetes have them connected to the cloud. Have our training happen? There? Have our testing happen in the cloud connect to our data services to our back-end, tor, scalable, back-end, and and provide and facilitate the intelligent edge as we like to call it?

A

Let's say a quick demo of how nucleophiles like.

A

So that's the nuclear UI in the nuclear. You are, you have a code, entry type, a runtime and a handler and the body function.

A

So at the runtime we chose Python 3, but nuclear supports many runtimes from the Box supports, go Java, JavaScript, Ruby and many others, and it's also bit built to extend so it's easy to write new runtimes.

A

At the code, entry type we can choose if you want to develop our function at the ID of the browser, the browser ID here, we can also provide it with a custom docker container that we pre-built either using the nucleus, CLI or some other CI CD flow.

A

We can choose to upload an archive of the function body. Let you go to deploy it from there in there it that it's not limited to one file. You can have a package of multiple files of multiple dependencies. You can also provide it with your github repo and have it part of your CI CD flow.

A

So before looking at the code, let's talk about the configuration tab.

A

And compare in the configuration tab, we have the resources book, which is uses native a kubernetes api for the resources, so we can limit a function to memory CPU GPU.

A

We also can specify how many replicas we can also specify how many replicas we want for the function can scale up or down to zero as well.

A

We can provide it with environment variables to inject to the function. We have labels and annotations same labels, annotation you're familiar with from kubernetes, so you can tag your functions easily has also the sense of what I said the shared volumes. So here we are defining a shared volume and mounting it to the function, to the docker container of the function, we'll see how it plays out at the code.

A

At the build block, we can specify a base image, so we can have our custom base image without we're told of our two links and the default is alpine since it's the most lightweight here, I'm overriding it with Python 3 6 Jessie.

A

We also can specify what Bill comments to run. So here we are installing pillow and request pillar doesn't work on the Alpine image. That's why we're specifying Jessie.

A

Right and let's take a look at the trigger tab at the trigger tab, we can choose things like Kafka for streaming kinases. We can have a message queue. We can have multiple triggers per function, so you can receive multiple events from multiple sources and handle them. We also have, of course, basic HTTP.

A

So let's go back to our code in the code we are importing Killah and then we have the main handler of the function in the handler. We have two objects, the contacts, object and event object.

A

So I said before you can have multiple triggers invoking the function. The event object holds the body of the of the invocation. It's the same for all triggers. So, unlike other frameworks, you don't need to write custom logic to know if it came from a specific trigger.

A

Also, you have the context object which which there you can store your shared data between workers and also has things like the logger object, which is common to all. The logger object can be connected to things like elasticsearch or STD out, if you like, or any other log forwarder.

A

So in this example, I wrote a really simple function, just to show you how it is to write one in it. We are getting a URL of an image downloading it to the shared volume resizing it to a 128 on a 128 size and saving the thumbnail on the volume itself.

A

So we are parsing the code at the URL, sorry down downloading it to our shared volume and using pillow. We are resizing it and saving it to our shared volume.

A

So, let's take an image.

A

In the nucleo you are, you also have a panel for testing simple things simpler to post a similar to postman.

A

Okay, so we got 200, so that's the shirt volume and when I will fresh it, it have the original image and the farm nail. So that's the original one and the funded.

A

A

Questions in the end, please, thank you. So we saw how is it is to write functions in nuclear is the same as writing a function for the cloud and the same to for writing a function for the edge. So, let's back, let's go back to our use case, the dimension. Previously we have retail stores and suppliers.

A

We have cameras netted at the store. Those cameras are providing providing our intelligent edge with the raw data.

A

We have functions added at the edge that process this data and and they communicate and pass to the cloud only the reports.

A

So let's take a view from the local store how it can help a local store to to maximize it profits so using nuclear function at the edge and cameras we can identify traffic's of customers know if to open another cash registry. We can identify what draws the customer eyes whether they go first to what shelf and what section of the store.

A

We also can foresee using some pre trained machine learning model. We can first see traffic, picks and and stock inventory based on that traffic, so that can be all done locally without relying on sending data to the cloud.

A

Let's take a look from the supplier example. From the supplier point of view. The supplier doesn't care about local view. He wants to get a more scoped out view. He wants to know about regional and global trends, so using the intelligent edge, we can aggregate data from various stores.

A

Do it at the cloud and put out a report about what trends there are there's another use case interesting. One suppliers today have agreements with various stores, various retail chains, about where to put their products in so it's important on them that the product bins in a specific shelf at a specific section of the store. But how do you check that? It actually happens if you have a retail chain, an agreement with a huge retail chain?

A

Well, today they send people physically to each store to do some kind of checklist to know that it really happens so using the intelligent edge. We again can aggregate data and automatically identify if there are breaches of the contract and where the and, where the deviation occurred, all that with small section of the of the data from the from the edge. So we can communicate only what needs to be communicated.

A

Let's feel some more use case here we have an intelligent edge and under control at the cloud. So in the control we have a CI CD pipeline, some UI, that we can view and monitor our edge devices. We also have a central container registry which keeps all of our custom logic and nuclear functions in it. Nuclear functions are essentially docker images, since it provides kubernetes the ability to orchestrated.

A

So, first, when we deploy a new version, we are going to pass our CI CD pipeline. In the end of it, it's going to push docker images or nuclear functions into a central container registry.

A

Using an MQTT event, we can say tell a function at the edge that the new updates occurred.

A

It will start pulling imaging from the from the clouds and using native Cuban 80s API. It will start a rolling upgrade all the while sending data to the cloud for monitoring for more entering.

A

So that's our demo UI in it. We can push software configurations, some set of command to the edge either to a specific device or a list of devices. We can view our edge devices. In this case we have free one free edge devices.

A

Each has two services, so I prepared a short video of how it looks like so in the video we are updating the edge function from point one to point two: we can see the logs of the mqt receiving their message and updating and starting a rolling upgrade, and then we choose a specific device to do a rollback.

A

So let's talk about another use case.

A

Let's talk about how we can down sample data from the edge and chew and choose to aggregate it and send it to the cloud so in here we are using the the cloud Agrafena dashboard or exploring it with the data science notebook and in the edge. We can choose what data to move to the cloud so forth. For example, if we are, if we have cameras, pointed out into a shelf at the store and nothing happens to the to the shelf, we can just send no updates.

A

We can do the inference at the edge note that there were no updates at the edge at the shelf and just send no update, no need to send all the raw data. We can also choose other data to aggregate using custom logic written in nucleo function, schedule it using a cron job or a kubernetes. Job have another function that does the metrics aggregation and send it to the cloud for further analysis in the data science book, notebook or graphing a dashboard.

A

Another new feature nuclear has its its integration with coop flow, so you can specify which cube flow. Your containers that, though the data preparation.

A

Dude, your training after the training is done to have a nuclear function deployed into the edge test it before that invoke. Some test test functions for testing and deploy the model at the edge.

A

Okay, so, let's just conclude using nucleon kubernetes, we can enrich data at the edge.

A

We can take actions based on the on date on this data we can choose where to persist the data, if persistent at all, as well as using nuclear, we can accelerate our application performance, we have the same tools at the edge and the cloud same kubernetes tools which cuts our DevOps overhead by a mile and thus reducing your costs and say. Thank you. Questions.

A

Okay, questions.

B

B

A

No, it's not inside the container. Okay. The question is: if the images that I showed are stored at the container itself, but no nuclear has a volume, a shared volume feature, so that's actually stored in a volume outside of the container, and then it can be in Indian it being read by the nuclear function. Using a mount point.

B

Hi, hey Great, Auk question: why do you think it a key to trigger updates and things of that nature doesn't simply mean so.

A

Mqtt is a message broker, so it's just easy to use it as a message queue if the H edge is not available or such that's, that's just its advantage over HTTP.

C

Great talk, thank you.

C

Are you running Cuban? It is on the edge as well. Sorry are you running Cuban in these clusters on the edge as well? Yes,.

A

So the edge is running a kubernetes cluster yeah. It's for you to decide. We are recommending like to have three nodes for a high availability and such just like an example of the chick-fil-a did at the US. If you're familiar with I read.

C

A

Kubernetes cluster at each stores like kubernetes cluster, at each store. So yeah, that's that's the pretty much the use case here. Okay,.

C

So you don't provide that, but you are going to have a Cuban Aires cluster on the edge as well. What sorry, the platform that you spoke about provides that or I as consumer have to install that Cuban. It.

A

Does provide you with a edge computer itself, including a managed platform for both your nuclear functions and data services and everything else, and.

C

How is my edge connected to the cloud using.

A

Mqtt or maybe some other architecture, it really depends on the your case. So it's really flexible there yeah.

C

So I understand and QUT because it's which will devices with low power and things like that. There's.

A

C

This size limit right. You can't really send a lot of data over into DT connection. Sorry.

A

C

Yeah you, the device phrase the size limit as to how much data you can send over an imp utility connection yeah. So what are the other? Do you also support.

A

Some kind of PPN yeah, so you're asking about data limitation and the the the traffic for your QT yeah.

B

A

The cloud is here is used using MQTT only for the control panel, just sending commands not.

B

A

Long long time, connections for passing a large chunks of data, it's only as a control panel to to check to to send out commands and such thank you thank.

C

A

A

Yeah, so your own here has a session at free about coop flow and nuclear you're. All welcome. Thank you very much.