Kubernetes IoT Edge Working Group, 12 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud-Native WebAssembly: Containerization On the Edge - Michael Yuan, Second State

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Cloud-Native WebAssembly: Containerization On the Edge - Michael Yuan, Second State

Speakers: Michael Yuan
The Linux container is a critical innovation that drives cloud-native applications. Developers are now deploying containerized applications outside of cloud centers on edge networks (e.g., serverless functions on CDNs), on edge devices (e.g., smart cars), and on SaaS (including databases) as embedded functions. In those use cases, the Linux container, with guest OS and support software, is too heavy, too slow, and not portable enough. WebAssembly (WASM) has emerged as a lightweight runtime for those applications. The WasmEdge Runtime and crun projects work together to make WASM a first-class citizen in the container ecosystem, running side by side with Linux containers in a single cluster, managed by existing container tools. In this talk, Michael will discuss how WASM fits into the container ecosystem, how to use container tools (e.g., CRI-O and containerd) to start WASM images, and how edge-optimized Kubernetes (e.g., KubeEdge / SuperEdge / OpenYurt) manage WASM applications.

A

Let's get started and uh um first of all thank you so much for coming to the last presentation of this coupon I wasn't expecting this many people. You know I I've done the last talk in conferences before and they were. You know it's typically was much less attended. So thank you all for coming here and I appreciate that.

A

um So the title of this talk is cloud native web assembly. So there are two keywords here: it's Cloud native and webassembly. So when we, when we think about cloud computing, what do we? You know? I? Think the key notion really is: there's there's really no Cloud. It's just other people's computer running your program right, you know so virtualization or isolation is the most important technique in cloud computing.

A

You know, so it started with the virtual machine, the VM, where you know um it's, they can virtualize the infrastructure and provide each user its own say um you know computer right and then on cloud native Revolution comes along and it makes provide tools and uh um and new design patterns to make it easier.

A

You know to based on container technology, so you know so you would have a Linux container, starting with stalker Linux containers, and um you know so makes it easier for um to have to run my tenancy applications and share the infrastructure, so that's Cloud native. So what about web assembly? So a little history about webassembly before we um before? We really goes on so webassembly started out as a Sandbox or virtualization technology in the browser you know so in the browser people, you know we have JavaScript, we can run code in JavaScript.

A

However, it's not satisfactory. You know people wants to run codes that faster, you know native code in C plus plus or in Rust, so they invented a new runtime inside the browser that provides a security sandbox. It's called webassembly and so webassembly started in the browser, and then it moves to the world of.

A

um We now call web3, but back then we call blockchain, you know that's uh so um many of you know that you, certain blockchain, but after you, certain blockchain, almost all the leading blockchain projects are using web assembly to run smart contracts which really, when you think about it, it has similar characteristics about application. You know programs in the browser which is somewhat somewhere else.

A

The program have to run on your infrastructure right, you know, so it's uh you have to provide a safe, sandbox faster on it, and it's also the same kind of problem that cloud native wants to solve. So then in 2019 you know there is a famous tweet that started all this is from Solomon Heights, it's the founder of Docker, and he tweeted that uh you know. uh If web assembly has existed in 2008, we would not have invented the docker, and you know that's he said webassembly is the future of computing right.

A

You know, so that's um really jump started. So that's a history, that's so by 2019. It's really jump started the idea that webassembly might provide a lightweight, secure and a very fast isolation technology that that performs superior to the current generation of Linux containers for cloud native applications. So that's the um you know the a quick background of you know how we get here right. You know so.

A

Cloud native web assembly is really to look Beyond Linux containers to have a new type of containers that are based on webassembly and uh um so in the in this talk, I hope to provide an overview of you know. What's this world look like and is it ready for production yet and what are the toolings and what are the libraries that is available today? That developers can get started right um right now right?

A

So these are some resources I'm going to repeat this slides at the end of the talk, so we're gonna skip it right now so um hard started, you know. So, let's look at the typical um Cloud native application. That's let's say a service mesh that is container based today right, you know! So it's uh you would have your application, that's written in a language like a go or rust or Java, or you know whatever, and then you know you compile that and you have a runtime and then it's you.

A

You package that into a container a Linux container, it's uh and then have the containers managed by kubernetes or similar. You know, react coupon, so you know everybody knows about that and then you know in the kubernetes world you have. um You know, you know you know say in a service mesh, you have Micro services in a part that has each part has two containers.

A

Typically, you know when the sidecar, the other, is application container and there's many many different technologies that you can use to manage this infrastructure and to provide services to the to the microservice right. You know so this is uh you know what we. uh What do we call container-based architecture. However, there are some pain points here. You know the first, when, when Linux container first came out, it was known for lightweight because compared with the virtual machine, it is lightweight.

A

However, when we evolve into the world of webassembly, um the Linux container or the docker container now appears to be very heavyweight. You know. So if you have a container application, for instance, and you need to drag in a lot of operating system, libraries and dependence, libraries in it, so it's not uncommon to see a container image. That is, uh let's say a couple hundred megabytes. You know, if you want to do um you know a machine learning or AI inference. You know things like that.

A

You could get a one gigabyte image or even 10 gigabyte image. You know. That's uh you have python, has pi torch and have everything in it right. You know so a lot of times it's uh you know, at least from our point of view from the web. We have a similar point of view. It's very heavyweight. It's so make it unsuitable for a lot of scenarios, especially on the edge you know on the edge cloud or Edge devices.

A

You know it's uh it's it's, maybe okay for the for the large cloud data center, but you know when you go to places like the CDN Network, the um the 5G 5G me stations. You know, and you know things of that nature- to become too heavy weight and also it's uh the container-based applications when it first came out is also known for being fast because compared with VM, it's fine, it's a lot faster, but from web assembly point of view.

A

We also consider it slow because, especially at startup, you know uh in order to start a container application, you need to load all the libraries you know from operating system and also the application dependencies. So it takes I would say you know 100 milliseconds or more. If you look at you know, datadox survey about AWS Lambda, which is not exactly a container-based infrastructure, but close it's a it's a firecracker based micro VM. It says half of the request that takes 800 milliseconds to complete. You know. So it's uh it's.

A

It was portrayed as a positive thing. You know but, however, if you think about that, 800 milliseconds is 0.0.8 seconds for uh you know, for uh you know, for essentially a service call right. You know, that's uh you know.

A

That's perhaps the number one problem people have been complaining about you know with this type of infrastructure, is that the startup time is slow in order to mitigate the the slow startup time you have to come up with strategies like keep warm, you know, so you would have containers that is running long running, that's running all the time and just to keep getting those extraordinary requests.

A

However, in that case the heavyweightness becomes the issue, because you you have to have those containers running concurrently and uh if each of them takes 10 gigabytes of space, you know that's, um you know, you know you will need a very large. um You know Computing infrastructure in order to support that, so the slow and the heavy are related because of the slow you have, especially at startup. You can't scale it up and down in milliseconds. So in order to serve those requests, you would have to.

A

um You know have other strategies like keep warm and that's um you know makes exaggerate the problem of being heavyweight right. You know so the two problems are related and the third problem is a little bit of controversial. Is that in general we consider containers not that safe, because you know initially to need kernel, space access and now it's much better. It can it's. um It doesn't need that. However, if you look at one, why AWS Lambda use firecracker VM, instead of say, Linux container it's because they consider it's not that safe.

A

You know, there's numerous ways you can break out into the you can break out of the container and if you look at other large Cloud, you know like Google cloud and also Azure, and you know, and tencent Cloud. They all have their own ways to have secure containers. You know like Google, has gevisor right. You know it's something that said you know beneath the container to make it more secure, but also make it even slower and also more heavy weight.

A

So the safety is a is a big issue for a container-based applications as well, and the other thing that's that's very interesting is that containers produce a repeatable development or deployment environment. However, it's not cross-platform, so you know you have a container image built for arm. You have container image built for um for Intel AMD. You have container image build for risk. Five. We now live in world, that is, that faces heterogeneous Hardware.

A

It's not only there's more than two CPU types, it also with the rise of risk 5, and it's also there's many other chips that facilitates the CPU right. You know you have the GPU, you have the GPU, you have all those stuff that needs to work with that need work in hand up and uh um so say. If you have a Linux container that you need to run a media GPU, you need a special type of container to do that right. You know, so it's not really portable across platforms.

A

So that's another pain points and uh so say you know, I've seen a study that says you know your typical service mesh, the setup on the left. You know the proxy itself consumes over 40 of the CPU overhead. You know so that is a very big. um You know um overhead that the infrastructure had imposed on the application right. So you know so that's what I can what we would consider some of the pain points in today's you know a cloud native infrastructure so how it's going.

A

You know why, and so that's set up the stage why we need uh a web assembly runtime. You know. So what about something around time essentially solves all the above problems. You know that's, uh so let me deliver the good news first and then the bad news later. Okay, so good news. First, it solves all the problems that I have just mentioned. So it's. uh If you look at the size in um I, um you know we as I will talk talk later. You know we.

A

We made an announcement with stalker on Monday and uh one of the overwhelming feedbacks We Got Is that how come the service are so small. You know instead of um you, know, 100. You know instead of 100 megabytes image. You know, I have two megabytes image. You know it's uh it's! It's literally two orders of magnitude improvements for the same functionality compared with the um you know, the Linux container images and at a startup the the the the contrast is even bigger. It's a it could be a thousand times faster.

A

You know so, instead of 100 milliseconds, you have sub millisecond startup. Now we submitted a second startup you. It enables whole new architecture. You can have applications that is not running most of the time and only runs when requests come in you know so and uh and the shutdown as soon as the request goes away. So you know you can um so it becomes a lot more scalable. You know.

A

So um one of the companies in this space is uh is, for um you know, also within the cn7 of umbrellas, there's fermia and there's a wasm cloud. You know those are the companies that provide cloud services for for web assembly based around times, and they all have the same setup you know is that when it's uh the the container or the runtimes only it starts up when the requests come in. You know, there's no need to keep warm and consume resource while there's idle.

A

You know with their Idol right, you know, so it's a it's a lot faster startup time and it enables new type of application architectures and it has near native runtime performance. You know, meaning that's uh you know. If you have applications that compiled into webassembly compared with the same rust or go Application compiled into native application, you can see. The performance are are basically on par within five percent difference. You know so there's a you know. We have done a lot of studies and you know that's um otherwise.

A

Some runtimes have done a lot of studies in that as well and it's secure by default, because it's an application that you write. It's not. It doesn't drag in the operating system and its libraries. So you cannot possibly forget to turn off the edge. You know the Apache server and leave the party server running or forget to patch the inject server or you know, or have the image containing a MySQL server. You don't know about it's impossible to have those, you know.

A

So it's a very small attack surface and that makes it secure and also because the security has been tested in a very hostile environments in the browser and on blockchains. So you know it has a very you know a solid security model. That's it's called. You know, declarative security or capability capability based security, so everything is turned off and you have to declare explicitly to turn it back on so and uh you know so secure it's completely portable across different Platforms. In that regard, it's very much like the jvm right.

A

You know the you have Java compared to jvm and then you can run anywhere. It's the same thing. You know you can accept. There's more languages supported on the front end. Now you can have rust JavaScript, you can have tiny, go Swift kotlin! You know, there's a lot of languages that spotted on the front end. You compile them into webassembly and it becomes automatically cross-up platform. So it doesn't matter. You want to run an Intel. You know um if you, if you want to access the media's GPU, you just need the web for some extensions.

A

You don't need a special type of container in order just to do that right. You know so it's cross-platform, it's programming, language, agnostic, meaning, that's um you know, like I, said, there's numerous we're going to talk about we're going to talk more about this. It's not entirely agnostic. It's not any programming language. You know it has different diverse degrees of support, but in general it has the capability to support multiple programming languages on the front end. It's unlike Java for a very long time. Java is only language supported.

A

Then you know the other jvm languages come later and they're all fairly similar right. You know, so you and there's many different. You know different languages that you can. You can have another place. Well, that's the important point. The last point is also important. It plays well with existing container tools. You know so people keep asking why why don't you use Java? You know you made the comparison. You know it's like CBM. Why can't you use the jbm?

A

Because um you know if you, if you think about the claim that I made a lot of those, are also the claims Java has made in 20 years ago before it moves uh to be. uh uh You know, a very complex platform, so you know theirs has been a huge amount of cloud native tools that has developed by the community. You know kubernetes and uh um and Docker itself and distribute run times we're going to talk about that in a minute, and it plays well with those tools because it's oci compliant it's uh it can.

A

You know, develop assembly emit web assembly, applications can be submitted and stored in Docker Hub and then be pulled down by say, container D or you know Docker itself, or you know, podmans crio. You know all those tools and then run, as you know, as lightweight applications in those in those environments. So those are the you know, benefits you know, that's uh we have a. We have a runtime that is very specifically optimized for this, um for this type of use cases right. So, of course, there's no free lunch.

A

You know um we have said you know. If, uh anytime, you hear someone say it's a hundred times smaller, it's a thousand times faster and then there's no trade-off. You know that's uh that person is probably a salesperson right. You know, that's uh you know, there's there's trade-off, you know, there's um um you know, so it is a web assembly is what we call. You know at least what I call our opinion in the very Market, meaning that it needs its own sdks. It needs its own way to develop software.

A

It's not like Linux containers which just give you the Linux operating system. You know everything that runs on Linux gonna run in the container you you just know that right, you know, that's uh um you know, but that is super easy. However, we also think it encourages the wrong Behavior. It encourages people to develop very wasteful heavyweight applications and then try to run them on resource limited environments. You know because it runs on my machine like I. Have a very you know, um you know powerful machine and and I need to write, I write application.

A

It runs on Linux, so you know, I expect it to run on the edge to a Raspberry Pi. You know all on. You know the CDN Network. You know that's uh um you know that would be um you know um for web assembly. However, you know you have your. uh We have your own compilers, we have our own sdks. Much like Java has its own compilers Java has its jdk right.

A

You know it's a it's a new programming model and it's a new programming tools that the developers would uh would learn and then you know, develop and develop new applications that are very specific that to take advantage of those features, that's uh the speed and the size and all that stuff right. You know so there's new things to learn. I, think you know it's a good news and bad news, and uh um you know- and there are common libraries that need to be ported right. You know so that's that'd be the trade-off.

A

You know so, there's um you know um it's more burden on the developers. Also I think it makes developers more. um You know, um you know uh more useful. You know you know that's um because there there are more things that need to be done by developers right. You know so um a little bit about um as it was a major runtime. What's the major on time is only a web assembly runtime, um you know um um that's in the cncf, you know that's where the CNF sandbox projects and we started the project in 2019.

A

You know that's uh right around the time when Sonoma hacks send out the threats right, you know, so we want to develop a web assembly runtime, that's very specifically optimized for the cloud native use case. We call it was a match because we see it's mostly it's. uh We call it Cloud native age native, you know so it's on the under Edge Network and uh you know we're we're resource are more constrained. So it has a lot of features.

A

We're going to talk about that in a minute, but you know just uh you know if you're interested in um you know this is a GitHub repository. You know just go there. You know there's over 100 contributors from the community. You know if you um try it out, you know see. uh If you see any problems raise the issue, you know talk to us so, like I just said, you know in two all this proud native web assembly started 2019 with a tweet from Dockers founder, um and uh then we were hit by the pandemic.

A

Then you know um then kubecon started again. You know that's uh so in this Cube car, you know, I think it's kind of come off the full circle. You know that's uh um so dark itself has launched a preview of webassembly tooling. You know the uh the way it works is that you know um from the from the docker desktop or Docker CRI. You know it would uh you would ask it to run a image. The image could be um a Linux image.

A

It would go to containerdy and wrong C and start a container and run the Linux image. If Docker sees the image, is uh it's a web assembly image from Docker Hub? It would go to containerd and bypass the whole wrong C, and the invoke was match to um to execute that that web assembly image and you know to to start another application. So you know we have a demo. That's uh you know in the URL here: that's uh um you know.

A

So, with a single command, the darker compose up, you can um it's actually three Linux, two Linux antennas, plus a wireless microservice. You know it's a one. Linux container contains a mySQL database and uh then um next to the database, you have a bottom Edge quote-unquote container, but it's really a runtime application. The application is what we traditionally call the middleware right.

A

You know so it's connected to the database through a connection pool and it's at the other end, it's lessons for HTTP request and then there's a um there's, an inject server that serves a static web page in another container. So if you um with a single command a Docker composer, you can build and launch all three things. All three. You know it's uh you know. Normally it would be three containers, but now we have two containers and one the the key application logic installing webassembly uh in the web assembly runtime right.

A

So you can, you can try it. You can see the size difference of those. You know the the uh the two containers that does much less one is the database that one is just a static web server, a hundreds times larger than the web assembly runtime. That's in the middle, you know so so you know. That's our dream, of course, is that it's maybe someday um you know a full-blown database you know like MySQL would be able to run your web assembly as well.

A

You know so that um you know so that we can get the size um so so that we can get everything that is really Slim and nice right. So you know so that's comes the full circle, so um on Tuesday, some of my hacks traded, it again to say you know, or all those years after he has left Docker- and you know the company has changed directions and all that- and uh you know that's uh um you know the the vision is trying is finally being realized.

A

So we are very, very happy about that and uh so yeah that's uh um so you know um if you are interested in trying all that's all just go to docker.com and it has uh it's currently featured Docker password. You know, that's. uh You can download a a technical preview version of the docker desktop and all the docker CLI and uh and try it yourself. You know just the demo. I just told you about right.

A

Okay, so in the rest of the talk you know so I I think I've covered uh most of the big picture. Stuff and I hope you know, that's um it's. uh um It helps everybody understand. You know why we need web assembly. You know, that's um you know.

A

What's the value proposition here right so um in the in the rest of the talk, I want to um uh go, uh go go over some of the you know, current state of the art you know: what's the status of the tooling support, what status of the library support you know how um you know um uh is that ready for prime time? Of course you know I'm biased, you know, I'd say it's ready for prime time there's multiple!

A

You know companies in the space like uh ourselves, you know, was marriage second state and also um you know uh our our um you know, um friends, you know like fermia and Walden Cloud volume Cloud by the way is also cncf project. You know, that's the um you know. So there's companies has been working in this space for a couple years now, and so the first I want to talk about is language. Support for cloud native was web Summit applications. You know so, um like I said one of the the points of web assembly.

A

It's uh it's polygard, meaning that's the language agnostic. However, it doesn't have sdks for other languages, and currently this is a survey from cncf. You know conducted in October, 2020 22. You know, which is you know right before this conference. It asks people, you know um their their programming language for, for um you know, for people who use webassembly or want to use webassembly as container whatever programming language are and, as we can see, rust is by far the most popular and uh followed by JavaScript and followed by go right.

A

You know so rust and JavaScript are both well supported on webassembly at this moment and gold support can really improve. You know, that's something that's um you know we are working and you know several big companies in this space, including you know, um I, don't know you know VMware and Microsoft doc, uh our working on that to improve goal. Support in uh in webassembly, so you know, there's a variety of different languages that people want to use, but rust and JavaScript are the most important ones.

A

So you know, um as people say, go is the language of cloud native. You know. um Java is the language for the jvm. um You know um you know webassembly Circle. We um we, we think webassemble. This fate is tied with the adoption of rust. So the rust is the language for webassembly. Rust. Compile to web assembly is, uh is a is a golden combination, so um I'll go through those quickly. So you know um with rust in one match. We support a variety of different libraries.

A

You know so um you know I would say 88 of the current. You know, uh server-side rust applications would easily just compile to to awesome. You know instead of the Native and then and also we provide um additional support for AI inference.

A

You know um when people do AI, they typically use Python, which um you know it's uh when you use that in containers, it's a it's a very big setup and then you know, and it could be very slow because you know there's python, that's delegate most of the stuff to Native, but for stuff that doesn't delegate. You know it's a it's a it's a slow language, so we have rust apis um through survazam. We have rust apis to run, say tensorflow, Pi torch open window. You know those are all running on the underlying Hardware.

A

You know. So, if GPU is available, we can run on GPU. So it's uh so it's for taking advantage for Native performance and- uh and so today, I think, if you want to use Rush to do AI inference, compiling webassembly would be the best choice, because those are the apis that only work on webassembly through the web assembly bridge to those to those um to the inference Frameworks.

A

So then there's JavaScript right, you know and when I give you this talk. I give this talk numerous times about JavaScript and people always are very confused. You know people thought webassembly runs alongside with JavaScript supplement JavaScript, but now we are talking about JavaScript running inside the webassembly. We are using webassembly as a container right. You know, so we run JavaScript applications inside webassembly.

A

So you know um one of the big issues um you know because we have announced you know JavaScript JavaScript support in Watson Edge, for you know I think couple months ago, then people immediately go to npm and put on a JavaScript module and try to run and of course, you wouldn't run because it has node.js API in it right. You know, node.js is the API that added to JavaScript. So now we realize if we say we are JavaScript compatible. We must support node.js right, I.

A

Think the the the um you know, the other runtimes on the market are learning. The same lesson you know is supporting node.js API now you're right, you know. So so you know um we have. um You know um we have three people working another. So we are. We are aiming to have the four node.js API support by the end of this year. So you would have um you know any node.js applications.

A

You would be able to run inside one match and take advantage of the like I said the Securities of the the small footprint, the the high speed, and all that. So you know that's uh that's our roadmap for for JavaScript support and then there's um a language is one thing like I. Just you know, so it's a little um you know intermingled. So language and libraries are the two things you know Java and the jdk. You know that's uh analogy. I want to. um You, know, I keep making right.

A

So you know so when, uh when the same survey ask people, what are the library features? You know, that's um access that you want from um from your um you know from your Cloud native web assembly applications, the number one is IO. Of course you know number one number two are both aisle right. You know, um I would say the number one number two number three are aisles right.

A

You know, so it needs to access the network because in the cloud native environment, a lot of things, including the database, the site, Car Service, the kubernetes sdks- are all available through the network. So you need. We need an efficient way to access the network. You know in a in a non-blocking asynchronous fashion. You know that's uh and then there's um you know other features that um that people want.

A

You know, there's, including um you know, threads and the HTTP is Network again, and you know so there's a bunch of things that people want to build into the web assembly runtime so that it can have access to those operating system features you know so so that's this is a survey result and for us, you know, is that um you know, because, if I focus on the case of microservices, so one of the first things that we need is to have a a very high performance, HTTP server, so that you know so that the microservice.

A

So you know it- you would have a proxy here or sidecar here that listens for the incoming https request and then it's after the load balancer to maybe you know one or a hundred. We have assembly run times that you, you spin up depending on your load right, so each of the web assembly runtimes can have its own HTTP listeners that come from the um that come from the the the the network proxy or the Ingress right. You know so in in Rust.

A

We we provide really good support for um the networking libraries like asynchronous Library, like Tokyo and Mio. Mio means a metal, IO and Hyper is rust. Http library right, you know, so both kind and server were supported, and in JavaScript we have the node.js server, API and uh so um I know. This is probably a hard read and I probably don't have enough time to go through that either, but it's, uh but you can see on the top left.

A

It's a rust Pro, it's a piece of rust program that opens uh um HTTP ports on localhost, the 8080 and when a new requests come in, it spawns a new task, asynchronous task that invokes the function called Echo and the the part then on the right side is the echo function. The echo function receives HTTP request and pass out. What's in it and then generate a response to go back right and it's so very simple, you know to write a web service in Rust. It's it's actually extraordinary simple.

A

You know it's, because there are a lot of new libraries that you can use. You can write it very elegantly, so you know so this whole thing is a web service right. You know, and uh um you know, within the web service you can make HTTP outbound HTTP requests to other web services in the same network or outside of network or in the side car.

A

You know to consume other services like key value database, and you know things like that and in Javas in Russia, and we have support for the for, for the entire HTTP stack and in JavaScript. You know the fetch API is really popular. You know, that's! uh You know. We support that and uh with some more code, examples to do HTTP, requests and consume web services, and then we have database clients right.

A

You know so because of the socket connection you know you can compile so those um those uh strongly typed MySQL libraries into uh into webassembly and right there, and so we support key value stores. We see we support. Mysql include that includes all the databases that support MySQL right. You know so. Database clients are covered here and uh so there's a complete example, which is I just mentioned with uh with a Docker example. So you know you can run it with stalker or you can run it with command line.

A

You know it allows you to build a a microservice that is entirely in Rust and comparing to webassembly. It has a HTTP listener and have the database on client on the other end. So you know the talks to database. It's uh you know it allows you to the code. You know operations to create, read, update, delete right, you know, that's, uh you know the simple things we all do when we first learned. You know server-side Java, you know it's the first example everything everybody has yeah.

A

So um then again you know: I talked about AI inference. You know we have. um You know um the web assembly standard has a has something called Duality and it was a neural network. It's a standard API in webassembly that allows it to access all the tensor-based. uh You know Frameworks, underneath that, so we have integrated the pi torch, which is now a Linux commodation project and uh open window is an Intel project and tensorflow as well. So you know that's uh so it's um let me see if I have available.

A

Yes, you know so, and tensorflow inference is also a couple lines of code. You know, I have four demo here, but I don't have time to do that. You know it's essentially take an image and turn that image call function, turn the image into a tensor and the password tensor into say, tensorflow or Pi torch and get a tensor back.

A

The tensor is array, and then you know, depending on what the model is, you interpret that the numbers in that array and come out with something you know so so for this one, it's an image: classification, um application. You send an image, it will tell you what it is. You know, so you can landmarks faces or plants animals. You know whatever you know, depending on your training, so you know so there's uh there are lots of um you know. um You know very useful, very exciting application. You can do so.

A

As you can see this this. This is the whole application. You know this is a whole web service that takes the input, uploaded image and classifieds. It fits into one screen. You know, I I, don't think a typical python application for machine learning would uh would fit into one screen. You know that's, uh but that's uh um you know. There's lots of um you know um easy to use apis that allow developers to do that. uh Ask a bit over that and then developer tooling. You know, that's also one of the major issues.

A

That's um uh you know that come up in the survey. That's people say it's really need this really need Improvement. You know because web assembly, like I, said it's opinionated framework. It is. uh It requires an sdks so for developers, um the the the tooling and the the need to learn. This stuff is huge, you know so so you know that's. um So there needs to be more materials, more tools, and you know things like that. So that's the refracts in the survey. You know most people say it is um you know it's.

A

uh It is area, that's current lacking, and so that's you know. Our collaboration with stalker comes in so you know um so now. What's the match um is um I think it's okay. It should be, it should say a Docker desktop and back her CLI. You know so it's a um you should go to the docker website. They have full documentation there. You know. So that's uh yeah. You know I really like Docker compose because it allows you to for a single file.

A

It allows you to build the whole thing and then you know spring up the containers and now you can specify which are Linux containers which are awesome, containers right, you know. So they can. um You know you don't know how to handle it properly. And you know it's a it's um you know takes.

A

You know three minutes to start up in her application with multiple containers. You know, that's, um that's you know, I think it is really powerful and uh of course you know. On the other side, there's uh um there's podman right. You know this is also a CNC project run by red hat, and it's uh it's also a container management tools and one of our community members. You know um a company called liquid reply is one of the large IIT consulting company in Europe and they provided integration for uh was amazing to um apartment.

A

You know that's uh to allow partyman users to use the same. You know, CI interface to start up, you know, um was image containers and together with with Linux containers, they run side by side. Again, you know. So that's uh you know. So that's where we're very glad to have a you know a 12-hour community members to contribute. You know this type of important work yeah.

A

So then um those are Developer tools, but then at deployments kubernetes is fully supported. You know so kubernetes because those in order to support those development tools, we we support containerdy and we also support serum. You know: that's a crown is the oci runtime, that is, uh the change to zero, has has merged up screen so that you know you can build a version of serum that that it would uh when it sees a image from Docker Hub.

A

It would know whether it's a Linux image, it would start Linux container to run or it's a lot of image. It would start was an edge to run it right. You know, so um we when we integrated at that level, we can have a. We can finally go to my original slides to say we have a um you know: uh service mesh with clusters with many different. You know applications running inside their own containers and they can you know, and they can all run side by side in the same kubernetes containers.

A

And then um you know on Tuesday. I gave a talk at the uh you know. Dappercon Dapper is also a sense of incubation projects and it's a it's a framework that very specifically designed to help people design microservices. So it's a it's a sidecars and then microservices. You know sidecars in containers. The microservices is another container. They communicate together with uh with a diaper SDK.

A

So we wrote that one of members in our community wrote the diver SDK into adapted that into webassembly, so that if you have a web assembly based microservice, you can call diverse Services. You saw diverse services, including service, Discovery, service, servicing location, and then you can have you know, um you know kiwi stores, you know a secret source and, and you know, messaging cues and all that stuff. So all those things are now available to webassembly based microservices as well through under the SDK integration with staffer and uh yeah, so um I think I'm.

A

um You know um I think my time is up and um you know so here are some resources and if you want to um you know, if you want to learn more or if you want to deploy that just um you know um please and uh um come to our GitHub repo, and uh um you know talk to us um yeah and uh and if you have questions you know, um come and talk to me and I'll stay here as long as anyone as the last person who lives at all so yeah.

A

Thank you very much.