Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Taking Envoy Beyond C++ with WebAssembly - Idit Levine & Yuval Kohavi, Solo.io

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Taking Envoy Beyond C++ with WebAssembly - Idit Levine & Yuval Kohavi, Solo.io

Envoy is the leading proxy for handling cloud-native application traffic at the edge and in the service mesh for its performance, speed and extensibility. The extensibility is in the form of filters that are configured in the proxy that shape, secure and manage the traffic that flows through it. These filters are only written in C++....so limited to those developers That is until WebAssembly (wasm) entered the picture, making Envoy filters effectively polyglot. Through wasm, the deployment of the filters is isolated making secure, resilient to failure and able to deploy without a restart This demo filled session will deep dive into - how WebAssembly works with Envoy - dev and ops workflow to build and deploy filters with wasm - demo examples of wasm filters

https://sched.co/ekAO

A

Hi everyone today we're going to talk about extending envoy proxy with webassembly.

A

My name is edith levine and I'm the founder and ceo of solo io, and I have here you valko javi, the chief architect of solo ayo, and we will share with you about webassembly and envoy.

B

So, let's start giving a bit of an intro a little bit about the control plane and data plane separation, just as it's important to understand the moving parts here, so you will go to the next slide. We will see that the way envoy is structured is that envoy itself is configured through a different component called a controlling plane, that's responsible of processing, user configuration and updating android in real time dynamically.

B

In an eventual consistent way of that configuration, and in this slide you can see that we have two examples: one over ico and one of glue both of these use envoy as the data plane component right.

B

The data plan is where a user data http requests flow through and they are the control plane component for onward, and the control plane is the channel where configuration messages go through and if we'll move on to the next slide, we can see sort of this visualization where a user configuration in glue's case glue processes, kubernetes, crds and one of the crds is a virtual service, goes into glue and then a if glue processes it through a glue list of plugins that understand the user configuration it gets, translated to android configuration and delivered onto android and now the reason for this uh extra api, because the user configuration that glue provide is use case specific and simpler to understand than the android configuration glue.

B

Also can automate a lot of the parts that android needs, for example, cluster membership and endpoints and making envoy easier to operate.

B

Now, if we'll continue, we can talk about how the data path looks like, and we can see that an http request comes into envoy and that's one of the key parts that make envoy very useful as a cloud native proxy. The ability to extend it right, the request, doesn't just pass through a envoy onto the upstream. It actually goes through a series of filters that can modify the request and impact glub impact their behavior of the proxy.

B

So in this example, we can see that the request goes through through on voice filter chain, first hit in the external filter, where it's con consulted with an external server, whether or not to authorize the request.

B

Moving on to the raid limit filter, where it consults a rail limiting server, whether or not to allow the request for rate limiting purposes and finally traverses to a grpc transcoder filter that allows you to provide a rest interface externally, but convert it to a grpc interface internally. So your microservices can talk grpc, which is very common in the go micro services ecosystem.

B

While your front end, your external facing api will be a rest, api, that's easier to consume from javascript, so that's kind of how android looks likes and one of the nice things about envoy. Is that it's extendable? So if you'll go in the next slide, we can see that we can also insert custom extension points, custom filters onto envoy and the rest of the talk. We are going to talk a little bit more about these custom filters and how wasm fits in now when talking about extending a product.

B

Usually we have a checklist of things we want to to get out of it right, so we want to be able a language to use a language of our choice. We want this extension to work fast. We want it to be safe that if we have an issue in the extension, it doesn't bring down the whole proxy where a lot of other traffic, potentially multi-tenant traffic flows through, and we don't want to now. We only want to build our code.

B

We probably don't want to rebuild all of envoy just to extend it, and we want an easy to use experience and with that in mind, let's uh I'll hand it over to edit to talk a little bit more about that.

A

Thank you well, so, basically, basically we'll take an example of the two uh control plan that basically extending envoy um and managing envoy and basically see how they decided to go about extending the data plane.

A

So the first thing that we will look at is glue and glue is an api gateway based on envoy is focusing majorly on the edge, that's the use case, and when we looked at how we can actually extend envoy, we decided that, because we won't care about a the performance right, we want to make sure this is a basically on the control pad on the request path. So latency is extremely important.

A

We decided that we want to make sure that those filter is going to leave still in the proxy and not going to do any round trip. But that means that it's hard, because the way to actually extend those uh filter is, is by writing a c plus plus asking um a a filter code and then recompile anyway, so by ask children to decide to take this vendor on us. In that case the solar company solo oil company. Basically we make it extremely.

A

You know it's extremely to the user itself. We make sure that it's extremely fast right, because it's in the control pad and it's built into the binary basically of envoy, it's extremely safe, because you know there is no round trip right. There is no network involved or wire involved in this, but of course it's not the easiest thing to do. Right, as I said c plus plus quad, is not something easy to do. We will. We are basically recompiling android for for the users. So there's nothing easy about that right.

A

We chose this we're taking the lore for us. So it's you know it is easy for our customers, but in the natural for us. It's you know it's a challenging thing and you know there's not a lot of people who can do that. um Sto took a different approach and when sto started basically their idea was we really care about extensibility.

A

We really want people to be able to take that and write it by themselves and not be depending on us and in order to do this, they basically came with mixing, and the idea with mixer is basically that every time that a request is coming to a to the proxy, it's going to basically take the request and route it to mixer and mix. Eventually we will get all the requests and we'll be able to do whatever you want from it. Telemetry or maybe you wanted to do some.

A

You know any adapter that people might want and basically return the request to the proxy after it. The problem with that is, you know there is a lot of advantage with this right, potentially right now, people right be able to use their own language because mixer can communicate grpc. You know honest to god. Most of the people were using.

A

Go um it's not fast right, I mean that's the majority problem that the if people's talking about talked about stability in seo, that was the majority of the problem, because there is a full request for constantly going on the wire. um It is safe right. I mean it is going over the wall. You know again, there is communication between two components. Here, it's not in the same binary. So potentially there's always a problem, but you know relatively safe.

A

You definitely don't need to compile anywhere, which is a very, very uh huge advantage and it it easy. You know I don't know it's still complex problem to solve, but it's easier than write a c plus fastassing and recompile. So you know when you're looking at this. um You know the google guys a and the invo community understood that this approach that the sdo community took is a little bit problematic. There was a lot of problem with that and tried to figure out what different way we can go.

A

So, as an android community, we decided to basically to explore the web assembly, so that's kind of like overview, a little bit of what is webassembly and why we thought we thought that this would be a good way to leverage that technology. So, first of all, webassembly funny enough: it's not an assembly code right, it's a binary structure format and it's not only for the web as you know, because we're bringing it to us to envoy.

A

So it's a little bit misled, but the idea with actually creating web assembly, the the the major reason for its existence is to extending the browser right. That's why people did it and the idea was that they wanted an ability to be able to extend the browser we, but you know with that. They needed to have some use case to make sure that, for instance, it's portable right because the browser can run everywhere.

A

So we need to make sure that it doesn't matter which operating system you're using they need to make sure that it's secure, because the last thing that they want is that the right, because some wasm extension? That is not you know. You know that there's some error prone or something like that. It will take all the browser down. That's something that we definitely don't want to make sure that's fine, so they created a sandbox.

A

We want to make sure that it's fast, because again performance is extremely important in the browser, as well as the sdm envoy, and you can actually see it that it's very interesting is that, if before, if you know before there was- and people were using, um javascript javascript is not going to be extremely fast and the reason is because there's just more things that should be, you know a process while actually it's running with webassembly, it's basically much more a slim version of what should be happening when the thing is running, so you only need to uh you.

A

Don't need to pause, you don't need to do a lot of stuff, it's already, basically packaged very minimum. uh The last you know the other thing as you've all mentioned before. You probably prefer that the customer will be able. You know your user will be able to use any language and for that in web assembly they created a very interesting. You know nice interface.

A

That basically mean that you can choose the language of your choice and there is some representative, a individual intermediate representative that can go to whatever process.

A

Like eight a you know, 86 or arm, and the last one that is, that was extremely interesting when they, um when they, where they came in, is the ability to actually run it outside the the browser itself and by creating this uh interface wazi. That's what basically allowing us to eventually bring it to endpoint, so all those functionality of fast, secure, portable and as well as as any language.

A

It's what basically make that attractive technology extremely extremely attractive for something like endpoint, and this is what we decided to bring in, because, if you think about, if you will leverage it, then we can use any function. Any language.

A

It will be very fast because it's very close to um assembly code, it will be safe because it will be on the same binary. There is no need to compile, because it's basically extended and but without needed to actually compile it, and it will be x relatively extremely easy or easier than do the rest. So I will hand out to you val to explain you how exactly we do webassembly and brought it to the envoy, together with the community.

B

Thank you indeed. So, let's talk about that. As mentioned before, the envoy has a series of filters that can act on the request and change it, and essentially the way wasm is integrated onto envoy is as a custom filter right. So as far as envoy is concerned, most of android core is not aware of wasm at all, but rather wasm is implemented as a native onway filter, as we can see in the next slide that filter actually talks with the envoy runtime and runs wasm code.

B

So most of core envoy is not really aware that it's running wasim and that also has the advantage of those filters having almost the full power of a native envoy filter, and that is accomplished using something we call the abi.

B

So the way the workflow works is that a wasan vm is built into an voice, so we have v8 and we have huaven and we have an album. That's not really awesome, but more for development purposes and testing purposes.

B

When you want to use a wasam filter in envoy, you create a an envoy filter or you instantiate, in that downward config and android filter of typewasm and tell the wasan filter to load your wasam code with a specific wasan vm.

B

The wasam envoy native filter communicates with your wasm code, using this interface called the avi and in the next slide we see sort of an illustration of what this api looks like it's all open source on github, so you can check it out yourself. Essentially, it's a definition of c like functions that pretty much correspond to the envoy internal interfaces today and allow you to do the same request. Modifications that filters can do today. So again, things like we saw before like external loads like transformation.

B

All of this a wasm filter can do it, has the advantages that the you can load it dynamically into android. You don't need to recompile android. You don't even need to restart android right. It's all loaded dynamically as as regular filters can be loaded dynamically and it has the advantage of you being able to externally write and develop it. If we go in the next site, we can see that today there are various languages already supported, so you in order to essentially write a filter.

B

You can write it using the raw sdk, but it'd be much easier to use a language specific sdk. That brings the native language concepts to your development flow. So today we have is the case for a c plus plus assembly script and rust. There is an additional tiny go and we have a list of them all in our a wasmery repo and we'll refer to that later and using the sdk allows you a much easier development experience than the raw abi because they abstract much of the low level.

B

Details of awesome of you know: memory management sandboxing, all that stuff.

B

So, if we'll move on to the next slide, we can summarize that, and the way it works today is that you use a language specific sdk to build your wasan filter and then the wasam filter that you wrote communicates with the envoy wasm native filter, using the api in a wason vm of your choice where currently v8 and huaweim are supported, and with that we can summarize- and we can see that we achieve almost all our uh requirements from before and the one thing.

B

The one point of friction is how to get the wasa module from your developer laptop into the cluster, and I let it did talk a little bit more about how this developer experience is solved. Using a tool we created.

A

Thanks you well so, basically, when we looked at this and we understand the amazing power that it's giving right now to the user, I also saw this tweet and that's what is from solomon hike, the founder of uh docker and basically link cloud is one of the the the the leader of webassembly ecosystem, basically announcing wazi solomon hike, mentioned that if wazi and wasn't was exist in 2008, that we will never created a docker right, and that's basically gave me kind of like the connotation of the fact that, as you well mentioned before, it will be extremely hard, like it's really nice technology, but it's extremely hard to go about doing it and when you think about it, this is what happened with the container ecosystem.

A

Because back then you know, google did an amazing job, google and the ecosystem to basically created the linux container, but to make the linux container as popular right that everybody can use it, including us. We need a doctor to make it very simple to use. So when I saw that I realized that this is something that we can help with and it, and there is a lot of of similarity between what happened with docker and what happened right now with web assembly and that's why we decided to create webassembly hub.

A

So what is web assembly hub? So it's very familiar. It's basically giving you the docker like experience for web assembly, bringing into android platform so in the natural is, is very, very simple uh interfaces. It's a command line and it's a registry, and I will let actually you will show you that instead of me actually talk about it, but by actually bringing this experience as well to the user. We kind of like checked everything every thing that we wanted on the list right.

A

It's use any language, it's fast, it's safe, there's, no need to compile, and now with webassembly app, it's also extremely simple, and now I will lend it to you val, to show us quick demo.

B

Thank you. Let me share my screen and I will do a quick demonstration of one's me are a tool for developer experience.

B

All right, I hope everybody can see my ide. Let me make it a bit bigger, let's start with the demo, so in this demo we're going to show how we can use wasmi to create a nice developer, workflow of creating building and distributing and running are web assembly envoy filters. So the first thing we'll start with is with wasmi in it and was me in it essentially has a template of a filter.

B

So we can, you know, have something to start with right kind of like the react, app template and when we run this command, we choose the language. We would like to use we'll use assembly script and we choose the platform we want to support.

B

The wasm has multiple avi versions, so this step explains to us which version of the glue and zeo this builder will support.

B

Once it's done, you can see that it has extracted a our wasam template and we can open a bit of code right now inside of the new filter folder the index cs in the assembly folder- and you can see this is a assembly script file, it's very similar in the syntax to typescript and now just for a fun, we'll change this a little bit.

B

So what this filter does it's a very simple filter and on response, headers right whenever a response comes into envoy, it appends additional header onto the response, and now I can just change this from hello world to a low sm. It wasam and save now do note that this. uh What we'll see is how we build and inject this filter onto envoy.

B

So the next step would be to build the filter, and we do that using was me build once we build, takes an argument, what builder type to use, in our case, assembly, script and the image tag to use- and this is a very similar concept to docker images and in our case the image tag is webassembly, have io, slash eval, which is my username or webserie, have a yo and then the imagery name, add header and a version and finally dot to build the local directory.

B

So this now pulls in a build container, with all the build tools you need, so you don't need to have npm pre-installed and builds the filter for us, and let's give it a second to finish.

B

All right it as you can see it created a filter with our image tag and that filter has our wasam already in it, and we can see if we'll do osmi list. You can see that we have this image listed as an image that was just built now. In order to distribute this, we can do wasmi push now. This is very very similar to how docker push work and in fact we even use the same code and was me push will push this to webassembly hub speaking of web assembly hub.

B

Let's take a look at that, so we have here the webassembly hub open, and this is our portal to essentially help you discover, deploy and pull and publish all your extensions and, as you can see, let's explore this a little bit. Let's view more repositories, you can see that there's a lot of existing filters already from users in the community, just sign up it's free to sign up and push your code and we can filter by label to kind of help you narrow down by use case.

B

If we look, for example, at the security label, we can see this filter that has a nice description to it, and then it also gives you the command. You need to pull it, so you can do was me pull to pull this filter from a web assembly hub locally into your local uh laptop now and let's log in just to show you how my user looks like, and you can see that I'm also a member of the solo organization- and these are fillers published to the solo org.

B

But if I go to my own user, you can see the add header filter that we just pushed.

B

All right, let's continue with that demo, so, once the filter is all built and all push we can deploy it. So once we deploy here I'll, just deploy locally to android to have it just run fast and let's run this command deploy.

B

We currently support a glue and hcl or local envoy. What it'll do is essentially pull in the image if needed and then run it and essentially give you a very quick and easy way to get your filter loaded and running.

B

Now, let's see that everything is working if everything is working, I expect to see this hello, wassum header, on the response. So let's do a head request just so we can get some headers and you can see here. The request was sent successfully and the response indeed contains the hello wasam response header that I just created in this filter and that's it that is our demo for wasmi. The tools that allow you to build, publish and deploy your web assembly envoy filters.

B

Take it away dude for some final words.

A

Thanks so much you well.

A

So guys, as you see, we are extremely extremely passionate about everything related to wasm and the extension of service mesh and envoy. Specifically, uh we think that this is the future of the cloud. We worked very hard to create a tooling to make it extremely simple to use it, and we really hope that everybody will use it running as envoy in production for a lot of big huge enterprise, our customers. I can tell you that this is so valuable and extremely desired.

A

um Look everything that we're doing most of the stuff that we're doing is open source, including glue and uh in a service mesh. All the work that we're doing is almost open source um come. You know, follow us on twitter, uh go visit, our repository and specifically the wasa repository there. You will find everything that solo is doing, including a spec for awesome, oci image, so please check it out and we would love to take you. You know to get any feedback thanks. So much.