Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Build Your Own Envoy Control Plane - Steve Sloka, VMware

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Build Your Own Envoy Control Plane - Steve Sloka, VMware

Envoy is a building block of many different solutions from Ingress controllers, Service mesh implementations, as well as functions as a service application frameworks. Any solution utilizing Envoy as its data path component most likely implements Envoy’s xDS API in some way. It is possible to only use a fully static configuration, but typically more complicated deployments incrementally add more complex dynamic configuration via xDS over gRCP. This talk will walk through how the xDS API is defined and what each component does. We’ll discuss lessons learned from Contour, an ingress controller for Kubernetes on how the team implemented xDS (and also struggled with it). Come take a look at how you can build a full Envoy control plane for yourself!

https://sched.co/ek9x

A

Hi everyone, I'm steve sloca and today I'm going to walk you through how to build your own envoy control plane, so we're going to introduce some some core envoy concepts once we have those concepts, we're going to look at how we can implement those concepts into different ways of configuring envoy and once we understand those we're going to go ahead and build our own control plane. Now, today, we're going to use, go line for our our source code and we'll have lots of examples that'll be available for you at the end.

A

Let's get started all right, quick little bit about me. So again, I'm steve silke, uh my my role is I'm a maintainer of contour at vmware. uh Contour is an open source, ingress controller for kubernetes. It's also a cncf project um so enough about that. Let's dig into the project. So so what is envoy so envoy is an open source edge and service proxy. uh It's designed for cloud native applications.

A

uh This is this is a quote from the onboard proxy website. um So it's a service proxy. uh We use it all over the place. um Contour uses it as this data playing data path component. Other projects use this as well: it's becoming a more and more popular service proxy out in the industry.

A

It's used by a bunch of people again. This is taken from the envoy website, so I'm sure there's much more in this list and it's also used by a bunch of different projects. So you'll see here. If I move myself, all these different projects are leveraging on under the hood again. This is taken from from the community website.

A

So let's dig into some core concepts and terminology in an envoy, and the first thing I want to talk about is the difference between upstream and downstream, so any request, that's in envoy and it routes somewhere to some sort of endpoint or other place. This is what we call the upstream and any request that comes to envoy from outside.

A

This is what we call the downstream, so requests will throw from downstream to envoy and then from envoy to an upstream, it's important to get this right, at least for the next 20 minutes, just so we're all on the same page, but this is what no envelope says upstream or downstream. This is what they mean.

A

Okay, so we talked about downstream. The first thing: we're going to introduce is a listener and a listener is a named network location and it's it's. What you know downstream clients connect to this will be a tcp or a udb connection, and then you can also apply filters to this and chain them together. So filters in envoy are sort of the extension magic they'll. Let you take these tcp connections and you can do different things with them. So in our demo we're going to go, build out.

A

You know an l7 or http type network proxy, which is kind of what contour is with that we'll be able to take the cptcp connection, run it through a filter and that filter will then make it turn that into http, headers and requests and responses, and so it'll make us easier to do this different type of l7 routing, there's a lot more to filters and filter chains, which you can dig into that on your own again, just to introduce that these are concepts that you can apply to your your your server all right.

A

Now that we have that we have routes, so routes are results of listeners, so listeners will then call different routes so again in our in our l7 our model, we're going to have a bunch of different virtual hosts that we can route to so again, you could have you know steve silke.com, and you know: vmware.com has two different virtual hosts and from there I can have them route different places.

A

I can also you know, modify the headers on each type and and do a bunch of different things. So again, this is this. Is super quick and not very much, but then just just to introduce that these things called routes are what makes you decide where things should route within your within your cluster.

A

Now, once you have a route and you're going to route to a certain place and we're going to route to clusters and a cluster in envoy is a group of logically similar upstream hosts.

A

So I always think of this as kubernetes right kubernetes has uh services and to me a service is like a cluster where it's just a name, and then it points to different set of endpoints. An envoy is the same way. So envoy has again it's a it's. A cluster which has members and members are discovered via service discovery, and there are a few ways that you can implement that service discovery.

A

So the first way is, you can just be static right. You can define a cluster and define all of the endpoints that exist in that cluster and then move along.

A

You can use dns and you can use strict or logical and they're similar in a few ways that they both look up, endpoints, dynamically or asynchronously, but strict will look for all the endpoints that resolve from a dns query. So say three come back from a query for a dns entry. Envy will then say all right.

A

There's three members in this cluster and it'll load balance across those three different endpoints, now logical, we'll use um similarly it'll, just as you can see, look up the at ip addresses, but it'll use the first dns entry as its result, so it'll just proxy out to that one initially and not without load balancing over all of the endpoints. In that back end we can have original destination.

A

I'm not going to introduce that today, we'll skip through that and there's also another one we're going to introduce we'll skip as well, but the one I want to really focus on is this thing called endpoint discovery, service or eds. This is our first xds kind of protocol that we'll introduce so for clusters. We can have again, we can statically, define the members or we can have the the members be defined by another lookup service, so kind of like how dns is implemented.

A

This will be a service within envoy that we can then feed in the members of that cluster. So eds is what we're going to do today in our demo and again I mentioned you- can have custom, but again we're not going to get into that all right. So now that I have these core concepts, I need a way to configure envoy with these these types right, these routes and listeners and clusters etc.

A

So there's a couple ways: you can do this, so the first way is to implement a static file, so I can just go: build a file program in all of the different routes and listeners and all the things that I want and pass that off to envoy on. We will load that file in and then it will serve traffic. Based on that, I can also have a place on a file system, so I can tell envoy: hey look in this directory for your files and it'll load them in dynamically from there.

A

I can use the rest endpoints for my management server or I can also use grpc. So this is what we're going to do today. So grpc is has an advantage of a rest. Is that rest? Has the poll right and it's going to be slow and it's got a lot of overhead because it's always polling for traffic.

A

Grpc is a rich connection right, so I can stream changes bi-directionally uh very easily. So that's what we're gonna implement today in our xds example, and there's two ways to implement these management servers. So the first one is called state of the world and the second one is called delta, so state of the world says hey, say I have nine clusters in my in my configuration and I add a tenth cluster, what the management server is going to do.

A

Is it's going to pass down all 10 clusters to envoy and say, hey envoy, here's the state of the world? This is all the clusters that I know about, um and so this is what you should use your configuration against and it works for all the different types right now delta is going to work like you might imagine, instead of it sending all 10 clusters down to envoy it's going to send just the one cluster that was added same way.

A

If it was deleted, it would delete one cluster and send that one deletion down if a cluster was deleted in the state of the world example. It would just you know, emit that from the list, so it would just only send eight clusters down instead of nine and then onward would say: hey this cluster is missing and it would, you know, delete it out of its configuration all right. So, let's talk about xds, we've kind of hit around it a little bit, but we've introduced these different things, called listeners called routes, clusters and endpoints right.

A

Well, they each have a different discovery, service or xds protocol that we can implement so over grpc. We can have each one of these return, a list of listeners, routes, clusters or endpoints, just like we just said, and if you strip off the first name, replace it with an x, you can see how this is. We get this name because there's a bunch of different protocols that exist that you can implement again. These are just four that I'm introducing. There are more and I encourage you to go research this more afterwards.

A

But essentially, my point here is I want to get across, is that this is where you get xds from now. There are four different variants of xds.

A

Just like we talked about, we have the state of the world as well as incremental. We can also decide on how we want to send out or set up our grpc streams, so the default way is to have a grpc stream per resource type so for listeners, clusters, routes and endpoints I'll have one stream for each type. So I'll have four total and that's in this top version.

A

Now, alternatively, I can implement ads, which is aggregated discovery service.

A

An ads lets me have basically one stream for all those different resource types, and I can I can send traffic across all different types over that one stream for today's example and today's purposes we're going to use the simple way, which is a stream per resource type as well as the state of the world type updates.

A

So having all that background right and that's a lot all these different concepts, let's take all of that and put it together and let's go build an example and we'll see what this looks like if you had to go, build this yourself.

A

So this diagram here is going to be what we're going to actually demonstrate today. So we first need an envoy. Obviously, an envoy will then get past a bootstrap config right, and this is a static file that will pass the on void and what what's in that bootstrap config essentially is anything you want to preload to onboard right. So we can set up static, cluster static listeners, any kind of static resource in that file.

A

Then we can also reference dynamic resources. That's what we're going to go today, so dynamic resources basically will say: hey. Instead of you, loading my listeners and clusters and all those different xts types, statically we're going to load them dynamically and they're going to come from this xds server here this box in green and what we'll do is we'll create a cluster, a static cluster which will point to this xds server. So when envoy starts up, it'll have enough information to go.

A

Look at that cluster and pull down all of this all of its information dynamically. Now what the xds server needs is it's going to need some source information right? What is its source of truth in terms of how it understands what routes and clusters and endpoints it should configure envoy with this is going to vary depending on your implementation, so for contour you know we run on kubernetes. So kubernetes is our source information. So contour looks for things like services for endpoints, for secrets, as well as ingress objects.

A

It takes all that different from information, and it makes you know envoy configuration and then fills in those envoy, xds caches and then passes those down to envoy.

A

So that is the nutshell. What we're going to do? Let's go ahead and let's poke around so the first thing I want to look at is this thing called: go control, plane right, so go control. Plane is a project in the envoy project organization and it's it's a go. Implementation of the data plane api.

A

So the data plane in envoy is a bunch of protobufs that represent all those objects we just we just described about um in the last couple slides, but this one they have they're all ghost trucks, so you can import them into your go project and use them from there. The second thing that this project gives you is, it gives you a sample implementation, xds server, which we're going to utilize today.

A

So instead of you having to go, build out all the grpc connections and all the extra overhead and routing you can leverage the the implementation here in this project to go ahead and build your own control plane. If you like now, you can read through a bunch of things here, but here at the bottom. There is this example server.

A

So shout out to this user here who helped build this out. I know we used to have to look at unit tests before to get a good example. So this is this is great, so this will give you a lot of enough information to spin up your own server as it says right, um so I'm going to show you basically the same exact example.

A

I took this and reused it, but what I did was I added a little bit more dynamic configuration so, instead of it being statically configured, I have my configuration, come from a yaml file right and we'll look at that next here.

A

So this is my project. I'll have the link to this at the end of the slides. If you want to check it out yourself so in here, what I have is a main.go and to get started. What we'll do is we'll go ahead and create a cache, a snapshot cache now. This cache is the core of this go control plane. You know xds server, the snapshot cache holds all of the different snapshots that exist that we've passed down to onboard.

A

So remember we're talking about setting this state of the world set of set configuration to envoy.

A

So what we'll do is we'll go ahead and look at build up our configuration based on our source information and we'll build out a list of listeners and routes and clusters and endpoints, and then we'll create a snapshot out of that and we'll pass that snapshot down to envoy and then onward, we'll load that in so whenever this, whenever those any of those different objects, change we'll generate a new set of caches, we'll make a new snapshot and then we'll pass that down to onboard until I'm like hey here's, a new version go load.

A

This in android will parse that in that's, essentially what we're trying to do here so, like I said I have this config file that we created and this config file here is just to help us. You know configure this server a little dynamic, more dynamically.

A

This server has, um I guess this is my source information again for contour and others. This could be kubernetes. This could be again depending on your environment. This is whatever it needs to be. But for me this is going to be this static file here, okay, so to watch this file for changes, we're going to go, set up this watcher, so we first created the snapshot, cache we're going to create a watcher and what the watcher does is anytime. The file changes. We get a notification back here over this channel.

A

That's all we're doing so we're getting a callback to say, hey the file changed, and essentially that's going to be our signal to go, rebuild a new configuration.

A

Now we have that set up we're going to start up our xds server, so here we're going to go ahead and build the server out we're going to pass it the cache again. That was the snapshot cache we created here in line 56..

A

Now that we have that we'll go ahead and run the server, let me get the pass it up port and for us this will come from a flag and we'll default to 9002.

A

So let's go ahead and run this.

A

Okay, so we ran the server. You can see we're listening on port 9002, like we said, and here this debug line says this- is the snapshot we're gonna serve up, so here you can see. We have some information in here and what it is. Is this information matches our configuration file here? So if you can look in here, you see, we've got this cluster name echo, and here I have.

A

Let me move myself over here for you here we have this cluster named echo and if we dig in we can see, we've got a listener zero, and here we gotta listen to zero. So the snapshot came from this file and what happens is anytime. We change this file. A new snapshot is generated. So let's take a look at how that happens. So here in this processor you'll see here is we have this new snapshot? I'm sorry this process file and when it gets passed in is the file that changed.

A

So the notifier again sends us an event whenever the file changes and tells us what file got updated. So the first thing we do is we just parse that yaml into a go struct so out here I have this api and this api defines our struct in ingo.

A

So coming back to our processor after we parse that in what we do, is we actually go and generate envoy types out of that that yaml file so we'll go ahead and loop through every listener that exists and we'll go ahead and build a set of listeners right and we'll pass in the name of it, the routes and the address and the port, and then from there we'll build a set of routes. So here's our listener cache and then here's our route.

A

Cache then we'll build out our cluster cache here so we'll add all the clusters and then we'll add all the endpoints. So essentially by time we get to this point. We have a cache full of listeners, endpoints routes and clusters.

A

So now after we build those all out, we have generated, and since our point in time or state of the world type cache now we'll go generate a new snapshot and what we'll do is we'll pass in a snapshot version. So this will go increment. Our snapshot version, basically just you know increment it by one from whatever the previous one was and then we'll also go ahead and select my place we'll go ahead and pass in the contents. Now this cache contents goes and converts my local type that I have here into envoy, xds types.

A

So if we look at something like listeners, you can see we have this resources package and this resources package goes and builds a new resource type called listener.listener. Now this type are actually on void types. So you see here I'm into this listener, protobuf. So again, now we're digging into that go control, plane and now we have. This listener go struct which lets us create envoy listeners.

A

So we do. Is we go ahead and build out so we'll go back in here, we'll go build out a listener, and one thing that's interesting here is that we'll say we'll build out this filter chain right? Here's where we're adding that http connection manager- and this is going to give us this l7 router.

A

So the same way in here we have routes, so here's make route and again it returns a route configuration and this route you know, has a match and an action again. These are envoy specifics, but these are building out all the different envoy prototypes.

A

We can create endpoints here and we can create clusters, so the processor goes and builds out a cache of that yaml file and then generates a new snapshot against that type, converting all those different types into envoy types. We then make sure that the snapshot's consistent meaning do. We have all the routes, referencing proper clusters and so on. If it is consistent, we'll go ahead and create a new snapshot.

A

We'll set the snapshot to envoy, and this is where we're actually going to tell envoy hey, go ahead and load in this new snapshot, and once you have that new snapshot, you can then process your update, all your caches locally to get the new configuration.

A

So excuse me with all that said: we have all that done now right. So the next thing I'm going to do is we're going to go ahead and spin up envoy. So let's go ahead and create a new window, a new tab here and let's take a look at what this environment looks like. So, as I mentioned, we need this. This bootstrap cluster, I'm sorry this bootstrap yemo and this bootstrap config will go ahead and load in static resources, and we talked about loading in.

A

You know a static cluster to point to our our xds server, and that's this so on localhost 9002 again, that's where our xds server is running, we're going to go ahead and tell it create a cluster in envoy to point to this.

A

Once we have that cluster we're going to set up all the xds endpoints, so we're going to set up our cluster cds config again we're going to use grpc type and we're going to point it to this xds cluster created here in this previous step. So envoy knows now that hey, I have dynamic clusters and they come from this endpoint here or the set of endpoints, and this is similar for lds or listeners.

A

So listener's going to come from this cluster name here it's going to set a cluster name and id and then down here we're going to expose an admin web page and the idle web page is helpful to debug contour, I'm sorry to bug envoy and look at all the different information that we've loaded into that envoy instance.

A

Okay, so now this is up there we'll go ahead and start envoy, so we're going to go, find the binary local on our machine and we'll pass in that bootstrap yaml file here using c bootstrap ymo. So I can go ahead and say: hack, start envoy and well. The first thing you'll see is when envoy starts up it loads. All the extensions that are compiled in so by we're using the upstream one here so by default.

A

What you'll see is we have all of the different extensions already automatically compiled in and just logged out for your information down here. You can see that we've spun up this, this runtime layer and we loaded one static cluster and that cluster is that xds server that we referenced earlier and then you can see that we connected to our server and we added a cluster and we added a listener here. So here's listener zero and that matches our config and there should be an echo server here right here.

A

This update cluster echo so that got added as well as my listener. So we connected to our server here. So um let's go verify our configuration real, quick, so we'll go look at again: localhost 9003, right I'll refresh this. So here we are, and what we can see in here is we've got you know a set of listeners so again listener zero, zero, zero, nine thousand and again that matches our config zero zero, zero on port nine thousand.

A

um We can also see clusters in here. So we can see that we have this echo cluster and it has two sets of endpoints, so dot 244 on 9101 and 24409 102. again that matches our configuration here. So it's called echo 244 on 9101 and I'm 102..

A

We can also look at this config dump and this dump shows you all the running configuration in envoy. So again it shows you all the different extensions that are compiled in down here, a bit further, we'll be able to see the static configuration we loaded. So here we see static resources and we have our static xds cluster, again pointing to port 9002.

A

And then here you can see, we've got our dynamic, dynamic resources and again it's that lds config point to grpc over that same static cluster. We created.

A

All right coming down a little further. We can now see our dynamic clusters, so this is what got loaded in dynamically through our management server. So here you can see that cluster called echo again. It comes from this xds server. I'm sorry our endpoints again are coming from eds. I mentioned that back in the service discovery slide.

A

So eds is the is the source of our our echo cluster and again it comes from the same xds server and again here's our dynamic listener because we load that dynamically as well, and then here we're loading in those those filters. So here's our filter chain and we're loading in this http connection manager, which gives us that you know l7 proxy routing okay. So this is enough to get running. Let's go ahead and query this and see what happens. So if I go ahead and curl localhost 9000, I get a response and you'll see here.

A

This is this: is that simple, echo server um that we have running and you see here's the host name that responds so we have two endpoints. So if we query this again, we should get a different one. Maybe 682 and then here's three fc right so there's those two, so I can go ahead and do a curl on this we'll curl every second and we should just load balance between those two endpoints 682 3fc.

A

You can see that right if I go ahead and come into here and I remove an endpoint again as soon as I hit save, I change the configuration our our callback. Will fire and going we'll then create a new configuration.

A

We'll then create a new snapshot, we'll pass the snapshot off to envoy and they will update so as soon as I save this you'll see envoy down here, updated. If I come back to our window here now we're only going to hit the same endpoint so 682, because now we have one endpoint in our cluster right cool. So now we can do.

A

Is we can add another another cluster in a route if we like, so we can go ahead and copy this and we'll call this new and we'll put this on 9 103 and then maybe 9 104.

A

and then we'll add a new cluster here. We'll call this one new route and a point to the cluster new. I mean we'll call this one slash foo that'll be our prefix right, so we can do routing on slash or slash foo again. Our goal here is to make an l7 load balancer, so we'll go ahead and save that again, our file changed. We created a new internal state of the world, configuration create a new snapshot and pass that back off to envoy.

A

So now, if we go ahead and do our curl, nothing should have changed here for the slash endpoint. But if we curl for slash foo, uh what we should get now is some two different endpoints.

A

Okay, all right, so there we go so we got slash, foo and slash running so, um which I think of what else. So I think that's all we have for slides here. So, um if you're interested learning about more um there's some resources here, this top resource is the sample we just looked at so it's the xds server that I wrote, which derives from the go control plan example, but adds the yaml parsing file. The go control plan is another place and obviously onbyproxy.io.

A

So again, I'm steve sloca, please reach out with questions. I'm happy to answer more, I'm happy to discuss this more. I know this was quick, but hopefully you've got a good understanding of how envoy works and how you can build your own management server. Thank you.