Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Contour, a High Performance Multitenant Ingress Controller for Kubernetes - Steve Sloka, VMware

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Contour, a High Performance Multitenant Ingress Controller for Kubernetes - Steve Sloka, VMware

Contour, a CNCF incubating project, is a high performance ingress and load balancer solution for Kubernetes. Contour offers a richer feature set than some common alternatives while maintaining a lightweight profile. At its core, Contour is providing a control plane for the Envoy edge and service proxy. This session will show you how to leverage Contour and Envoy for Kubernetes workloads in a multi-tenant environment, demonstrate recent Contour features like authentication, rate-limiting, service-apis, as well as preview our near term roadmap.

A

Hey everyone um welcome to the contour inverse controller session, so let's do a round of intros. First uh we're trying to get pretty much all the maintainers for the project here. So um I'm alex shu, I'm a product manager at vmware in the modern applications business unit, I'm the product lead for contour over you nick.

B

Hi everyone, my name, is nick young. I'm the tech lead on contour, I'm a staff engineer at vmware over to you, steve sloco,.

C

Thanks nick yeah, hey, I'm steve, I work for vmware. I work in contour since almost just inception. uh So let's see y'all pass it off to daniel.

D

Damian hanson principal software engineer with with red hat and excited to talk more about what we're doing with contour steve chris.

E

Hey everyone, I'm steve chris, I'm an engineer of vmware and I'm a maintainer on project contour I'll hand it over to sanjay.

F

Hey everyone, I'm sanjay battia as well, and working on the contour team.

A

Thanks everyone um yeah, so let me just kick it off here with what is contour and what we're all about so contours the ingredients controller for your kubernetes cluster, which means that it's a piece of software that you have to install into your kubernetes environment in order to leverage ingress ingress is a kubernetes concept. It's one of the preferred ways to bring external traffic into your kubernetes environment. You might have heard of service type newport or low balancer.

A

Ingress is just one of the options, so we are built around another really popular open source project called envoy.

A

So contour is the control plane for envoy, which is acting as a data plane, and the envelope instance, which is deployed by and configured and managed by contour, is essentially functioning as a reverse proxy and a layer. 7 little bouncer and the communication back and forth between contour and envoy are all protected by mutual dls.

A

So there are a lot of ingress controllers on the market today, and the purpose of this session is to just highlight why you might want to use contour, given some of the things that we're working on given our road map, our extremely prolific roadmap and some of the thought leadership around ingress that we're involved in so there's something here that you don't quite understand or you have neither on ingress they're not being captured in contour today, please reach out to us in in github and slack through email, we're always wanting to hear more.

A

So if I may, um I just want to highlight some of the most compelling features in contour today, so we support dynamic configuration updates. This is driven largely by how contour functions as a control plane for envoy. We have very strong focus on multi-team ingress delegation or multi-tenant ingress.

A

What this means is that basically, the behavior of how traffic lands into different applications in your cluster or different sub-domains, all the domain cannot be customized by different personas.

A

So your kubernetes cluster admin can set some basic rules on how traffic arrives at your cluster and then each team can further customize the behavior of that request once it lands in your part of the application. Contour is very lightweight.

A

You know we leverage pretty much just pure kubernetes objects, so we create a bunch of cluster rules and spin a bunch of crds there's no additional software or licensing here, there's no proprietary or paid version of contour with more functionalities.

A

Of course, we can do tls termination and pass through there's http to https redirection, there's, also session affinity, there's all kinds of low balancing options available to you and you can also modify head request headers as well.

A

Next slide, please.

A

So you've probably seen this slide before um in previous coupons, but we already on version 1.14, we released monthly, so you can always expect a pretty stable, build, very fluid release, management and lots of contributors, and in the last couple of releases you know, we've really been focusing on a couple things: external auth, so your incoming requests can be secured against not server.

A

We've added local, as well as global rate, limiting we've, also added a support for ingress class resource which was released as part of the kubernetes 1.18 and we're adding support for gateway api, which is a new kubernetes networking sig project that is attempting to re-architect, not just ingress, but a very large broad set of requirements around dynamically sourcing network infrastructure.

A

So you can think of ingress classes in recipe, 1 and ingress and gateway apis and recipe. 2 and contour has coverage for both. So please stick around to learn about how we're actually supporting gateway api and with that I'll hand it off to steve silco. Now.

C

Thanks alex alex, touched on a little bit beginning some architecture, overview components of how contours put together this slide here will help you dig in a little further into what that looks. Like you know, with with some some graphs. So again, contour is typically deployed as an edge proxy. So the goal is to route traffic from outside of your cluster into your cluster. um Here you can see we have traffic from the internet, hitting some sort of load balancer. That load balancer then targets a set of envoys.

C

So, depending on your environment, you can deploy one or more envoys in your fleet again to route traffic. So android is our data path component. Its job is to actually route traffic from point a to point b within your cluster contour's job is to be the control plane server, so contour implements an xds server and xds is an envoy type protocol which lets us configure envoy, dynamically and contour and envoy talk over a grpc connection.

C

So again, that's a rich data connection, so we can pass updates quickly and easily, so contour's job is to watch the cluster for resources. So to look for things like services for endpoints secrets, ingress objects, all kinds of different things within your cluster when any of those things change, contour will rebuild a configuration and it'll pass. Those changes down to envoy when the next request comes in envoy will be able to process that request and route traffic into your cluster.

C

However, you see fit again so typically, this is. This is an edge proxy. This can also be done for internal routing as well. So here you wouldn't have this external traffic. You just have internal traffic routing as well, but again the generic example is is how this is works with envoy and contour working together. You know to help route your traffic, so I passed out to sanjay next to talk about so the next thing.

F

uh So, as alex alluded to a little while ago, um we've got some new features that we've been working on in in contour. um Those include rate limiting support, so ability to set per envoy request rate limit.

F

As well as a version of a rate limit service to for you to be able to more granularly control, your rate limiting requests, your applications um across a full deployment, um we're starting work on uh and getting initial alpha support on gateway. Apis. A new api that the sig network group is is working on in kubernetes.

F

We're trying to round out uh the ingress v1 support so supporting all of the various features in terms of filtering ingresses, using ingress classes, various types of path, matching and other other features of ingress v1, and also we're working on a project that we have called a contour operator, which is an operator for a contour that is currently in the alpha state.

D

Yeah, so let's talk more about gateway, api, so gateway api is an open source project that was established around 2019, um actually from kubecon san diego 2019 and is managed by sig network. uh It defines a collection of resources to model service networking in kubernetes. So what are some of those resources to cover them real quickly?

D

You've got a gateway class, which is a cluster scoped resource that defines infrastructure, there's a gateway resource that defines an instance of the infrastructure, the infrastructure being a load balancer or a proxy in the case of contour, and that, of course, that proxy being on void proxy and going back very quickly with the gateway class is really represents.

D

An abstraction of the details of the infrastructure, and so, as you see here in this diagram, uh the different resources map to different roles and and the role orientation or of uh of gateway api is one of the benefits of the api.

D

If we look at existing resources prior to gateway api, like the ingress resource, there is no concept of uh having different roles to express how an application and the infrastructure is modeled for service networking, and so one of the last pieces of the resources represented by gateway. Api are routes. The most common is the http route.

D

As you see in this diagram, and the http route represents how an application developer would model advanced routing, for example, sending a certain percentage of traffic to one service over the other, and that's just a quick summary of the different resources that make up a gateway api.

D

We we first introduced gateway api support in 113 that was introduced as alpha. It's still in alpha, so still very new um still have a long road ahead to go with this, but this at least gives users an opportunity to kind of keep the tires with gateway. Api uh provide feedback not only to us, but also the gateway api community and we're looking forward to get that feedback and excited to to have gateway api as part of.

E

Contour all right, thanks danian and now I'm going to talk a little bit about contours, support for rate limiting, which is a feature or series of features really that we've added progressively over the last few contour releases and so rate limiting fundamentally is about uh defining policies for how much traffic is is allowed to reach your back end services in a given period of time, um and there are, you know, a number of different reasons why you might want to have uh control over this and have enforcement of this.

E

So a few examples are are for protection against denial, service attacks from malicious actors, where other folks are trying to overwhelm your backend services with extraneous requests.

E

It might be useful for enforcing usage quotas for different categories of users, for your services um can also be useful for protecting against uh bugs and client applications that create unintended traffic.

E

So lots of different reasons why it's useful and um it's it's definitely a really valuable tool to have in your toolbox uh and so we're we're happy to have support for it in contour now um and so envoy and and by extension, contour support, kind of two main modes of rate limiting so they're called global and local rate limiting and the the terminology really comes from the fact that you know usually, when you deploy envoy you're, not just going to have a single instance of envoy.

E

You typically are going to have a kind of a fleet of envoys, multiple instances that are running, and so, if you're, if you're running a service mesh, you might have an envoy sidecar for workload or if you're, in an ingress scenario like you, are with contour you're, probably going to have a daemon set or a deployment that has multiple instances of envoy. And so then the question with rate limiting, is really when you're making a rate limiting decision.

E

Is it taking into account all of the traffic that's coming into all of those envoy instances which would be kind of a global rate, limiting decision, or is the decision made locally by an individual envoy just based on the traffic that that envoy has seen coming in without any coordination with other envoys?

E

So that, in a nutshell, is kind of the difference between the two approaches, like I said, envoy supports both and contour supports. Both the key operational difference is that, with global rate, limiting you, the user are required to deploy a a separate external rate limit service. That's responsible for making those rate limit decisions, and then each envoy, within your envoy fleet, is going to be communicating with that external service.

E

Providing some information about requests coming in and getting rate limit decisions back and then global also provides you some additional flexibility in terms of how you rate limit so you're you're able to rate limit by things like the client, ip and and header values on requests and various other things, whereas local rate limiting is pretty simple and basically just allows you to limit the total number of requests that hit a back-end service through any particular envoy instance, and so just to to dig into the global rate limiting architecture a little bit more since, since there are some more moving pieces there.

E

um What you see here, uh first of all, in the center in the the purple box, is envoy and then on the bottom. We have the the rate limit service. So, as I mentioned on the previous slide, this is something that you, as a user, would have to deploy yourself um envoy, publishes a grpc interface that the rate limit service is required to adhere to.

E

But as long as your implementation meets that interface, you can use whatever implementation you want. There is an open source rate limit implementation, that's published as uh as part of the envoy github organization, so that's definitely an option for you, but you're not tied into it by any means, uh and so then, on the top in the in blue.

E

We have contour and contour is, is kind of made aware of the existence of the rate limit service through an extension service custom resource- and this is a part of the contour api that we introduced as part of the external auth work last year, and so this tells contour that the ray limit service exists and should be configured with envoy.

E

And then each of your http proxies within contour can optionally define a global rate limit policy which essentially says that information about requests for that proxy should be sent to the rate limit service and the rate limit service should then return a rate limiting decision back to the user and so envoy, then, as its processing requests coming in we'll reach out to that rate, limit service provide the information about the request and then based on the decision that the rate limit service makes either allow the request to proceed to the backend service or return a an error code to the user.

E

So with that, I'd like to jump into a quick demo of global rate limiting. So let me switch over to my terminal here. So uh first of all, I'm I have a cluster here. That's running uh three node cluster!

E

Give it a second. So I have a three node cluster here and let's take a quick look at what's in the project, contour namespace uh get pods and so you'll see. I have my two contour pods as part of my replica set, and then I have an envoy daemon set. So one envoy per node and then additionally, I have this rate limit pod here, and this is part of the the rate limit service that I've deployed into my cluster and then, let's just take a look at the contour config map.

E

And this is where uh you as a as a contour user, would tell contour about the existence of the rate limit service, and so you can see. I just have a simple yaml block here that defines the rate limit service.

E

It identifies the extension service that that kind of maps to that rate limit service and provides some additional parameters for that rate limit service, and then the next thing I want to take a look at is the config file for uh the rate limit service itself, and this is where the actual rate limits are.

E

E

And so I'm not going to get into the details of the exact configuration format here. This is specific to the the envoy rate limit service. But if you have a different implementation, your config format would look a little bit different, but the the important thing I want to point out here is that we have two different rate limits to find so the first one here you can see that there's a five request per hour rate limit and that's defined for each remote address, which is effectively the client ip.

E

So this is saying that each client, ip can have five uh requests per hour and then the second one is is similar. So it's a two request per hour limit, but it's saying that if this, essentially, if this header value here is present so if x, contour rate limit uh is defined on the request with the value of true, then only two requests per hour should be allowed.

E

So basically, if the header's present, you should get two requests per hour and then in total each client should get five requests per hour, and so uh the last thing I want to show just before. I start issuing some requests uh really quickly. Here is the actual http proxy I have defined and I just want to focus kind of on the the rate limit policy.

E

So this is a new part of the the http proxy api, and so you can see here that I have a global rate, limited policy defined and again I'm not going to walk through this in excruciating detail. But this is basically telling contour and envoy that for any requests coming into this particular virtual host, we should send the client ip to the right limit service. And then, if this header exists on the incoming request, we should send that to the rate limit service as well.

E

So what I'll do now is is demonstrate those rate limits being enforced. So the first thing I want to do is is use that to request for our rate limit, and so if you'll remember. Basically, if this header is is present on the request, we should only be allowed two requests per hour. So I'm going to issue my first request here and we can see we got a 200 response. Everything worked.

E

Okay, we got a response from my echo server and so I'll do one more request and we see we get another 200., and so now I've issued my two requests. So this next one should be rate limited, and so we get a 429, which is a too many requests response, and you can see a number of headers here that describe some information about the the rate limiting that's taking place and tells you that it's it's been rate limited.

E

But we know that if we don't have the header, we should be allowed five total requests per hour. Now I've already issued three requests and those have counted against this. This additional five request per hour quota, so I should be able to issue a couple more requests here where I get a 200 there's another 200, and now I believe on this one. I should get rate limited, because I've now issued a total of six or six requests from this client ip.

E

So hopefully that gives you a feel for uh for what global rate limiting looks like in in contour and envoy. There are uh many more knobs and many more ways to configure this. That I don't have time to get into, but definitely check out the docs and and play around some more if you're interested and with that I'll flip back to the slides and hand it off to nick.

B

Hi everyone, um so I'm uh talking about the road map today, so we've got you can see here that there's a few. uh These are the bigger items on our roadmap. um Our whole road map is public. uh It's published in our community repo, um there's a link in the slides if you can see it, but if not it's uh github.com project, contour community uh and then there's a roadmap.md file. um So the uh the first thing we've got here is um we want to bring. We want to uh get our operator up to scratch.

B

We want to get it uh all the way to ga. um Obviously we're gonna make it beta. First, uh it's currently alpha um in my mind, uh you, I think we've talked a little bit about this, a large part of the the things that we need to graduate to gi. um The biggest feature that we need to do that is support for support for upgrades. um The operator is intended to help you operate contour. That means it helps you install contour.

B

It helps you make sure that it's uh that it upgrades to the latest version, and it should do that upgrade in a safe way for you, so that you don't you don't have to encode the knowledge of how to do a safe, contour upgrade into your deployment tooling. The operator can do it for you um as part of that. um We also want to, or and a little bit associated with that we want to expand the gateway api support.

B

We want to make sure that we bring that um to as production ready as we can make it, obviously, with the gateway api still being an alpha api, there's a limit to how far we can take that until it moves up. uh It's the production readiness chain itself, but I think that we should be able to get good support for the basic sort of workflow and the two route types that we support being a http route and tls route.

B

They should we should be able to get those in in a reasonably timely way. Like you know, in a few months, hopefully uh that's what I'm hoping for anyway, um the next one's a bit there's a the next one I wanted to mention here is we've definitely had some issues with this in the past. Contour currently has two options for um the code that actually handles xds. So xds is the uh the control plane protocol that envoy uses uh to talk to its control plane.

B

So if you are deploying an envoy in your cluster, you are running a control plane. It may be a static control plane from a file, but if you're running any of the other products that use envoy, um they provide an onboard control plane. um Currently envoy contour supplies two options for that. um There's our own sort of organically homegrown, uh slightly slightly elderly implementation, and there is one based on the upstream go control. Go control, plane, implementation, um the uh sort of homegrown one it is aging.

B

As I said, there's been a few advances in the xds code that we haven't had a chance to build out yet um things supporting things like incremental xds and the ads anything discovery service, basically and a few other uh things there and the reason to do those is that it should help when, when you're running a large contour installation- and we have heard from people who are running thousands of http proxies- that envoy can use an awful lot of memory in that case and we're hoping that by modernizing the xds code, we should be able to help a lot with that, and last thing: that's on the roadmap that we, you know always intend to keep pushing along with is the further feature work.

B

um So some of the ones that we've got in train we've got some work for our als, which is envoy's automatic logging service.

B

So you can configure a remote logging service to have envoy send all of its logs to, um but then also there's a few things like tracing and other stuff. That's well requested features that have been around for a while. The requests have been around for a while and we are now sort of just working our way down to getting to them so yeah with that.

B

I think I've hit as much as I can in the time that I have I'd love to take more questions about this at some other time, um but over to you alex for to sum up.

A

Yeah, um okay, I just wanted to also add that we're also working on uh windows support in narrow map right, as well as making sure that contour can deploy onto ipv6 clusters, and so these are the two big ones.

B

Thank you alex for uh doing your pm job and making sure I hit those things.

A

um It also fits compliance right, which is a lot of customers, have requirements um open source users, community members, as well as customers, have requirements around delivering um fips one for 140-2 compliant ml images, so making sure that contour works using a specific set of tls cypher suites. So all of these are coming um yeah. The last thing is the community. The community is doing really well, as everyone can see um so we're at 2.7k github stars.

A

We have really lots of contributors, 450 forks. We have maintainers from vmware from red hat and we're taking a very active position in driving the design for the gateway api project upstream.

A

So a few ways to reach out to us, you can reach out to us on slack in the kubernetes channel. You can also follow our twitter account, find us on github project contour and watch some of our community meetings as well as office hours on youtube.

A

So um yeah, if you guys have any questions about you know contour the project if you're like, like I said earlier right, if there are needs around ingress that are not being met today in contour, definitely reach out to us. So we can discuss them anything else from from anyone else.

B

I guess I'd just like to say thanks to everybody, who's contributed in the last few months, since we last gave an update. um It's been amazing to see such great uh contributions from the community.

B

It's been really great to bring daniel on board as a maintainer, that's not from vmware um and to just start getting a lot more uh external contributions. So, as I take later contour, I tip my hat to the community, not actually wearing one, but you know what I mean and yeah just thanks thanks to the community for everything we're doing, and we're really excited to see where we go from here.

B

So I think we're going to move to some questions and hopefully we'll talk to you soon.

A

B

A