Cloud Native Computing Foundation KCD Amsterdam 2023, 6 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Gateway API The new way to travel north south by Ara Pulido

Description

Since its experimental introduction in 2015, Ingress has been the preferred way to manage traffic into the cluster (north-south), with a lot of different API gateways and proxy vendors implementing controllers for it. But its popularity and the different controller implementations also showed the limitations of its API, as vendors started to introduce their own APIs to complement the Ingress one. In 2019, with the knowledge gained from Ingress, a group of developers from SIG Network gathered to carefully design a new API from scratch, with the goal of keeping the simplicity, but to make it richer and extensible. In this talk we will introduce the new Gateway API, its benefits and how it compares to Ingress. By the end of the talk you will be able to understand the difference between the two APIs and the path to migrate from one to the other.

A

All right all right, all right, so we're starting again with uh ARA pulido uh the developer Advocate from datadog with the kubernetes um Gateway API.

A

We are after hour we're going to have a 30 minutes break and we're going to restart again at 3 30., uh but for now ARA will you join me, give it up Ferrara, please.

B

Thanks thanks William good.

A

B

You so yes um thanks, thanks for having me and and thanks for the organizers for this amazing conference. It's super well run the venue. It's super cool. It's like a Club come on. This is very hack, hacker, Club, 90s kind of feeling, which is great, but today I'm going to be talking about kubernetes, Gateway API, which I consider it to be the new way to travel north south.

B

um So just just some words about data dog is a platform that helps companies improve observability and security of their imprint applications, and that, of course, includes kubernetes as a platform, but also the the workloads, the applications that you run on. On top, we actually have a brand new office in Amsterdam they're, very keen to host uh community events. So if you have a Meetup that you want to have a space to run, just ping me and I'll put you in contact with the right people, but today North South traffic.

B

What is North South traffic and I wanted to call this talk north south, and also to explain with this term, which not everybody knows and I didn't know when the first time that I saw North South traffic I was so confused about it.

B

um What is North South traffic is just external traffic into your cluster um I. Don't know we why we come up with terms like that, but that's basically north south, um as opposed to West East traffic, which basically means the tracking between the services into your cluster. So this is more or less how it looks like north south, but because of the slice format. Most of my graphs are going to look like this, but always nurse out um good. So how do we do North South traffic in kubernetes? So usually what we do?

B

We have to get a load balancer to get traffic in and if you're, in a public Cloud. So if you're running gque or AKs, AKs and others there is all these clouds provide controllers. So when you create a service and Salman is playing a little bit this morning all about this. So thanks, thanks to that, I can be a little bit faster in this.

B

If you do a type load, balancer service, the cloud internally is going to have a controller that is going to provision a load, an actual load balancer for you and that's how you get trafficking, your classic. You know public Cloud. If you know, if you're running your own infrastructure, you can actually write your own controller to do the same thing.

B

But what happens in reality is that you may have more than one service that you need to get traffic from from external um and having a single load. Balancer per service is not a great idea because they are very expensive because they are scars. They actually give you an IP that is routable on the internet and we know that there are not that many of those so they get pretty expensive. So what we do in kubernetes up until now is that we create this Ingress objects and basically an Ingress object.

B

It's an abstraction of things like reverse proxies, for example, L7, reverse proxies, so you can. You are able to Route traffic based on HTTP application levels. So you can say things like if the traffic is coming to corporate.com Route, the traffic to my service called corporate, but it's it's shop.corporary.com route, the traffic to service e-commerce, and that's that's usually what we do- and this is how the Ingress object looks like um it's.

B

Basically, that there are a couple of more things you can do things like TLS termination and ex put it there, but it's very, very simple. It's a very simple spec is part of the networking core kubernetes API, so very official, it's GA um and but but the it's. It's super simple.

B

It's that, basically, you create rules based on the, for example the host, and you can do reroute space, for example, URL prefixes um again, you can do a little bit more than that with Ingress, but that's that's more or less uh what you what you can do in this example. If the traffic goes to fubad.com it routes the traffic to service one on 4200, uh if it does slash bar, it goes to service to poor 80. and that's very simple, to understand it's, it's a very, very good abstraction.

B

To be honest, um so we're good yeah we're right. Where we we are, we we can finish this talk today, um not not quite um if we step back a little bit. uh We know that in kubernetes there is this reconciliation Loop. We have a set of binaries called controllers. That, basically, would they do this watching for changes on your cluster for specific resources and do something about it, and there is a component in kubernetes called the controller manager that has a set of controllers.

B

You have controllers for replica sets for deployments for other things and those come from kubernetes itself so from the kubernetes distribution itself. The Ingress object is not like this. The Ingress is a is a core kubernetes API, but it doesn't come with a default controller. You have to deploy it on your cluster externally, so you only you only get the API but know the controller with it. So what happens?

B

Is that many options I just put here, four of them, but basically any open source project or company that they are on the realm of uh I? Don't know APA gateways or reverse proxies all that area probably are going to implement one of these Ingress controllers, and this should be fine, like the the API its core. Is there, so you should be able to move from one to another, and everything should be fine.

B

The problem is that the Ingress resource is pretty simple and you may want to do things very classic things that you do with reverse proxies like Shifting, the traffic based on weight like if you want to do canaries or a B testing, or things like that, you may want to say: okay, send 90 of the traffic to the first version of my application and 10 of the traffic to the second version, but the Ingress object that sing a lot for this, um but because it feels like it should all these implementations started to actually accepting to do these things in several ways again.

B

The Ingress object only allows for many, like very small number of things, but uh here comes nginx and says: okay, my users wants to do calories. So if you want to canaries using the Ingress object, you're able to hack it a little bit with annotation, so we are going to provide you with annotations. There are English controller, understands and you're able to do these things that you want to do that's great, but then comes contour and said: okay, if, with our Ingress controller, you're able to do anything that you want to do with Ingress.

B

But if you want to do something more you're going to use these crds that we came up with instead same with Ambassador, they came up with different crds and cone with a plugin that is not even part of the open source release, so you can start seeing the problem already, and this is not only happens at a L7 reverse proxy level.

B

This is where the problem is really really apparent, but it also happens at an L4 so because the spec of service in kubernetes only allows to say type load, balancer and nothing else.

B

Cloud providers also do these things because they say: okay, I can provision for you a load balancer, but you actually have three types of load balancers, which one do you want so because you cannot say that you, for example, in this case, is an example with G key. You can create an annotation to get, for example, internal load balancer.

B

So we have a problem in kubernetes and this is why they came up with with Gateway API, so Gateway API to the rescue three about three years ago, so it took a while until this was more or less usable, they a lot of people in the networking space and kubernetes coming from Google coming from Contour, with all the same people who created the problem, acknowledge that they had created a problem and they started trying to fix it.

B

So they decided that they wanted to build a set of new kubernetes apis to deploy L4 and L7 routing, so both both levels, not only L7 and from the very beginning they they had this.

B

This North Star to focus on while designing it, and this North Star was like the Gateway API is going to be designed to be portable, meaning that if you have a resource that you have created in a controller, you can move to a different controller, and it's going to continue it's very simple, but it's clear that we need a little bit more, so we are going to build it. So you can express all your routing needs, but it's also going to be extensible, because we know that some point.

B

We cannot design everything for now and the future, so we need to design it in a way that it can be extended in the in the future and and also role oriented with the idea that, because we are going to create so many different uh different resources, each of those resources are going to be designed for a specific Persona in your cluster. So not all your users, all your the users of your cluster are going to be using the same type of resources, good, um so without further Ado.

B

This is the most important resources that you're going to find on the Gateway API. Not the only ones, but basically it's a very simple Gateway classes, gateways and routes, and there are many type of routes again. This is L4 and L7, so we have TCP routes, UDP routes, erpc routes, but the most mature, obviously, and the the really the first one that was designed is the one that we are going to be seeing today in the talk which is HTTP routes, which you can consider really the the replacement for for the Ingress object.

B

So how does it look like um the Gateway class and thinking about this role? Oriental model is going to be created by improv providers and nobody else. So if, for example, in this case, if you're running in this example, if you're running a key cluster with Gateway API, which already you can do, uh they are going to have these Gateway classes for you. So, instead of having to create an annotation for the type of load balancer that you need, they are going to have those created for you. You can see there is a typo external.

B

This is actually on GQ. So if you create a gke cluster, you can see that there is a typo under definition of Gateway class, at least this time, I check, which I find it very funny.

B

um But then, once you have those Gateway classes that you can use as a cluster operator or as an application developer, if your policy allows for that again, this is the different resources allows for a very specific rbac rules to control who can do what you're able to create those load balancers by creating gateways on your on your cluster, and you have to specify basically a Gateway class. If you do this in UK again, I mentioned GK a lot because they have a very, very mature implementation, not like other other clouds.

B

At this point, uh but today we are going to do a lot of examples on HTTP routes, which I think are the most important ones for most people, and most people are application. Developers um I, don't I didn't have time to do. Demos, I only have one demo for today and it's and it's recorded I have actually three examples: I'm going to show two one of them with one demo, I didn't have time while I was rehearsing.

B

This talk, I was already over the time, so I decided to reduce a lot, but you can run everything all the examples on a kind cluster. If you clone that Reaper, you have all the all the examples, so you don't need anything else on those examples, I'm using Contour as the controller and the reason why is Contour is because it's open source and also they they have a very, very up-to-date implementation as well.

B

um We are going to be using an application, so this is the app for the example, so it has a front end and a couple of of services behind discounts and and that's backed up by a database. uh This is how it looks like if you run the demo, uh you can see it. So this is an e-commerce example. Application uh on the left. You see what the discount service is going to bring, but which is a discount and on the right hand, side.

B

You can see the the ads that you get and the reason why it's it's just says one point: something is to distinguish two versions, because we are going to do routing of traffic, so you will see that you can get version one in blue or version 2 in red and that way, you're being you're going to be able um actual visually how the traffic is being rerouted and everything by the way. It's uh if you clone the repo all this application is already instrumented with open Telemetry.

B

So if you use an open, Telemetry backhand, you can actually actually see the how the traffic is is actually being routed uh without um having to to do all the tests. So you can use it. It's open Telemetry. You can use it with whatever backend uh okay. So, let's see a couple of examples, the first one is something very simple that you usually do with Ingress is, like you say, with a particular host. If uh the URL is slash, Foo go here, slash bar go there. This already, you can do with Ingress.

B

So there's nothing special here, but the problem is that if you do that with the some Frameworks uh some web Frameworks like rails, for example, they will get confused because they expect the application to be on the root URL.

B

So what you do usually on on reverse proxies rewriting that URL and that's something that all these implementations can do, but the Ingress object doesn't have an abstraction for that, and you want to do that rewriting that URL. So your application believes that is running on the route.

B

So you can do this with the Gateway API specification. So the first thing we need is a gateway and a listener. In this case http and one of the things that, from a security model, point of view in Gateway API, is that because gateways may be created by operators in a different namespace, they need to say okay I'm, going to allow this gateway to be used by routes that are being created on a specific name spaces.

B

Obviously, you can allow all your cluster if you want, but you can specify only certain namespace in this case the ones that match this label can use the Gateway. Then you need a couple of HTTP routes. This is how it looks like this is very similar to Ingress. As I said, um you can do path prefix, so anything a slash. Foo goes to fronten or name space, one anything a slash bar to front end on namespace 2., but this is the interesting bit.

B

The interesting bit is that things interesting things like URL rewrites now are part of the spec, meaning that you write it the same way, no matter the controller that you're using.

B

So once we do that, you can see that if it's you go to slash food, it goes to version. One slash bar goes to version two and the application continues working and the reason why condition is working. If we check the trace that is being created, you can see that the application thinks that that is coming from from the root URL.

B

Okay, so uh the another example- and we are going to see a a recorded demo for this one- uh you can see in this case uh the example that we started with something that we want to do and Ingress doing allows to do it, but the implementations uh did, which is uh routine traffic by weight.

B

So in this case we want to Route traffic 80 of the traffic to our first version and 20 of the traffic to a second version, and in this case they have a new object, called the reference Grant, because I'm saying that a route is going to across boundaries to the next to the next name is space. In order to do that, you can do that with nginx and Ingress already, without any permission. So unless you have Network policies or things like that, it allows you to do that. In this case, they thought. Okay.

B

If you want to do that using the Gateway API as the owner of name Space 2, you are going to accept traffic coming from routes. So the way we are going to do that is first, we create the HTTP route that has two backends one in namespace, one one in name space two and you can see that routing traffic by weight it's in the spec. So it's part of the spec, something it's not an annotation.

B

It's not another object that we don't know is coming from is actually par part of the spec, so something that you can do in the spec and then we have the reference ground. So on name Space 2. We are going to create a reference grant that basically says I'm going to accept routes coming from namespace 1 to send traffic to my backend services.

B

Okay, so let's see the demo, let's play this, so basically, this is the same thing that we saw so we are creating first, the reference Grant and we are going to also create the routes, in this case the route, and once we do that, if we start refreshing the page, we are going to get 80 of the times version one and twenty percent of the time version two.

B

So I fast forward this a lot of times and um after a while I can use this open, Telemetry traces to think about whether the Gateway, API and the controller are doing what I'm supposed to be doing. So, if we check the traces for the ad service we can see, visually in purple is version. 1.0 in blue is is 2.0, you can see visually more or less.

B

80 of the traffic is going to the first version, and 20 of the traffic is going to the second version, and you can also see some latency issues on the second version. That probably you need to to check out.

B

Okay, cool, um so that was uh an example of you. What you can do, but I wanted to talk about as well um on a little bit of of some of the design decisions on the Gateway API that I found them very, very cool.

B

So we mentioned the beginning that it's designed to be portable, so you you can move things from controller to controller and they continue working as expected. How do they do that? How they I as an implementator of one of these controllers? How do I make sure that it's going to be portable, so basically they offer support levels for features they. The Gateway API have a set of core features, and those core features mean that you, if you want to be compliant, you have because they have a set of compliance tests.

B

You have to implement everything on the core level following the spec. So if, if you don't want to be compliant, that's fine, but if you want to that's how you do it Then There are a set of extender, extended features, meaning that it's optional for you as an as an implementation to implement this set of features. But if you do, you have to do so by following the spec, you cannot create annotations or tricks.

B

You have to follow the spec, but they understand as well that they might be things that are very implementation specific, because these things are very broad, some not, for example, in the case of of Gateway classes, not all clouds are going to have the same the same set of resources, so there are things that you can Implement that are specific to your cl to to your implementation, but you have to do so Following also, some rules, and the other funny thing about the design of of the Gateway API is that they have a different release model.

B

So we are used to apis and kubernetes to be part of the release of kubernetes. In this case they decided early on that they were going to be released as crds. That doesn't mean that it's not an official API. It's an official API, it's Gateway networking cates.io, but they decided early on that because there is not going to be a default controller. Part of kubernetes is going to be always external.

B

There is not that need of type that with specific kubernetes release and also crds are so much here now that people know how to deploy them, how to maintain them. So they say that this is what's going to be the way to to release the Gateway API, but because they're official they have to have some rules, and this is uh the way they manage those um those releases. So, first of all, you have API versions. In this case we have V1 beta1. Some resources are already better. Some resources are still V1 Alpha two.

B

We only saw three of them. There are many of those um and uh once uh nothing is GA yet, but it's going to be probably pretty soon, at least for for three of them, um but inside those resources they also have standard features and experimental features and standard features, meaning that they are mature enough, that they believe that they are not going to change that much. Obviously, it's a better object, so it still can change a little bit but they're they're, pretty confident that they got it right, but then they have experimental features.

B

Things are, you have to know that they are going to be changing and anything in Alpha is everything is experimental, so they get all of that and they they release versions of the Gateway API.

B

The latest one is 0.06, actually one, but more or less zero. Six, the only thing it changes a little bit of compliance tests and they basically release a version based on what they have. But then you have as a consumer of the Gateway API. You have two ways of getting two channels to get that because you it's not part of kubernetes release, so you get it through these channels, depending on what you need and the level of maturity that you require for your clusters.

B

So if you get zero, six zero from the standard, Channel you're only going to get V1, beta1 resources and only standard features of those resources and actually on the demo. If you, if you clone that Reaper in the demo, we are using experimental feature. Url rewrite is an experimental, Feature Part of HTTP route, which is a V1 beta one. So what I did is to download 060 from the experimental channel in which you get everything you get. V1 beta1 resources also alpha alpha to resources, and but you get experimental features as well.

B

Okay, so, coming to the end. um So just a recap: uh Gateway API is a new. It's a new communities, AP extension to extenderize vendor implementations for both L4 and L7.

B

It's an official kubernetes API, but it's based on crd and this may confuse people at the at the beginning, but I believe that it's it's big is going to become a pattern so I they are the first ones, but I believe more apis will be, will be written this way and if you use it watch out for two things: support levels if you're choosing an implementation and and also the maturity level. So what type of experimentation you you allow in your in your cluster and and that's it? Thank you very much is hiring.

B

So if you see anything on our careers page that looks interesting. Let us know- and that's that's the repo in case you missed it thanks.

A

So, thank you. uh Thank you very much. Ara it was brilliant. Do we have any questions for ARA? Yes, we do.

C

Two questions, actually the first one is: is there a way to define the custom traffic split like based not on the weight but on the request itself like headers? Yes,.

B

And maybe some.

C

External decision logic.

B

Yes or everything that you can think.

B

Or already allowed to do with Ingress on the hacky way on the all of that, obviously they they had that as as the as the base to Define. So yes, headers is definitely one of them. Okay,.

C

And the second question: is there, like version compatibility, Matrix between Gateway API versions and kubernetes cluster versions,.

B

Again, it's a crd, so the it's, if you can use whatever, whatever version you you're in uh you have it's I think it's more important that to to think about whether that those implementations have any problem with one version of the other.

D

I notice, one thing regarding the crds that you mentioned: basically, the first time I saw the the first time I saw. The API was that in order to include into the cluster, you had to actually install these manifests to make the crd accessible.

D

And now you mentioned that there are actually no plans to introduce this into the standard kubernetes API and who is governing this Gateway API, then, is this cncf or.

B

It's a the sick, it's a very good question. This is part of sick networking, so the same people who are um managing the Ingress, for example, API. So this is completely official part of the Sig networking and, and one of the reasons uh I think why it took a little bit longer is because they had to come up with a good set of conformance tests, a good set of release model, because they are a little bit pioneering on on how to release this.

B

But again, I really believe that there are going to be more six, creating creating new API extensions. This way, because again, if it there is no controller that is based. That is part of kubernetes like like Gateway API. In this case, there's really no need to type data to to um to a release model, and it gives them a lot of flexibility. They they. They can have a different pace of release if they, if they need to.

D

B

One thing that is important is that, once it becomes GA and probably the first uh resources that are going to be GA are the the same ones that are better right now, which are gateway, gateway, class and and HTTP routes. uh Reference run, I think, is also better at this point um once it gets to GA it's not it's not changing. So that's why it's taking a while to to to get to that level, because they want to make sure that um they are not going to say a month later. Oh my God.

B

No, we are GA and we want to change it, so they they won't change the the resource.

A

Thank you very much Ara um we are. um We are going to have a little break now until 3 30 at 3 30. We are going to have Christina the vodka I, don't know if I'm pronouncing it right, pretty pretty sure not with managed services with manage kubernetes service, so you're free for a flower. Thank you.