Kubernetes Network Gateway API Special Interest Group, 20 Feb 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Network Gateway API meeting for 20230220

Description

SIG Network Gateway API meeting for 20230220

A

Okay, everyone welcome to Gateway API meeting for February 20th 2023, although, as always, it's the 21st for me, um let's get started um Shane. uh Do you want to run through your couple items.

A

Awesome sorry yeah I forgot to mention the usual. uh The usual code of conduct disclaimer be nice to each other everybody. This is uh subject to the usual uh kubernetes to contacts Goshen.

B

Where is that um so I'll be basically just saying this at every meeting until then, but recently we decided we wanted to try and do an earlier morning, or rather an earlier time meeting one that would be more friendly. Is the EU time zones Brazil East Coast, that sort of thing so March 6th is the selected date that one will be at 9 00 a.m. Pacific time just it's on your calendar already, if you're using the Sig networking calendar I just wanted to remind people and then another General call out from the Greater kubernetes Community.

B

If you are using any uh images that were on the previous case.gcr.io, there is a big movement right now to get everybody off of those as it's being deprecated and moved to registry.katesio k8.io. So if you're using it in integration tests, anything like that, please go through and replace the GCR dot IO with vks.io registry. It is I, it is identical, um but the old one will be shutting down. Eventually, that's it for me: okay.

A

Cool um yeah I had a couple of reminders and a couple of things that are that have come up for me this week, uh as I have go back up to speed after sort of traveling for a few weeks.

A

um The first one is this discussion about um allowing TLS to Route using uh alpn negotiation, um the um yeah so um I think um it would be nice to this feels like a nice to have to me, but uh yeah I'd be interested to have more people comment on here. It's been me and uh David and John Howard so far are talking about it, but I'd definitely like to get some other people to have a think about it.

B

Yeah I read this one I thought it was kind of interesting I mean as long as it's I think it sounds like it's going to be something that ends up an extended and that's fine yeah. Try it out for a.

A

While yeah I think it'll be yeah, I mean the TLs route is currently still Alpha, but even if even if it moves to Beta, this would be uh you know, experimental uh and extended field right to that.

A

um Does anyone else have any questions about this before we move on.

C

Awesome yeah I was just waiting for more feedback. There's like one concern around. You know how many proxies actually support it, but um yeah we're not too worried about that right now, yeah.

A

Yeah I mean I think with uh yeah. If you say you, you've tried it with nginx and Envoy um and uh yeah it's traffic as well supports it. That covers a lot of implementations. I actually have something to to show you for that in a minute, but yeah, um that's. That seems pretty good. That seems certainly good enough it to me to put it in extended, um yeah, um yeah uh yeah. That's really it's a really cool idea, I think um yeah, so yeah I definitely will keep an eye on this.

A

One uh David make sure that uh it doesn't fall through the cracks. um I do think that if we can just get like I, just like a few more people to sort of give us some feedback here before I tell you to go and do a get because doing.

C

D

A

All the feedback, if you haven't uh but like I I'll, be honest, I think this is pretty close to like hey like you know, kick off, kick off and get and uh and let's see what happens when we actually propose doing this for reals.

C

A

A

uh Cool uh I'll give uh a small count for anyone else to have anything to say here.

E

A

uh We'll leave that online. If you uh the the discussions there, though, if you want to add something to it later, okay, another one from me, um this one came up while I was away, um I did keep a bit of an eye on it.

A

um Look I, think yeah as one of the primary culprits uh responsible for policy attachment. um I think that a lot of this feedback is very fair.

A

um I think the policy attachment is not very usable without us solving the way to get feedback about what's attached where um and we definitely need some patterns and some guidelines about how to do that. um I think that the concern of Keith has here about CID for information is 100 true, but I. Don't think, there's any better way around. That I'd put a big comment on here. Further down, um yeah like I I, really only see three things like you.

A

We either add full-fledged fields or we add more CRTs like there's, there's no and adding full-fledged fields for this kind of sucks, because um you know you'll be kind of building up stuff. That's uh you know very optional into the API. um That's why we wanted to do uh extra crds.

A

To start with, I think that the thing we need to keep in mind here is that we want to try and work towards um having you know, having the cids be as standardized as possible, um and so maybe- and so that's why um I've moved on to the next uh item in the agenda about timeouts before I get on to talking about timeouts um uh uh Arco I see you're. Here um you raised that great.

A

You raised a great issue um with um with those things, but before we get on to talk about that, um yeah I'd like if everyone could have a look at this, this PR well can I get a few more reviews on this paper. Please Shane, you've already done yours, um but um yeah it'd be good to get a couple more people comments or feedback on this PR.

A

um The idea here is to just update the language around policy attachment um no I'm gonna fix the um yeah to add the fact that um policy attachments are meta Resources, by which you know a meta resource is a a resource that changes and modifies the functionality of another resource um for all policy attachments and meta resources. Not all meta resources are policy attachment uh reference. Grain is another meta resource that changes the functionality of the targeted resource.

A

The important part about policy attachment is it's about this, having config flowing across the the hierarchy. um That's how I think of it anyway. So that's what this update is about is about clarifying the language there. So if we could get some more feedback on this, one I would very much appreciate it. That will be very useful for us talking further about policy sentiment, uh so yeah that's most of what I had to say about policy, this policy judgment thing.

A

Does anyone else want to say anything before we move on to the specific example of timeouts.

A

Thanks Shane, that's a thumbs up from Shane for the record.

A

Okay, okay, cool we'll uh leave that online. For now um so yeah uh I have um I've done a first uh first cut of a finance Gap I'm, going to go straight to the deploy preview um I only this is this was done like at 11 PM last night. My time so I have there's some more stuff I'm going to add. Today, uh Arco I saw your great issue about um what you proposed. We should probably hit um so I.

A

Think I'm, definitely gonna address that uh in here, but the most important part about this was that um yeah I've done I did a survey of all of the current listed implementations and what data plane they use.

A

um I'll do some summary numbers and stuff like that, as well in terms of what data plane they use. It is important to note here that I'm talking about the layer, 7 stuff in HTTP route, not more General, uh timeouts um I, think the layer 4 stuff is interesting, but we'll have cute. There are interesting places where you can put timeouts there um yeah so anyway, um and then the the most important thing is. He uh I've made these flow diagrams I'm, just gonna open this bit go so this is um this.

A

Is the basic flow diagram I made? It's literally like you know, hey here's. What happens when you create a client connection proxy sends the connection to the upstream and then gets a response. You know pretty basic stuff, but that gave me then a place to put uh where all the timeouts are. So this is the envoy timeouts diagram with the 14 available timeouts. That Envoy allows you to configure.

A

um Envoy has the biggest number of timeouts, um but the thing that I was hoping to do is to sort of take that uh those timeouts and sort of wash them against the nginx ones and uh proxy ones, and the traffic ones, and this way we have some better data on um on what the timeouts are actually available and what timeouts we can like it's possible for us to to configure like a standards or sort of as John said on uh on that policy thing, uh a more vague sort of description of what a timeout should be so yeah I've talked a lot.

A

uh Does anyone else have anything to say about timeouts.

D

B

C

D

Great uh Nick, thanks for yeah, drawing all drawing it all out, yeah.

B

Seems good yeah, I.

D

B

I had the day off today, I'm here for the meeting, but I wasn't at work today. So I haven't read through this yet, but it will tomorrow, yeah.

A

Yeah no problems, uh yeah, there's quite a bit to look through uh in terms of these diagrams yeah I must admit that part of the point was for me to say to people. You know you know how I told you this was. You know harder than you might think these diagrams sort of make it a little bit more clear. I think that it's uh yeah there's more there's more to timeouts than just a request.

A

Timeout um and all of these are there for a reason like you can't people don't add stuff to their programs for no reason uh so yeah it's it's harder than it might seem. Sorry, okay, you go.

D

Yeah next I think I I in the issue, I'm forgetting the number um I I think I described four kinds of timeouts. uh There are two at the HTTP level. One is close to request timeout, so yeah the time it takes for a request to complete so I think each proxy calls it something different based on right, so I think that's that could be a high level um uh term, another one it's idle timeout. So that's between consecutive requests. What's that yeah um and I. Think I think.

D

Therefore, timeouts are important because they affect layer 7. So something like a connect timeout. So how much time do you wait for to sort of set up a connection with an upstream and the other part? I think this is an open question, is do keep alive fit into this model or blue keeper lives as in live on at a top level, a different sort of noun.

A

Yeah yeah so I think again, cable lives, cable lives become more complicated because you've got both like four people eyes and last seven keeper lives uh and they are important um more or less important, depending on whether you've also done stuff to upgrade your connection to a long-running one like a grpc one or a websockets one.

A

um So yeah I think uh there's a few um there's. Definitely a few uh things. There I think I agree that it's likely that some sort of request timeout will be um pretty reasonable uh and some sort of idle timeout and some sort of connect timeout.

A

Most things support something that performs those functions to some extent: um mm-hmm yeah, the and I and I think that I think John's John said that you know his point is fair that as long as there's something that's roughly analogous to it, you know probably okay, for it not to be exactly the same. I do think it'll make it complicated if we want to add conformance for timeouts if they don't work close pretty closely to the same, um but hopefully the we can make the definitions work but yeah.

A

The the point here is just to start off on the the data Gathering to sort of say: hey you're like if we do a connect.

A

If we do a request, timeout and a connect timeout, then you, the top three or four uh data plane implementations can do something like it and it's possible, um and so that's that's the point of what of what I wanted to start here so I will I will uh take the stuff that you put in that issue Arco thanks again for that and uh update this uh update this with sort of some suggestions um that yeah about things that we can handle uh and then and then leave this for you all to have a bit of a read-off.

A

Awesome um yeah I know it's uh I, know it's holiday, so I, don't wanna hold you all up. That is the end of um all the stuff I had on the agenda. So we got a bit of time left. Does anyone have any questions they want to ask or any other things you want to talk about?

A

If not, then we'll probably call the meeting early.

E

Hi there one question um I'm Thomas nice, to meet you um first time. I joined just want to know. If I want to talk about something, I can add something to the to.

A

The agenda absolutely, for example: yes, yep, yeah, so yeah, you feel free to add the agenda anytime. That is a general rule for everybody. um The agenda is open. You can add things to talk about um now or next week. um Absolutely yeah, perfect. Thanks, no problem.

B

Yep and next week, we'll probably be a little more active lots of people off today. Yeah.

A

Yeah we are us, we are a slightly U.S Century group. So when there's U.S.

E

A

I'm in Barcelona, so it's pretty late here, yeah, oh yeah, wow, that's pretty rough! Hopefully uh the the meetings that uh Shane is Shane is talking about hosting free. All should uh should be a bit helpful. So let's not keep you but uh yeah like uh if you, if you do have a minute to talk about what uh what sort of stuff you're looking at doing, we go. Api I'd be super interested, but if you want to do that at a.

E

Time I can talk briefly yeah sure, like so um I'm involved, I'm from Red Hats.

B

E

Doing projects um using policy attachments so when it came up, I still have to read: I didn't I didn't see. I didn't know that issue about the complexity of policy, testment existed, I, guess I could have written it myself, maybe, but um uh so we're doing rate limiting and and off and we're trying to make sense of. You know policy attachment, and there are there's this. uh You know of course, for example: Envoy Gateway, going the the filter.

D

E

And we're just wondering okay, so we we really like the policy attachment thinking of different roles and overrides and defaults and the whole thinking behind. It makes a lot of sense to us, but in the details there are problems and and and yeah. We have to fear like that. It's some things are easier to do through filters um and how and yeah well so like it's like discussing that topic, the topic of policy assessments and how to make more sense of it and how to make it better.

A

So yeah I think the to summarize the comment that I put on that discussion. uh Polysat attachment is complex and it's unfinished. um You know like it's. It's the skeleton of an idea, not a fully expect idea.

A

um The the idea here was to sort of have us say yeah. This is a thing that we're that we'd like to be able to do. um The defaults in the overrides party, in my mind, was a large part of like the reason why you would do it this way um and I think Arco um you and the other you and the folks who are actually actually active on Envoy Gateway.

A

Unlike me, um you know, did a great job there of sort of making the call that, for the sorts of things that you're working on that filters were better um one of the updates that I did in the pr uh in 1565 is one um that I mentioned is to actually change the language in the initial. In the as it stands today, uh policy attachment document, it says you can't do um yeah, you can't do policy over custom filters.

A

I think that was a mistake, um although the um the like an extension filter, I, think that the complexity there is definitely around knowing what config is available, but that is a larger problem.

A

um I think that it makes sense to me that if you are going to have custom filters, that it would be nice to be able to default those custom filters and that probably something that looks like polished attachment is close to being the right way to do that.

A

um The um so yeah, that's part of what I wanted to update here is that I have changed. The wording um right. uh I have changed the wording.

A

I'll read up yeah yeah I've changed the wording to say like hey you can you can do um you can do um it's okay to do um Custom, Custom filters with custom policy attachment objects to sort of set defaults for those filters, because I think the biggest downside of using a extension ref Uh custom filter is that without something like policy attachment, you have to specify that in every place that you want it to apply and that kind of sucks that uh you know for a rent limit.

A

For example, you you have to specify the rate limit on every single service that you want to be rate limited. You can't just say all these services are very limited in the same way.

E

A

E

um Yeah what you do have with the filters of course, if you have like you, can have a each row, can have its filter in an HTTP uh route, for example, and with the policy you attach, you attach it to the to the object. So you you have the route and that's it like so you'd have to repeat, for example, all the rules. If you want to have that specific specific specificity in the uh you know, what.

A

I mean yeah, absolutely yeah, and so so that's what what I'm? What part of what I was aiming for here was to be able to say Hey, you have the specificity if you want, but if you don't want it, you can also default things easily easily for a given value of easy um I. Think the massive the massive open big problem of all of this is how, as a user, how do you see what policy is an effect on your object? This is a this is a very unsolved problem.

A

Yes, so yeah 100 I acknowledge that this is a big unsolved problem. It's actually surprisingly difficult, because it's really easy to create fan art fan in problems for the API server.

A

um If you, if you're, not careful about what happens when one one of your resources gets updated and if that generates 20 other object updates, because a policy has fallen in or out of scope, then that's not great um yeah, so it's really cool yeah yeah. So anyway, I have a read of that posts are, and please yeah feel free to come back to. Maybe the March 16th meeting might be a good one for you, um but.

E

A

Do that yeah, yeah and uh I encourage everyone else to get involved on this one. This is going to be a sort of a long-ranging lot, probably a long running and long-ranging uh and important change that will impact a lot of stuff in the future, so um yeah I definitely encourage people to um to post your thoughts on there like I said like I.

A

Would label myself as sort of well Rob and I are jointly together, but sort of I did a lot of the the stuff for policies at the defaults and overrides, and so, like you know, Point finger I'm happy for people to point the finger at me that this is not finished um but and I agree.

A

It needs to be finished now that we're actually getting to the point of end where we're trying to use it, um and so yeah like I, really want as much people as many people to be involved as possible so that we can so it could not just be me writing this down and doing things. The way my way.

E

Cool, that's good, so I joined to the right moment. Yes,.

A

You did absolutely.

A

Okay, um so yeah uh does anyone else have anything they would like to add or ask questions about. uh Remember it is an open Agenda. You don't need to. You, don't even need to put the item on the agenda if you just want to ask a question now knock yourself out, but otherwise I will give everyone their holidays or evenings back.

A

I'm going to give you a hand countdown.

A

Okay, thanks very much everybody. um We will uh see you all next week remember about the months March 16 being at the different times. That probably means I won't be able to make it because that puts it at two o'clock in the morning. My time I think um I will see you all around. Please feel free to ask questions here or on Slack thanks.

E

A