Kubernetes Network Gateway API Special Interest Group, 22 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Network Gateway API Bi-Weekly Meeting for 20220822

Description

SIG Network Gateway API Bi-Weekly Meeting for 20220822

A

All right welcome to the gateway api uh sync for august 22nd, um just as a reminder. This meeting is covered by the kubernetes code of conduct, which pretty much boils down to being nice to each other, um and the agenda is open for anyone to put anything that they want on it. If you are looking to add something to the agenda, just make a comment and chat. If you don't have a copy of the doc and we can send you a link.

B

Did you intend to share your screen shane.

A

I am about to cool, was I already doing it? I didn't know.

B

No, no, you were not. I okay.

A

All right, sorry august 22nd, 2022 all right, so let's go ahead and get started. I had a couple of topics um that I wanted to talk about. um So I'll start with. I was thinking about this from the perspective of uh the implementations that I maintain. So it would be nice if there were scaling semantics within the gateway resource itself or within gateway.

A

Api potentially- and I wanted to just throw this out there- something for discussion if other people feel like this could be a cool thing or feel like there's significant pitfalls to doing this, but the basic idea that you would have instances- and I won't say pods in our case- it's pods- but I know like in rob's case with google it'll, be instances somewhere else, instances of gateway that you would potentially scale horizontally.

A

So should there be something within the spec to allow you to basically do that curious. If people just have thoughts on that.

C

uh I'll say we being console added scaling semantics to our gateway class config. um It feels.

D

Like we're gonna do the same thing: yeah.

C

It feels like one of those traps where you might get stuck.

D

Just proxy and config values through.

C

Forever but I don't know.

B

Yeah, that's uh one of my concerns would be just that that it's hard to decide where the limits are on what you pass through like I know uh there are a lot of reasonable requests in this realm and one of the other ones I've seen was basically how on a gateway, you could communicate what kinds of service type load balancer configuration you wanted. For you know, a lot of implementations are provisioning, a service type that'll be in front of their gateway, uh and so trying to pass config through that way was also.

B

It seems like a very similar concept here of gateway, specific configuration, and I think you mentioned you know like what we would do on what what google would theoretically do. uh If we ever made this configurable, uh I think it would be probably a different gateway class or you know, a different gateway class config, so that overlaps a bit with what it sounds like console is already doing.

B

um I I don't want to say this doesn't make sense. I just uh we have to be careful uh with anything. We add to a gateway, because you know uh we have to be very calculated in what the scope of a gateway is and what is we're just templating other resources, um yeah but I'll hand it off to. I think keith you had next.

E

Yeah um I share some of the concerns that that rob brought up. um It feels to me, like the gateway api up to this point, has been very much um describing ways describing actions that the data plane should take when handling traffic, traffic, routing or policy attachment. uh It's describing actions. The data plane should take management of the data plane.

E

uh That is, management of the compute behind the data plane seems to be in a related, but still to me it's related, but it's a separate concern um and you know, should the gateway api have, you know, should gateway management fall under the purview of the gateway api?

E

Maybe um gamma has some interesting use cases here when it comes to um psi carlos mesh uh or other mesh configuration, uh but that's a can. My gut tells me that outside cam we likely want to kick down the road until we've got more concrete.

E

Well, the use case is clear. um I I guess what I'll say is that if we, if that's decided to be a part of the gateway api, it should be a completely separate thing, um because I can just already see the collisions between the existing routing apis and then management and crossing of concerns. And it doesn't look fun from where I'm sitting.

C

A

Yeah I see where it comes from.

F

So is the is the question I would want to ask you shane is, is the thing that you would like to be able to do is to be able to be like cute kettle scale gateway.

A

I mean that would be cr cool. uh One of the things that I put up in the notes here was I was and like again I didn't even make this a discussion, because I already realized some of the problems with it and more or less just thought I'd like to talk it out, uh but I did think it would be really cool if we could have gateways be the target of auto scaler so like, for instance, the hpa could actually target gateways in addition to like that might be wrong too. But it was a neat.

A

It was a thought that I had and it was kind of a little bit where this was all driven from, so I'm actually in the middle of figuring out what scaling is going to look like and I've done exactly what I I'm sorry.

A

I don't recognize voices very well, but the person from council said we have a gateway, configuration, that's used through parameters referenced in gateway class, and that's how you do it today, um but in our operator like you would deploy a gateway and then theoretically you would have let's say traffic, or I mean you could have some of the basic stuff that you get with like the non-custom metrics for the for an auto scale, or something like that. But you could definitely have like traffic based custom metrics.

A

That would lead you to need additional instances of the underlying pods for that gateway right and having that something that the spec accounts for seemed like neat to me. Does anybody just like that? Just would not work at all in their setup. I'd be curious to hear is like nope. That just would break everything. There's. That is not interesting.

A

F

To me to me, it sounds like it, it probably would work, but it really.

F

I think, the thing that it scares me for some reason right like, and I think I think it scares me because um because it feels like there's going to be a lot of definition required for like what do you mean by loaded, um you know how do you, how do you define like what traffic is important and you know what metrics are you gonna use and stuff like that and there's I mean we've had some stuff open for a while about like how do you produce standard metrics for for the sort of stuff that we're talking about and you I think I.

D

Know there was.

F

D

F

In smi about that, um but and and.

C

F

Think, like it kind of feels like I agree that this would this would be cool to have, but, like the um the uh I think, it'd be cool to have. But I really.

G

F

Like it's almost like, we need, we've got a tune, an egg problem here- that we need implementations to try it out to be like that works that doesn't work to me. Sorry rob yeah.

B

Yeah, I was just going to say you know as far as things that just don't work this. This is something that is fairly is not meaningful to the implementation at gk right, it's something like I don't want to get in the way of something that is clearly useful for a large number of implementations, but it's something that uh it would not be meaningful to us.

B

uh It's not it's something that we can't translate to anything.

A

Yeah, your use.

B

A

Is actually the biggest attracting thing from my from my perspective so like on one hand I was like it makes some sense because, at least in our implementation, a gateway ultimately becomes a deployment right and you can scale a replica set directly if you felt like doing that versus the deployment, which is a higher level higher level api, and it would seem to make sense that, as you go higher, you know, there's still some things that you can figure. On the other hand, not everybody is modeling gateways.

A

Do not uh ultim automatically become deployments that everybody set up.

B

Yeah, I think, that's different. You know it's not just limited to cloud providers right there. I I hope that there are going to be implementations of this api, that model to all kinds of different uh low balancing infrastructure, on-prem, etc.

B

Like things that are are not currently in our set of implementations, I don't want to back ourselves too far into a corner and say that you know in the vast majority of cases, gateways are going to be represented by a set of pods, because I think there will be other mechanisms other than just cloud providers uh to deploy a gateway. At least that's that's my perspective. I could be wrong in this.

D

By the way, this will be amazing, cucum talk, so there's your ticket. um I think, like the the struggle I have with this is how useful it is to generalize it as like an api construct or like something that the community is required to implement versus, like hey. Here are some really good best practices and we found that this works, and this doesn't and we shouldn't be reinventing the wheel among the different providers.

D

um Also just it probably needs some experience to inform the design. I don't know how far people have gotten.

D

But again, this is your ticket to kubecon.

A

um The we've gotten, as far as I wanted to have this conversation, although I'm confused by that, maybe that's not what you meant it sounded like. You were like how far is this thought gone.

D

What I meant is like, if we, if we turn it into I, I guess I went one step further than you, which is like okay and then what kind of like api or like what would that be the result of that and that one I was like do we have sort of the experience with the user base to know what makes sense in terms of trade-offs, but I do have I do echo robs like how generalizable is this.

D

I think um it's definitely useful to understand how people have been doing it, but to like mandate it as a requirement. It's less clear.

F

Yeah, I guess I guess in the end that I think when I said before, that I found it scary.

C

I think that's that comes.

F

Down to what that's really what the heart of it is for me that um it's that it feels like something that would be difficult to generalize correctly uh and um yeah and at the same time yeah. It would require a lot of specific.

C

Experience to then be.

F

Able to generalize it correctly, because I certainly don't have a good intuition about how I write something about that. um So I think, like I guess from my point of view, I'm like I'm not. I don't know. I'm uh I'd like to see I'd like to see some more about this, but it does also feel like yeah like.

C

F

We've got a lot on already as well.

A

If nobody else has does anybody else have any? Oh sorry, I can't really see you guys right now go ahead. Somebody talking.

A

I was going to say if nobody else has any burnt.

C

Yeah go on the question: when you talk about generalizing to me, it seems like it's either an implementation supports it or not, because the scale api is extremely specific to make a resource scalable, you need a slash scale subresource. That has, I think, it's a selector to identify the thing. You're scaling and um I think, like a field saying how many replicas do you currently have something like that. So when we say generalizable, are we saying how generally applicable or is there something we could do that would make this api more or less general.

B

I think you know what since you're mentioning that I I think one of my questions about this would be how meaningful is it. You know this. The scaling api is something that is well known and there's I you know, I believe some efforts to make traffic based, auto scaling thing right or or some concept of that uh for deployments right. So if you have a deployment, that's backing gateway that can support auto scaling.

B

Is that significantly different than a gateway that maps directly to pods? That I'm not sure- and I don't want to- I think, there's more. uh You know somebody needs to sketch this out in a little bit more detail. I don't want to go too far down this rabbit hole quite yet, but uh I'm a little curious. If we can, you know model the same thing a little bit differently.

A

Yeah, so what I'm gathering so we've we've gone at this for a little over 10 minutes. So I don't want to hog too much time with this, because it was just meant to kind of test the waters. I would characterize the response that I've heard so far is that there is interest.

A

Nobody thinks that it's the worst idea ever and generally people would like to see more to be convinced that they'd want to keep going with it is that does anybody feel like that's a bad characterization.

F

A

About right to me, but I think.

F

John, I had his hand up.

G

F

G

Sounds right, one thing: I would just notice that I don't want to scope creep things too much, but I do feel like there is a very large uh general problem of like customizing improv in cluster proxies right, it's more than just auto scaling. Oftentimes there's you know resource requirements and basically the whole deployment and service spec, potentially.

G

I don't know how to solve that or that we need to, but we should at least take that into consideration when we're discussing anything, that's you know customizing any part of those. I think we did have.

D

G

Long while back, they had some discussion of like classifying like types of deployments, um but we never really standardized in any way. For for how this would work.

A

Did I paraphrase what you were getting out there well enough.

G

Yeah, I think so: okay.

A

That's a good thought: yeah, the the in cluster, the in cluster proxy thing is my life. uh I don't have. I don't work on anything that does you know anything like gke, so um I have found that the configuration for the gateways has been. I mean right now, one of the ways that we do it is.

A

We have a gateway configuration crd that you attach via parameters ref, we toyed with the notion of having other things, but that's just what we have for now and it's not great, um but it is the way you would do like some of the scaling stuff right now. Theoretically, so it's a good thought, like maybe solving in general, better how you do the configuration of a gateway might be something that would inform this more, but also just kind of solve a general problem. We're all solving different ways there may be.

A

There may be ways to standardize that I think that was a really good point in thought.

A

um I can't see so somebody else has their hand raised just start. You know interrupting me, but.

A

I think that's, I think the action item for this then is thank you all for the feedback on the thought. I think that action item is, I'm gonna, think about it and maybe bring up a discussion with something a little bit meatier like start or or even reach out to individuals and just kind of see how everybody's doing this and come back around to this with another discussion with more meat and potatoes to it. So.

A

Cool I'll do that and if nobody else has anything else on that topic, I'd like to jump to the next topic.

A

All right, so, the next thing that I got here is that I wanted to talk about is just I've had two people uh recently that are your in european time zones tell me that they want to come to this meeting, but it's too late, um and so I know we've talked about it before and I just wanted to like. What's that?

A

Is there anybody on this call? That would like an earlier like to do the alternating uh thing kind of like what we're doing for gamma. Anybody opposed to it. Trying.

C

To switch this to.

A

F

Oh, uh switching this to alternating means I'll only make every second one um yeah making it.

C

Seven hours.

F

Earlier puts it at one am my time um and when.

E

F

Time in your in your summer,.

C

F

C

C

So yeah so look.

F

I mean I'm only one guy so like it doesn't really look to some.

C

Extent, it doesn't make.

D

Sense to hold everything up.

F

For me, but yeah, that's why I'm.

B

I think I would also uh for a little bit of historical context. We for some period of time we did rotate between times that were friendly to europe and times were friendly to aipac.

B

I what ended up happening- and this was at least a year ago now I is that the people attending the uh europe friendly time zone uh meeting were a subset of the people attending the apac friendly time uh and we didn't miss anyone by switching to the apac friendly time. I'm not saying that would happen today.

B

That's just what happened historically, it was a lot of um it was challenging in the sense that I there were meetings that were significantly larger than other meetings. So, if you had a big proposal, you try and save it for the bigger apac meeting as an example, because more people would be there.

B

Another concern that I'd have is, if it were monday morning, pacific like if we just took this meeting time and bumped it up seven hours.

B

That would be asking people like myself in pacific time to come to a meeting at eight a.m on a monday morning, which may not be fun. For me, at least that is a tiny, tiny complaint compared to what everyone else is dealing with as far as time zones. So I can't you know whatever my personal preference here uh would be to uh wait and see a little bit longer.

B

How gamma does with this since they're really kind of pioneering this idea again, and I think they have had decent turnout for both, uh but maybe someone who's more involved. I don't know what keith I think you're on this call, john. I have you any any thoughts on your experience so far with alternating time zones.

E

uh Yeah, so this is actually part of the status up that I was going to um I've got on the agenda here, but we've seen a pretty pretty good attendance uh for both meetings uh as a cursory glance. I don't necessarily see uh folks that we consistently gain with the morning meeting, but um there there like there are some times where we have more people show up to the morning meeting. So um I guess it remains to be seen.

E

um We've still got around 12 to 15 people uh each occurrence, so that tells me that something is going well um so, and the cool and the thing with us is that we've got um main we've got leads across the entire continental united states when it comes to time zones, so uh we're swapping duties, which makes it a bit more sustainable. So I guess our my takeaway is to keep that going until we see reason not to. uh I know I I think john had to step away about mike. You have any thought.

C

Yeah, I think we don't have a lot of evidence in terms of like folks that are only able to make the the uh amir friendly, but it certainly.

A

Your dog every time.

C

Every time yeah uh but yeah as far as rotating hosts is I'm I'm working on hosting those ones so start off. The dock.

A

So what I I kind of missed what you were adding to the to the thing here mike sorry,.

G

C

I'm gonna just type.

F

C

F

C

F

That's probably a good idea.

C

A

Okay, actually yeah, if you just put your thoughts in there that way. Next time we come back to this, we can actually I'm trying to take notes today, because I realize there's some things we come back to every once in a while, and so having notes could be helpful, um I'm okay with waiting. I think the idea of just waiting a little bit longer to see what's going on with gamma gamma looks promising right. The meetings are working, but it's mostly in terms of it looks promising.

A

Attendance is actually happening, not necessarily we are gaining anyone, so maybe just take a little bit more time and keep an eye on that sounds like keith's already, specifically trying to kind of keep an eye on that and use that to inform a decision in the meantime for the couple of people. I know that are like uh want to come, but it's prohibitively late. I will uh I'll talk to them a little bit more and tell them that so.

B

Yeah, maybe if you could ask them to add something to the agenda, like maybe comment on this thread or or somewhere just so. We have an idea of the people that are just waiting for an earlier time to attend that that could help too.

A

Yeah sounds good.

E

You also gave me an idea, rob I'll put this as a uh I put as engine item in our next uh gamma uh emea meeting just to ask folks uh if they're enjoying the flexibility of the other time to maybe more directly gauge. So I should have an update in a couple weeks sounds.

A

Good all right, and on that note keith, do you want to uh do the what rest of your gamma status, update.

E

Sure um I made a slide deck just for folks who might not be able to uh who might not be here today as well as anybody who wants reference back to it. uh How do you want to do? Do you just want to continue sharing your screen? um No I'll uh I'll, stop and let.

A

You share your screen cool.

E

Do the screen sharing dancer uh you gotta, allow me to uh as hosts gotta. Allow me to screen share. You got it.

A

Does that work.

E

Yes, it does thanks so much and I'm not going to blow it up just because that always sucks for anybody trying to do the full screen thing, um but yeah.

E

So just a quick uh status update on gamma and where we are uh we, when we kind of started the gamma initiative, uh it was suggested that we do these kinds of recurrent status updates, so that folks, who I might not have the time to attend, gamer meetings or uh just people who are otherwise interested in gamma, can get a update every so often of what's going on. um This is kind of a high level overview slide of uh how things are going. We've had four meetings.

E

This is about a month uh since our original uh original meeting kickoff meeting on july 26th, uh we are as as mentioned all settings time slots uh different time zones that solid attendance uh between both time slots about 12 13 people. um The first meeting had around, I think, 32 attendees um when we started so that was really cool to see that kind of interest uh of the stuff that we're doing for service mentioned gateway api.

E

We started keeping track of organizations across different game meetings and uh I went through and did account of all the different organizations that are represented here. um There have been 14 uh different organizations, um I think, with uh nick's new orc change 15 uh once he starts to come to a good meeting, so we've got pretty wide uh wide interest across the community as well, which is very exciting.

E

uh We've got our first get merged. As of uh last week, we got uh get 1324 merch into get your api. Great yeah yeah get the claps going. um This is a high level account just talking about the goals, non goals and the places where we're explicitly cutting scope and deferring certain goals um for things and uh we'll get more into detail about that, uh and then just a quick kind of up next section uh that we're going to more detail about.

E

We've got two more cornerstone gaps that are in progress because I say cornerstone, because these are kind of the foundation of what a mesh needs whoops. What a mesh needs to be able to be. uh You know to express it in gateway api.

E

So where are we now? uh Can you does someone say something? Go ahead: okay, um so where are we now? uh The? What and the y have kind of have been clearly established? uh We were trying to be very intentional um about establishing what we're trying to accomplish and why we were solving those problems before digging too deep into implementation detail, um and this version 24 covers that um I'll I'll say I'll call up to pay special attention to this glossary.

E

uh We went through this on the last game of call, but we've got this really cool glossary here. That goes through a lot of terms that are going to be important for having mesh conversations, um and some of these might even be broadly applicable to uh gateway api uh there's, also a deferred goals section, um which are things we're explicitly not excluding, but we're going to revisit.

E

um Some might be disappointed to find their favorite use case here, uh but we're helping to iterate quickly and uh having cutting the scope is going to help us to get something out there. A lot sooner um and now, at this point we're ready to set our slides on the how uh we've got two gaps uh around next representation and then measurements binding that we're going to start really digging into the meat uh meet towards.

E

I don't even know if that's correct english, but whatever um in their next meetings, so uh we'll probably start with mesh representation first and then, once we decide how to represent a mesh within gateway api, we'll figure out how to start binding routes between services and back to the overall mesh construct, um as a as in general, we're hoping to bias towards action in these early days. A lot of conversations about um striking that balance, making sure we're not going to regret. um You know some of the precedents and actions that we uh take.

E

um But the kind of answer to that is, we hope to try to keep scope, small uh and iterate very quickly. uh We've got about two months from now until kukan, and we hope to uh you know, have these two gaps knocked out by then and have something that implementations can start implementing at least generally generally opportunities for alignment.

E

I think it's really important because of the nature of kind of these two work streams between ingress from the main api meeting in the service mesh uh with gamma, that we call out uh opportunities first to align and work together um and so calling a a few of those out here.

E

uh We're currently evaluating the need for a get or some other clarification around dns uh dns has popped up a lot in some of our recent conversation and uh the greater gateway api kind of the ingress use case is kind of being able to sidestep that it seems because uh the binding happens directly to serve to service resources or to other, like kubernetes resources that exist within std.

E

When you start dealing with host name, though, um in in reference and intercepting traffic for destinations like you do with the mesh you, you might not be able to get around some sort of clarification uh for for uh dns, but they're.

E

Not a gap is the right place for that, not sure yet, but uh we're going to be exploring that um as well in some of our uh gam meetings, um the next one- and I think we caught this out in the last uh last week's skate with api meeting as we're gonna- need a gap around ingram uh egress. Excuse me: um there's been a lot of questions and conversation around egress for get your api just for like the egress gateway use case, um external name, services and things of that nature um and gamma.

E

As a service mesh exercise use case uh group, uh we naturally have some some concerns around um or some questions around egress as well um and we're seeking a champion. I think that was the consensus around the gateway, api, uh ingress use case meeting and I think there's even commented on a discussion or a or an issue somewhere.

E

If you haven't really, if you happen to be really passionate around the egress use case, that's a pain point for you and your implementation. Then please we would love for somebody to come and to take this all the way and teach me some things, because uh it's a complicated problem and to do well and the third one is get 1282. that knicks kind of pioneering you're on backhand properties. um It's uh the provisional pr has been submitted um and it will likely impact our service attachment uh gap.

E

So perhaps we're binding to this spec and properties resource instead of an actual service, um because that's gonna depend on some of the implementation, some of the how that's not yet uh been been decided upon when it comes to the 1282 get so again. I would very much like to be a part of that conversation um love to try to work together around this, and then um one thing I'll call out here is that uh specifically a part of the gamma glossary for this gep?

E

Is this duality between a kubernetes service being a front end and a back end and I'll call this out because shane um on the uh on this gap? 1324 pr? You call this out as something that's been talked about before within gateway api, um and it might make sense to start with the backend properties get already in progress. Maybe it makes sense to to bring this over to the ingress use case with with uh gateway api.

E

I don't care about the name, but this concept of you know the kubernetes service resource serves two functions, one as a a front end that can be addressable uh in reference and the back end representing a set of endpoints uh or addressable hosts depending on your proxy terminology.

E

um So that's something I'll call out there for some folks who are really embedded within the ingress use case of gateway api to to see if this might be useful in some upstream uh conversations. uh So that's the end of kind of my status. Update for uh for everybody is uh I'll, go ahead and keep sharing my screen in case there's any questions or clarifications uh something for folks. I want to see for next status, update um or anything like that.

F

I'll just say: great update man, cool, slides, yeah.

A

That's great, thank you.

B

Yeah echo the same thing. I really appreciate uh how you all have taken gamma on and just make run with it, and this is great progress for just being a month in um yeah. Thanks for the update.

C

A

Problem I like.

C

A

You're, looking at all the pieces connecting to each other, the backend properties service front and back end, which is something that like I I want to help with at some point, because I have a use case for so good job kind of looking at the whole picture and taking it piece by piece.

E

Appreciate it um yeah, so I will like I said this. uh This slide deck is in within the meeting notes for uh for this meeting so you're watching this recording and you want to see this uh or add comments. It should be commentable to everybody, um so feel free to take it. However, uh we don't I wanna ask: do we want to figure out when the next uh status update should be uh establish a cadence?

E

Perhaps I'm I'm fine with it being ad hoc, um but I don't know if anybody has preference for more structure or not.

E

A

B

Sorry rob uh no sorry either way, but I would say uh you know every month seems like a fine cadence, uh but also, if you wanted to you know, if there's significant something that you want to bring and discuss in more detail, then we could do that too, but at least in these early days making sure it's you know once a month or so at least so we're not going too far in different directions uh seems reasonable.

A

Yeah, that's what I was gonna say I was gonna say once a month is a good goal and they don't need to be huge if they can't be huge right. I think this was a great update, but if, in a month's time we don't have an update this big, that's fine like.

E

Yep sounds good to me, um uh um even if uh you know the update's, not big, I do want to try to make sure. We call me whoever's making these calls out the opportunities for alignment here. So I think that's one of the big benefits of having this kind of status update is: how can we bring these workstreams together to make sure that we're moving in the same direction? So all right appreciate the opportunity to share I'll turn it back over to you, shane.

A

All right, thank you, keith back to it good update, rob you want to talk about versioning updates.

B

Yeah, okay, so I've got a few things going on here. I hopefully got them in the right order. They're, all pretty related one is trying to document our versioning plan for future releases of the api.

B

I I don't think we've documented that anywhere right now I have a rough proposal of what I would write in this issue uh and I've assigned it to myself so I'll work on a pr that does this. But the high level thing here here is maybe worth discussing a little bit more, and that is that uh you know talking to some people in api machinery around how they would recommend we do this.

B

uh The the the overwhelming reaction I got was to avoid supporting alpha two longer than you have to uh you know I I'm fine with supporting beta one for a long time, maybe forever, uh but alpha 2. It doesn't seem like there's any reason we want users to keep on using v1 alpha 2 anywhere. Our docs have all said: v1 beta 1 for month, plus now um so with that.

C

B

The recommendation would be to set the new version uh v1 beta 1, as storage in o60, and stop serving the old api version, which is v1 alpha 2., so the old api version would still exist in the crd.

B

So that means that if anyone has resources that haven't updated, yet you know whatever they're still supported, but it means that if anyone tries to write or like to update or create a resource using v1 alpha 2 of api, it will get rejected using the v060 crds.

B

So old resources still work uh controllers that are reading from v1 beta 1 still understand, v1 alpha 2 resources, and it's all good. It's just starting an o60.

B

If you use those crds and you try and write or update a resource with v1 alpha 2, it's not going to work. That's the proposal for some context. This is something that we are planning to get ahead of with gk. With the uh you know, we we have no plans of of installing alpha api versions in in our managed product.

B

So if you're using gateway api you're going to see this experience a little bit sooner on gke of just you only get beta as the exposed api um there's, also a reference to the idea of cube storage version, migrator uh we're going to need to work on a something that makes this a little easier for users to understand.

B

But in you know, if we ever remove v1 alpha 2 from this crd completely, instead of just not serving it, but just also, not including it at all. We'll need to make sure that users that created something with v1 alpha 2 have done something to update their resources. When v1 beta 1 was a storage version, uh storage version migrator. Does that automatically just bumps a resource and says hey you're, not using the right storage version? It just bumps you up automatically.

B

There may be a simpler tool to use whatever, but we'll need something like this in our documentation before we remove v1 alpha 2 completely.

C

I took a look at using cube storage version migrator as part of our recent upgrade for to get users from v1 alpha 2 to the comparable view on data on resources, and I think the most unexpected thing for me was to just automatically upgraded everything in the cluster.

B

Yeah, which I I think we probably want a more type of feature yeah, you know I, you would hope, that's a low-risk thing, but I think we probably want a more targeted thing that just targets gateway resources because yeah, um that's uh that's something that I'm intending to work on and the related pr's. I guess we can go through. Those now are really the same thing right. Well, actually, uh maybe let's go to the next pr first, because that lines up more closely with the documentation.

B

This makes that change. uh My goal is to get it in before o60. I think it's ready to go now and I don't see any reason to hold off on it, but it ended up being a cascading change and that what turned into you know it started with I'm going to change three lines in the types files and then I realized well hold on a second.

B

I need to change the examples because we don't need v1, alpha 2 examples anymore uh in many cases, and then I need to update the docs to make sure they're, including the related things. I tried to cover the very specific changes that I'm doing in this pr.

B

um It is massive, but most of it is just one line: changes in a variety of files, um yeah and the removal of a bunch of v1 alpha 2 examples that no longer useful.

B

That's the big change that kind of implements what I was suggesting documenting uh before I go any further any questions or concerns about this kind of change.

B

I know it's a big one, so I I don't want to rush it or anything. Oh he's awesome.

B

Okay, cool um yeah great, so then the next one uh we'll go to the supported versions, one that yeah. Thank you. uh This is more of just a if you are launching, I think many or most of us on this call are planning on launching or building a product around this api, and it would be very helpful to have some kind of guarantees that the api will be stable.

B

So this tries to make some commitments in that in that vein, so one uh right now, our docs just say well right now, just because we support up to 118 backwards, compat 116, if you're not using that protocol. You know a variety of things like that, but we don't have any commitment that that will forever be the case.

B

So first step is: let's say: we're going to support at least the most recent five kubernetes minor versions. I think the most relevant thing that could be frustrating with this is there's a new thing called cel coming out for crds, which allows us to represent a lot of the validation we have in a web hook in the crd itself, and by saying this, we're saying we're going to wait. Basically, five kubernetes minor versions before we make that transition to ceo instead of webhook validation. Yes,.

F

uh Yeah, it's the better part of two years, just to be clear.

B

Yeah, it's it's! uh It's a long time so, five years times, four months, yeah at least.

A

Yeah, two years: that's like a yeah, that's a good reminder. Do we have like an actual like issue reminder somewhere about that? I.

B

Don't think so I think cl is just getting to beta in this in 125, so there's not really any usage of it yet, uh but you know a year from now we may be regretting it and saying: oh man, I wish we could use ceo today.

F

I would, but personally I would bias us towards not not having five. I I think at the maximum we should do is the same as the upstream kubernetes support ie3 one year, um the um because um yeah, just just because, like I.

A

Could talk, I could talk for a whole hour about.

F

How long it took us to get to that um in the lps working group? um But you know, I think, that's that's about the best that we can do, um because otherwise we're going to be stuck, and it's going to be it's hard enough to make changes already and we're going to need to push changes to some stuff. That takes a long time to change. And if we also have to not drop that until later. It's going to take us years and years and years to be able to make solar changes, and I think.

C

G

Is going to be a very big deal.

F

It's going to make the web hook a lot easier. um Yeah. Yes, that's I think that's it. uh Makaya has put a link in the chat um yeah, so.

C

F

That uh yeah being able to being able to do the um being able to use the um being able to use that will be really helpful. We will be able to drop a lot of the stuff, that's in the admission book as soon as we start using that I mean there's, there's some stuff to be checked about like hey. If you put those into the into the cid, you just see the cid specs. What happens if you apply it to an older cluster and stuff like that?

F

um If api machinery has done the usual thing like, hopefully you should, uh you should be able to put them in there and say if you're running a version older than this run, the initial.

F

But if that's not the case, then yeah we're not going to be able to use.

C

F

The ceo feature is in you know, ga in all the supported versions of the kubernetes that we support and the more the more supportive versions we do. The longer that will be.

A

Yeah, I should get better to comment on the pr. Then I think that sounds wise.

B

I have a bias. You know this is a personal bias to provide longer support uh than than three for the reason that you know, users are coming to crds, with the expectation that they can install these crds in any kubernetes cluster.

B

And if you look at what the majority of kubernetes clusters running today are, they are likely older than three versions old. If you say 125 is out now we're about to be out 125 24 23, I don't think the majority of kubernetes users are on 123 today uh and I I hate to artificially limit uh where this can be deployed, just because we want to use the latest thing, uh and I know that there has to be a balance here and I don't want to push too far the other direction.

B

I just want to be clear that, right now we have an api that we say if you're running a kubernetes cluster, our api works uh and what we would say if we say the latest three kubernetes versions would be as maybe half of kubernetes clusters are supported by our api, which is a change and may not be expected with the crd.

B

um I don't know: keith you've got a hand.

E

Yeah, um I guess my putting this another way.

E

I don't know how I feel about crds inc because you're not encouraging usage of out of support clusters, but you're not discouraging the use of out of support clusters, and I I get what you're saying, but it becomes tricky because that's a switch for or that's a knob for users to to have them adopt to a safer, more secure version of kubernetes, and I know my my little experience makes me feel like we should, as a community, try to use those knobs where appropriate.

E

um That's trying to say that.

F

Yeah, I think to add to that this is only an issue because we are delivering this with crds. If we were building this entry, then there would be no question right like this would have to be only three versions, because that's how entry works? We are an upstream api.

F

I you know the fact that we're being delivered with crds feels like uh implementation detail to.

G

F

And so that's why I'm pretty strongly on my case on the side of three only I.

D

F

That it will, I know that so I know that you're 100 right rob that crds do have a slightly different support expectation. But equally you know. I think we just need to manage the expectation like in our release, notes and talk about it in these meetings and talk about it in calls and make sure that it's clearly communicated um that hey. You know these. These might work in order clusters, but, like we'd, like you to try and encoura we're encouraging you.

G

F

Try and move move a bit faster than that, um so that um so that we can use this newer stuff that we need to be able to provide you with a better. You know with a better experience at the end of the day, not having to run on admission web hook. I think, is a very big deal and would be really nice.

B

I yeah I get that I maybe a question for other maintainers on the call, uh including eunic, would be you know for your implementation. What version range do you support with the latest release? What kubernetes version range? Do you support trailing three or is it a wider range than that.

F

For contour we've, uh we I'm not a maintainer there anymore, but for con contour supports uh three um and has for a long time, because because of me, so I I was like not three. Only in the story: uh it'll work, it'll work back down to like 116 or something like that. But but the explicit support is for um is for only three because of this sort.

C

F

C

F

You don't want to have to wait. You know in the case of english. V1 is the classic one here right, like you, don't you know um the transition from v1 to beta1 to v1 took forever, and then it still screwed people, because people weren't looking for it and didn't know that sort of thing could happen.

F

um You know the removal of v1b to one really surprised a lot of people and people were shocked, um and so you, I think that the way to stop that from happening is to ensure that you're that you're telling people you can't just keep your 1 12 kubernetes crusted around forever. Right, like you, have to keep upgrading it. Just like any other software, and I know and like I know, rob that one of the reasons is that all the big cloud providers don't usually stick within the within a supported window.

F

Right like it's, it's very uncommon.

G

That all the big.

F

Cloud providers have their managed, kubernetes clusters within that supported window like and so yeah like it that that kind of sucks, too um and so like, but from up like wearing my upstream hat, it's better for upstream for us to keep it to the to the three things. I acknowledge that that the practical problems of you know uh managed platforms needing to qualify new versions of kubernetes.

F

You know and it taking a very long time, and so it can it's very difficult to keep to the managed platforms to keep their managed versions within the within the supported uh kubernetes versions. But, like I mean I'm going to be facetious about it, but like that seems like a you problem like as far as wearing my upstream.

D

F

I was wearing my upstream hat right. Like you know, it's like hey, that's uh um yeah, that's like that's like we got.

G

To do what's best for the upstream and then.

F

Everyone else needs to feed themselves out right, like yeah and like I, I don't want to change the upstream stuff just to like yeah. I don't want to acknowledge that too much or anything I mean like you know it's like hey. I know that. There's a lot of work there. I know there's practical reasons, but I think it's better for the prep for the official support position to be three same as the same as the rest of it um and for us to manage the expectation issues separately. Personally, sorry, I've talked about.

B

I no that's, that's very helpful and I just I want to share- and I I think you alluded to it earlier with ingress right. I I am very familiar of the pain we caused. I caused with ingress, uh and I know we had multiple implementers come to us and say you know this really hurts uh the idea that there is really a by the way. Sorry yeah there that there were three support versions that had both b1 and v1 beta1, and if you wanted to support anything outside of that range you needed to either.

B

You know, provide parallel support or you needed to have different. You know different versions that you released, and you know big implementations ingress engine x and I several others that I can't remember the name of right now. I had issues with this where they supported a bigger range of versions than that three and it was a really rough transition for the community as a whole. As a result, there's no perfect number here, but we do have prior experience that three can be painful uh and I you know we.

B

We can continue this discussion on the pr, but I just I wanna avoid that kind of painful transition again and especially with crds, where the thing is we just don't want to maintain a web hook for say four more months or whatever it is right.

B

You know, maybe that pain is acceptable if it means a better transition for the rest of the community. I I'm not sure um shane. I think you you're up.

A

Part of my thing is we're at time.

A

I would like to I mean the action seems pretty obvious to have the conversation uh in the issue, or rather in the pr I'm leaning next way. To be honest with you, um I mean it's to me. It seems like the what we're really we'll always try to support. Far back like we at kong, we can get you running on some pretty old versions of kubernetes.

A

We don't want to be beholden to that and that's kind of the key here right. That's at the end of the day, we'll put some effort, but we don't want to be beholden to an inordinate amount of effort to try to keep you on an old version. At some point you got to give and as the end user you got to be like. I just need to update my cluster.

A

It's five versions behind uh you know, so I kind of lean that way, but um I'll make I'll comment on the uh on the thing, because, honestly, I kind of thumbs it up at first and then now after hearing this, I'm like yeah, no, I kind of agree with a shorter span, because five is a lot so.

F

I think that, as key said, best of it is a magical phrase. I think that that putting some stuff in there to say, hey three three versions being the supported version doesn't mean that we're gonna just that we're gonna, always hard drop things. We will make every effort to make sure that that a longer support that the.

G

Api works on longer.

F

Versions like we'll do our best efforts to make sure that the api works on longer versions than that, but we're only going to guarantee three it like is. I think the thing that we need to say like because.

C

F

C

I think you're right rob that.

F

That the um the amount of effort in maintaining the extra web hook as well is probably not that much. What I'm worried about is, if there's no easy way that your crds can have ceo enabled and not have it enabled then you're kind of um you know, then you're kind of screwed right, like you know, if you, if you can't have the ceo stuff in the crds and have them apply to older clusters, then that means that we then have to write like.

F

Not only do we have to have standard and experimental, we need to have the you know the ceo talk of the crds plus the non-ceo fork of the crds and and that that we will then have to maintain for a period of at least a year, and that will suck even more than you know, having to manage the transition from people.

B

F

So that's one yeah.

B

I'll follow up on the on the issue. I I'm not convinced yet, but I want I I understand, uh and I and I also understand I'm outnumbered right now, so uh I'll I'll try to make my best case but uh we'll defer to the community for sure.

F

A

All right: well, um we don't have any time for anything else, but all we had left on the agenda was triage, so um please just make sure to check out uh the recent prs and issues and stuff like that. If you have time, if there's anything that piques your interest and get some feedback on them, but we'll check in again on them on monday next monday, thanks everybody for coming and have a good one thanks.