Kubernetes API Machinery Special Interest Group, 1 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Gateway API Bi-Weekly Meeting (APAC Friendly Time) for 20210401

Description

Service APIs Bi-Weekly Meeting (APAC Friendly Time) for 20210401

A

We are recording. This is gateway api meeting for march 31st 2021.. I I wanted to spend. I know we mentioned this uh a while ago, a few weeks ago, that it would be good just to get some feedback on what we were thinking about presenting at kubecon, and uh we need to have kubecon this presentation recorded basically in the next week, and uh so based on that I just wanted to.

A

This is not going to be a formal presentation or anything like that, but I wanted to take maybe 15 or 20 minutes at the start of this meeting and just run through uh what we've been thinking about, presenting and hopefully get some feedback as far as areas that we're missing out on things that maybe we don't need to highlight as much or should highlight more or anything else you can think of so yeah. This should be visible to everyone. If it's not, let me know- and also hopefully, my screen sharing is actually working.

B

um It is working.

A

Wow, it's amazing, okay, cool, so yeah. This is a really simple. I just structured the presentation with uh really the first half going through an introduction to the api where we're talking about the goals, the resource model and some of the key concepts around the api, and then we want to spend a fairly significant chunk of this presentation going through demos. So because, at least for me, that's how I learn. I you know.

A

I really like to see it in action to understand the different components, and I think we have good demos and I've kind of scoped out- and I spent yesterday running through and before running through uh different implementations, to make sure that the demos I wanted to do were possible.

A

uh So right now I have scoped out basic demos using traffic, istio, contour and gke and we'll get into those a bit later and there's there's a lot of different content. We could choose for demos and I'm not sure I chose the best or most compelling content.

A

So when we get there again interested if we should be highlighting some other functionality or whatever it is um so for history, we go through and talk about the the origins of is of ingress in general, and you know what it prioritized and how simple it was and how widely used it was, and also the idea of how service started simple and it became larger and larger and started to uh include configuration for you know all kinds of advanced concepts that just kind of got tacked on, and so the combination of those things led to the question.

A

Can we build a new api that can represent those low balancing concepts in a better way and so go back to the history kubecon san diego? I know many of you were there uh and you know just trying to answer. What's in scope, what problems are we trying to solve and do we need new apis to solve them?

A

I had this somewhat classic picture of one of the first meetings uh meetings or that brainstorming sessions- I don't know I know we had several at that kubecon uh and they evolved from there. I think, shortly after that kubecon we started setting up weekly meetings and yeah here we are so uh and feel free to interrupt me at any point. uh Harry you've also been involved in uh just you know, developing these slides.

A

uh If there's anything you want to add- or you know give context on why things are the way they are or what what you think should change uh yeah all yours.

B

Go ahead, rob yeah, you don't want it! You're too generous. Okay,.

A

Cool all right, uh so the other thing is, uh and a lot of this concept is uh taken from docs, where uh we already outlined some of the goals of our api. We want it to be role oriented, we want it to be portable, we want it to be expressive and we want it to be extensible. I know we have probably more goals than this, but I think this is kind of a good high level way to describe what we're trying to do here.

A

uh Roll all of these feel like pretty core ideas that we're trying to solve for uh so the next few slides I go through. We go through the different roles, we're considering as we designed this. uh I kind of I had this in a couple. Different places and harry suggested uh harry improved this a lot uh we'll say, uh but I I think uh we want to describe the idea of our compatibil.

A

Yes, our our conformance levels at this stage, um so I think that's really a key point of why we're trying to be portable and how we're trying to do it. uh Then the expressiveness just covering some of the features that this enables. I know there's a lot more uh and extensibility.

A

Obviously we want this to be an api that can be extended broadly across a wide.

A

You know across a wide set of use cases, and we know that we can't have everything in the core api, so we want to provide lots of extension points um if there are other extension points we should mention here or if some of them are, you know whatever definitely add a comment or let us know.

A

Okay, so uh moving on from here, we've got a resource model.

A

This is, uh I think, a not terrible way of explaining how things work that a gateway class. I can reference prams and this kind of all animates in so you start with the gateway class, then params and then gate. Each gateway class can be referenced by multiple gateways. Each gateway can select multiple routes.

A

Each route can have forward traffic to multiple services and a back-end policy can apply to multiple services.

A

So that's like a really high-level overview of the resources involved, and then I uh take this. We take this idea of with that resource model. Let's overlay the rules and show exactly how this relationship works.

B

Rob how do you think about what do you think about adding like a tcp router, another protocol in there? uh We we talk about protocols, but we don't mention them in multi different places. I was thinking if we can reinforce that. This is not http anymore, which isn't a big complaint. We've had hold up so.

A

Yeah you're, that's that's a really good point. uh One thing that you'll see throughout this presentation is: uh we we don't cover tcp. We we cover. We, we only cover http route in the current presentation, uh so yeah that that's a good point. Let me.

B

I can I can do that.

A

B

A

I forget how to add comments on slides but yeah. That's a good idea.

A

Cool and then uh I just uh take it- would just take some time and run through the decor. You know resource type, so we've got gateway, class gateway routes, etc.

A

And then some key concepts and the key concepts. I had a lot of ideas for what could potentially belong in here, but I'm trying to avoid putting anything in this section that we're also going to be demoing that we can cover with effectively with a demo so because I think a demo is a better way than slides wherever possible, but some some concepts are a bit more complex or would require more than one demo to properly show it so and we we do have pretty limited time here uh so gateway route.

A

Binding, I think, is a key detail, uh but I don't I'm again trying to trying to find a right, the right balance here, because we don't want to go too far into details for any one feature, because, although I think the resource model as a whole is probably not going to change much the the specific details of say route binding- I imagine that's gonna change, I I don't know- maybe maybe it'll just be tweaked, maybe we'll overhaul it more broadly.

A

I I don't know, uh but I I'm hesitant to go too far into the details here. um There are some slides where we could you know like. I think this is a good way to describe the current relationship and say: okay, so in this case kind matches namespace matches labels match, but the route is preventing it because it only allows you know from the same namespace and so that that is the thing that's preventing this from working.

A

I think this is a really good description, but I'm I'm not 100 sure I'm really interested in feedback here, because it. This is a good way to describe this works, and this doesn't, but it could also make this presentation uh less relevant, say six months from now. If we change anything- and I realize a lot of things in this presentation are pretty specific to now.

A

But this is one area where it feels like it's likely to change at least somewhat.

B

So yeah, so so I was sort of I don't know. If I did this first and then saw all the activity on the route binding selector updates, so I think I did it in the other order, but uh I completely agreed up. So I think maybe here we can gloss over the details since this is like you know, this is probably going to change right. So, like I saw the slides, I was like okay. How do I explain this?

B

I fill that in so let's maybe use the demo, because I think in the demo there is one place where you do explain how transfer.

C

B

Works so we could use that and instead, you, you know, focus on another aspect. You know look at.

D

B

Protocols or dns, or something like that.

D

Yeah that makes sense yeah. Is there like a use case, we're trying to cover with this, because those should always match, regardless of how we implement the api, you would mean like use cases. I have an operator who wants to do this and a user who wants to do this, and this is how you would restrict that, or I don't know what I mean like thinking about high level of use case kind of things versus the actual api. But maybe this helps as well. I don't know.

A

Yeah, that's that's a good point. We we really could be focusing more on specific user stories and how and how they could be achieved uh with this kind of api, um and I think this really right now, we've covered just the things that you can do without really going through many user stories, we'll get a little bit closer to that in the demos uh but yeah. That's that's a good point.

A

We um could go into that a little bit more in the in this section as well.

A

um Yeah good idea, the other section- and I I have nothing nothing here at all, but I was just trying to think of what other key concepts do we have that we can't really cover well in a demo and tls came to mind we, of course we could, but it tls feels like something: that's a bit more difficult to demo easily, especially because there's you know a few different it it's.

A

It can be relatively complicated to demo the idea of say a route override, and I don't think we need to go into lots of detail here, but just saying, oh, by the way you can specify tls here or here- and you know uh so, given that I think this tls may belong better in this kind of section in this kind of format than a demo. But uh this is the one section we haven't gotten to yet.

A

um So with that, uh I scoped out we scoped out a few demos here, the implementations that each each demo uses are relatively random. I I think I started going through them in like an alphabetical order. I don't I don't really know whatever. uh If you know it that this may have to change. I know some implementations uh in my experience play nicer with other implementations than some. You know like that. You know my. My testing, with these different implementations shows that the current state uh different things do better for different demos.

A

So these specific implementations that are I'm we're using to demo make change slightly, but the content, I think, will stay similar unless but again, very open feedback on this um that the key goal here was to demo with a wide variety of implementations. I didn't want to reuse the same implementation for a different demo kind of thing I wanted to show this is a community project, a community api, and you know you can use these same apis with contour or istio or whoever it happens to be so.

A

The first demo I wanted to kind of just let's say, create a very simple use case. I wanted to show the yaml that was involved in your most simple configuration. So let's say we start with contour. It targets a route which has a service deployment, etc, really straightforward and then show how that all works and then take a step back and say.

A

Oh by the way, you could also spin up an istio gateway and target that same full stack and it will work the same way and show requests going through istio or contour and getting the same response. It's a really simple demo, but I think it it helps, show a couple concepts, one being the the loose coupling between route and gateway, but two the the idea that uh these same apis work for the you know for different implementations, uh which I think is is a key goal here.

A

um But again uh the specific implementations here can change and if there's a you know for any of these demos, if you think there's something that's more valuable that we should highlight. uh Let me know, but this was just I wanted to have one demo that started very simple and showed end-to-end support kind of thing, so yeah this uh this walks through the approximate process for this demo, uh but yeah open feedback on these.

A

uh The next one I wanted to. I think you know header matching, is kind of a unique concept or a new concept as far as core kubernetes apis go, so that felt like one that was useful and interesting to highlight here. uh So in this case uh we do some. We have a single route that and two services a stable service and a canary service and we're using header matching to target the canary service.

A

So in this example, we have a simple gateway, it's using the traffic gateway class, uh and you can see that uh yeah. This also allows us to highlight the route selection process and then shows us how header matching works and how path matching works.

A

So I I think this is really again a relatively simple thing to demo um and again the implementation. That's actually doing this could change. I don't think, that's critical. I just think it's important to have different implementations covering these.

A

Then uh the last demo, I think this one is relatively unique to gk, but I think anyone you know these are using entirely open source apis and I think it's cool to show the intersection between what sig multi-cluster has been doing and sig network has been doing and kind of the new apis on both fronts and how they can kind of merge together.

A

So this is a more complicated demo, but we have three clusters. In this case we have a config cluster that includes a gateway and a route and then two clusters that that route targets I the cluster one, has a service and deployment and when you create a service export in cluster, one that results in a service being imported in the config cluster and the same in cluster two. So the http route is targeting the service import in the both service imports in the config cluster and splitting traffic between those clusters.

A

So this is our multi-cluster gateway. You can see the gateway class, that's being used here as a gke one. In this case, uh and then you can see, our route is again showing the selection, but then our route in this case is forwarding to services representing two different clusters and weighting traffic evenly, so you've got fifty percent. I I probably shouldn't use 50 here, because that makes it look like percent, which it isn't um anyway so store and cluster one and store in cluster two is what this is targeting.

A

There's many many different ways to do this, uh but this is one way that works using the existing apis.

A

And so again, just throwing a screen recording here of actually applying that yaml and showing the traffic getting split across I'm using, uh who am I pods, which show you kind of the cluster? Each request is going to pod name and many other things and then the other concept I wanted to highlight here, and this can really go in anywhere.

A

But I think this is also an interesting thing. So, instead of what we had before, which was splitting traffic between two clusters, you might want to uh you know, send all traffic to cluster one and mirror it to cluster two, and so you can do that with request, mirroring so similar concept and again it just kind of works and another screen recording and that's the extent of the demos I currently have thought of. But there's a lot more uh and I'm trying to find you know the appropriate amount of time here.

A

uh We have probably between 25 and 30 ish minutes to try and cover as much as we can so any additional demo. We add we have to cut something else, I'm I'm thinking unless we just talked really fast, uh and so I you know some things, uh header modification, that's another another new concept.

A

uh The idea of you know selecting routes across namespaces, I think, is really cool, advanced tls, config, custom route filters uh and then really, like I think harry had mentioned earlier. Anything to do with l4 would be great to demo or show or talk about in more ways than we currently have.

A

um So I want to at least mention these are other new things uh that are included here. But if any of these feel like things, we should try and you know, push up into a demo uh yeah. We should. We should talk about it, but I've. I've spent a lot of time talking, there's a few more slides left, but I think the demo content is something I'd really like to get some feedback on uh so of the of the demo content. I've shown. Does that seem reasonable? Are there other things we should be highlighting?

A

Instead, um I don't know open it up.

C

I think it's pretty impressive. uh The demo list that you provided there especially doing the multi-cluster.

A

Cool okay, yeah! Well, I will yeah. No, I I'm hoping that's a good way to kind of cap it off, and you know anytime, you get to demo two. uh You know new sets of apis to get working together. It kind of shows, you know how the kubernetes community uh is, is a pretty cool thing.

C

Yeah I like that you're showing the different implementations and also starting off with a very simple use case. uh Just you know, important to crawl walk, run, yeah, yeah, exactly.

A

A

Well, if you, if, if anyone thinks of anything here, any changes, you know we, you know we're gonna probably have to start recording this in a couple days, so we're not far out, but if anyone has ideas for changing things, adding new things, whatever this we get, this presentation should be visible to anyone comment away whatever yeah and- and thank you also, I I know I've reached out to some of you as I've had questions as a as I'm trying to you know, demo, your specific implementation and uh gotten a lot of help, so thank you to everyone.

A

Who's helped along the way. uh So roadmap- uh I don't know I I don't know if logos are the best thing here, but this is definitely the most visual way to represent this, and this is really just showing. These are the organizations that are either already supporting or working on supporting these apis, and we expect there will be lots more in the months to come.

A

um Hopefully, it's okay. I know everyone has like their own little brand guidelines. So hopefully, you know something like as small as this isn't too big of a of an issue, um and I don't think we need to you know because we're recording this a month before it's actually going to be presented.

A

We can't really get into specifics of what is actually live or not live, because we could be wrong by the time it actually rolls out. So I think something generic of you know these are all being worked on and yeah, and uh this time this road map is probably a thing where we'll get most into trouble uh for sure I think I'm I'm way too optimistic here.

A

I I like to I like to set, you know, ambitious goals, but uh I think especially this one, I'm going to regret um we'll see, but anything else that you think we should include on a road map or things that we should just not talk about. uh Welcome here.

A

And yeah, that's it! That's uh that's the extent of this, uh this presentation, um yeah. If you have thoughts on things we should include or not uh let me know, but I know we've we've gone over the time. I said we're gonna take for this, so we should get into actual api work um thanks again to everyone who helped along the way and uh yeah. Hopefully we can get uh get the last little bits of this together and to get it recorded soon.

A

B

Is great stuff, this is mostly uh mostly all of it is rob and not me. uh I've been out for a little while due to covet so but yeah. This is great. I think rob you're going to do pretty good job with the demos right like just talking about an api, but then you know actually showing that people are serious about it and they are already implementing it. I think that that gets the message across that you know this is happening and it's a big deal.

C

Yeah thanks a lot rob for it's obviously put a lot of time and effort into this and appreciate you doing that and highlighting the implementations as part of the presentation.

D

Yeah I'll do to that thanks thanks for working, I think it's good. It looks good. Another thing I can think of might be helpful is, and it may not be fitting for this presentation, but I know folks ask a lot about why house is different than ingress. That's a whole different bucket of worms but um like. Why would I use this over ingress or how does this evolve, or how does this you know differ from that, but that could be a whole different talk in different places. Yeah.

A

No, I that's that's a really good question uh and we, you know kind of cover it as we see these problems in ingress and these problems in service and here's how we think we can help with this new set of apis, but we never really come back to that. Like you know, we talk about it a little bit in history, but we never come back and say so.

A

After all that you know, ingress is going to exist forever for a very long time when it. How do you decide so that that's a really good point? uh That's probably worth at least giving a high level overview of in the talk. If we don't it's for sure going to be one of the first questions, that's asked after the talk, uh so yeah really good, find uh yeah I'll make a note of that.

A

Yeah so, and and then I and then yeah thanks for thanks to everyone for the feedback here, you know it's it's pretty, I feel lucky to get to. I feel like I'm standing on or we're standing on. You know shoulders of giants here. You know I get to give a presentation about all this work that so many people have done and just kind of you know showcasing. You know sometimes at kubecon I've I've given presentations that are work.

A

You know covering work that I've mostly done and that's okay, but like this there's just so much that the scope is so broad and there's so many different people working on it, uh that the the demos become so much more cool because, well, you know, there's so much more, that's possible! uh So yeah! I I'm just appreciative to the community too, for enabling all this really cool stuff um yeah. It makes me excited about uh where we're headed- and this is only v1 alpha. One.

C

It's nice. We have things.

A

To demo, yes, exactly yeah, thanks to yeah I I know uh it was when we made this uh cfp back in the day it wasn't. We weren't sure we hoped that we'd have at least a couple implementations ready to demo and yeah we've well exceeded that so great um yeah. Let me keep on running here. I wanted to again remember that the 0.3 uh milestone we've got a few things that merged in in the past uh past week.

A

We've covered just about everything in here in the last meeting, so I don't want to bring anything back up, but just to highlight that, hopefully, in less than a month, we'll have sufficient content to release an 0.3.0 release and again at least my vision for this release would be kind of our our last major additions to v1 alpha 1 before we switched gears entirely and started moving to v1 alpha two and started taking on kind of those bigger, bigger changes that would potentially be breaking, such as the changes to route gateway selection, and I know we have a variety in the v1 alpha 2 milestone, but I kind of wanted to separate these out and do additive.

A

Non-Breaking changes first, just so that we could make that v1 alpha 2 release a little bit smaller and more manageable, because we'll already have at least some breaking changes in there.

A

So if we can do as much as we possibly can before that, that seems like it might be a bit more manageable, so I don't think, there's anything particularly new in here other than that we've closed. You know a few things out, um but if anyone has time capacity to take on some of these things, uh that would be great, and- and I also want to re reiterate here- that uh not everything in this milestone needs to get in for the for the release to happen.

A

It's just these are things that could be relevant and could make it and probably sometimes soon-ish.

A

We should add some kind of prior prioritization here as far as what we absolutely need to get into this milestone, but at least right now what I'm seeing is almost everything is, uh you know, additive and not critical, just nice to have um yeah.

A

Cool well, let me uh move on over to issue triage because there's been a lot. You know a fair amount of uh good discussion, uh both in issues and pr's. I want to move over to pr's, because I think we we got a pretty significant pr from you, damien and uh since you're on the call. Maybe you can give a little bit of background on what you're thinking here.

C

Yes, as part of the contour implementation, there is kind of two controller components, and so you know we discussed within the contour community. Oh you know. Maybe we should create a package, the validation package that can be used. um You know, since the uh the web hook uh is not implemented, then, um as part of that discussion was like. uh Well, maybe it actually makes sense to have this upstream um in gateway api, um and the thinking was, is you know again the web hook?

C

uh Implementation is is in the works, um and this is a validation package similar to, as the description shows, the networking, api's validation package. So if implementations like the web hook could utilize this package, along with implementations like contour and others, that don't use the web hook and again it's it's modeled off the networking, apis validation package and right now, by no means is it comprehensive right. I just wanted to kind of put something together and get feedback from the community, and I think right now.

C

It's only doing the the hosting validation of the gateway list center.

C

Now, with the intent, if we move forward with this, that myself and others contribute to expand this validation package, and then I believe I linked the the web hook.

C

Implementation pr and I've reached out to to chris to kind of get his input as well of of having him either update that pr or if we submit a follow-on pr where that implementation would leverage this package.

A

Yeah, this is really cool. I I wanted to have a bit more discussion around this when I first thought about well, when I first heard of the idea of a validation package here, uh one of my first questions was, it seems like this is trying to provide information that would be used both for a validating web hook and also to populate status. Is that accurate.

C

Yes, so um any my thinking is is that an implementation you know would make the call to validate a gateway in this example and um and then yeah use the results of that as as part of uh gateway status.

C

um If they're you know, if there were any errors and be able to use a status message to to pull in any of these errors to provide more context within status.

A

Okay, yeah, so that yeah, that's where I was trying to it, seems like it would be difficult to.

A

You know, return an error that is suitably you know suitable for both a web hook and- and I think this is possible- I'm just I I'm imagining that you want, in some cases at least the same kind of validation, to apply to either uh so some like either the web hook doesn't exist in a cluster or I don't know what, and so you want something that can also be meaningful as far as status.

A

But you know if, if you run some specific validation, how do you know what status to set if it fails and and where that status should be set? And maybe maybe I'm thinking you know too specifically about this, but uh it it feels like it could be really cool. If we could do this in a generic enough way that you could have both here, but I'm not as sure about the specifics of how we could accomplish that.

A

I guess is what I'm saying and if we- and I guess to take that a step further, if we're not completely sure how implementations will use this yet I guess it would just be helpful for me to see an example of an implementation using this kind of library to populate status as well as the web hook. Hooking into this, and using this kind of library to you, know, populate responses.

C

All right, I could um kind of dig into how an implementation as well as a web hook, could utilize this.

A

To for status and this- and this is to be clear like I think this is a really cool idea- I just want to make sure that before we merge this thing, we have like a clear idea of how it will be used and that the you know interfaces we have in here. The patterns we have in here uh work well for both, um but I I'm a huge fan of having this kind of thing.

A

Any other feedback on this.

A

A

All right cool, I will leave it at that uh thanks. uh I know. There's already been some some people that have reviewed this, but yeah that this is exciting. I'm glad to see some work on validation here. um Let me jump back over to issues. I know I'm kind of going back and forth, but I want to at least cover the things that came in in the past week that we have missed, or that maybe haven't had enough discussion.

A

I think a number of us commented on this one already. I got a good question about, so if weight is zero, what is the standard response here and the you know? Should we you know if that's the only back end what happens, and I think the uh conclusion we came to here is that uh 503 is probably the standard it. It's not clear that we need to make that uh something that is conformant uh yeah so should, instead of must is, is there?

A

Is there really a significant difference between should and must in terms of conformance.

B

uh I mean we only conform for must right.

C

B

Should or I'm not sure or maybe it's like conformance test is allowed to feed, we don't have a framework, so I'm not sure how we talk about it.

A

A

But I yeah whatever our vocabulary is. I think this is something that should be a recommendation, but not a requirement. um Correct and uh yeah. So traffic should never reach a back end if weight is zero and but traffic should go through any filters or must, I think, must is what you said. Harry.

B

A

A

This feels like such an edge case, it's hard for me to it's hard for me to understand why you would have a back end with weight zero, but then also apply filters before uh I don't know, um yeah.

B

Yeah I mean to me: I mean why would not just remove the back end and then put the back end in when you actually need the backing yeah. So it's sort of weird.

A

Yeah, I guess one way, I guess one way of saying it is we have if I'm remembering correctly, we have considered a back end optional. uh We don't require forward two to have a length of one.

A

I don't think anyways and.

B

A

B

That's right, and that was the case that we thought about where you know we wanted to run some filters and not forward the request and uh we used to have. I don't remember the what's the case right now, but we used to have weight to be minimum one.

B

uh I vaguely recall that we updated to allow weight zero. I'm not sure why we did that. I don't recall.

A

Yeah, I think I think weight zero was really just uh a way to simplify.

A

You know you want to leave something around in your config, so it's a little bit easier to spin something back up or down, but it it is really identical in concept to a back end, not existing, that that is my understanding of it and it seems to match the comments here that uh the the way we treat a back end with weight. Zero is if the back end didn't exist.

B

um That makes a lot of sense that yeah. That makes a lot of sense. It's we have so many places where we have. These sorts of you know edge cases due to you know different ways of representing the same thing.

B

Yes, is there any? Is there any guidance from like in general api designer to solve this.

A

Yeah, I mean that's, that's a good good point and I don't know uh but you're right. We, we are struggling with this. This concept in in several places where we have more than one way to to do the same thing, and when we do, we may be adding unnecessary complexity.

A

It is a a tough balance.

B

Yeah and yeah: let's.

E

B

I mean I think more implementations coming in will expose more such areas and maybe we need to get to get to simplifying the api more yeah. Maybe I don't know yeah.

A

Right, that's, that's. You know if we simplify it too far, we'll be right back at ingress, uh which you know, but.

A

Yeah, the other side is we we go way too far. The other way and no one wants to use the api, because it's just way too complicated. uh So we'll see I I don't know what what the correct balance between those two is.

A

um Yeah. There was a little bit of follow-up on this issue: the gateway route, binding discussion- um I don't yeah, it was just. uh It was just a couple comments here.

A

um I I definitely agree that the complex use case this enables very complex configuration for better or for worse. I think harry's comment is that in the most verbose example of this, you have four selectors that can all be relevant. You can have uh route and namespace selector from the gateway down and a gateway and namespace selector from the route up, and that, admittedly, can feel quite confusing um and then and now that I'm thinking about this further, it's actually per listener.

A

So one thing that I don't think we've properly accounted for yet in any you know, regardless of this, is that we consider a gateway having admitted a route when any listener has admitted that route. But it's not obvious right now that you know this listener did, but this one didn't or vice versa in the multi-listener use case um yeah, I don't know, did you have anything any any comments harry know you.

B

A

B

I I I read the talk, and I read this and I was like okay, I was still like. We had some examples. I was just trying to put the api in my head together, so I wanted to confirm. Like is that really the case? I think we have discussed this api in this specific solution a couple of times. I wanted to make sure um so that's good to know it's certainly a little complex, but then what I'm thinking is maybe how danian has proposed to be like a package for validation.

B

Implementations could share, you know like give a listener, and you know the shared package gives the set of routes that are being that should bind to that listener, and you know all of the maintainers can share that piece of logic. I think that's pretty cool to the api, and you know, implementations wouldn't have that difference, and you know that's how we can make sure that we have correctness and we have low maintenance burden.

B

If we decide to go down this route.

A

Yeah I've been thinking about that too. You know we even even the current model is relatively complex with all the different ways you can do this.

A

We don't show the most complex scenario very often or at all, but there are a lot of different factors in play in our current system too. um So I I would agree that some kind of shared logic here is going to uh one simplify life for implementations, but also help ensure that implementations are consistent.

B

Correct yes, there was one comment that I did not put in I'm not sure about this, but I think uh so we are going to remove the from list. You know so right now. What routes can do is they can specify the list of gateways to which that specific route can bind to that feature won't be possible in this model. Is that right.

A

Yeah that was yes yeah, so go ahead. Yeah so yeah that actually came from the dock has a tiny bit more context, but not not much. I think some of the comments that led to that got removed- or I don't know, as I changed things around, um but uh the idea was this. This all comes to the idea of loose coupling right.

A

So uh one of the selling points of this relationship, in my mind, is that you have different roles, a gateway admin and a route admin and the route admin does not need to be aware of what the gateway admin is doing and the, and vice versa. So in a let's say, more traditional okay day two operation, let's say right: you, your gateway. Admin is trying to replace a gateway with either a different implementation or a newer version of the same implementation.

A

Whatever it happens to be, and so their solution is, let me spin up an entirely new gateway beside this existing one. It will select the same. Labels do all the same stuff, but I just want to test traffic going through this new gateway. While I have the old one in place and then gradually, you know, send traffic over to my new gateway as I'm confident that it works the way I want it to and it's configured properly.

A

That is, I think, a really compelling use case of our api and it falls apart completely if routes are specifying gateway, names, okay, I shouldn't say completely, but it requires everyone who owns a route to update their route to reference, a new gateway and it it makes that transition very painful.

A

And admittedly you know not. Everyone is going to be replacing gateways in that manner by you know, spinning up a new one, testing it and then spinning down the old one, but I think it's it's possible, especially since we're trying to sell this, as you know, a role-based api, where these roles can operate somewhat independently and they each have their own contracts of if you're a route owner label your route.

A

This way or select gateways of this class and we'll take care of you like that's what the route owner needs to know and the gateway owner needs to know.

A

Okay, I need to do this and that's that is the contract I'm fulfilling and that kind of loose coupling, I think, is what we're trying to do here and the reference by name got in the way of that um and and from from my perspective, the reference by name is not particularly powerful, because you know if a gateway admin can change, what a name means they can, just you know, create another gateway of the same name and there's so many different ways to to do this right uh that I think the slightly looser coupling that select label selection enables is more powerful here.

B

Yeah, okay, so that was a very, very good answer. So thank you so much rob for that. uh All of that makes a lot of sense and I think I just wanted to know like we're. You know explicitly doing that and I was missing this context, so I think I'm late to the party, so I'm making you repeat a bunch of stuff here.

A

But no it's this! It's cool, yeah! It's it's good! It's good! To revisit these things, because I I will be the first to admit that this proposal is quite complex, especially if you fill out every potential bit of config trying to do the math of what actually matches what is going to get complicated.

A

You know, I think this simple example is fantastic, but the further down the tree you get the more the messier it becomes. So it's good to reevaluate these these ideas as we go.

B

Yeah, I think, yeah, I think the most the most verbose case that we did talk about. I think nobody's going to use it because nobody can possibly use it, but people will use a subset of those, so people will use like subset configurations of those, and I think the bigger challenge will be properly testing it. So you'll probably need to do some performance test which can help all of us to make sure that we are implementing route selection and binding correctly. So, but.

C

B

I'm going to go back and read a little more and spend a little more time to understand this and maybe come up with some questions but yeah. This is where we are. It's not the best thing, but it solves most problems, so yeah yeah.

A

Yeah sounds good thanks uh thanks for good feedback here um I don't know if there's any particular most uh most of the rest of the content here I know we've had a variety of pr's that have been updated in the past week.

A

You know in the last is: I think we have time for basically one more pr uh in our window here. uh Is there any anything that uh someone wants to call out as needing more discussion.

B

I'm a little split on the adding that the the first pr in this list fire recently yeah. So we already.

A

B

Discussion but I don't have a problem adding that field and calling return. It's just the reasoning behind that is a is a slippery slope. That's that's my concern.

A

Yeah you're you're right. uh It is a very slippery slope. uh It is a frustrating problem not just to us, but anyone who's using crds, uh so an upsc upstream fix would be ideal.

A

The recent comment I added to damian has more context here, but so short answer is. This is intentional in upstream, and there was a pr that is now basically two years old uh that would have fixed this and it got some.

A

I don't want to say negative feedback, but not positive feedback and ended up getting closed uh or became stale and got closed, and I need to follow up and see what exactly happened here.

A

But what we're doing uh right now is, you know, we're doing a full json path match and we're getting uh the very first match and outputting that every single time. So what that means is we get this and and unfortunately this is not a cube cuddle thing: this is a api server thing.

A

So, even if this were to get fixed, it would be get fixed in say, kubernetes 122 at this point, and and this assumes that this is actually considered a bug and not a intentional feature um and then what you have for anyone who's using an earlier version of kubernetes is very confusing. Coop cuddle output, because you'll get the first item in the list always and nothing more.

A

So in this example of connected gateways, what you would get is the first gateway that was connected to a route which works in some cases, but is very confusing if you happen to have more than one gateway, and you still only see one because it it's enough to make you think it's working, but that's not actually what it's doing.

A

It's just it's just showing you the first one on the list. So even if we fix this in upstream, it results in a very confusing experience for anyone using a version of kubernetes before the fix, um which leads me to think. We probably need some kind of custom fields, at least as a short-term patch for this, but maybe these aren't mutually exclusively things. um The fix itself seems straightforward enough. We just need to find you know this is this is an area outside of a cig? I work with. You know like.

A

We need to understand why this didn't get merged in maybe there it was seen as too much of a risk. I I don't know um but yeah. This is this is where we ran into some issues.

A

The fix itself seems relatively straightforward to me: um oh yeah, so that this is a complex thing and I don't have a good answer for it, but I think because any fix is still not really going to solve it for us in upstream.

A

We probably need some kind of custom thing uh for now, and probably until everyone is using a kubernetes version, that is newer than say 122 or whenever we're able to get something fixed upstream. um I don't know.

B

That that's fair of my concern here is, I think there is. There will be further places where you know cube, cadillac gets in the way or us being out of tree gets in the way yeah. So we need to at some point draw a line. You know these are the things we are willing to do, and these are not. I don't know how.

A

B

That's that we need to do that first, rather than later point of time. You know future us or somebody else, uh trying to add in another field. You know or one of the features saying that, oh you did it in the past. We need to do this and you know it improves user experience, and it makes a lot of sense that it makes a lot of sense to me, but we know that we are already adding hacks.

B

Still relatively early.

A

Yeah I've been meaning to show up to an api machinery meeting but as as you would guess, both sig off sig apm. This is a very common time for cigs to meet, and this is this overlaps with auth and api machinery at least uh so anyways. With that said, I we do need to reach out to to them and understand.

A

You know what plans are if this experience will get any better anytime soon as an example. If this truly is considered a bug, if we can pick it back, but I don't, I don't think it is considered a bug, um etc. But the the other thing you you mentioned is also entirely relevant. I and that's why we have to be out of tree, and I don't think we do uh being out of tree being crds, definitely gives us a lot more flexibility.

A

It allows us to move at our own pace outside of the you know: cap release, cycle, etc, and it has enabled us to move both faster and slower, which has been great as we're iterating on an api, but as the api stabilizes.

A

Maybe it is going to make more sense for this api to move into tree and then you're right. You know the the capabilities we get. There are so much better. The uh you know, you write a function for your couple: get output and your coupe called the scribe output that has all the custom logic in it. You could ever want uh so yeah that that makes these kinds of problems go away, uh but it introduces the other problems that are being tied to a kubernetes release cycle. um So uh this this is a bigger topic.

A

I probably a good starting point. Is a dock or an issue or something that starts tracking the pain points we are encountering when interacting with crds, and that could help us make a better decision on where these apis belong longer term and or if we start making more and more exceptions like this.

B

That's fair, okay! Let me start just basic parent issue for tracking you know out of free crd problems, and then you can start to figure out what to do about it.

A

Yeah, that sounds good awesome. Well, thank you or go ahead.

E

I'm just wondering it seems like the options here are confusing output, um an awful hack to get good output, but there's also the option of just not printing that output in those. So you get a kubectal get output right.

A

Yeah yeah you're right. uh The kind of the third option here is to do a kubernetes, plug-in or coupe pedal plug well yeah, yeah, that another option is just don't print anything at all, and, and you know, people can rely on yaml output, etc uh right or uh do a coupe cuddle plug-in, uh and you know I've written coop, cuddle plugins before they're, not particularly difficult to create.

A

I, like everything, they're a lot more difficult to maintain than create, uh and also you know it adds an extra usability burden. If somebody has to you know, jump through those hoops to get uh get usable or understandable output, um but you're right uh this. This is the kind of thing that probably deserves a doc or further discussion.

A

uh Before we actually go ahead and implement. You know make this kind of compromise. It would be good to outline the other options we have here.

A

Yeah I can take that as an action item cool all right. Well, I know we're we're past time here, um so I thank you. Everyone for staying on. uh We will talk more next week, we're making good progress here. It's been fun to actually try out everyone's implementation here, uh thanks to everyone for your work uh and- and it's just it's cool- to see this uh all coming together, um we will talk to everyone next uh next week have a good one.

B

Thanks rob thanks everyone, bye-bye.