Kubernetes Network Gateway API Special Interest Group, 17 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Network Gateway API Bi0Weekly Meeting for 20221017

Description

SIG Network Gateway API Bi0Weekly Meeting for 20221017

A

This meeting is being recorded.

A

Thank you all and welcome to today, October 17th Gateway API community meeting, um just a reminder that we are under the kubernetes code of conduct here, which boils down to be excellent to one another, so be nice, um we're going to go ahead and get started with. We have a few things on the agenda. If anybody here has anything to bring up the agenda late feel free to add it. It's kind of a light agenda today, although we'll see how it goes.

A

um If there are any newcomers too, if you want to take a couple seconds to introduce yourself and talk about like uh what you're doing with Gateway API or anything like that, we can stop and do that I see people are still joining in. So if there are any newcomers interested in introducing themselves, I'll give a couple seconds for you to pop up here.

A

David wave hello, David.

A

How's it going not too bad. Are you? Are you new here, I, don't know if I've seen your name on here before.

B

uh uh I'm uh gonna be a passive listener, I work on key native open source and we're consuming Gateway API, so I'm gonna just do the vetting.

A

Cool well thanks for joining good to have you here.

B

Thanks all right, I.

A

Am having trouble with sharing screens, apparently what's up, come on.

B

Oh yeah um also kind of a newcomer uh but uh I don't really work with anything, but I uh just wanted to contribute I guess and uh saw that this group was active and kind of jumped in.

A

Yeah cool thanks for joining happy to have you. There are a few things labeled, um a few issues that are labeled like good, first issues, but not a ton. So if you, if you run into any roadblocks, please please feel free to jump into our kubernetes slack and like ask us, like you know where you can help out and stuff like that. This meeting is good for that too, although I, don't it's a little harder to do it off the cuff, usually I have to go digging for something yeah thanks, yeah, no problem.

A

Thanks for joining.

B

I am just want to win I joined last week with uh Phil Collective platy. um We kind of introduce us at F5 we're trying to see how we can integrate uh the Gateway, API and kind of like standardize. Our implementation, like make it more consumable for particularly for service provider that that's our Focus right now, but um yeah.

A

Yeah you uh going pretty going pretty well so far.

B

Yeah I think so trying to follow the you know, kind of like examples that you already have and we I've been able to.

B

um You know, integrate it, just a proof of concept right now and we'll see how we can try to make it more consumable, yeah. So far, so good, um some some hiccups, well I saw like some things have been reverted from from when I started, the POC, so yeah.

C

B

And things like that, but I guess that's normal.

A

Yeah, there's definitely been some breaking changes. um We're we just released our first Beta And I. Think we're going to do one more breaking changes. Related breaking change relate related to status soon, after that things should start to level out yeah.

B

No worries, I, I, just didn't see any like bullet points of any breaking changes, but then, when I dug into it, I found the like the like the revert and all that so.

A

Gotcha, maybe there's something better: we can do there be happy if you run into anything, that's just like you're, not sure what to do with. um Please do feel free to open up like a discussion in the GitHub repo and say hey. This isn't working right or documented right for me and we'll be happy to check that out. Okay,.

B

Yeah sure on my phone, I I didn't know: what's there yeah.

D

I mean the most important thing is that you get it recorded right? If you, if you find something hard to understand or you know something doesn't make sense, the most important thing is to get it written down somewhere so that we can.

D

You know so that we can do something about it for some things, some things uh are just artifacts of you know history, but other things like there are some decisions that we've made that you know we only arrived at by like a lot of discussion, so yeah, it's worth writing down your your thoughts and then we can enemy and talk about.

B

um Sure yeah I saw the trail of like you know this was kind of debated and then eventually pulled back, but I'm just talking about the matches right. There's this um yeah I forgot, but yeah.

C

Anyway, this guy, it.

B

Got reverted and I had it and it broke. Oh.

D

That that's the um the source and destination matching.

B

A

Yes, we did pull that yeah, so your use case is mostly layer. Four.

B

Yeah I mean I, didn't need it. So it's fine, but oh okay, yeah.

D

Yeah I'd say that yeah that one was a good example of one where we, where we sort of made some suggestions and then, when it got to time for the Upstream API review, Tim Hawkins raised some concerns that none of us could really could easily could have had a good answer for, and so we sort of opted to just you know, take the feature back for retooling.

D

um I can't remember if it.

B

D

B

It just happens like it got pulled. It got reverted like one week before I tried it again kind of funny.

A

Yeah, we could do some actually I'd be happy to work with you after this I have a thing that I'm gonna bring up in just a minute here. That kind of vaguely relates to this. Actually. So, um if you'd like to follow up with me directly in kubernetes slack after this I can give you the links to like show you where things happened and I can also talk to you about like how we get that working going forward.

B

Okay, actually, thanks.

A

Yeah no problem.

A

Pure talking, or is that background noise.

B

Here, Louise.

A

Okay, I guess I'm gonna mute here, Louise uh hit us in the chat room. If you were trying to talk, it sounds like you just had somebody in your background, trying to talk. Okay, um give Pierre a second to chat here. If he's trying to say something.

A

Okay, he just sent me a direct message. Says apologies, I think it was just background, no worries Pierre Okay. So if, unless anybody else wants to uh pop up and introduce themselves, it's really good to see all the new faces, give it a couple seconds and then we'll move on to the agenda.

A

Cool okay, I'm gonna try to share my screen again, and hopefully it stops stopping me.

A

It told me no like a few times all right um cool, so I have something that I wanted to bring up today. I almost had a demo uh to bring up today, but not quite um basically uh we're at a point just to kind of recap where we're at we're at a point where v060 um is just is about to go out the door. It's pretty much, not gonna make it for kubecon at this point, but like for a while, we've had a name on every issue that was required to get it closed.

A

It's almost done. um One of the big things coming out of that are like the resource grants going to Beta and stuff like that and there's a few other things.

A

I am interested in trying to push for L4 stuff to be beta at least TCP UDP route beta by v070, which I would assume land somewhere I'll, be optimistic and say early next year, but probably you know like a little later than that.

D

um I would guess that it there's a good chance. It would probably be closer to qcon of you yeah that.

A

Might be fair, I I want to try to see if we can I mean.

D

I'd, like it I'd like it to be sooner but but.

A

Yeah yeah I know same I'm, I'm, really hopeful that we can start getting our releases faster, but we'll see how it goes. You know um suffice to say: go ahead.

E

Tls throughout or or udpro I mean there's a big difference between them. I think.

A

E

A

And UDP I'm not sure about TLS route. Yet um I didn't want to invoke that one just yet any.

E

And it isn't for that, because UDP Rock in particular seems the most difficult to support of all three and TCP on the Internet. It's not so common, maybe I, don't know.

D

Yes, I mean so, let's, let's actually um as part of talking about this- let's, let's quickly address uh Candace's question, that's later on, because it's directly relevant um that Candice has later like how come we didn't yeah bring uh TLS right up to uh up to Beta.

D

um That's because uh you we need a certain number of implementations to so our graduation criteria for beta are, there needs to be a certain number of implementations have done it and we need to be confident that people are actually using it and we need to have performance tests.

D

um We don't we don't have any conformance tests uh and we don't have good data about how many implementations have it. So um that's, basically the reason um I don't see that the spec will change that much for for TLS route. There's not a lot to to do. uh You know like there's really the only things that are available to use are. You are the the TCP properties of the connection and then the Sni.

D

That's the only things that you can use to select that's what the Lesser ad is for is to you know, forward things to a back-end service without uh unwrapping the TLs session, uh and only looking at the server name indicator uh as using um yeah to use for discriminating between different things. So you can have one port, listening, uh listening on a TLS thing and not terminating the session, just looking at the S9 and forwarding it off to other to uh some other packet and yeah. So that's. Why is that we just don't?

D

Have uh we don't have any common assists? uh And um yet we don't have any sort of solid. This many implementations have done it, and people are generally happy with the API.

C

Thanks Nick, but is it correct that there's no other way to do pass-through.

D

Yeah, that's right, yeah, so pass through pass through is clearly tlsr is full pass through. Yes,.

E

ah Okay, so UDP route, I suspect it's even worse, because it's.

D

Oh yeah, like UD period, is going to be a robot gonna do unless you are um unless you're in the position, where you're controlling the cni as well, um like it's going to be very difficult for an in-cluster implementation to to do UDP without uh without having some ability to mess with the cni and and a whole bunch of like layer, three stuff as well as layer, four stuff, because it's so easy right.

E

But you said you said you want to promote UDP route to better.

D

So yeah so Shane is saying he would like to. But you know the same conditions apply to promoting TC period and UDP around the beta. We need to have a couple of implementations. We need to have conformance tests we need to you know we need to have a few people doing it, and so I think Shane's point that he was going to say was. You know he has a thing that he started working on. That's going to help that maybe we can yeah and that's yeah.

D

The idea is to have the goal to promote a debater, but to promote it to Beta. We still need to meet the the beta requirements. The same as we would for anything it being promoted to Beta does not mean that you have to support it. An important part of this API is it's 100, fine, for your implementation to be like I only do HTTP route and maybe TLS rail right. That's that's 100! Fine! You do not need to support TCP route and udpr. If you do, if you don't.

E

But yeah, but if nobody is implementing your details today and I'm, not sure if any implementation supports gdpr alternating equivalent, but maybe udproc should be the last one to promote when and wait until at least what the implementation have support for UDP. Well, so.

D

A

D

Like I think we did.

A

Not have said UDP route I should have just said, TCP right.

E

A

Thank you very much. Thank you very much.

E

Udp I got skipped: okay,.

D

I mean I, think I mean I do want to, but.

E

Yeah, everyone wants GDP, but once and Ken is different.

D

Yeah, like sure I I, a thousand percent agree that doing UDP reality is not straightforward and is very Earnest. A lot more tricky than people would assume at first glance. People like, oh, you could do TCP. Why can't you do UDP? And people have literally said that sentence to me.

E

Quick question for TCP route: are we planning to do termination or TLS termination for TCP or is just the old? You know plain tcpe, because one of my question is that a lot of on the internet is not so common anymore to have plentycp, usually it's TLS and some protocols yeah. So.

F

I mean we need to be careful because we we say this, but, for example, looking at the games, video games, industry right, like they're, very interested in kind of raw protocols like we cannot. We should be careful about blanket statements.

E

Yeah not blanket statement, but it's you know, maybe half and halfway maybe I mean I'm, not saying we should not support brain TCP, but we should not probably promote sap without having some TLS. Maybe I mean same thing with the http.

D

Yeah, so look to be clear: tlsr is far closer to being ready for V1 beta one than than TCP rail. I. Think, like you, there's not so much. We need to do for TLS route aside from make conformance tests and do some other stuff like that. Like the you know, um I think uh Mike asked a question.

D

So Mike asked a question the thing about why a TLS go to tsp are separate. Sorry, someone arrived in my front door and my dog doesn't like it um the so I think. So what you're talking about with.

D

um So uh yeah so I think so what Costa's talking about is TCP are out with like terminated TLS, um so where you're terminating a two, uh a TCP connection, that's wrapped in TLS and then passing a bare TCP connection on. Is that what you're talking about custom.

E

uh Yeah I mean Gateway 39 stls, so it has options, but I want to make sure we we consider completely the use case. I mean you know, that's a nice lpnology.

D

Yeah, okay, um the um so yeah look I, think so I think for for things that don't for any for any tail that's passed through. The idea is that's what TLS route is for right. It's it's made! So that's if you're, not terminating TLS, that's what it's for I, don't think we have spent any time talking about uh terminated, not like terminating TLS for non-hdp connections as you at all. This is the first time anyone's mentioned it. So I.

D

Don't dispute that we need to talk about it at some point, but I think we can talk about it further down track. So I think in terms of in terms of the. What we're talking about here, like the whole idea, is that we want to be able to have. We want to be able to add these extra objects in we need. We want to be able to have a path to how we get them to Beta, um and then you Leah.

D

We can totally have discussions about exactly what we include when it's in beta but like. We don't also need to boil the ocean and have everything included before it moves to Beta like we can. As long as the things that we're doing are additive, we can add extra things once it moves to Beta, so we don't need to have everything sorted before it can go to better.

D

Does that sort of answer everyone's question say? Oh actually, um Mike asks why why TLS around and TCP right separate because uh TCP route doesn't so the distinction is that TLS route lets you discriminate based on the Sni in the handshake um and.

B

So yeah yeah I'm, familiar with like the how it's implemented. Now it was more of like an API design question of like. Why was this pattern chosen? I remember like I was a few months ago and I think somebody like was able to link to something. Maybe that had some more context on like design decision at the time.

D

So I I, don't I, don't remember if this was what it was, but I know that I would have been pushing for sure, uh because for Contour, when we did HTTP proxy, um we called the TLs routing feature that we had TCP proxy, which was complete, misnomer and really took people up the Garden Path, because people were like well, why you know how do I use this to? How can I use the the TCP proxy stanza in uh you know HTTP proxy to forward my SSH sessions and I'm like well?

D

You can't that's not what it's for it's actually for forwarding thing: forwarding TLS sessions based on the Sni which, which is a real, which is a much more common use case than wanting to like forward SSH sessions based like based on some property of an SSH session right, and so yes, there is no other way to do pass-through except TLS right. That is what TLS right is for pass through is done, is accomplished with TLS well um yeah, so.

B

E

So I think yeah.

C

D

Not SNR routing is weird um it's you know it works in a really weird way. It is not a clean, it doesn't happen at the TCP route level. It happens at some spot in between just above um just above, uh left layer, four but like it doesn't fit cleanly into the OSI layer model either, um and so that was the idea. Sorry hang on I believe I'm getting a package as well. Okay, thanks.

C

So Mike did you get an answer to your question about why there are separate.

B

Yeah yeah I I think so it was mostly just making sure that you kind of like had an opportunity to like share the same um rationale that I remember this Phoenix thing previously I just want to make like. Does this make sense to you too.

F

Foreign, so I think the General take is that TLS is, is a different protocol right, it's on top of TCP, and it has to talk about l7ish kind of I mean it is kind of an L7 property L7 protocol. It has a handshake with you know, metadata that could be used for routing and so on so forth. So I think. That is why it was separated from just the TCP route, which can only talk about TCP properties.

F

Now, after you get through the handshake, maybe you just view it as like a bite stream so in in that sense, like the two are sort of similar, where people can kind of throw that bite stream over TLs, but the pertinent bit which for the API, which is all the routing Etc, that one is actually very different.

F

So that was that's the general mistake.

E

I'm a bit confused here uh when we are talking about DCP route. It's you, you have a Gateway with a listener and The Listener does have a TLS section. We can say to specify certificates, and that would mean that it terminates TLS since you're not much TCP routes to it, and that means that Gateway terminates TLS with a certificate. So.

D

E

D

All of that behavior is currently not defined. So yes, the spaces. Well, I mean yes, but there's it's not defined in the sense that we haven't written down exactly how that will work or have any conformance tests about it or anything like that like we just so so.

D

The spaces are there for that to happen, but we haven't spent any time, actually writing down how that would happen right, like you know, that's what I mean by it's not defined like there are spaces in the API for that to happen, but we haven't written down like what happens. You know if you're terminating TLS like and you have multiple TCP routes. How do you match on them? We don't know like we don't have we haven't. We just haven't spent any time designing it. That's all like it's not that we don't want to do that.

D

It's not it's just there's a space there, but it's a it is a un under implemented spec at the moment.

E

Interesting I, I I wasn't the impression that it's you know that's by this I mean it's pretty obvious. I mean that you have, since you have a TLS in The Listener would know it could be obeyed. Then it doesn't say anywhere that it's only for https or anything like that. Yes,.

D

Okay, yeah, we didn't specifically say it, but again like it's, not that it's not that we have specifically not designed it. It's just that we haven't specifically designed it right, like we haven't, spent any time or effort on figuring out how that would work or what it would look like if someone wants to go and Implement terminating TCP sessions and forwarding them with TCP routes and showing us how you showing us your work cool but like it's more just we haven't put any focus on it. Yet that's all.

E

Pretty sure this just does it I mean it's, you know, I, think it's implicit and that suspected that the people are doing the same. uh I mean uh okay, I'm perfectly fine to promote PCP route. I'm very happy, but we should address this before we do. It I mean at least.

F

This is, this will probably be very useful as like graduation criteria. It's given you know, thankfully these are still Alpha, so these kind of discussions actually are possible.

F

um We should I think as uh Shane like and other people have commented in the comments like what is that graduation criteria, it's it's clear that there are some things that we should write down very specifically and also looking at how the experience of different implementations are, because uh we can't discuss things quite a lot, but then, when someone goes to implement it right, they they have like a very real understanding of what what the consequences are.

A

Right um Candace: did you get the answer that you were looking for kind of and what you need to move to go forward, or is there more that you needed out of your question for today? Your agenda item.

C

um Well, it was in so I know it was in V1 Alpha 2., um but it just didn't make it into V1 beta one um I. Just wasn't exactly sure. If there was you know some big issues so for us, if we wanna, if we want to do something that uses pass through I mean we would need to use an implementation that you that that supports both V1 Alpha 2 and the one beta1 for other things that.

B

C

Better one, yes, so.

D

Right now, the way that you do, that is by using the experimental, uh install yamls, they will include all of E1 Alpha 2 resources and the V1 beta 1 resources. So they'll include the the Gateway HTTP route and stuff that everyone better want, but they'll also include the V1, Alpha 2, stuff and they'll also include the experimental fields in the V1 beta1 resources.

E

Or maybe we can push you know theater's route to since it's the easiest, as least amount of sure.

D

Sure that's what yeah I think that it's quite reasonable to look at pushing to lesser out to v1b to one in uh 070. uh You know, I think that there there is work to do you've got it. We need to fill out the fill out the performance test. Suite we need to you know it looks like there are a couple of issues that Candace noticed there um that probably probably need resolution um yeah a couple of other things like that.

B

Yeah and they say.

C

Yeah, thank you. Thanks, I think we can move on past this one. Okay,.

A

um What I have to say has some bearing on it too, so where I was where I was trying to start with? Was um you guys defined kind of the problem pretty well? In that we don't have?

A

We don't have things in good enough shape right now to really push uh for a V1 beta, 1 release of some of the L4 stuff or the TLs route stuff um I have kind of realized for a while that there's been kind of a missing push, there's been no champion and I want to kind of take up. The I want to be one of the Champions to try to push for L4 stuff, UDP route, TCP route, don't forget at UW route, but I want to do it.

A

um So one of the things that I noticed about this problem is that, okay, so like a Kong, we have an implementation of TCP route and UDP route. We have a couple customers that use it it's very minimally used and in general it seems like there's not a whole lot of implementations of it.

A

So what I've been thinking about doing is what I've started doing actually is trying to come up with what I thought would be an example project of doing uh an L4 implementation of Gateway API, the intention of which would be to help to through experimentation uh start to Define these things that have been undefined. All these things we just said we're just not sure it's not all coming together.

A

We don't have enough implementations or we just haven't like figured it all out yet use an ex an example or an experiment of building an L4 Gateway implementation. That's specifically for L4 to try to do that. To that end and I have a link in here, I started working on a project called bleakst which is Swedish for lightning, because Tiki it made I liked the name.

A

um So this is meant to be, and it's just the beginnings of an experiment. I don't have a demo today, unfortunately, I was really close. I think I will have a demo for some of you at kubecon, but this is meant to be an L4 Gateway implementation, basically TCP route, UDP route that uses ebtf, uh uh basically Linux kernel as the data plane. So it's very generic. It's not tied to anything, Kong, specific or anything like that and um I've got most of the pieces working to where you can actually create UDP and TCP routes.

A

It's just a little bit more work needs to be done. I can tell you about it a little bit more, but I. Don't know that that we really need to spend time on that. Suffice to say uh my company is like if this is something that we wanted to do is like an example, implementation of Gateway or something like that, or even just as a toy project. My company is like on board with potentially just giving this to kubernetes six.

A

So it doesn't it's it's in there it's in there right now, because that's where I've been working on it, but it could be something that we use it's kind of. If people are interested and I know that people don't always have time for things that are more like toy projects, but I'm going to continue to work on it, so that's was my take on okay. How do we get this L4 thing going? We build something that needs it. We basically make it. You know.

A

uh Necessity is the mother of invention- um or in this case I guess invention is the mother of necessity, I don't know, but if you're interested feel free to ping me. Basically, if there's enough interest, I'll probably try to make the move to like push it into kubernetes six, but the basic idea is just right. Now. All it's doing is just piggybacking on um service, because Gateway doesn't have like a load, balancer implementation, yet piggybacking on service to hijack Service, uh IPS or load balancer service IPS, and then on. The nodes.

A

Do ebpf routing of those IP addresses based on? What's whatever is in a UDP route or TCP route specification, which is very little right now. This is one of the things that came up was like doing: Port uh based routing like uh destination port and destination, IP and stuff like that for L4.

A

This is maybe a place where we can exercise that and try to actually get that back in and figure out how that works, so not necessary, I'm open to conversations and questions about it, but it was more just like a call out like if you're interested in this. We can go async and talk about it, some more, but it go ahead. Bowie.

F

Yeah thanks. um So one thing that was sort of like a Far Far Away goal was what, if this replace service type load balancer, because yeah service type load balancer is one of those things that, because it's just sucked into the service resource, it's just been very hard to evolve.

F

So that's something I would put in your mind. The other one is the absolutely.

A

F

On kpng, so it's like did those intersect somewhere potentially.

A

What the, if kpng, makes it in some of the pieces of what I'm working on could potentially just become kpng pieces.

A

um On the other hand, I'm also very aware of the like, the we've always had the north star that we would like you to be able to create a Gateway and get a load, balancer IP with no service involved. So there's that dichotomy um I'm very aware of that. That's why I kind of pointed out right now, it's just based on service load balancers, because that's just what you do, but very much in favor of potentially using this as a driver for like this is a really good use case.

A

Something that really really doesn't want to have to use load. Balancer type Services would be a good use case, maybe for driving forward the load. Balancer provisioning for Gateway thing, which has been around forever.

E

I'm, a bit confused, you're saying that it's uh attaching to a service, so it's more like Gamma or or is it.

E

A

If my uh sorry, if my what I my language wasn't clear um at the moment, you create a service type load balancer and then on the nodes. It just hijacks the load, balancer IP on the on the uh on the uh yeah on the interface.

E

A

Routing by by redirecting those packets with BPF to the virtual interfaces for the cni, the Pod Network, it's all very POC- it's not really meant to be. It is whatever it is. But at the moment this was my like I started it to start working on this problem. Space like let's build something that is from the ground up a layer for load, balancer, a layer, four Gateway implementation. So.

E

Your goal was to avoid the external balance of the type I mean if you have service type mode.

A

Eventually, I would love to avoid.

E

A

E

Exactly because initial, we just you know, use a service type load, balancer PCP, instead of use. You know we have our gateways that support PCP and https and everything else, and then we just supplies the other automatically. It's pretty boring.

A

That would be I, it would be ideal, but it's not necessary to get this POC going. Okay,.

D

So yeah, uh it looks really interesting to me. Shane um I am also uh interested uh once so. I am still onboarding, but obviously psyllium can do like a bunch of evpf stuff ourselves. So you know I'm this. It's you know TCP route and you do piano, is definitely something that psyllium has the pieces to do it's just that. We've got to figure out how to wire all the how to wire the you configure this with TCP route and wire it to you end up with psyllium config, somehow um so like yeah.

E

D

E

Would anticipate.

D

That at some point, uh psyllium will look at doing uh TCP round and ukip or no actually no I, anticipate I know because, if I'm in charge of making that decision but but like the uh you know so I don't it will be a little while before I can get to that. Because I got to do.

A

That yeah yeah, like I, said this is kind of a pet project at the moment. So this is not meant to be like. We need to do this right away, um but I would be I'd, be happy in the next time after Cube gun, let's just say after kubecon, maybe before Thanksgiving. uh Let's see, if we can maybe talk about it tomorrow, I'd be happy to work with you on it and I'm using psyllium's ebpf go I actually started.

A

Writing this thing in Rust and I found that the rust ecosystem just wasn't quite there yet and I used to do c for years more than half a decade ago, I'm dating myself, but um so I was just very comfortable in the sea ecosystem and the go uh celium's ebf go is pretty great. It's pretty easy to use so um cool yeah I, that's I'm, I'm glad people had some interest and stuff like that. If you have further interest, please do reach out to me directly on slack Shane at um I'm.

A

Gonna Keep I'll probably bring it up again after kubecon enduring kubecon and just see if people are interested in jumping in on it and building it out. I should have a poc in a demo at some point, maybe that'll be for the next meeting after kubecon I can kind of demo it for a few minutes, but that's the the general idea. Let's, let's make lo4 happen by like building something together. That's all out, for that was my idea, all right. So Nick you want to talk about back-end capabilities again.

D

Sorry, um you know I think we're getting close to here. um So if you could pop the um pop that link open, please, um oh that's the that's. The link to actually get um so I think it seems like from the discussion that we've had on the thing um most of the discussion has been about um like. Why are we putting everything in this one new object, and why do these fields live here and not somewhere else and a bunch of other stuff I think I would summarize um I would summarize the so.

D

This is on the pr for um not for one three six four, but for um uh the pr is 1430, and this is Candace PR Candice. Sorry, do you want to say anything.

C

No, no, that's! Okay, yeah, take.

D

It away yeah Okay cool, so yeah I'll summarize, you I think the the uh the way I would summarize. The discussion that we've got on this PR is I mean you can see it's a lot of comments.

D

um Is that nobody's really just reading that we need to do like something like this we're kind of just talking about, like the shape of the API and where the fields live and what Fields they wear and what bits get configured where. So um you know the the pr that um Candice has. There has basically uh we're adding a back-end capabilities, uh object that that is um that will that has a that's a little bit like a it's a wrapper object.

D

um It has a reference to a specific service by name and namespace, and you say it's basically intended that when you create this back-end properties resource it, um it has that reference and it wraps the um the the service. This is because we can't make changes to the specs, so respect very easily um takes like at least a year, even to get them looked at, um probably like two or three years to actually get them functional and in a number of versions of kubernetes.

D

um So the idea here is to try something out and to figure out if it's actually viable and to see if um this is a thing that we want to try to push later on to to move into the service spec, um then the idea here is to like to have these backing. Have these backing capabilities be things that the person who owns the service wants to say, like my service, supports this yes custom.

E

So the intent is basically to have this as a big bag, with absolutely everything that we want to put and we keep adding films to it.

D

Yes, yes, so it's going to be it's going to be like you know like so. The initial thing in this in this uh initial PR in this initial implementation is just TLS details like so it's back end TLS details.

D

So this covers the re-encrypt use case where you have a HTTP route um behind the gateway that has a TLS stanza a listener that has a TLS stands up a HTTP route that then, when it goes to the backend service, you want the Hop between the Gateway but data plane, proxy, probably uh and the service itself to to be encrypted with TLS, um and so we need somewhere to put that information. Bowie.

F

Yeah I had a question I'm looking through this. um One of them is that we should be careful if we need to do service comma port, as opposed to just service.

D

Yeah yeah I thought about.

F

That, yes, unfortunately, kubernetes has sort of grouped services with multiple ports for better for worse, so that just seems to be carried everywhere in the system. The other one is that there is I.

F

um You call this capabilities, which I think is very key term here.

F

Is there there is a use case where the server says I need to handshake with a particular CA. Otherwise, I say you are not authorized. I I don't want to talk to you. How do we express that in this this scheme, because wouldn't this be controlled by the producer of the service versus the yes? Yes,.

D

The server side yeah, so this is controlled by the person who owns the service. That's the intent here is that this is the person who creates the service object, creates the backing capabilities object um in that one. You can actually see it in the example that we have there. The ca, certs um variable is a is a CA certificate, bundle, the the uh the server that the service is telling you should be used to validate the um the uh the connection to the server.

D

Now there is a light implication here that, if you're supplying a client cert, then it needs to park, it needs to be valid with that CA we kind of haven't so one. That's one. One of the things that we are arguing about is where the client-side information should live.

D

um Candace originally had it in this in this resource, um but, like you know a couple, people raised the point that it's like. Well, maybe you want to have different certificates for different clients, and so it doesn't make sense to put it on the on. You know for that to be owned by the producer. Yes, custom.

E

So initial here we have this initial rule for this purpose and I think many people would agree that destination rule combining TLS settings with a bunch of other things. I mean balancing and other stuff was not the best idea, and it may be better to call this TLS.

E

Especially since we don't know what other capabilities we had and it's not really a capability, it's a requirement, it's kind of a you know, back-end TLS settings or uh it would be confusing to to start such a broad term only have TLS, which is not really a capability, but I.

D

Don't know yeah so I understand the name is definitely needs to be changed. um There have been a number of other comments to that effect. um I am welcome. I, welcome, other suggestions, but like the the the key part here. Is it's not just ALS, like the other one that people have that people definitely want? Is web sockets? How do you say which paths on your service support websockets right now, when you're using Ingress, there's annotation on Ingress that says or annotation on your service? That says these paths support websockets.

D

The idea is that there will be a websocket stanza added to this. That says that has that same information, the.

E

Point of this, my point was that it's better not to I, mean TLS settings and web socket and other things like timeouts are pretty different things and may not be very easy to to have one such we said this. We are back into service. We are implementing service here with one.

D

Cell, yes, we are 100 re-implementing service because we can't change the service spec, like I.

C

B

D

We could change the service back tomorrow. We would stop pissing about with separate resources, and we would have separate step. We would add a stanza into service for TLS and a standard service for websocket and a standard of service. For, like any other property, you can care to name right. We can't do that. It is out of the question it will take at least three years to get those changes in. It is impossible right, like so.

F

So I I think it is worth discussing and especially in terms of naming which we know everyone has lots of opinions about.

F

um You know please comment on the pr I think this is like a great way to gather it, uh whether or not we need to split it on functionality. I I think I do have a question about particular ownership models and just to make sure that we we understand whether or not this matches those use cases, for example in in this current case, where we're we're saying the client, the the producer of the service, also determines how the Upstream communicates with that I think.

F

That's that's an a valid sort of uh use, case and kind of semantics.

F

Is there a case where that is not true and the two the producer of the service and then the person who's routing to the service are not necessarily cooperating. What happens then, like yeah, so.

E

F

There space is there space to sort of allow that, like what happens, if we have to compose the two together.

D

So yeah, so this is one of the problems with doing. Tls is the first thing we do with this. uh Is that there are all of these weird like it gets a bit funky right, because you've got because what this thing is essentially saying is. This is saying like as the service owner when you connect to my service I'm. Only gonna I'm only gonna respond to you if you're using tls13 and decipher suite, and this CA, or something like that.

D

Like all it's, it's a shorthand way for people to say that, rather than you needing to figure this out by by like trying to connect to it and failing until you do TLS 1.3 the.

F

um Yeah I think that some some aspects of it that makes sense like uh especially the ones that cannot be discovered in the handshake, yeah but I- think the the tricky one for me is the ca search, something like ca. Certs, where the server has to like the server cannot create a policy that says so by server I mean the the person who's referencing. The service cannot create a policy. That's saying, like I'll try to communicate with you, but actually I need.

F

You need to appear in the ca which, which is one aspect that I think is actually fairly common.

F

Yeah or you can imagine it's it's not even a a physical list like a file of CAS, but you know somehow Provisions from some infrastructure to say that that router can has a trustable set of things and untrustable set of things, and if a service appeared, That, Couldn't kind of produce off.

F

um That is trustable, then the server will basically kind of fail. The connection yep I just want to make sure that, when we take in this proposal that we make space for that, because it feels like a very a fairly reasonable use case as well.

D

Yeah so can I just interrupt.

C

For one second I think it's the same uh discussion that that John had posted to the pr too.

C

um You know making sure that we recognize the producer and consumer roles um and I have to say I wish there was kind of a more uh descriptive way or or more succinct way of describing this, because I'm I'm still not sure whether you're talking about Bowie. If you're talking about the same thing that John's talking about um and if we could sort of succinctly, write it down and take notes on it. That would be great.

F

Yeah so I, maybe I'll, look at John's, sorry I'm, just looking at the raw PR, so I don't have the comments, but I will follow up on John's, because I feel like that is that is sort of key to this is that the service we're describing what the service advertises, but there might be some things that actually the control doesn't doesn't necessarily belong there. So that's why General comment.

D

um Down at the bottom of this comment chain, uh I put in some stuff um this is we had this exact same problem um on Contour um for similar reasons. Right, like um that, you know some of the sometimes what you want to be able to do is to have the service say. You know: hey I only do TLS on this port or something like that, and then sometimes you want uh the gateway to say well, I'll only connect, two services that do TLS 1.3 right, no TLS 1.2- is allowed in this cluster.

D

If you advertise your service with TLS 1.2, sorry, it's not gonna work. um You know and that's like a admin level thing um that is a good sort of fit for the policy uh construct, but policy constructs are kind of intended to like the way that we built the room. Originally was the policy constructs are kind of intended to either default or override settings that are already available at like some more granular level right.

D

So for this to work, probably what we would need is we have the service owner has a way to um uh uh has a way to say my service supports TLS 1.2.

D

Only or you know, let's, let's stick with that for now um and then the uh the then on a on something in the on the Gateway side, probably in a HTTP route, there needs to be a thing that says: okay, the connection to this service should use TLS 1.2 only or TLS 1.2 or above, or something like that, and then then, and only then you can have a policy that you can attach to things that lets you default or override that City.

E

So so one dot tls1.3, it's you know, property of the code. That is.

D

Sorry, but so yes, that is 100, so, okay, so TLS one point so accepting TLS 1.3 is a property of the code that is running in the service that is determined by the service producer. Whether or not the connection from the gateway's proxy to the service consumer uses. Tls 1.3 is a property of the Gateway implementation. It needs to be configured on the Gateway implementation. There's an another way to think of this is, if you're going to use a client cert to connect to the back end servers. Where do you specify that client cert?

D

You know it is reasonable to assume that people will want to Specs best that service producers will want to say each thing that connects to me needs to have a separate client search, so I know so. I can say this client belongs to the Gateway. This client search belongs to some other consumer right like so.

D

There needs to be a space in the API for us to say how do you specify a client search per backend right like, and so the only place where you can put, that is it somewhere in the Gateway resources right like in Gateway in HTTP route. I would argue.

D

The most natural place is something like in the HTTP route in the back in there, for something like that, you can also have it be that you can specify those in a params ref or something like that, but then you're going to need to specify a lot of them and you can and you're going to need to have some way of looking up which CA belongs, to which CA and a bunch of other stuff like that. So so.

F

I think um just.

A

To time check that we're running.

F

A

F

uh Just one comment so sorry to add more work, I think it's worth sketching through at least the the non-cooperating cuj just understand, because I'm fairly certain that, like let's say we get this in in the future.

F

Probably that use case will come back up, so we just need to make sure, for example, any language and any kind of conformance that we put around this is going to be compatible with like a future proposal based on policy that is sort of in the non-cooperating case, yeah I, just don't we I was very careful like we don't accidentally go over and kind of lock it out.

D

I understand the things that you're thinking about um I've thought about it. A bit. I do think that so one of the other things that John raised in one of his comments was that I wanted to talk about. I know we are running out of time. I'm sorry um is that um John's like well?

D

How do you know how many layers of TLS there are, and you know what, if you're have got something in between doing um you know doing magic TLS for you, um so your TLS and your TLS again, um your double encrypting, or something like that? Oh yeah. Yes,.

F

Exactly right, I'm.

D

Like the the point that I'm trying to make, is you don't you don't care the whole point of that's? Why it's so important that this back-end capabilities name to be determined um objects like it is saying from the point of view of the owner of the service, you need to use this TLS. This TLS details, I.

F

Have a question about that because uh sorry I'm like way over time, so you can cut me off if you want so the notion of capability is an interesting one, because that's like hey I'm telling you up front, you know I I, don't accept Mastercard, um yeah, sure yeah. So, like that's, okay, um I, wonder what we talk about in terms of conformance, because you know the server can can choose something else. It might just get rejected. It's like.

F

If the server wants to talk to you, they should use these, but it's not necessarily a game about. They must does that. Does that make any sense like.

C

E

C

F

Didn't want to do it I guess they could do they. They have some flexibility or something I I've, just trying to that way like, because if we mandate that the client can force something on the server side that seems to that seems to conflict with later on.

F

If we came back- and we said, there's some policy on the server side to me like let's say the client up here said like yeah, you can use SSL one and then the server has a policies like I cannot use anything lower than TLS 1.3 like what happens in that case. So.

D

So yeah so yeah I agree. We need to specify that in my mind, for that one that is, uh you know that starts to be in the direction of things where we would be like. Okay. That means that the the back end isn't valid right, like when you start like losing back ends and ending up with 500s and stuff like that right.

F

It would be good to at least go a little bit into that um yeah to kind of explain. You know. To what extent is this uh mandate for the server versus like this is sort of extremely useful information for the server to communicate the client, but not necessarily that it's sort of like mandating something I? Would.

A

Be happy to to I'm sorry to interrupt I would be happy to say overtime to like go further into talking about this cap, for anybody who wants to stick around I would like to give Dave P a chance to do his agenda item. If we could stop and do that real quick, would that be okay.

B

My my item was just there for documentation, because that was the initial discussion we had. So that's why I put it at the top and then I saw so I'll move it to the bottom I.

C

Just wanted to capture.

B

The links people share it in the thing. So, if you want to spend the last five minutes going back to the previous discussion and then go for it, okay.

D

Thank you I think, given that I started that can of worms as well, um it's probably fair that I talk about it a little bit, so that is basically um Mike thanks. Mike did a great job of raising a whole bunch of issues that there were with the existing status design.

D

Basically, there are a whole bunch of places where the status design was inconsistent, vague or contradictory, uh and so, in order to fix that we've raised, get p1364 um we're sort of working through making all the changes we need to do to handle the Gap that I actually opened, um and so that's why there are a bunch of breaking changes there, because those are changes that break status when we break status, we're going to break performance because components status is used in conformance, and so um that's why those are breaking changes in terms of the actual spec of the resources.

D

There are no changes. This is just a. This is a um like a a breaking change for um you know for how the status flows work to make it so that, on this across different objects and stuff, they they're more similar. They they make more sense.

D

They're they're, uh you know they're more system because we designed the objects at different times and did the status at different times, and so they've kind of they were very weird and inconsistent because we had built the objects at different times, and so this is a sort of an attack to make us more standardized patterns for doing status.

A

So implementations will need to update and then, if implementations are familiar with their users using status like integrating with the status they'll have to let them know to update as well. Yes, so.

D

I'm aware that making breaking changes like that, absolutely sucks and I am sorry, but we but the the status of it now sustainable, and so we were like, let's break it now and break it hard and get it done.

B

D

That we only have to do this once hopefully so yeah, that's the purpose of those changes yeah, and so yes, um everyone who has participated in discussion about the um back-end properties back in capabilities, whatever you want to call it. um Please add those comments to the pr um you know the to 1430, um the uh yeah. We like I think that yeah talking about the the name like bike shooting on the name is useful.

D

Like you I think, capabilities definitely seems like it has implications that I, don't think we intended, um if you just do a space after that chain, it'll magic it for you, yeah I did that with the other one yeah, the um so yeah, so everyone has commented here, it'd be good. If you could put your comments. Put some comments to that effect on the thing I think we most of the comments at the moment are happening in like a comment thread on a specific line of the file.

D

I think we should try and walk those back to the main comment stream. Let since we're talking about sort of very general things for the um for the pr um yeah. Yes, I definitely think that we could spend a lot of time arguing about exactly on the name I'm kind of in favor of Brewing on the side of having something to play with, rather than spending too much time arguing about the name.

D

um But if people have strong objection, stick capabilities again, I would urge you to record them on the pr please so that we can, if we're going to change the name. Let's do it but like we need to agree on a name.

A

Right to that, does that mean you don't want to try to do some overtime today we want to stop in here and keep going on the pr I.

F

A

F

It over the slack, I I, think I'll put together some comments as well. Nick.

D

E

I definitely owe.

F

You you guys that if I'm going to ask you to do any additional work.

D

Yeah sounds good and I. Think yeah price is a good point that next makes waiting is canceled for kubecon um yeah. Obviously you you saw me get up on my crotches before um I'm I will not be at kubecon, so I'm very sorry to miss you all, um but that does mean that I will be I. I will have time to uh spend working on this sort of stuff, so um yeah so um yeah.

D

If you have documents, you need to review I'm going to need things to keep myself distracted because firmware will be real, um yeah and um so yeah. Please uh comment on that. Pr. um You know it's Candace's, PR, I'm, sorry Candice to sort of derail uh and you know be the one to drive the conversation, but I figured that I started this. This uh Shenanigans I feel like it's on me to help you no.

C

Yeah, it's just the so Nick started it and um he put together the how and no sorry the why and you know for what reason part of the document and then um that was exposed to the community for a little while, um and there was a lot of a little. You know a little bit of changes based on that and then it was in PR form for a little while and the pr was merged and I. Don't know how long it's been around for, but um I just picked up the implementation part of it recently.

C

So if we need to go back to the you know, why are we doing this um part and andry and take another look at that? Is that something that you're willing to do Nick.

D

Or totally yeah, if, if that's what we like, it seems to me that people aren't really arguing that we need to do something like about this. It's people are arguing like what we're doing and the the sort of the the more the conceptual model that we're talking about could be that when, uh when I've done this before, like I thought, I thought I was clear on the conceptual model, but evidently I wasn't um so you know.

D

Obviously the conceptual model needs a little bit more clarification about like what's owned by who and why we're splitting up the config in this way and how that how that stuff works is the bits that we need to add. So it could be that I need to make an update to the sort of the. Why the? Why and the goals section um with more information about the conceptual model, but then that will clarify some of these questions, but I think we need to get all the questions.

D

I think that PR is a good place to get all the questions written down and the comments written down on and then we can decide like you know how we take it from there. Maybe maybe I go and do another PR um to the other bits of it, and then you sort of rebase onto that and then and then we make the you change the bits that you're adding based on that. That.

B

Definitely feels it could be helpful because yeah, it feels like a lot of kind of the like there and back with the actual implementation is really driven by like not having everybody on the same page, it's like what are the use cases that we're trying to enable with this. What is this intended to represent?

B

What is this not intended to do, and like is this reflecting the things that's been figured elsewhere, or is this a way to actually explicitly configure something so yeah getting the getting those questions all in one place, I think would go a long way towards making the implementation part a little bit more straightforward.

E

I I, don't think anyone disagrees that we need TLS settings, I mean TLS. Settings are probably the most one of the most important most complicated things. uh The argument is how to do it in a way that you know it's not repeating mistakes that other people did in the past, including history.

A

We are over time, um I. Think, there's a lot more to be talked about.

A

uh Unfortunately, I'm really, sorry that you won't be a kubecon this year Nick, but um there will be I'll put I'll I'll get something posted if we haven't posted it already in the slack channel for those of you who will be at kubecon, I'll, probably post it like Monday, we will have some space and some time on Tuesday just for us so uh more details to come on that, uh although I think Rob might have posted at least that we're doing that before so.

D

Yeah, so I will try and have some stuff done on this before next week, so that you'll have some stuff to talk about. Obviously, I can't be there, and the time would be even for me to dial in it'll, be like three o'clock in the morning or something like.

A

That right, so we could portal you in, but then it would be the other side of the earth. So yeah.

B

A

D

And with all of the lying around and healing we're not supposed to be doing, I should not probably be getting up at.

A

Yeah lie around and heal, dude I feel so bad man, that's a pretty bad injury. All right cheers everybody thank you for coming and if you have any follow-ups, feel free to jump into slack and just ask away have a good night. Thank.

C