Kubernetes Network Gateway API Special Interest Group, 6 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Gateway API GAMMA Meeting for 20221206

Description

Gateway API GAMMA Meeting for 20221206

A

A

And welcome to the December 6 meeting of the Gateway API gamma, uh uh as always kubernetes code of conduct for alternative effect. So please treat each other respectfully and with that uh we can get started my screen and if there is anyone who was present at uh the last gamma meeting, who was interested in recapping uh I would certainly appreciate that as I was out of office, foreign.

B

Ly, do a quick recap.

A

That that sounds great feel free to leave yeah.

B

I'm I'm, looking over the notes and trying to remember if there was anything that didn't make it into the notes that I thought would maybe be worth expanding on I. Don't really think so. I think the notes did a pretty decent decent job of it.

B

um Let's see poll 1493 we kind of got to the point of going yeah. We should just go ahead and accept this one and realize that we will probably have to uh I, guess I think. The way it was put was along the lines of.

B

We expect that when we start actually using this stuff in Anger, then we will find that there are things we have to change, but we're at the point where there was a reasonable consensus that it would be better to start trying to use it in anger and see what needed to change rather than trying to guess what would need to change before using it.

B

There was a really good point about the fact that we don't really get to mess with HTTP route as much as we might like to. um There was a little bit of discussion about the Gateway API 0.6 review.

B

um The most interesting thing that came out of that was some discussion about the possibility of the Gateway API people thinking. It would be nice to split out the front end and back-end functionality of a service in some different ways. So several of us were interested in learning more about that.

B

I'll spare you all the rabbit holes around HTTP 3 that conflicts are weird point in the dock um boils down to nobody's terribly happy with the idea that we just say a consumer route always takes precedence over a provider route. But again it's probably a reasonable way to get started right now and then see what things change, um because conflict handling is strange.

B

uh uh What was the other thing.

B

Headless services.

B

Headless Services had some interesting conversation because it's not entirely unstreetforward to think about how to deal with headless Services, where only one headless service maps to a given set of back ends and as soon as you start talking about overlapping Services, it becomes somewhere between not straightforward and impossible, um depending on lots and lots of things. There's a note in here about we're going to be chatting asynchronously about this I.

B

Don't think that actually happened, um everybody was okay with saying and well everybody was everybody accepted that defining the overlapping endpoints Behavior as implementation specific was okay, and there was some question about whether the non-overlapping endpoint case should be implementation, specific or extended, or what.

B

uh There was also a note that implementation specific and extended have carefully defined meanings, namely that extended means there are compliance tests and implementation. Specifics means there are not compliance tests, so there there's an important difference between those two and since Shane is on the call I'll ask him to keep me honest about my sense of those two definitions.

B

Okay, maybe Shane's not actually on the call, maybe.

C

He's not I'm here, I'm, sorry, no, that that definition is correct. Extended is when implementation specific is, when you don't have conformance tests extended is when you do plus it's where we put things that actually have more than one implementation. So it's not just conformance tests, but it's implied.

B

I believe also am I correct in remembering that a given implementation can be conformant without executing the extended the conformance test for extended Behavior um correct. Do I, remember, okay! That was the other thing that I found important, because that was the difference between extended in court right.

C

Yeah correct it can be just core yeah sorry. Somebody else was.

A

Talking about the extended you can pick and choose like, we might Implement one or two extended features, but not all of them.

B

Yeah, but that was a thing where um it was clearly not something that everybody on the call last time had already internalized. So I thought that distinction was worth calling out again that when we say things like core and extended and implementation specific, um there are very specific meanings attached to those it's kind of like the difference between should and May and must, if you're reading, an RFC and it's important to recognize that those don't always overlap exactly with their English definitions.

B

um I think that covers everything am I, the only one who was here last week, I think Lehman was right, um but uh you.

C

Were the only one on the call just did the whole agenda yourself.

B

Sorry I believe that uh last week, well, okay, looking over the lists.

A

Here looks like Leon was here too, but yeah yeah.

B

But also uh John was there cage? Was there um I'm just looking over the list of attendees this week, so I'm, not the only one. This week it was here last week, excellent.

C

um I just wanted to watch the recording of you just talking to yourself. That would have been.

B

Well, you know I mean you: can we can set that up anytime you're, like um it'll, probably be a.

C

Really Gateway API I.

D

B

Lot to say, but yeah if anybody else was on the anybody else who was on the call disagrees with my my recap there. Let me know.

C

Great recap: thank you for that, because I wasn't there either! Thank you. It's very much appreciate it. Cool.

B

Makes me really glad we kept the notes from last week. So all.

A

Right, yeah yeah definitely uh I tried to check out the reporting, but it looks like it. Hadn't quite been posted yet last time, hopefully, that'll be all Now by hours.

A

E

A

That is our agenda for today, um uh yeah I guess um Shane. Is there anything that is relevant to this group that we're trying to get wrapped up before the Gateway API 060 release.

C

I can't think of anything off the top of my head. That's specifically relevant, um hang on now, you're making me rack my.

F

Brain yeah I know I, can't think of anything either. um I I think we're in a reasonable spot right now, but uh there's some longer term discussions to have uh in preparation for 070, where I think we had a goal of having some mesh. uh Some mesh Concepts well thought out and formalized and fully approved at that point.

C

Yeah and then just for this, it's kind of not a uh a v060 thing, but we did decide last night between the gamma maintainers and the Gateway API maintainers. So we're going to make a gepster just for gamma to help uh kind of keep things focused on gamma in one place, which also will help with making sure we can assign reviews more uh appropriately.

C

A

What's that um this is an open gender. um If there's any topics that anyone uh would like to discuss at this time, uh please feel free to uh add them to the agenda. uh We're happy to make some time for that or I will find some stuff to follow up on, but uh yeah we can certainly uh we have time so any topics that are on people's minds that we would like to spend some time discussing. uh Let's go John, you will start with conflicting protocol roots.

E

um Yeah actually I realized I, maybe have two things um so there's been kind of two open PR's for a while on enhancing the Gap. The first is the namespace boundary stuff, um I think Keith approved it, but it's not actually approved I, don't know if that's because either through GitHub, instead of the slash commands or the Nissan house, um but I think we're. We established that we wanted that to merge. So if someone could take a look at that, uh we could do that.

E

The other thing, though, is what I want to talk about was the other one which is allowing non-hdp types, um which is this conflicting thing, so the main open question for that is what happens when we Define multiple different routes of different protocol types for the same service. So you can imagine it. You know TCP route and HTTP route for the same service and same port, I. Guess specifically, this related issue on different ports. It's probably pretty easy to resolve right.

E

um So this is something that's not really discussed in Gateway or Ingress, um which is largely because you have the listener to kind of tell you which protocol the routes need to be um I. Think it's not currently supported, like you, can't actually have a TCP route and HTTP route for the same listener and Gateway, because the protocol and The Listener defines what routes are valid and we don't let you attach a TCP route to an HTTP protocol listener, although technically it would work if we wanted to Define it that way, we didn't so.

E

We sidestep this problem. um We did reintroduce the problem in Ingress with grpc round HTTP route, though both of those can be defined on the same port in we need to do something there. So that's left undefined um and mesh is entirely undefined. So now the question is: how do we want to resolve those conflicts?

E

um I've been talking for a while, so I'll put out some ideas, but then I want to see to the other people. um You know. One idea is that we just have a strict hierarchy of some protocol, takes precedence over others like issues who are out then TCP routes, then Qs routes or whatever order we want um I, don't know we could do something else.

E

I wanted to get some feedback, I think for HTTP route and grpc route. Well, that's kind of outside the scope of mesh, specifically how we should handle that it should probably not just be like one takes complete precedence over the other, given that to your PC route is effectively an HTTP route, but I think we probably don't need to decide that here and we can inherit the decision from the English side. I don't know. Does anyone have any strong thoughts on on this.

B

John do I remember correctly from last week that I seem to remember vaguely some discussion about the Gateway API generally seeming to favor more specific things over less specific things am I, just imagining that, or was that a thing that we talked about last week.

E

I, don't remember it, but it's been a long time since last week, so I'll. Let.

B

You know it's been a whole week: I mean yeah yeah I. Have this I? Have this really vague feeling that one of the you know, one of the principles we talked about was that Gateway API already had this more specific, ever less specific idea, which could provide some help in terms of resolving conflicts like this.

E

Yeah I guess the question that I would have is: do we? Let's? Okay, let's say we have an HTTP route in CCP drop, the same port right now, basically, appearance, obviously more specific I would say um so we want to match the first now.

E

The question would be: if the route doesn't match, then do we fall back to the TCP route or not um it's technically implementable, either way it becomes potentially a bit weird, because, if I'm defining a TCP route, I might assume that my traffic is being TCP proxied, but obviously we need to actually terminate the HTTP in order to tell if it matches HTTP routes.

E

So you end up with some kind of weird Behavior.

E

um It should be implementable. I would imagine at least an Envoy um like I guess how you would implement. It would be like if there's no, like you just add a wild card to http match, which then becomes a TCP match um again I'm, not trying to focus too much on religious. What I'm familiar with I? Don't think it could do that for TLS, though uh like if you missed the all the HTTP matches, there's no way to then like go back to do a TLS match an Envoy.

B

E

B

Because you have to already terminate the TLs connection to be able to even start receiving the HTTP headers so.

E

That's what I thought at first, but in this case I think the TLs route from mesh would be passing through the user's TLX right, not terminating. That's interesting, we're not terminating egress TLS in mesh.

B

Yeah, that's a very interesting question. Actually I.

A

Feel like chaos. Fruit is almost like a separate question because it's so weird.

B

D

I think and kubernetes are kind of acting as a Distortion for for very simple things in the internet and uh the the fact that we can Implement whatever written, voy and kubernetes may allow all kind of weird things. You know that's interesting, but in reality, when application Port, you know you see that HTTP or TLS or tcpl has a clear intent. And then that means whoever is providing.

D

That service can tell people what it is and they don't need to guess from other things, I mean I, don't know how it can be expressed, I mean with application portal, but clearly that whoever the person who needs to decide what Portage is a person who's publishing that particular product and they need one way or another to express.

D

What was that word to declare what the port is, and there are different ways to do it, but I, don't think guessing and resolving complicated route and and using this authenics or whatever the employee can do is is really the solution.

E

B

D

E

That means that you would be in favor of. If you have an HTTP route, then you only use HTTP routes.

D

I think I think the owner of an HTTP service should somehow declare that it's an HTTP service and it doesn't matter. If you have a route, you don't have a route. You have multiple routes, it's whatever the owner of the service declares because they know exactly what they're exposing they know that their application is expected and whoever else wants to talk to TPP doesn't matter, I mean it's, it's really content of the publisher, because nobody else knows what applications that the publisher.

A

D

A

D

A

Fallback Behavior by just having a Splash World on the HTTP route, like if that's behavior that you really want you, don't you shouldn't, need a separate TCP group to do it. I think.

B

I tend to agree with Customs point that all the prior art I can think of involves a port having a single protocol associated with it and being able to do things like. Oh, if we can't route TCP or account route HTTP, let's try it as a raw TLS route seems or a raw TCP route, either way. That feels a little strange to me.

B

um Having said that, the grpc versus httpsing is a little more surreal to me, like I could conceivably imagine, maybe possibly a little bit that if you don't get a valid grpc method, you just route the rest of it as HTTP, um but even there I'd probably be inclined to say, that's asking for trouble more than anything else.

D

But the grpc's https I'm not talking about what you do over HTTP, so you can have an https Port that is multiplexing multiple things and that can be HTTP because again, as there was a discussion about uh who's, you know over TLS who decides what protocol it is if kubernetes, whatever route or whatever or is there a negotiation and on the Internet? Is your negotiations? I should decide. So once you do a LPN, you know it's HTTP I mean if opportunities to stls, then you know if it's HTTP it can be.

D

Tcp can be a lot of other things and then you go to to H2 and it could be grpc and grpc just means that you use service, slash method. So it's really HTTP I, don't think we are talking about. uh You know kubernetes apis overriding, the internet standards, for you know, LPN, negotiation or or for whatever user wants to expose on HTTP port with grpc or build plane http.

D

In other words, you should defer to the internet standards and just you know, specify the minimum that is necessary to express the user intent.

E

So would it be sufficient to just say that if you have multiple different routes, route types for a single service, we pick the most exact type, which would be HTTP, then TLS and TCP I would imagine and on the least specific ones. We emit some sort of conflict rule the one concern I have with. That is that you maybe have a TCP route. That says, like I, don't know natural 480 and send it to XYZ, and someone adds an HP route for something specific and suddenly the TCP one's wiped out.

A

So I think it wouldn't be wiped out if we follow the guidance from the like route, like the other conflict rules that the oldest.

B

Song first prioritizing.

A

Existing things.

B

What are the oldest thing? First rule yeah, yeah well,.

E

In the existing logic, it's that's kind of the discriminator of last choice. True if two Protocols are or two routes are completely equal in terms of priority and then it's based on time, yeah I mean we could do that, though we could say the oldest protocol wins and conflict, and then, if you want to change it to http, you apply your HTTP routes. They do nothing and then you delete the TCP routes.

E

That would probably be safer. um Yeah.

B

The I guess the.

B

Sorry I was thinking you know imagining a case where you have. uh You know a catch-all HTTP route with a slash as a prefix, and then somebody adds a more specific prefix in that one. Suddenly, your catch-all isn't routing all of it, but I think that's a fundamentally different thing. Sorry Rob, you were going to say yeah.

F

I would just you know, one of the things we've really struggled with in API usage elsewhere is how we handle or what we expect from stateful controllers or if we expect controllers to have state uh one of the concerns of any approach that you know requires. On the you know, the the it's one thing to say: the oldest route we've done that it's a different thing to say the route that was that matched this first.

F

Basically right, uh you know you could, you could say the oldest route resource, but it may not have been used to match a given service or destination until more recently, um similar with Gateway similar with anything uh class tonight, maybe I'm misunderstanding. So all you're saying routes are not the source of Truth. Here.

D

Yeah, that's my point: I mean it's not just the route that determined what there was a protocol. That was my point, so it doesn't matter if the TCP route is attached or not it matters. If the user says that hey I have a port that it's TLS and on TLS I have HTTP, 2 and and other things, that is a sort of two things returning for out is allowed or not.

D

So if someone specifying a router is not compatible with the intent of the user, it should be rejected not to the other way around where it's.

F

So to clarify, are you talking about the use? The service specifications and and app protocol am I missing. Something.

D

Is that I don't know I mean I'm, not clear? Where is the right place to put the intent of the user in the service in the Pod itself or some other? uh You know attachment I'm, not sure about this, but I. My feeling is that it needs to be something where, as an owner of application, publishing for 8080 I can specify that what 88 is the http.

E

But we do have a mechanism do that without protocol on the service right, which one the service port app protocol.

D

uh That's that's a good one, yes, but is it what we want to do I mean do we do we want to require a protocol to be used and specify HTTP or something syntax to to identify? Well.

E

That was that was my concern like what happens if they either don't set it or they set it to I, don't know, say TCP, but then Define HTTP rep. Do we reject that and make them sync them up? Do we except it only if it's unset or you know, we'd have to it.

C

Could be done for.

E

Sure we just need to Define what it means to have those interactions between the.

D

E

Around having a protocol and the service having a protocol, yeah.

D

Given that there is security implications, I would say you know: I mean if it doesn't matter, try to put an HTTP route on a port that is declared as tcps and safest thing is to reject the HTTP route. No.

E

E

And like one yeah like one case, I could think of is if you are a consumer route, you may want to apply HTTP route, even if the producer of the service doesn't declare themselves as HTTP right, I. Think.

B

E

It's just some third-party service that they don't. They didn't bother to set up the protocol, but you know that it's HTTP, so you want to do an HTTP route. You can't really tell them to go set the app protocol I.

A

Think that's the same as how it would be with listener. If you just don't set a protocol, then it allows anything to attach to it if it's unspecified.

E

Isn't it required end listener, though,.

A

I, don't recall.

D

Yeah is that maybe a good compromise.

F

It's it's a little confusing to me because it it seems like there are going to be some Services out there, that you could use TCP, grpc or HTTP route, all to route to the same destination service and all could be valid even if they're, not equally useful or configurable, and only say, I.

F

I think what we're describing is just a way of a way of defining preference here, but not disallowing other route types, but I just want to make sure that we allow any valid route type to route to a service and not like prevent that based on that protocol or something something like that. I I think that's what others are saying, but I just want to be sure.

D

I completely I mean if it's an HTTP service, it should allow both erpc TCP and HTTP routes. But if the TCP service, you should only allow TCP route, because it's clear is the user explicitly say it's a TCP. That was my but.

C

E

D

The TCP route and then HTTP routes and apply and because it may have Port based you know, ranges and whatever key ideas throughout on as an HTTP will do the higher level routing, so they are compatible with each other foreign.

E

If I had an HTTP service and I had an HTTP route, matching slash, Google and TCP route, just matching the port in terms of how we would configure that in, let's just say, Envoy that would become a two HTTP matches, one on slash, goo and one on Match. Anything essentially like we basically turned the TCP route into an HTTP route. Effectively applies.

D

D

E

Yeah, my only concern with that is in general, the API favors more specific match, first, which I guess I mean comparing app was burned, is used to say, what's more specific, but to me HTTP seems more specific than TCP.

A

I would think so and I would think gearbox.

A

Also, the like more specific thing, applying conflicting out the like TCP one like it isn't the worst, because at least it fails fast like.

A

I would I would think that you would see the change immediately when you went to apply it and assuming you're following Quest practices of buying stuff in the state requirement. Of course, like it.

D

Should be an apparent.

A

Change, rather than something that is like implicit Behavior, that's difficult impact.

B

uh Mike FYI checking the Gateway API Gateway type. It looks like the protocol is required in The Listener, section. Okay,.

A

Thanks can I have multiple protocols or is a single one required.

B

Looks like a single one: okay,.

A

That makes this a little easier, Maybe.

B

Yeah protocol type is a string. It can only just be the one, although I don't know, maybe there's some. uh No. You can't do anything like HTTP, comma, TCP, comma. Whatever.

F

Yeah right now we are tied to a a single protocol and you can imagine a protocol like uh I I. Don't think we have a way on listener right now to specify a grpc anyways. But if we did, you can imagine something like that would be consider you know compatible with multiple route types.

A

I mean if, if we already have prior art around, you should have a single protocol attached to a port. I. Think that kind of points Us in the direction of doing something similar for gamma.

A

Rather than trying to do something new with murder,.

B

Yeah I would tend to agree with that.

B

Just on the principle of least surprise.

B

B

All right, I gotta, ask John what kind of keyboard are you typing on.

E

uh Sorry I was some.

D

E

Yeah um so I guess if.

B

We go speaking of somebody that I'm pretty convinced my keyboard is audible too so yeah.

C

If I'm not mistaken, that's just a basic plastic keyboard. You just hit it like crazy with your phone all right. What my.

E

My phone's, like literally an inch away from it so I'm kind of weird, set up right now, um so I guess. If we go with what Mike said, which I I think I'm tending to agree with that um it at least makes things simpler. We could always expand it later right right. This is kind of a more restrictive option and we still just need to decide. Do we go with all those protocol wins or most specific protocol wins right or a third option, but those are the two that I've heard mentioned.

F

What what do you mean by oldest protocol.

E

uh So let's say: I first applied TCP route and then later an HTTP route and when I say later, I mean in terms of creation, timestamp, okay, okay, that's so yeah, which one wins basically.

D

E

It's more specific or is it BT speed.

C

Route because first.

E

And we don't want to wipe it out accidentally right.

D

For both I mean or both apply, that's the other option.

F

Yeah sorry go ahead. Rob I I would step back a second and say that even though we we hit, you can specify a single protocol on Gateway listener.

F

We specifically have put a lot of effort into supported route kinds right, so you you, as a consumer or as a user, can specify these are the kinds of routes I want this listener to support and the implementation can tell you what kinds it actually supports in status, uh so you can set an HTTP protocol and I.

F

Imagine at least some implementations are going to take that and say I support, both HTTP and grpc Route I, so I think there probably is going to be some scenarios where a single listener supports both I, don't I I, so I, just I, don't want to go too far into this idea that we only have a a single protocol per listener because it is a single protocol, but not necessarily a single route type.

A

Do we have any thing explicitly written about how that should be handled for listeners in north south Gateway, yet.

F

I have been trying to dig it up. I somehow I thought that we had said it was mutually exclusive to start, but I can't find that written anywhere.

F

uh Grpc route was really the first thing we ran into and it's experimental uh so I. You know as much as you technically could mix and match. Tcp and HTTP I don't see a clear use case for it um maybe I'm, just not creative enough. Actually.

D

No I I'm starting to to see some use cases I mean given that TCP route is based on cidr ranges like the addresses and comparative City level. There is a valid use case for someone to apply PCP routes for an HTTP to hdb traffic, so that is deny or send traffic from Russia to devnal or to some specific server to really apply some TCP routes first based on cidr ranges, and then the HTTP route should apply like with normal.

D

You know viral separation when you apply the route statistical routes, maybe at the router level, don't even.

D

Just just translate them into firewall rules and to into routes basically, and then HTTP also combines and grpc it's effectively. You know you can have an HTTP setup exposes both grpc and HTTP, and it really is not under the control of the routing crpc route can be translated into HTTP routes. They are equivalent synthetically.

D

So basically you apply them in order from the from the you know, the layer order, basically they apply first apply. Tcp is an approach, HTTP routes, then you apply grpc routes.

F

Just to clarify we, although we very temporarily uh experimented with IP range matching in TCP route, it does not exist today.

D

F

Agree: it's like it's very likely to to come back so I that that's a good example.

D

Yeah but we'll go for security, I mean you want to to. You, know Russia somewhere over, for you know the region or you need to keep traffic Regional process Europe traffic. Only in Europe and whatever is called the regulation.

C

D

John, what do you think are you, okay, with this approach.

F

Maybe we lost him I'm, not.

E

Sure, sorry, if you're talking about me, I did go to a different meeting room. So you did lose me for a minute.

F

uh No worries uh yeah, we're just uh curious, I get well, maybe costing you can. You can ask better than I. Can.

D

I know I I wonder what do you think about about applying guys apply for CCP routes, apply zonage to keep your routes and Xena plays grpc routes, just like Keenan boy, and you know how it will happen on TCP routes are applied by a different system and a router, and then HD browser applied by an HTTP load. Balancer that doesn't know grpc and finally, grpc routes are applied locally.

E

Yeah I, it's part of me likes it, um because it's much easier to apply TCP routes first and then fall back to http routes. Part of me doesn't like it because it kind of conflicts with the idea that most specific route wins and I I mean I, don't think we need to solve HTTP and grpc Route precedence in mesh, because it's more of a general problem but I tend to think that they shouldn't be so orthogonal.

E

It seems a bit odd that, like all HTTP routes are, have higher precedence under your BC routes or vice versa, um but I don't want to get too hung up on grbc semantics here, necessarily.

F

I I, like the direction you were going, I think it was you John that you were going of. Basically anything from a lower level can be translated to some kind of common understanding right. So that's especially clear with it. You know grpc and HP, which are basically going to be translated to the same set of matchers yeah, and you can kind of stretch that and say well. Tcp could largely also be translated to the same set of matchers.

F

uh In most cases, anyways.

B

So TCP could be translated to the same set of measures as what.

F

As as HTTP, if, if TCP right now is limited to Port matching I, don't know it depends how far we go.

E

Should always be a superset of TCP right, even if right, because again.

B

I point out: HTTP 3, you guys what Fair yeah.

D

B

I mean the the basic idea that an HTTP route should be more specific than atcp or UDP route makes an enormous amount of sense to me.

B

um The idea that a grpc route you could consider to be more specific than HTTP or you could view it, as you know, their peers, because grpc is HTTP, um but yeah H3 really does mess with a really ridiculous amount of stuff. In what we're talking about um brief aside here, Rob was at gep 724. That was talking about the supported route kinds.

B

F

B

Check but yeah that that's the only thing that I was able to find, but um the the concept that.

B

The idea that a Gateway can say hey these are the kinds of routes that this listener can support. That kind of does make a lot of sense to me, too. I'm just not sure how we would convey that in mesh land.

F

If there ever were a Global mesh resource or a cluster-wide mesh resource, that would be a really convenient place to say hey by the way. Here are the things that I support.

B

I certainly do like the idea of adding a status to say: hey, you know this particular route is going to be ignored because you did something incompatible with other stuff um I'm kind of leaning towards we should Define a specificity and then let the most specific thing win, because that feels to me the most consistent with existing things out there in the world, but yeah I, don't know it's. A tough call is.

D

It consistent with other things that work, because in the world you know, TCP routes apply at router statements that that's my point here: that in the world there is no the most specific winds, but they're combined I mean they are playing independently and in.

B

My world I have seen a lot of places where you can't have an HTTP route and a TCP route on the same port.

D

Pronouncing different equipment, I mean different servers. Different admins different series of TCP ad means that it's dealing with bgp and routing and routers and firewalls, and there is no HTTP.

B

Yeah in my world also I am most accustomed to looking at things from the perspective of an application. Developer and bgp is not something an application developer thinks about. Bgp is a thing where they trust that the person who set up IP underneath them did something sane right.

B

um So from the perspective of an app developer, it's very natural to think about things like oh, hey, Port, 573 is already in use for a TCP route, so I should probably not try to put my HTTP server on that port, and you know that's a that's a pretty sane thing. Likewise, people generally don't try to do raw TCP routing on Port 80 when there's also an HTTP server there. So um you know I'm, I'm, biased, 100, biased in this, by the way, like I I, recognize that.

D

I only agree with you and- and that used to be true when the internet was plain text and supports were simple, but nowadays with uh you know, require you know a lot of people Multiplex, almost every single over https for gateways, that's actually not for gamma, but uh where, because it's very common, other ports are blocked because friendly isps and and.

C

Yeah everyone is.

D

Sending everything over over https and and you know, even websocket, on top of it and other things so and by the way websocket route is probably going to be one important. One.

B

Every place I've seen something like that happen, then, by the time the Ingress controller gets a hold of it, it's being demultiplexed.

C

B

Because, and that implies that by the time gamma gets a hold of it, it's being demultiplexed as well, because by definition, gamma is dealing with the inside of the cluster and I. Don't see people doing that kind of multiplexing inside a cluster.

D

I think that's the difference between gamma and Gateway I mean for Gateway. Probably is a composition. I mean.

B

I I could certainly imagine an Ingress controller that needed to do the demultiplexing for sure um I haven't seen it in practice in my own career, just because of honestly, because the stuff that I've been doing in my own career is from the perspective of the application developer, and we tried really hard to not make them think about stuff like that.

B

um But yeah I. Don't think that that's a thing that gamma is going to need to to worry about. This is, of course, the point where John and Rob stand up and go.

D

Oh no, no ambient.

B

Mesh has to do it, um but maybe not.

B

Foreign, so again, my my strong bias is start with the perspective that.

D

B

You don't get to run multiple protocols over the same ports, just pick a protocol for report and you know, go with the most specific protocol if there are conflicts and tell the user that something went wrong. If there's a conflict, that's that's driving.

E

Too, as well, I think it's very uncommon for this situation to actually happen, and so rejecting it with a clean status message seems probably sufficient.

D

I would agree with you. My only concern is when we need to implement that cidr based stuff for everything, because there is a customer is required by regulation to do whatever. How do we do that, then? We start cutting crdr ranges to http route to express this requirement, or what do we do because normally is a cidr, it's better to be isolated. It's IP routing in that particular use case. It's you know clearly exist because.

E

Well, which one takes precedence, is it the cider route or is it the HTTP route yeah.

D

Of course I mean that's that, because cider applies for them in in normal internet. If you forget kubernetes, that's how it works, you have a router that is applying cidr running ages and then it goes to load balancers. That is doing HTTP.

E

I, don't know I mean if you have like Linux routing table and you define a route for uh 32 cider and a graph or a slash, 16 cider, the more specific one wins right: basic peer routes effectively bound to a single IP address, so that seems more specific than a range.

B

uh Sorry can I just to double check cost in. Are you and I apologize I feel like I missed some of this earlier? um Are you thinking about cider for Ingress, routing or egress, routing or East-West routing.

D

uh All of them is a use case with keeping you know. European traffic in Europe, for example, applies for everything, including in in in network. So you are saying everything coming from Europe will be routed to supporting Europe and that apply. It is best expressed with cidr ranges or network policies to deny everything that.

B

Implies that you have a single IP space that spans multiple Geographic sounds.

D

Ipv6 probably allows that long term I, don't I'm, not so worried about the space, but uh my my real I mean. Let me let me tell you why why I'm I'm prefer this option? Is that what we are doing in history with Envoy? Combining everything is one option, but in reality you may want to to split it to different layers, so you may want to put all the PCP routes to translate them into.

D

You know either program whatever below you know, public you have for for kcp and cni, so your VPC and then translate the HTTP allows to program some. You know front-ends, that are you know, multi-tenant or whatever, and then applies the grpc routes at a more specific uh load balancer.

D

So, basically, you can call in a way I'm behind this, a good model for that, where you have you kind of separate the layers again instead of combining them, and for that, it's better to to to to to have a way to clearly separate what goes to the you know underlying fabric. What goes to the load? Balancers that are sharing what goes to the most specific load, balance waypoints or whatever? We call them.

B

I think I would really like to see a.

B

I, don't know if it's a use case or an example or case study or something but as I think over what you're talking about.

B

It's really tricky for me to figure out how exactly that Maps into the kinds of Concepts that the people who are doing work with existing kubernetes resources are going to be lining up with I'm, not sure. If I'm doing a great job of explaining this but I feel like there's a.

B

Yeah I feel like the existing okay first off I feel like we are in fact talking about kubernetes at this point, and second I feel that the um the existing kubernetes resources, possibly implicitly, are modeling some layers that they are assuming are not going to be combined and then, and then you know, then we split out.

B

um So there are a lot of ways that I feel like the case that you're talking about May itself be a little bit at odds with the way kubernetes is put together. So I would really like to understand the use case that you're talking about better to to understand where I'm missing stuff.

D

Yeah I can try to write something down there. I mean I'm, not strong about them, I'm, okay with because, in reality, I suspect. What I want will happen anyway with separate apis, because no matter what we are saying, people who want to configure IP routing will be able to do it at a higher levels outside of kubernetes, because that's more multi-cluster and- and uh you know, yeah regions- you don't have to do that. So it's not necessarily much into kubernetes.

A

Yeah I mean I really tend to like skew towards IP based reality like that is a liar but outside or outside of the typical mesh concerns. Yes, that's.

D

Actually, a good point: that's a very good point, widely bother.

B

With it, I mean a lot of my gut reaction to the problem of using cider to keep European traffic within Europe is okay. Why don't I just have a Europe cluster and then I have a cluster, that's in North, America and then I have a cluster, that's in Asia and then I could just get to rely on Kube. To already do this for me in a lot of ways, um then you get into many more interesting such a nice situations around. You know your one of your services in the Asian cluster has died.

B

So how do you route? Those to you know and that's an entirely different animal and.

D

Some Services may be required to stay. Other may not be what kind of use cases yeah! That's why we need some control, but.

B

Yeah, but those are things that you could do already with HTTP routing as long as you understand the way, multi-cluster Services work and things like that, and and that's an entirely separate Rabbit Hole to be dealing with external services and multi-cluster and mirroring you know, and all this kind of stuff, but I, don't think that we need to build.

B

You know full-on cider routing into the HTTP route to make that work. No.

D

No, absolutely absolutely. My point was that to avoid about bending the idea routing in HTTP, we better to combine them instead of having more specific, win.

A

I think so we as time travel about five minutes left so definitely want to try to like wrap this up.

A

um I just want to kind of close with something of I think it may be helpful to get like a example of like how would you express an application that serves multiple protocols through a spec like that through aspect like this, whether that's um separate routes, um whether that's like different Cube Services, whether that's a single service with multiple ports, specified like let's work through like a kind of concrete example like what does this actually feel like in practice to write? um Does that feel useful to folks.

B

I, pretty much always think it's useful to do that exercise.

A

A

um Is there one or more next steps with what we want to do like.

B

A

E

Long discussion that covered along the.

A

Ground here and I'm indented like six columns or something like that, um Are there specific next steps uh that folks want to take on to drive a decision of some sort on this and again this is all falling out from which issue not that one, the other one.

C

D

Seems to be sorry, it seems to be to to have more specific and I'm happy to withdraw my uh Concepts. When we can fix it, I mean we can. We can change it later. It's nothing compatible! Change.

B

But that's pretty much what I was going to say. I feel like we have a consensus towards more specific wins user status and.

E

uh Let's see, those are the only two open issues right now for the mesh stuff, so we're probably out of time here but next week we should probably consider, if there's more things we want to do or if it's ready to Market as implementable or what's kind of the next steps.

D

E

Have three more minutes, John.

D

Let's do it now, does anyone know anything else that will prevent fast move forward.

B

Anybody who thinks we shouldn't just go ahead and try it out and see what happens.

E

Is what is the actual General pattern Rob or someone else, that's more familiar with like yeah the market is implementable and then go implement. It do I, implement it first and then Market is implementable once I prove that it can be implemented. uh What do we do.

F

Yeah so I I would say we we would want to Mark something like as implementable and uh actually usually we would move it to experimental, Channel I think one of the things that we still need to follow up back up with is when we did the 060.

F

When we started off with o60 review with the broader Sig Network group of reviewers, which were specifically Tim and Cal, we got some feedback that here attaching to a service as a parent was on unexpected, uh and there was you know there was some related effort uh coming coming through a Sig network of pulling out some of the pieces of service.

F

The API like uh IP address allocation as well as DNS resolution into something else that could be attached to Gateway or any other thing that could be very relevant to mesh I, not sure that that's a blocker to implementing something in an experimental state, but it's definitely something that would be a blocker towards moving much further than that uh I. Have it as an AI to try and set up some kind of follow-up discussion on that there are some people like.

F

Basically, we need to Loop in people from Sig Network that are working on these things, with people from gamma and Gateway API, and try and have some big discussion about all this. It's hard to schedule, something with that many people, but that that is kind of The Next Step. There I think we need to clarify how much of a concern that is, whether that's a blocker or just a let's.

F

Let's keep on discussing this while we consider it experimental.

B

F

B

Before about that, that work with splitting service out implied that that was a timeline of months which feels like it would be really rough to block everything. Until then right.

D

Yeah I I would it help to to change the language to say itself we're attaching to service? We cannot touch to service or equivalents teachers, so basically it's do not say which strictly a card to a service, but we can attach to any object that represents group of workloads yeah. So in future, because even if we Define this brand new split people take a very long time to adopt and we still want to be able to express that you attach the service. If only service is available.

A

I I think that's something that we have to design space to be able to do like the the field in the spec is not named service. It's parent wrap.

A

um If and when a DNS name, parent graph or DNS name resource becomes a thing we could add something to gamma to allow attaching to that instead, um but I would definitely lean towards that we're trying to block something on something that may be months or years.

E

D

I mean I, don't think service will go away, so D1 API it will never be removed. So we'll always have some use case of an actual service.

F

Yeah so so to clarify: I I think that's unlikely to block, uh but we have not considered anything uh for the sake of o60. We've considered this an informational, only review on gamma. We have not tried to include any part of gamma in o60, so I I, don't think anything's a formal blocker, but it you know we will not consider any gamma spec part of the 060 release as a result, so we can provide some guidance of hey.

F

This is a thing that we are considering experimental, but it's not going to be part of the l6o release, so yeah.

B

I think that makes a lot of sense.

A

B

A

To me, cool all right uh with that, we are out of time so we'll wrap it up uh as always thanks. Everyone uh definitely appreciate discussion and thank you.

A