Kubernetes SIG Network, 25 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Network Policy API Meeting for 20230425

Description

Network Policy API Meeting for 20230425

A

Awesome, uh hello: everyone today is Tuesday April, 25th 2023., it's the week after kubecon uh EU, so folks are still recovering from that I'm sure this is a meeting of the Sig Network policy API subgroup to sing network uh as all other cncf meetings. We are sponsored by them. So please be nice and kind to everyone and let's try to follow their rules um but yeah. Otherwise, that's about it we'll go ahead and get rumbling today.

A

um Just before we dive into some stuff from the agenda, uh I don't want to put you on the spot, Syria, but or Shane yeah. You all went to keep con last week. Were there any good conversations around uh the admin, Network policy, API or anything else. We're working on uh that may be applicable to the group.

B

Yeah I think it was definitely an English experience during this right, so we had our Sig a talk for the policy AKA spots of sign Network as well, which is on unfortunately on a Friday evening as usual. So a lot of people had left but I think.

C

The value that.

B

I found from the conference in relation to at least the A and P bits is just talking to the other cni plugins like sodium Calico and trying to get them to use this or convert their bits to us and see if they're having issues so it's kind of trying to track down them on their boots and tracking issues on their side.

A

Awesome good to hear yeah I'm sure, like a lot of the values. Just in the conversations you have so glad, I got a shout out um and let's just keep the momentum rolling out of it. I know we'll probably try to submit a um admin or policy talk for Chicago. Do you know when that uh cfp deadline is or is it even.

D

A

B

D

B

D

A

Sweet and our goal is to get that in not on the last day. This time.

B

Yeah I was also talking to Shane about the conformance profiles right if you can start and get that done for our group as well, it's something that on a common ground that we could showcases. So, let's see how it goes.

C

That reminds me, the Prototype that I wrote with Matia is now in a PR, so I'll send you that real quick.

A

Amazing yeah: that's his agenda.

A

Yeah I mean if we can adopt like that from the get-go instead of trying to raise our own path with cyclonus or something I know, it's solves slightly different issues, but still that would be awesome.

B

I'll, take a note of that. Yeah I feel inclined I'm actually doing it for Network policy, but I wanted the other part of it. The integration to be a bit more seamless like once, you run Cyclones and you get the tests. Then we use Shane's framework on publishing.

A

E

I think we're still seeing IDE rather than the agenda right now.

A

That's my bad thanks for letting me know no.

E

A

Too many windows going on Okay.

E

A

Shane, did you send that PR in slack or was it in.

C

um I just sent here in Zoom, oh right here and I- think uh contour and um possibly Envoy Gateway- might help us test it out as well. But we got it working at Kong with like it's producing a conformance report which, ultimately, in time we're going to have sent back to the Gateway API.

C

So most of like the just the really fundamental Machinery is there so might be interesting, since you guys are talking about maybe doing something similar here and then Surya and I talked at some length that kubecon about possibly needing to get this out of Gateway API kind of abstracted. The test Suite basically pulled out and abstracted a little bit and then put into kubernetes somewhere more General, especially if Network policy um starts to use it. So just keep in touch with me on what you're thinking and like what your I guess.

C

The more important thing is what your bandwidth is for doing such a thing, and we will uh I'll be on top of like making sure we we do that, because I.

D

C

Hate for us to all end up with separate things that do effectively the same thing. So.

A

No 100 um I think, as these implementations are getting done, I know sorry has made good progress and I know. Yang is cracking away at it or pretty much almost there. uh This will become more important, so I think once those are kind of at a stable point, then we can kind of recenter together here in upstream and say like how can we test these in a uniform way?

A

That's kind of where we're at I think.

A

A

Anything else on testing. We want to talk about.

A

Cool okay, so uh let's toss it over to Nadia to talk about a security Zone use case. So is this another new use case you're going to propose Nadia or you're thinking about, or you came up with recently.

F

uh Yeah, that uh is, can you hear me all.

A

Yeah we got you okay,.

F

Yeah, so that's uh one of the topics around the current API implementation right and um that specific one is around tenants and same and not same labels, and um you said that well we for now we, the API, only applies on the namespace level for same and not same labels.

F

Right because we said that we expect tenants to be based on the name spaces, and it doesn't allow you to do that on the pod, selector level and I, let's say, found the use case where um customers actually using this kind of tenancy, which is in its Essences a security Zone.

F

So uh only pods in the same security zone are allowed to communicate with each other and that kind of that forms something similar to a tenant. But it is based on a pod on pod labels. Instead of namespace labels.

A

F

A

Like you're you're, basically saying in the API previously, we only defined a tenant as a namespace or a set of namespace. We didn't Define it as a uh set of PODS that span May span, namespaces and you're, saying that that you found kind of a use case in the wild. For that.

F

B

We had a question on the use case right, so why are they not segregating those pods as namespaces, which is what 99 percentage of the users do right like? Why do they want to do this across namespace thing, yeah.

F

That's a good question: if how many users actually do that- and there are some notes around that so I know a lot of examples when uh cluster admin creates all the namespaces right. So there are not multiple owners of different namespaces that create their own pods.

F

So what they are doing is let's say you have five security zones, but you have thousands of pods in different namespaces and in that specific case, especially when a cluster admin owns all the namespaces, they may use namespaces a bit differently and I've seen them being mostly split, not on the ownership basis or something related to networking, as we are suggesting right, but based on some logic like a back-end service or some application that is running and they may have a lot of namespaces that are.

F

That logically means something to the workload and it may be difficult for them, especially for customers that have large applications or sets of applications running in the same cluster to actually restructure all the namespaces. They have now to just fit that requirement and I guess. That's also the case in this example: I'm sharing now.

B

But also in case, let's say we go ahead with this like we agree that there is a valid use case here, right, like in that case, even if same and not same is applicable on a namespace level. You still have that pod selector after it that you can use to then select a subset with them right from the API. Oh.

F

That's a good question because we haven't decided on how to use this same and not same labels. uh Finally I guess or like at least we have this discussion going. So it's hard.

B

To tell yeah that's yeah fantastic questions right I wish. The adventure was here because Dan and I talked a bit about this at chipcon, oh and and I. Think it's a bit early for me to even say this right, because we wanted to do a like done, wanted to do a proper presentation. I, don't want to speak for that part, so just stream, some ideas that we spoke about and I I think Yang already has a PR to implement same labels and answer yes.

B

I, don't know if it's too late to bring this up, but we wanted to restructure and redefine tenancy a bit. So we wanted to replace same labels and not same labels with us with a field called tenancy right at the top of the spec in the API right like along with the subject and peer that we Define and then the labels that we use to define tenancy right. So that will be the subset that you define your peers for. So you cannot do this cross uh relationships with multiple labels and stuff.

B

So, if whether it's the same or not same right, that relation will be very much tied to the tenancy that we are defining, which will be a set of levels, but I know it's a bit confusing, because it's still in the early stages like we just it was literally a random thought that we were writing on a pen and paper. So maybe Dan can share more thoughts in the upcoming meeting, but this is good discussion once again and it's it's not fixed in stone.

B

It's just an idea that we spoke about right and one thing that I think we can kind of agree on I, don't know young if you have thoughts here, is that there is still some discussions on going around these two things right same and not same, not being really clear at this point, so my question is: should we rethink this in the Viva alpha one part of the API, or can we maybe do it as a so remove it now, but then re-add it in once?

B

We have like um a better way of a definition here.

E

E

Go ahead, sir I I need to think through it a little bit more yeah.

A

I'm I'm kind of processing, I I, tend to agree that most of the um API challenges that like Nadia, has brought up recently and a lot of what we've been discussing over the past couple weeks have been with how we Define and select tenants. So if we could think up an easier way to do that, I'm totally all ears, um I think from a generic standpoint like procedurally, we need to release something soon regardless, like right. Now we still haven't like packaged or released and done it.

A

um Obviously, we have V1 Alpha One up in our repo, but we've been making like small fixes to it. If we're gonna start thinking about doing bigger things like I'd really like to go ahead and say, like we're done, small fixing, let's do a small release like everyone, Alpha, One, proper release and packaged unit, um and then we can move on to either everyone Alpha 2 or V1 beta 1., that's kind of where I'm at but I think. This is totally something we should think about, because it's leading to.

B

Yeah many problems, I had the same thought right. I, don't want the same and not same discussion to stall the release or getting an implementation up right, because right.

A

B

Percent of customers are actually happy with selectors, at least that's my understanding, right yeah.

D

C

B

10 person we still want to do that use case for tenancy on, but I. Think that's why I was saying if we can put this discussion for the next version of the API, but still do a release with the stable parts of the API that we know are like good. That's something.

A

Yeah I mean you and yang have already been working on the implementation, so I don't want to keep like switching it up like I'd, rather go ahead and cut hard cut, rv1 alpha one and then keep con. If we think we're ready to after you all have started implementing it and everything seems to be right so far, and then at least on the API side of things and then just keep iterating Upstream.

B

But do you mean you want to cut but same and not same the way? It is.

A

If at least the mechanism, the way it is right still using it, um if you all have already designed to it right, if you've already implemented, Melvin, kubernetes and Yang's already implemented it and Andrea.

B

Hi yeah I haven't but I know, Jan has I, saw a sphere so I. Let him speak.

E

uh Yeah, so I I just want to clarify some of the things right. So so first things. First um we're talking about the security Zone use case. um Something is still I, I, don't know uh I'm, not saying this to you know discourage uh Nadia or something, but something doesn't really sound right to me about this use case is that um is there actually people who put who, with different tenants actually put you know different tenant workloads in the same namespace and then do that across a bunch of namespaces.

E

So what you're saying is that you know for a admin right? Maybe he created the namespace ABC and then three Tenants come in and they just decided, oh for some, for my workloads for 10 and one workloads. I just wanted to put these workloads in the names is, you know across names, ABC and then the second tenant and the third tenant decided to do the same thing. Then you know it. It doesn't really make sense to me anymore on. You know why the admin had to create this namespace in the first place right.

E

So what does you know this namespace actually signal and how you know the cluster? Actually, you know look like that. The reason why I said that is that you know when we designed the adminable policy at the very beginning. We were kind of like everybody who jumped in the car. I know this is way before right.

E

Two years maybe ago, um Everybody kind of agreed that you know a namespace or a bunch of namespace combined, do Define tendency in you know most of the use cases in the in uh what we are seeing the customers do as of now so for whoever is asking for adminable policy. You know this is the thing and- and we also have you know, different implementations or companies come in and say hey. We also agree that you know the name.

E

Spaces be in tendency is good, but we wanted to have a easier way to do: selectors to select Tendencies and Define how tenancies can talk to each other, so this is sort of like the founding stone that we we build, the amp on and I'm just not feeling like really comfortable saying that you know we're challenging that at the moment, because I personally haven't seen a lot of the um use cases where um we actually mix tendencies in namespaces. Like that. So you know this is my first observation here.

E

I I, don't know, I, don't know. If uh Nadia did you find the security Zone use cases somewhere online or you you do know that you know um you have worked with customers or whatnot who actually uses this sort of like construct and and is there any a little bit more context you can provide on why you know a single tenant may want, to put you know, workloads in different namespaces and all the tenants will try to do the same.

F

Yeah, so I didn't get too many details about why the workloads look exactly like that specifically. But I know. As I said this customer they have a lot of namespaces and much less security zones. So.

D

F

Namespaces in general, I, don't think namespace has any networking meaning. Yes,.

D

F

Can try to use this tendency right so.

C

And that's also.

F

I guess what they follow and they use name they like use namespace as some abstraction that was comfortable for them to adopt whatever they were running out of the cloud like to just put that uh into our clusters right and they have, as I said many namespaces that are like they. They are logically separated workflows but and I specifically asked them.

F

If how difficult it would be to change how these workloads look like uh to make sure that the tenancies like names namespace based and they said that would be very difficult for them, so I thought like if and again so what I I'm not like I'm, not sure if we want to see if there are possible use cases like that right, so I have one. Now we can ignore it. We can say yeah. We want to introduce that for customers.

F

There is another thing: I actually wanted to focus on a bit here, because in general, I guess, namespace labels is something that is set by cluster admins right and then pod labels. Every namespace owner can redefine right.

E

F

Is like admin? If you create admin, Network policy based on pod label, then the namespace owner can just relabel the Pod and then Network policy will not work right as we want from our admin level. So yeah.

B

That's all all the more reason to not use like this kind of mechanism right like this customers.

F

Well, you want.

B

To do a tendency on a namespace one that users cannot change so.

F

That makes sense and then actually it in I guess that it should go along then, with the subject selector, also, because if we say that that's not safe to apply policies based based on pod labels, then you shouldn't do that for the subject. Also. But then what I'm saying here is that I know of some customers that are using clusters like that, and they are like the owners of the cluster and they are owners of every namespace right and then that fact that someone can change the namespace owner can change labels.

F

It doesn't mean anything for them because they are the admins and they apply labels right. So.

A

F

I mean I guess there are just like two decisions on this level. We can it to stay more or less consistent. We can either say that we just don't allow pod label based filtering for admin, Network policy, because that's how we expect namespaces and pods, and this name spaces to be labeled, or we say that we kind of acknowledge.

F

There are some cases, as I said, when cluster admin owns all the namespaces, and that may be like useful for them to have this kind of functionality, and then we just tell everyone and I guess that also that's a good thing to say, even with the existing API. To just have this warning about pod based uh filtering that namespace owners can just I, don't know, remove or re-label the pods, and then this network policy will not apply, as you expect anymore.

B

I I think the there is a valid point. I've seen, another customer run into the same issue right like the power levels, changing which is there for Network policies as well. Today, the the way I see it is.

B

We can have docs that say best practices right around using namespace selectors for tendency, and that is what we have in mind as we Define these use cases, but the Pod selector is there just to give that freedom, but if they are using it in a way after which the users are going to change it, then that's on them right.

B

They can stop using the Pod sector like if that's a problem like it depends on your use cases and the kind of environments you've set up and what kind of permissions your users, the cluster, have right.

A

I I think I I think it's that, like you're saying sorry, it's like that subtle interaction right like generically I, agree with you Nadia, but then what? If the admin owns that namespace like the cube system namespace and they want even more fine grain control about you know being allowed to say, talk to qbns only, for example like in that case, you need.

A

You need pod selectors, but when we talk when we start talking about defining tenants, I think that's totally different right, like tenancy I, think we want to err on the side of tenancy being hard right like like When, an Admin defines a tenant I can't think of many times where they want that to be solved right. They don't want to Define soft tenancy boundaries. They wanted to find hard ones. So if there's some way for us to ensure that when we're defining tenets explicitly, you can only do so at the namespace level.

A

That makes sense, but taking away the ability for an admin to specify pods, specifically I, don't think make sense at this time. But I I totally see your point um and again. Documentation is everything here.

A

Right like if we expect users to use this in a certain way and to set up tenants in a certain way like we have to document that accordingly and if there's some tiny subset of universe, users, which I would say, people setting up tenants that are sets of PODS across namespaces are a smaller set of users like as long as we Define. You know what we're doing with good documentation. We don't have to design to every little corner case.

A

B

D

One actually yeah.

B

For Jan, have you met any challenges with same in not same on your side like either implementation, wise or even like your users? Thinking it's a bit complex or anything like that.

E

um To be honest with you, I think, uh within Andrew, the same labels is still a PR, not merged, because I think a majority of users who are using ensure right now are actually happy with the self and uh maybe not self-implementation, of this in in the sense of you know, we would I know we talked about a lot of um times. You know Tendencies are defined as a namespace boundary.

E

Well, instead of a bunch of namespaces, the majority use cases, we've seen and sure users have which obviously you know there are a lot of uh VMware integrated product, I would say, and a lot of these um products use namespace as a like a single namespace as a hard um tendency boundary. So that's why you know for entry use cases, usually um when we we have implemented self right. So let's say I wanted to allow traffic coming from my own namespace, but nothing else right. The other namespaces I just wanted to drop.

E

So this has already satisfied. I. Think the majority of the use cases that Android users have and I'm implementing the same labels just to sort of like be Upstream, um comply compliant right. So so, when would you like admin? Our policy? We wanted the ability to have a bunch of namespaces as a tendency boundary, which is also from what I remember.

E

uh Maybe a year ago, a lot developed, the other guys coming and say this is what we envisioned the tendency boundary be we we don't want to only support a single name, special boundary, but a bunch of um namespaces, which was uh uh same labels as the tendency boundary, which also makes sense to me. So this is where we add- and you know, obviously we also. We also implemented the uh the the the same labels, um uh which is in the PR, but as I said before, it is the not same labels.

E

That's a little bit trickier for us because of the uh no no cells uh selection, but only East-West thing. So essentially you know we. We have a it's a little bit more difficult for us to implement that in a very, very you know, scalable way, I would say because for us it means that we needed to sort of like calculate the the complement of the the same label selection in the cluster, um because we are explicitly living out. The north-south part right so yeah.

B

Yeah, that makes sense thanks and for sharing that and I think it's also valid, like in my head, also for tenancy I either had a single namespace or a bunch of them, but it is interesting to see that there are users who want a mixable.

A

I think there's going to be an infinite number of user user Corner cases to be honest, I I don't want to go as far as saying this is a corner case, but like there's a lot of ways to set up kubernetes clusters and and Define your users to find your groups to find how they're organized the concept of a a full a fully admin cluster right, where you're not necessarily I, want to say like shipping a service to customers, but instead like using it as a platform for yourself to just run workloads on I, guess, that's the one more kind of that audio is kind of running into and seeing so.

A

We just have to think if we want to moving forward like have change our definition of tendency to to expand to that or not and I think the place to do. That is probably in a dedicated issue um for this, and we can like hash it out there and ultimately come to a conclusion based on well more folks than just us. So.

E

Sorry about that sorry for interiming, you're, good uh I. Think that the the more uh the one thing that I take from Nadia's uh thought is that you know uh in in a sense of what.

C

E

Do when users sort of like change the uh pod labels in the um unexpected way right what if the user does just trying to escape um the policy that that me has said for them? um I I, don't know that. um um Do you do you have any idea, an idea on? Is there any potential Solutions or something you see in terms of solving this, or do you think is this is just a um sort of like loophole in terms of the uh um policy definition here.

F

Domain solution to that specific use case right.

E

F

Yeah, it actually can be like worked around, because the number of security zones is pretty predefined and more or less static. So you can just set regular admin, Network policies um matching on pod labels to do that.

D

F

That's kind of fine, it just was um so since we are still kind of defining the tenancy right and how it should work, and we just said that you didn't hear of a single use case where someone would use that the cross name spaces I, just thought, I'll bring it up. So that's just like as an example of how something like that and C can be spread across.

E

Oh yeah, yeah, I, I I, mean I, mean the the thing that you talked about on you know. Seeing a a user can actually change the Pod label to escape adminel policy thing.

F

Yeah, you should not well, you should not Design Network policies in a way that can be escaped like that right. So that's what I was saying. We need to add documentation, saying that if you, you kind of, can only use pod selector for admin, Network policy, if you control the namespaces, this selector applies to because otherwise pod labels are not under your control.

E

um Yeah that that that makes sense and I I think uh that's a that's a really good point, because uh to me I do think this is sort of like um out of the scope of the uh the adminary policy API itself, but because, if you think about this, you know if user do wanted to change pod labels, uh there's nothing really. You can do about it and other than the labels I, don't think of it like a good way for people to Define.

E

You know uh I wanted to select specific Parts in the namespaces, um which you know no matter how the users change the power labels I will always be able to select that as a subject or whatnot right. So this is not like a problem. That's specific to A and P itself.

E

um I I would say like for Adam being I would imagine right if they wanted the whole package um I wanted to enforce amp on the product retinarity, then we needed to have something like I: have a admin label, maybe on a pod saying that something like you know, kubernetes.io, you know admin or like labeled this part as uh admin control, which is function, blah blah blah and then I.

E

We use this specific label to select the part and when users tried to modify or delete the label, we have some other Solutions in the cluster, maybe a web worker. What not to prevent user from doing that right.

E

So so this is sort of like a r back or other thing that are completely I, feel orthogonal to the admin Network policy API, like the admin, needs to make sure that if the admin is to make want to make sure that you know for the policy they have been set, cannot be um sort of like abused by users in certain ways. I do feel like there needs to be. Other mechanisms are, are back wise even for namespace story right.

E

The admin needs to make sure that the users of the namespace, the tenants, cannot just come in and modify the namespace label so that they just simply escape the amp right. So this is something I personally feel like is a orthogonal to the API design of the amp, which is we just provided. You know the way to sort of like help admin to set. You know these guardrail rules so that we can Define the uh the tendency boundary, but you know we we actually needed some.

E

You know maybe some other things to make sure that um it is not sort of like bypassed intentionally because user just wanted to you know sabotage. The I was the sabotage, but they wanted to sort of like bad bypass. You know all the rules that have been set up by the meetings. That's that's what I'm thinking on this.

F

Yeah well I mean that that's.

F

Finding the cluster-wide objects, it's still nice to share this kind of knowledge and possible drawbacks right and I. I also think there are some use cases where actually admins May use that pod based labeling, just saying to their clients that they can label their pod with this label and something will be allowed to them, for example right, so it kind of it can be used both in a good and bad way. So if we uh just can clarify that a bit I guess that would make sense.

A

A

Okay, cool so out of that I think what we should do is open a issue for kind of looking at tenancy in general for like the next iteration I, don't think we should redo it in V1, Alpha One. So uh that's probably a good takeaway here.

F

I guess there is said that then is kind of preparing something around that already. Maybe he would own that or.

B

Not, oh, that's I, I, don't think it's more about who owns it or not. I think it I think I agree with Andrew on the fact that we can release the API asses, given um Yang's done his part and I can try to do the same deals if needed or and then we can reiterate on the tenancy on V2 on how we want to change it. But.

C

B

I'm not sure if, like we just finished, keep going we're just back so I think he said he will not be here this Tuesday, but maybe the next in the one. In two weeks you might be able to get something.

E

So so so sure um do you think Dan's idea on you know? Maybe we wanted to change the uh the tendency uh thing it's still based on the namespace labels right, so do you think it's vastly different than the API? We have right here or it's just sort of like semantic change. Like uh sorry announcement, just like like a API type of change but semantically, you know the implementation would basically be the same.

B

I think implementation might not vary too much, but at the same time the the part that it's really vary for me is right. Now, if you use the same and not same, you.

C

B

Have different combinations of matches right? So let's say you have label one a and label to B right on your subject and then you can have peers that are selected using a relation of label a right, but.

E

B

Subject itself was selected using label B right, and that was the.

E

One of the confusions.

B

That Nadia had.

E

Raised ah I, see I, see, I, see a lot of different yeah.

B

So the same rule, the same policy C can generate multiple Dynamic rules, so the peer to subject relation is not really. You know, as it is for selectors right because of this, so that is the real part that Dan is trying to fix because he thinks it can be confusing, and that is not something that we intended to do as tenancy. It's like we said right.

B

We we wanted a full name space as a tenant, and it's mostly is is the same tenant or is it not the same tenant, and this can either be a single namespace or a bunch of namespaces.

E

I guess it makes sense, so so they're she's trying to cut down the freedom to to differ whatever labels that can go into the same label in.

B

E

Sense of whatever label you use to select the subject uh you're, you should be using the same labels to select. um uh You know same label tenants here.

A

B

Make sense yeah what he what he was proposing is just like we have subject today. We'd have enough the field called tenancy and tenancy will have a list of labels, but then we won't have any more same labels or not same labels in the peers. What we'll have is what Andrea has today, which is like, is same or is not same right and that relation will map to the tenancy. So all it means is the labels that you use to define the tenancy asses.

G

F

Can I ask, do you have some users for the network plugins that are implementing this API now for this tenancy.

F

Or do you only do that because it's in the API.

D

E

Well, it's it's not really like in the API, because we kind of like designed the API, but you know I know for entry at least um we, as I said we do have customers who use um namespace as tendency boundaries, and we kind of like uh did want to go with this when we designed the API in the first place is because uh I think a really long time ago, maybe more than a year ago, and we have these initial people who is interested in doing the developing the main Network policy and I think those are from different companies, and they were talking about.

E

You know the use cases, and you know some sort of like quote-unquote ad hoc network policies that they come up with uh to solve the use case, and we look at these policies and say: oh they write these because they wanted to group these namespace together and they wanted to group those names basically together and then the takeaway, with from all you know from all these talks, was that you know for the for all the you know: uh customers or implementations we have talked about in terms of tenancy.

E

The namespace boundary seems to be the uh the mainstream of you know how a tendency is defined in a cluster uh these days, so so I I think this is you know, uh I, guess the history of it.

F

Yeah that totally makes sense. Thank you.

A

So I think for me, like that sounds that sounds great. We'll make an issue. We can think about refactoring that for the next version for this version, literally the for view on Alpha One, like I, would like to cut just cut it right. So go ahead and do a release of it, um and this is the only PR I think we need to like figure out or resolve before I can do that. I, don't know if y'all agree um the one just to fix the description of same not same labels.

A

We already removed namespace relation I. Believe right. Sorry, yes, so.

B

A

B

Think I'm, okay, with that I, can go and update the description alone, and for now we can cut it and then I think, like Shane said for Gateway for conformance testing, we don't have to mandate the implementation of these two fields for the cni plugins to make them performance. We can keep them as optional, but keep the rest of the API Fields is not looking.

A

I like that, so then we have our cut release of V1 Alpha One. You two are both using it in your implementations um and we can move. We can like officially quote unquote, move our all this conversation into working towards a V1, beta, 1 or V1 Alpha, two right, um I think that'll just make everything a lot easier. Yeah.

A

Okay um Nadia: this is another separate one. I think you want to talk about.

D

F

If we don't have too much time left, we can maybe postpone that if there are other topics for today,.

A

That's fine, I, don't think there was really too much else. um Syria just poking for review on the.

F

Oh okay: well then, we can I, don't know quickly. Look at that if.

A

Yep, so just folks here, like we're, trying to figure out our user stories for egress cluster control, Siri is open to PR I've been reading. It I haven't had any comments yet, um but we definitely need like complete consensus on this uh before we merge. It. I think.

B

I appreciate it I think Dan gave it one pass, but once we have the use cases nailed down, I was thinking of doing a proper. What network policy enhancing proposal? I don't know. I was joking with Shane about this. They should have our own and peps like the gaps in Gateway yeah.

G

B

Yeah to see how the exact APS semantics would look like and have that shorted out.

C

But this is something.

B

That we Edward had would like to have as soon as possible, and we want wanted to be an upstream first thing right. But if we sense some delay, we might it's something that we might have to do. Downstream, which I don't really want.

E

Yeah I know my bad I'll. Take a look at today.

B

Thank you, yeah cool.

A

I'm gonna pass it back over to Nadia.

F

I actually have a question about that: uh egress use cases we don't do anything for Ingress because we didn't have any use cases for that right.

B

A

B

A

Like a lot harder to Define I think it's that's the other kind of an issue there right, because what most clusters are using some sort of Ingress mechanism, I.E the Ingress API or the Gateway API. um That's another interaction that we need to start exploring. But it's not like as simple as defining egress.

F

Yeah, but about nodes ingressing to pods that is not covered now, but it's inside the cluster right. Don't we want to do to include that.

B

Do we have valid use cases from users or customers because I know egress is an important aspect. Multiple customers has anyone on our side actually asked for this idea. If, yes, we should definitely think about it.

F

While I know some examples of people using that in regular Network policy and I think they, some of these cases would make sense on the cluster level. I yeah I, don't.

B

F

B

Regular Network policies do they use IPS of PODS to do node to pod, or they just use selectors like. What's your just, you use selectors as a subject to the whole central part, gets Chosen and then.

F

Yeah I think something like denial is quite popular in general for Ingress, also and.

D

That includes just zero zero cider.

G

Well may I step in way quickly on this topic, so I actually see a very strong use case, for instance, if we can imagine some kind of small cluster uh in front of some, let's say scientific instrument, which is basically a bunch of Hardware outside the cluster, and then we wanna some kind of controlling applications inside that cluster and those applications will definitely need to access Hardware.

G

So I think that's something like like a strong use case. Maybe.

B

G

Mistaken sorry, no.

B

No I think that's a great use case like I, actually had a sorry, Android I'll.

A

Just think no no you're good yeah.

B

It was a good use case Igor. The question I had is like your Hardware is outside the cluster, and you said your apps are inside. The cluster was the other way around.

G

Yeah, so imagine it's like a boxes, so one box has a cluster with a lot of applications. Running I mean in this particular case, and the cluster I mean kubernetes cluster right. So.

D

G

Deploy a bunch of applications there like, uh which may let's say, build some dashboards or whatever some alerting stuff, or even some controlling things where users want to interact with the hardware on the other side. And then there is a bunch of Hardware which is effectively outside that cluster and it might have some kind of network endpoints. And we want to talk to those from the yeah.

B

That's it makes it makes sense totally like the only thing that doesn't really work for me with the southbound use case, always is that pods have private IPS, so anything coming Ingress from outside the cluster is already kind of protected, so I'm thinking. What do we want to really deny? Is it's anything coming from the internet to in to to the part that we want to hit here or, like that's the one thing that Ingress bothers me about, but.

G

Yeah I think it could be well I, I couldn't imagine, for instance, if you have a single plaster for a multiple, let's say, research groups, which is a research facility and then the hardware on the other side.

G

So, for instance, if we think about some kind of think control facilities, they usually have a storage ring where electron spinion and then they have beam lines where actual science happens to say, and uh they it's usually well in, maybe not usually, but at least in facilities I used to work, it's kind of one network, so technically one scientist from one group can add access directly, Hardware from which is completely unrelated and so on. It would be nice to say: okay, I have application, but it's a kind of dashboard and that particular application.

G

My access only this range of whatever IPS or this you know so limits somehow uh that Hardware access and- and this you may think about the hardware accesses basically as Network endpoint, it's premium elsewhere, yeah.

B

To me that sounded more egress, you said your apps are dashboards that are trying to talk to the hardware which.

G

B

Of ips, so that's egress, right, yeah.

G

But we are discussing right now.

B

No in Ingress Nadia was saying why we don't have the opposite scenario like from.

C

B

C

So let the hardware.

G

B

To talk to the apps scenario, being blocked kind of is nothing.

G

Yeah, that's also a valid scenario, because you obviously want a data from the hardware to come in in those applications and then show them to the users. But.

F

G

F

Sorry, if the data you're describing coming back is a reply to a request coming from the Pod that.

G

Is inside the cluster that is.

F

G

It's not necessarily, it can be that the hardware is kind of pushing data.

D

B

Right yeah, that sounds like a this case. We might want to consider comment on the issue here. The 86 that I really has highlighted.

G

And yeah I can do that sure.

B

Yeah and thanks Nadia and everyone for bringing this up, what I can do is either we keep the egress and Ingress separate, but if Ingress so Ingress within the cluster seems easier right like what I'm concerned is ingress from outside the cluster, which is tricky to implement, so I might even suggest Igor to open an issue with the use case right, a new issue on our repo and let's take it there, because then we can keep egress separate from Ingress.

A

Yeah I think we definitely should keep eating grass separate from egress, because that was one of I think that was one of the mistakes with network policies. They just lumped it all together. So.

A

Cool okay: we have a couple minutes left um Nadia. Do you want to get started on this next one, or do you want to save it for next time.

D

F

I, don't know uh I'm not sure how long it will take. That's not a big topic.

A

Okay, yeah go ahead, then.

F

Yeah, it should just be on that slide. I'm, just I I had this question about asymmetry about our subject. Definition right, I've asked before I guess uh so that we can Define subject as namespaces, with namespace only label selector and then as possible. That have has both and then basically, when you define namespaces, you implicitly Define empty pod, selector right uh well at least that's what it means.

F

So what I just made a little comparison here of uh how you currently can Define something versus like if you just had a symmetric version of that without like name spaces or pods Fields right so.

A

So yeah I I think the one thing that was like previous history here and I'm, trying to reboot my mind but like right now, a subject is a basically a one of right: it's either pods or namespaces. You have to Define either a namespaced peer or a namespace pod, pure whatever um that design schema was done explicitly for extensibility in the future, um so meaning like.

B

Like you could have services, for example, if service mesh comes into play, that can be it that we are also trying to consider some of the expansions, and actually this is also something that Dan and I talked about, and we were like: okay, any addition to namespace and pods. If there is something in future that comes up, then we want it to be separate entities.

A

How does this, how does this doing it? Just like this break things? That's.

E

B

E

B

Sorry no I was about to say having said that, I reached the same conclusion right like for me: it's not really about symmetry or asymmetry. For me, it's the same thing. It's just the way we do it. You can either do the name.

C

At the end of the day, you're.

B

Having a namespace, selector and a part selector the current way and the other way, so it for me doesn't really matter how we Define it.

E

uh I I think I can speak a little bit about that. I think it's the whole fell. Open, fell, closed story that we had like wasted.

E

I would say you know hundreds of minutes on you know discussing, but the, but the short version is: if we do that, like this, on the on the right side, let's say like in the future, we introduce a service selector uh on in the parallel of namespace selector and the power selector right and um the the the issue we found out is that if we introduce that and a old implementation, let's say it's oven, kubernetes orange or whatever implementations that does implement the API, uh which doesn't know about the service lecture yet so that implementation will not look at the service selector.

E

uh So let's say people put a namespace selector with a service selector with the intention of I want to select specific services in that namespace right and what the implementation sees is that it only sees the namespace selector, because it doesn't really actually know about the service selector. So the implementation looks at the policy and parse it as I want to select everything in the namespace because it doesn't even know about the service selector.

E

So that uh is the not the intention of the policy writer because he doesn't really want to he or she doesn't really want to select everything in the name space he they just wanted to select specific service in the namespace. So then we have a discrepancy between the intention and what the implementation actually does. So, in order to avoid those kind of problems, we have to be really really explicit, like what we do.

E

On the left hand, side where you know, we wanted to make sure that people either select a namespace or a pod, or once the service comes in right in the future. They they need to say that I wanted to select the serp subject and the subject type is service and that they, you provide. You know different selectors on how you actually select these services, so so, in short, this is by design.

E

It is really explicit really redundant, but you know uh uh it is the way that it is because we want it to be future proof, so it.

F

Makes perfect sense, and that explains why you would need one more level so like pods right under that that doesn't explain why you can still select the same thing. I've just updated that to reflect what you said right.

F

You could just do this so because I, first of all, I can't imagine people coming to me and asking what this namespaces thing selects if pod selector is something different, but namespace selectors still selects pods and we do have still two ways to express the same thing in the API, which is usually not a good thing for the API right. So you still can have this extensibility with one more level here, but have only one way of expressing things.

E

I I feel you I feel you I think I think what we think is that uh in, for example, in the documentation itself, uh on the left hand side you have the the part, the name Suite selector equals labor selector the power selector being an empty one. This should never occur in the documentation because, even though it does the same thing as the above, uh essentially whoever writes is just trying to select namespace.

E

So so, even though we provide the flexibility to do that, we should never see the policy written like this, because, by the end of the day, you're trying to select namespaces right. So so we're sort of like given the flexibility and when people do actually wanted to select parts. We expect that the power selector to not be empty. We don't sort of like require that um and the reject the policy if it it's written like this, but you need to really.

E

You know know that by the end of day, you're selecting namespaces and you should not write pause as a subject. Does that make sense.

F

I am not sure if that if you're saying that empty selectors should not be there because they still will be there in some other cases, right.

A

They'll be there because they're built in the types I think like this goes back a long way and another big part of this was like building a yaml. That was really explicit and easy so like. If a user wants to select namespaces and there's a tool called a tool, iea yaml field called namespaces. Why the heck? Would a user ever use the tool called pods yeah.

E

That's that's my point. Yes, I.

A

Think that's what yeah and yang and I hatched our teeth over this for a long time, that's kind of, but I fully understand what you're saying Nadia um I'm thinking. If we really want to get down into it here, we're over time too so I'm gonna try out this up. If we really wanted to drill our teeth down into it here, we could do some validation, especially with kubernetes, X, validation, to say, pod selector must be set, but cannot be empty right.

A

That would fix your problem. Then you could not explicitly select a namespace easily by doing what you did on the right, but.

F

Then you cannot select. Oh the way to select all pods in the cluster would be through namespace, but I want to select pods, not all namespaces.

E

You're actually selecting part in the end right. We just exactly just group these parts in the namespace and then you need to understand that you're actually selecting the namespace every part in the namespace. That's what I'm saying.

F

I mean I every time I see namespace selector in this case I'm thinking about pods, because I know I'm, selecting pods right. That's why? Maybe it's like some, my perception right of that. So uh if you thought that about that a lot already I'm like just I've, had some answers right. If you think that doesn't that that's not worth keep discussing, I I think that's fine I mean it looks like I I, just I I definitely miss some history right as you're saying here so.

E

D

E

See no no I'll just say that you know I'm happy to sort of like discuss a little bit. You know offline with you if you have any more concerns or something, but you know uh um so to me. uh I I need to be really honest here, because you know Andrew and I have been in the in the game.

E

For you know over a year and to me I feel, like you know, the more important thing uh right now is sort of like we presented an API in a way that makes sense to the users and it can achieve the use cases uh that most majority users want to see at this level, but not to be too sort of like fix it fixed it on what the API look like exactly because sometimes some apis can be really confusing.

E

um But looking at this, as a you know, selecting namespace all parts as a subject, I don't feel like this is really confusing and it achieves the sort of like use. Cases that we have. You know came up this far. So my intention is that it, unless it's sort of like really confusing to the users, maybe we should not, you know, look at and changing it. That's what I'm you know. Thinking on on this matter,.

A

Yeah and I'm gonna, plus one on Yang here.

B

Yeah me too plus one especially because I've implemented it change. It. Sorry.

A

But it's a really good point, so it's something we should think about, and I also think it's something we should talk about. Right like this should be some sort of note in our documentation about this um ambiguity.

B

Yeah and I think we should also document what Yang was saying right like while we do provide a way to have that uh the part on the left side, with pods, with empty name space and part. We should not let users do that as an example, and they should just use name spaces.

E

That's my point: they technically care right. So so, if.

B

E

Wanted to drive somewhere there's only one mile away, they can take a highway and take a detour tens of u-turns and they get there right. They they're free to do that. It's just that in the documentation. What we should do is that we should provide the example saying that you should use the namespace selector and be done with it, and if they really wanted to go that route, um it's fine. We provide.

B

E

It's ability, but that's not you, know what we envisioned people would do right. That's that's.

A

And think of us, as as programmers the number one thing we do is Google kubernetes Network policy pull the default Network policy they put there and then edit it to.

E

Right exactly yeah.

A

um Okay, that that was a good conversation, though it is really interesting and API design like this. You run into it all the time so I'd like to have a note on it, um but we are way over time. I'm super sorry for keeping you all late.

A

uh We've had a ton of good conversations, so I'm gonna go ahead and end it here, let's continue over to slack if we want to I'm happy to keep talking and uh yeah I'm gonna work on getting a release cut before next meeting for V1 apple one, and then we'll set our eyes squarely on implementations there and um on to the next iteration so on to the egress stories and any other user stories you want to Target for beta, but thanks so much everyone for all your time today. I really appreciate it.

A

Thanks for all the help and we'll talk to you all soon have a good one.

B

Thank you. Thank you. Thank.

G