Kubernetes WG Policy, 25 Apr 2022

Previous Meeting

⏯

youtube image

►

From YouTube: Network Policy API Bi-Weekly Meeting for 20220425

Description

Network Policy API Bi-Weekly Meeting for 20220425

A

Okay, hello. Everyone today is april 25th 2022.. This is a meeting of the sig network policy, api subgroup to sig network. This is a cncf certified meeting, so please be nice to each other and yeah. Let's have a good meeting today. So, if you've been following these recordings, the past two meetings we've basically been working mainly on the admin network policy api, and we will continue to do so today. Unfortunately, between this meeting and the last one, we haven't had much more additional reviews.

A

um We had some good follow-up comments and and stuff added in the last meeting, um but there's just a few things. I think we can go over here today, the first of which, being the work to break out a new object in this type or a new object in this api for baseline rules, which I drafted a commit for here.

A

um Basically, the only questions I had were does it make sense to have I copied what gateway api did basically, and I created a new file called shared types that basically holds most of the actual types that are used by both baseline, a baseline admin network policy and an admin network policy. um Is that the right way to do things, or should I be more explicit in that? I redefine types totally or is there any opinions on that.

A

Actually funny here see it rendered it rendered as a rename, even though it's not.

B

So so, essentially, you're saying for both crd types. You know for the structs that are shared by both crd types, it's in disco file and you have two separate other group files for for each crd. Basically right yeah. I think that makes sense.

C

Yeah I mean we don't really need two separate files. Most api groups only have a single types.go. Okay, but you know whatever works.

A

Cool, I can do that as well. I I didn't know I just that's what I thought gateway did, um but I will.

C

It yeah. Actually, I think that is what gateway does yeah.

D

Yeah, I was gonna say that in gateway, actually it's easier to find things as well, because they split it. So I think it's fine yeah.

A

Okay, cool then I'll tidy this up. um There isn't, I mean nothing, really changed from what we had discussed in the previous meeting. um We just basically removed any references to priority equals zero and there's like one extra note in here, did: did we consider.

B

Singleton, though, like I, I remember this being a discussion.

A

Yeah, that was my next question: um singleton in the sense that, like we, that we should. I think we agreed that we can have multiple baseline admin network policies, because that's all right um but yeah, I'm not I'm. I guess that's not singleton, but I'm not sure.

A

Like what do we think do we have one admin, baseline admin network policy um for the entire cluster, and if that is the case, um do you just define the name as restricted to one name? I don't really know how how singleton's looking the actual api.

E

If we want a singleton, that's how we would do it.

C

E

C

I don't remember exactly why we were arguing for or against singleton.

B

Indeed, I think the the argument was brought up by uh rahu, who is not here today, but I I think I do remember his argument being because we have a single power right.

B

So if we allow, you know people to create a bunch of baseline policies, uh maybe they're we're basically just allowing them to step there on their own toes because it is a single poverty thing and if you define you know uh conflicting rules that might end up not looking good, whereas we, if we just do a single turn, you know everything will be listed in a single file so because we have the rule all the right thing right. So.

C

Yeah- and I remember I think we we talked about- we can make it singleton now and then, if we decide later that's a bad idea, we just relax that restriction and- and we can do that, whereas the other way it would be harder to change right right.

B

And the other and the other thing is about use cases, because I think the last time we discussed it, you know we only think there might be one specific use case for the baseline rule, which is you know for every namespace, that's not coupe system. Basically, I wanted to start it out uh uh denying everything stuff like that. So it really doesn't. uh It really doesn't strike me. As you know, we have more use cases than just 2.1 single baseline policy, although this has been.

B

You know a discussion also for me in our policy because we're saying hey, this is the only use case we can think of, but people will come up and say you know what, if, in the future, there are other use cases we need to be future. You know right so um there's another. You know discussion, uh sorry decision to make. I guess, um but I'm I to be honest: I'm fine either way. um If we do want it to go with the singleton, then uh probably we wanted to.

B

You know, choose some good names for it. I guess um I actually don't know.

A

Yeah I mean it can be as simple it could just be. You know, I don't know default yeah default, baseline admin, network policy, that's a mouthful or just default, yeah, that's fine, um and then it to do that in the api. Do I guess I just literally validate that it can only be default, but it's as simple as that right, yeah, okay, cool.

B

A

B

I get, I guess this is also a good topic to to bring up to tim as well um in in the discussion uh to to understand how what he thinks about this guy.

A

Cool yeah, no, I agree. um I plan on basically bringing up everything. That's still left uh to tim, there's only a few things actually really. So I think that's all I had on the baseline admin network policy. uh It wasn't super complicated.

A

um This will be a big jolt, I think, to everyone following this pr like if they haven't been following really closely so I'll, probably try to put a comment in there or something just saying like this is what happened, and here we are, I already kind of did on.

A

The engineer from calico's comment, but I'll try to make it a little more straightforward as to what's going on there.

A

Cool okay um next thing was refactoring amp ports. I think we should just table that until tim, that talk with tim, because he was the one who kind of pushed us away from the um what we thought was more understandable sort of thing. So I don't think it makes sense to keep talking about that here. At the current time.

A

um The next thing was looking at the refactoring of the lovely self, not self etc, and dan had a good um kind of additional way to do this without boolean, so that we don't have a you cannot set this bull to false setup, because that obviously is kind of confusing, um but it still doesn't really talk about redesigning.

A

You know how how we're setting up namespace set anyways so like we designed designed all these selector structs to be quote, unquote, extensible and quote fail close, but we kind of disprove the ability to fail close or to define what fail fail. Closed is within the api. So I think we need to take a step back and look at the design of these trucks and ask ourselves like.

A

Is there a better way to do this um and I, for one am not super sure yet uh I'd like to think about it a bit more, but I didn't know if anyone else had any opinions.

C

Well, so I think one of my first comments in in my old review was: oh, you know this would be so much cleaner if we did blah blah blah instead and somebody responded well. We can't do that because we need to fail closed, and so you know if we're if we are changing the theory on that, then.

C

You know yeah, we do, I think, sorry forgot where it's going.

A

I got you that makes sense. um Let me see yeah, I think I know what comment you're talking about.

E

And it goes on subject.

E

Let's go even further back than this.

A

Oh, it might even be it's probably even out of the diff. Unfortunately,.

C

Oh, oh wait. It was my first comment, but it might not have been um like you know the earliest line in the file that I commented on yourself right. You commented out of order.

A

I will find it, um I think, that's it needs to be done, though um yeah here it is yeah.

C

I think this one yeah like I think we can make it a lot cleaner and and like easier to write if we don't have to worry about what happens when we add a new field right.

A

Yeah then we don't have to have that nested extra level of um um you know nested fields you could just use.

E

A

Selector and pod selector flat out um again, that's obviously reverting a bunch of forward quote-unquote forward progress. I think we made at some point to get this cut merge, but I mean the logic behind it. Is there um I'm just trying to think of the best way to do it in the sense that you know we do it, while also bringing in kind of your rationale here into that change?

A

um Obviously, I can't really document it in the api itself, um so I'll either have to put a comment just pointing to this, or maybe even write a medium blog adapted from your comment, because you wrote enough to put in a blog so um something along those lines, um not 100 sure I'll. Do that yet, but.

D

Definitely I was going to make a joke, or maybe expecting this to became a netflix series.

A

I think it should be at this point kind of like a dramatic saga right.

A

I have a vlog started and the, but I obviously have been able to finish it because we haven't merged this kept and the title is, I think it's it's in line with the you know, you know a hundred something commits or two years 100 something commits, and you know 800 comments later here we are, and now at this point it's even more than that. So yeah it's been a saga um but yeah. I guess really. We can't solve this on this call.

A

Specifically, uh I will take a look at kind of refactoring the admin network policy, pier along with the admin network policy, subject, because clearly those had a lot of overlap and if we can remove that extra level of indirection to allow for quote, unquote extensibility, I think it would be a lot more straightforward.

A

Do you agree yang in terms of like refactoring those bits.

B

Yeah, I think so, I'm just trying to sync on you know dan's proposal there, because essentially he's saying there are only two cases: one is namespace and the other is pod right. So for namespaces, the only thing that can be provided there is namespace vector, whereas for pods you can do combinations, I guess um yeah that makes sense, but for pause you can also just do pause vector. Oh no,.

B

Right right, yes, I I I got you what you're saying so essentially, if we don't worry about the default close thing, this definitely is the more um readable api design. I would, I would definitely agree, but uh I'm just not sure how um the the fading open fading clothes discussion will. I just I'm just worried that we'll just tear that discussion bring this discipline up again.

A

Right and that that's what I was trying to to contemplate how to make the change, while also um you know, pushing some of that rationale, we've come up with in the sense that it's it's.

C

I think the key part of the rationale is that there's nothing saying that you can do with a network policy or with an admin network policy that you don't recognize. All of because of the possibility of combined allow and deny rules means that the safe choice could be anything so.

A

So don't do it and tell tell the validate to the user that you can't do it right yeah! That's I mean that's always been the truly the correct solution. The the problem was it was. It was hard to do and it was hard to enforce um now.

A

Basically, even with all the you know, the rationale you've laid out dan, um it does fall on the job of the you know: admin slash, cni provider. You know users a little bit and I don't think that's wrong at all, but it's going to need to be documented in a way that you know wasn't there with network policy right, so I'm not against it. I don't think I've been against it from the beginning, but here we are now do we need to yeah and I guess at the end of the day, it's it.

A

It is this that you know talking about going from un unstructured to the actual resource, that's validation, but not even using a web hook or anything that's just built in to the api or to this well.

E

It would be built into the implementations.

A

Right exactly yeah, okay,.

C

C

If the implementation also provides a web hook, then they would want to do that in the web hook as well, but yeah.

A

B

Okay, sorry, I haven't actually read this, but I'm actually just right reading the second point here about the uh about the unstructured type. um Do we know that for.

B

I might be horrible around. Do we know for the current uh you know cni's, for example, if we read let's say networking b1 uh net network policy do do implementation actually do that or from my understanding, implementation. Just you know, grab and now policy object and assume it's better one, because it has been. You know, accepted by the api right. So I from what I remember. I only see this instruction thing if we are actually implementing a web book for the resource in which we don't use.

C

Normally, you wouldn't use unstructured for something like this, where you know the type at compile time right. The the idea was just that by using unstructured, we can get access to the raw value right or it gets parsed, which you wouldn't be able to do if you're, just using the typed clients that put it into the go objects for you right. I see.

B

So so is the idea here that uh maybe to get around this problem, we advise you know implementations to to look at the now policies, object using unstructured and see if, okay, I I got your thing.

C

Like I think you can just do a a round trip like unstructured to what your idea, admin network policy object, is and then back to unstructured and if there are any extra fields that your version of admin network policy doesn't have they'll get lost in that round trip.

C

B

A

And that little bit of code, even I feel like, is something we could write upstream or at least like a reference like this is what we think you should do right.

C

I think, theoretically, that's something that maybe we could add to the dynamic network or the dynamic listener or client code, because yeah that's. There are potentially other apis where people might want to be able to do that.

A

Right, I wonder why that it wasn't we didn't follow that schema? Currently, I guess it's a little more inefficient, because you're parsing well either you're parsing it on before or after you send it over the wire. I don't know.

C

I think it's just in the beginning. We didn't really think about this problem of you know what happens when they? If the api server and the client are using different versions of the object.

E

F

Apologize for joining late and missing some of the setup discussion. um What exactly problem are we trying to solve? I guess like I. I understood that there's the versions queue, but is there like a particular field that this is in reference to.

A

So it started with a comment from you. Actually, um this is kind of an old comment around the admin network policy subject and how dan proposed you know. How can we make this easier to read and our feedback was we can't because we want it to be extensible.

A

However, now we've flowed into basically arguing that extensibility, I.e, failing closed, quote, unquote, isn't something the api can enforce or specify. We can't specify what failing closed is because we have allow and and deny rules.

A

So now that you know, that's quote: unquote, no longer a constraint, we're looking at redesigning some of those ugly nested types we have and talking about how we can do that without. You know folks, coming back to this and saying what wait? What happened.

C

Basically, if we can't design the api in a way that implementations can safely use objects that they don't recognize, then we want to have some way of the implementations always being able to tell when they're getting an object that has fields that they don't know about. And so that's what what I was talking about in there in point two.

F

Got it uh did just because it's alien did anyone have a chance to see the uh discussion that I linked in the slack uh I linked the link to a kubernetes api reviewers.

B

F

B

I skim through it, but I was really hoping you can. You know, give us a high level idea on how you think it's similar into our discussion.

F

So it's it's addressing sort of the same problem, which is that you have this struct, this top level struct that has a bunch of basically modes. It can operate in. So in this case it would be our subject selector and it has a bunch of modes it can operate in. It can select name spaces or it might be, selecting pods or you know in the future.

F

We might have a third selector or fourth selector, and the idea is how do we write these sort of structs right these structs that are essentially a one of and so they're discussing? Do we want to have a bunch of sub selectors, the kind of kind of the way we did, which is? We have a namespace selector, a pod selector and then we add new ones.

F

Do we want to add a type field um at the top of the struck which says you know this is a type namespace and then you go read the namespace fields. This is the type um pod selector and then you go read the namespace and the pods like so on and so forth. So it's it's basically dealing with the same. I think problem that we are approaching here, which is how do we handle these potentially extensible list of you know: sub sub, selectors or subfields there.

F

The recommendation seems to be uh to use the format that we have today, which is each logical um thing essentially like each logical. Selector in our case gets its own struct um and then they all uh you know just operate independent of one another. There's no shared fields between them.

F

I might be doing a poor job of explaining that, um but did that? Did that make sense to anyone.

B

That makes perfect sense. I think, since tim isn't in spread. I would imagine this is also you know what he suggested, because I think when we get on the meeting with him, he suggested the same thing, but uh we're just having a really hard time.

B

uh You know naming these drugs, so that's why we came up with namespace and past lecture and then namespace selector, puzzle actor underneath so.

A

Right, it's just ugly. I mean it's.

B

A

Like let me go look at it, it's it's! It's a little clunky like you know something along the lines of of what dan was was talking about. Like reads a lot better, so something like this or even like removing that extra level right like. Why should we have to say, subject? Well, I guess you still always have to say subject with a certain types of subject, you know the extra level of nesting here or maybe it is just a name for naming problem and I'm wrong.

A

Let's go look at what it looks like today.

F

I mean I, I don't think that their guidance is by any means finalized. um I just think it's very interesting that they're having the same discussions that we are more or less they may not have considered the fact that naming is really hard and it's when you try to specify this. It gets a little bit tricky.

D

F

Being said, personally, sorry guys, oh sorry, glad I was gonna say I'm I'm sympathetic to the argument that, while the extra layer um like the extra level that you have to add in the yamo is annoying, it's preferred to the ambiguity that comes about when you can reuse fields with sort of not arbitrary rules. But you know rules that you have to mentally compose in your head, like if I, if I, if the namespace field is reusable a bunch of times- and you know like this other label, selector is reusable only some of the times.

F

I have to understand what gets parsed when, which is a little bit um more confusing than just having everything packaged nicely. For me,.

C

So my original comment I had two suggestions: one was was mostly just shortening names um to to try and make it look nicer and then the second one was well. Why do we even need two copies of of namespace selector? Why can't we just have one so yeah, so the one that I actually wrote out the details of here that does still preserve the a separate object for each kind of thing, with no reused fields, thing.

B

Right right, but the problem is, uh I was just trying to say this because the problem is, um let's say we have a specific example. We had in mind when we designed this right. What, if, in the future that we add a service account selector, for example, right right.

C

No, no, no I'm I'm saying now keep keep name space selector separately in each of the two different things. So so, if we we had service collector in the future, then there would be subject colon services call in whatever you need to do a service selector.

B

Okay, I was, I was just gonna say that if we do have a service account selector to be coupled with namespace sector, it is essentially also selecting parts if you think about it right. So, if you do subject pods blah blah blah, then when we introduce that combination, it's really hard for us to name that struct. If we have named this guy pods right. So that's.

F

By that logic, everything's just selecting pods right, I mean that's the whole point: we're just we're just giving you nice groupings on how to select pods. So I mean you're really specifying a service account. That's the subject that that that makes sense. Yes,.

B

But if that's the case, then maybe you know what dan has suggested here is actually the way to go in terms of you know, shortening these uh uh these sub uh destruct names. I will be totally fine with that.

A

And then we get a little bit of both. I mean, I think the only thing rahul was actually disproving here was like the the last sentence. Is, why can't we just have the one level of nesting um right.

C

And I think I think that is a.

A

Reasonable argument: okay, okay cool that works for me, I'll, look at at least changing the subject field, um and then it also kind of applies to um the.

E

A

The peers, now the peers, basically uh where's, like a good way to look at this namespace and podcast- I even get confused after I don't look at it for a while. um We basically do the same thing in peers um and so I'll try to shorten that as well to make it more readable. I guess um now.

A

The only other thing with the pier was was refactoring. These booleans, which I think before you hopped on or all we had touched on, the possibility of just making these uh constants instead of booleans to to signal um self or not self. I don't think there's ever there's been other argument around that, so shouldn't be too hard to fix.

A

But as long as we're maintaining the same nesting, the changes are definitely pretty small.

A

Are there any other points here that folks would like to bring up.

A

Cool, so that's pretty much what I had the last bit, I guess which is still open and we've gotten. One comment on from dan: basically, is the north south, south traffic selector issue? That's a separate issue I made now. I can't find it where I put it.

A

I can't find it.

A

A

Is issue 28, which is essentially just trying to reopen the network policy, the north-south debate? um I think dan already had a good point here. Basically, one of our rationale for reopening the debate was that cna cnis may not be able to, or cni's already are distinguishing between east-west and north-south traffic, um specifically I.e like technically an empty namespace selector selects all only east-west traffic and north-south traffic. I would say that that's not implemented correctly by at least oven kubernetes.

A

I know that for a fact, um I think most most cni's today see an empty namespace selector and they apply it to all traffic. I could be wrong. Does anyone else know otherwise?.

C

I feel like obn kubernetes does get that right. I mean it would have to explicitly specialize it to get it wrong.

A

Well, I have to go look now because I'm pretty sure like if you so so, if you block or just say like egress, to name empty namespace selector, it's blocking traffic to everything. It's like you can't go to google. You can't go to 8.88.

B

Is this because the dns service is blocked or it is actually the no nor sauce traffic.

A

Because say that it's actually the north south traffic or like it's actually everything.

E

A

Just east west, it's it's, I I will go check because it's a single acl, that's doing that, um but I'm pretty sure generally in in how it's used today like when folks want to block everything to a pod or from a pod. They literally just write, basically an empty network policy. They don't write.

B

Well, the the the difference is, you can say, egress and don't provide anything down there. That by definition means block everything, but if you pro provide the rule egress rule which says I wanted to select namespace selector empty curry places. That means a different thing. um That's my understanding on the network odyssey current network yeah, we're.

C

We're not talking about having no egress rules, we're talking about having an egress rule that explicitly contains namespace selector empty, which is different.

E

C

A

No, not totally, I I got it.

C

It looks to me like, like network policy. Very sorry, like oven kubernetes does implement it correctly.

A

Okay, cool, so maybe that that is a valid I was just saying like. Basically, if this is the case, then that kind of removes one of our reasons to even bring this back up again um because cni's already doing it if they're implementing the api correctly, but otherwise I don't have any other thoughts um and I totally get it if no one else here does dan's already gave a comment. um The only thing I I would advocate for doing now is adding an ip block selector, maybe for egos traffic.

A

If we wanted to do this um and see what the feedback is, I don't think we should add we. We should add a ip block selector for ingress and egress traffic. um That's for sure, but.

C

If we add ip block, I would say, do not add accept just let people mix allow and deny rules, because the accept semantics and network policy ip block are really annoying.

A

Yeah and they're hard to implement, at least for obs.

A

Are there any other thoughts here? I mean it's totally all right for a thought to be. We don't want to deal with it now, but we are going to have to deal with eventually. So our rationale was we might as well revisit it because I I'd say the uh the voice. The number of voices in this has has dwindled down to now us us three or four.

A

So it's a lot easier to get good agreement on things than it was.

A

Okay, I'm gonna let this soak a little bit more. I would like to get tim's opinion. um I think I, the summary of our opinion, is up to date here. If it's not please add a further comment um like dan did.

A

Okay is there anything else? Folks would like to talk about now. um Comments, questions, concerns.

B

Not I think not really related to you know policy api seek, I guess, uh but I was just wondering I haven't been able to. You, know tag along with the multiculturalistic discussions. I'm just wondering is there anything related to the multicultural policy, that's bringing them that has been brought up there? Do you have any idea on that or no I've.

A

Been going to the meetings uh nothing's been brought up yet.

B

A

Sanjiv put together a a pock of sort of a really basic um multi-cluster network.

B

Right right right, I've been looking at these documents, but I do feel like, as of now, it's more written in a way like function, those backs rather than use or user stories. Yeah.

E

B

Just really curious on you know how people think they needed to use. You know multicultural policies right.

A

And as that matures, hopefully that conversation will keep moving like. I said it hasn't really yeah. I would.

E

A

Let this group know, um because I go to those meetings- they've actually been getting canceled a lot recently because of empty agendas. Oh.

D

A

Yeah not much has been going on in the last couple weeks, at least thanks andrew yeah, no worries um cool and then so after I'll kind of go back and make some updates based on what we did today. Sorry, I didn't do it until today for last time's meeting, just because I was super busy with some other stuff, um but I'll try to get that done sooner rather than later um rahul. Just you weren't here yet, but I'm also to set up a meeting with tim.

A

He couldn't come today, so he has free time on friday, I'm going to set up another meeting, so we can hopefully kind of squash some of these final little things out, um such as basics, basically just syntactical refactoring of amp ports and also talk about north-south stuff. So I'll, add you to that meeting if you're interested as well? um Yes,.

F

Yes, please thank you.

A

And hopefully, after that, if we can come out of those two with a plus one from dan and tim I'd feel really good. So those are my next steps for now sound good.

A

Cool awesome well I'll, give everyone 20 minutes back thanks, so much for coming uh per usual and uh yeah. I hope you all have a good week we'll talk to you later.

B

Thanks andrew bye, yep.

D

See ya bye, folks.