Kubernetes SIG Network, 26 Sep 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Network Policy API Meeting for 20230926

Description

Network Policy API Meeting for 20230926

A

Hello, everyone today is Tuesday September, 26 2023. This is a meeting of the Sig Network policy, API subgroup to sync Network. This is a cncf sponsored meeting, so please be nice to each other and follow the code of conduct um and yeah we'll go ahead and hop straight into it. Today.

A

um Didn't really have much on the agenda, so we'll use most of this time for npep review unless folks have any other things to bring up. um I only had one thing to highlight, which was on the dev side, are netlify.

A

um Previews have been broken for a while ever since I renamed our main branch, our Master Branch domain uh I fixed that I got that fixed yesterday. So if you want to render your website previews again just go ahead and re-push your commits and it should update automatically um apart from that I. Don't think I really have anything else.

A

So I am happy to hand it over for folks who want to talk about mpeps I, don't care if we start with I know rule wanted to talk about fqdn and I'm sure Nadia wants to talk about tenancy. um Who wants to go first.

A

I will pick, then, uh how about we start with Raul then, because he was here. First.

B

All right sounds good um yeah all right thanks uh on the FTD, instead of things I pushed in update yesterday. I hope uh addresses a few of the comments that I got, uh but there are two major discussion points uh that are happening in the comments uh that I figured we'd bring to the broader discussion here.

B

Maybe um the two are uh whether we want to have deny rules and whether we want to or how we want to handle uh DNS records with short ttls or DNS records, whose IPS uh change very rapidly uh I think those are the two major ones uh Nadia and Dan. You guys were in the comments. So if there's anything else, you want to talk about, um I can shout that out as well.

B

um I guess we can start with the deny uh rules. um This is something that we've talked about uh a decent amount in the past and I know dance had a great write-up that hopefully people have had a chance to read. um Basically the long concern of it is that writing denial. Rules based on DNS trees can get complicated because you know IP DNS is an incomplete uh spect in the sense.

B

So, if you're not guaranteed to get every single IP back into your domain for any given uh request, and so you have a hard time guaranteeing that, when you're trying to enforce a deniable you're, actually denying every single impossible IP a client could be reaching that's. Hopefully, I did that Justice.

B

That sort of the cliff notes, version of the security concern with denials and the thing I'm struggling with is I haven't, come up with any good user stories for denials um and that's why I uh I'm leaning towards not having having no different STDs but I want to open that up to other folks. So you know help me fill in the blanks and things I've missed, especially Yang, because I know Atria has did I am PDM rules or at least I think it does so I'm curious.

B

If you have any good user stories, maybe we could think about.

C

Well, well, let me trip in on that I think um to me: I I, don't think you know we implemented deny rule because of there is any sort of like outstanding use cases for it, but I I, guess for the sake of consistency of the API right, because we put it in the same sort of like custom Resource as the other uh uh as the other sort of like in cluster rules of enter policies, and that's why we wanted to have the same sort of like looking view when user you know apply these policies and I think I I have read.

C

You know, there's comment uh on the on the end pad that I have yet to respond, um but I think you know my question would be uh yes. I I understand you that there's a problem where, in the two different fpdns could result to the same IP. So if this is a L4 based solution, then you know um it might be that you know two different rules create at two different Power. These have sort of like conflicting actions which is causes the problem.

C

My question for this is that it doesn't really matter if we implement the knife or fqd or not. As long as this is like a L4 based enforcement, we will have this problem because say if we only Implement allow rules right and the allow rules only make sense. If there's some sort of like deny all rules lying uh under which says I wanted to deny every egress traffic other than the fqdns I'm allowed.

C

So if that allows rule, that's fqdn um resolves to the address which can be associated with a bunch of different fpdns. Then, essentially, your allowed rule doesn't only allow what's written in the fpdn policy, but allow maybe potentially a couple of other domains as well right. So so that's why you know the deny Rule and their needs.

C

uh However, that is implemented, doesn't sort of like have the default be or or expected, behavior that the users are syncing them to to act like, but that's a problem that I don't think we can solve as long as the policy only operates at The, L4 level, I'm, not sure if I'm making sense here.

D

Yeah, that does make sense um and and that that is a good point- um it's it's not on the part that Andrew has on the screen right now, but I think I brought up later. The real problem with denial rules for me is that um this is still the same one. um You know it.

D

It seems, like you know, like you said: oh well, the API already supports it and it's easy to implement in the same way, and so you can do that, but it just it creates traps for the user, where um you say, I want to deny everything to google.com, but then like what, if they figure out the IP via some means other than the DNS resolver that we're snooping, um and then you know they can sneak around the deny rule.

D

um So you know it's more just that it's it's easy for users to misuse, and maybe that's just a documentation issue, um but given that we also had no explicit use cases for it.

D

Maybe we want to leave it out.

C

Yeah I I agree with that too. uh It's just that. You know.

C

um We also wanted to make sure that without explicit design rules, we don't go with the quote: unquoting pits and isolation Behavior like a network policy right, so we we definitely wanted something, at least like you should create to rule uh underneath so that every egress traffic is blocked other than the fpdn force, or we come up with some other sort of like API designs to to sort of like explicitly signal that, rather than uh rather than something like that or policy where, where you select a bunch of pause, if you and you impose fqdn rule on it, then it comes becomes implicit isolated for every egress traffic.

C

I think that's the one thing that I don't want to see, but other than that I'm I think I'm fine, with static limiting the entire rooms. If there's no better use case for it.

B

Awesome thanks thanks for the context here. um Does anyone else want to chime in or like use cases or thoughts about denying um I mean from what I'm hearing I'm going to propose that we stick only to allow for now um and yeah, we'll probably want to include a best practices section that says, if you're using uh fpd in a lot of rules or really any type of allow rules, you probably want a casual deny at the bottom of it.

B

um Otherwise these policies may not do anything.

B

Definitely you should have some money.

A

uh Yeah documentation is going to be, and documentation is going to be important. There I'm trying to think because I don't I, think that's a broad statement right like it's not going to always be the case that if you don't have an underlying denial like like we designed A and P explicitly did not have to be always a white list. Type policy set right, but we made it expressive enough so that it could be with Baseline kind of because there's holes there with network policy right.

A

um So it's it's a little bit more subtle and I. Think that leads to a lot of confusion. So I know Dan's are sorry. I know, Yang's, writing something that should help, but documentation is going to be really key.

A

um I, don't have any any feelings for or against this I think not doing deny for the first pass is fine, I think the cool part about the cycle we have now is like if we Implement um allow on DNS and then we ship it as an experimental feature or even an implementation specific feature, and we get a bunch of people saying. Oh, we want denies like we can reevaluate that in the future, with another npap so like, let's, you know, keep moving forward um and see what the users want.

A

At the end of the day sorry Nadia I saw you had your hand raised.

E

I wanted to say a similar thing like one of the use cases when you don't really need to have an explicit deny after if codian rule is, if you just want to make sure like namespace Network policy, it's do not deny something you want to always allow right. So you can't just say: I want to allow that F goodn and then have zero denies in the end, because there are network policies, there is BNP lower level, so that can be like an example use case.

B

Yeah, that makes sense so yeah.

C

uh I do have a follow-up question on that, though, uh like are, are we thinking of I know this is very API detail, but are we still thinking of having you know? Fqdn rules in the admin level policy objects itself, because right now, I know that you know on the website. We say that I mean our policy doesn't at all touches. You know sales traffic, so I'm, not too sure how when fqdm comes in, you know how that changes things, because now you will have to say something like for egress rules.

C

uh I drop, our rule would affect north south as well right, but otherwise you know the fqdn allows them make any sense.

A

Well, so behavior is changing with not with uh serious egress cap right so like for now. If this goes, oh okay, I would assume this is going in after the egress, and it will only apply to egress right. um Okay,.

C

A

Gonna have to think about that on on surya's uh and pep, like in the API design, when we're writing the API about how deep, if default, Behavior changes which I'm hoping it doesn't but I'm gonna have to think about that more.

C

Okay, I I need to I need to look at that, but a little bit more, but thanks for influencer.

A

Yeah it merged so we're that npep merged last week, so we're in the process now now surya's opened. Some uh PR is for actually getting getting the API implementation going so we'll see we can hash out in there.

C

F

I'm just going to add to the support of not implementing it like with our clients, the ones that do enforce, like egress-based, deny, are doing a blanket like cover and then punching holes to specific egress endpoints, usually if you're like, as you egress rule, but similar sort of functionality like they're, going to say which endpoints they want specifically and then just deny the rest, I think that's pretty common. The reverse.

G

B

That yeah I mean may not be necessary.

A

Sweet awesome, yeah. That seems like a common practice, especially for admin, so at least we kind of provide. We provide a bunch of different layers of being able to do that. Right, like you, can provide it like Nadia was saying you can say like you, could make a hard deny everything and poke holes, or you could make a soft deny at Baseline and allow admin allow Network policy, admins namespace admins to override those rules with uh Network policies, so should just fit in there.

B

Awesome um and the other one uh other uh topic I wanted to discuss, was um I'm kind of conflating that you, but the idea that either a domain would have rapidly changing IPS or that the domain would return records with extremely short ttls. um You know potentially even zero, but you know sub minute sub. Second, or you know individual seconds and whether or how we want to handle that from an API perspective.

B

um I think the major concern was that- uh and you know not either correct me if I'm wrong but and Dad you, but that if you, if we make a claim of how responsive the system is, are we shutting out certain implementations? You know? Are we restricting users to um only implement it? One way and one other um I personally am not too concerned about that so long as what restrictions we're setting make sense from a user perspective, but I'm open to hearing encounter involved skin.

B

My thought is: if we all think that, oh you know, sticking a random number, a one minute. Ttl is more than reasonable and you know you don't need to respond faster than that, but you have to respond to one minute to Deals. Then you know it's okay for implementations to pop down. Then.

B

They don't have any thoughts on whether we wanted enforce something at the API level.

D

So um generally agree with what you said: I mean one thing it sounds like we probably do want to include Wild Card support and that certainly imposes requirements. You know on the implementation right because you can't do it. The proactively, resolving IP addresses way when you have wild cards, so so I think we we already are going down that path of making certain implementation requirements um on on implementations.

D

So you know we don't need to be shying away from that if we think it's the best plan, um I, guess well and and and I think, maybe also we're going to want to have like implementation suggestions, like you know like stuff like that, that you know uh chart and the the comments that I pulled out from the Google and openshift docs saying you know, look like here's what people have done before and and we have found that certain things don't work well, so we, if you're implementing this, you know you should think twice before doing it.

D

You know in in certain ways: um I guess: I would be okay if we had only weak guarantees for the end user, that you know certain things would be supported um and I guess well. If we make weak promises initially and decide that we need stronger promises later, we could do that. Yeah.

A

That kind of should come with our new um with our new release channels, slash support level work right like like, if we feel like there's parts of this feature that don't necessarily this.

D

Isn't about parts of the feature this is about? Is it legitimate, if say, open, shift ship ships a version of this where it detects? You know any changes automatically, no matter how fast they are and Andrea ships a version that only detects changes. Every five seconds like is that a legitimate outcome.

A

That'll have to be tested in our conformance now in our conformance testing.

D

Well so yeah so I guess yeah I, guess that would just be a conformance thing then that we would say you know you're required to gets changes at least this often, but not necessarily any faster.

B

G

Yeah I think we could make a timer based on extended core something like that, like levels of support or something, if needed to say a default value and then compare it or something.

A

Yeah I mean it just means that, like this field, you know a wild card selector as a as a field or as a feature would would have we'd provide extended support, meaning our conformance tests to thumb spec. um That's not that some implications may not be able to support, and that's fine right.

B

Yeah I mean it wouldn't just be ball card, I think even specifying a single domain. If it's I think the use cases, if it's behind a cloud load balancer that you know just decides, IPS change every second, you know everyone gets into the program, then yeah, even a single domain. um To do this.

A

B

But yeah, that's fair enough. I! Think.

A

Nadia's had her end up for a while.

E

So did I get that we that we actually do want to make all these things optional and not predefined. In our API.

D

I, don't think so. I think we were mostly just talking about the the like. Well, the the conversation sort of moved, but I think we were originally talking about uh guarantees on on how responsive it is to rapidly changing.

B

Ip addresses and what.

D

We want to require, for conformance there, yeah.

E

I I had a point like around that, but it's also maybe easier to use these wild cards as an example. But it's also like a part of implementation. Let's say so. The problem with strictly defined requirements is that if I have a network, plugin right and I can do non-wealth card fqdn rules, uh and if we say that wild cards are absolutely required, then I cannot really use that API, but also I, don't have any other way to define any policies in the same priority range. Let's say: I cannot Implement like another, completely different API.

E

That would be somewhere between A and P priorities, maybe because that may be extremely difficult. So that's why it's kind of it may be important to allow for such cases, so that plugins actually I mean that may lead to not being able to use a and PS in general with that, if I need just one F good and Only, Rule and I cannot use this API because it requires wild cards. Then I cannot really use anything after that. If that makes sense.

B

So you're saying a split. This feature I've, essentially two individual intercontrollable, optional features like one is just mqdn. One is just wildcard.

E

Well, I, yeah I, don't know what is actually a good way to implement that now, I'm, mostly on a use case level.

B

I mean yeah I mean I'm, not opposed to that I.

B

It might be more complicated than it needs to be, but I I don't know I'll. Let maybe answer someone who understands the optional thing. I'll talk more about.

A

Both I think for this stage of the end pep. That's not as important right like we can figure out how the API looks in the next stage.

E

uh Yeah I mean that actually that applies to this rapidly changing a piece right, there's um I think we've discussed like we can't just predefine a very uh wide limitations. Let's say that, let's say like within a five minutes, you should be able to reflect the change in the IP right, but then like what is frequently changing, IPS like how often that actually is and also different plugins may have different limitations on that. So we don't want someone slow enough, let's say, but if my slow API is still like serve some of my customers.

E

I probably still want to use this API. For that.

A

Yeah and I, don't think that's something we can figure out right here regardless, like that. This seems like if we're going to constrain anything, it's going to be done in a reported via conformance test, so probably something we need in a conformance testing section or in like the actual PR that that uh adds conformance tests. I, don't know if we can Define that here, but sorry go ahead. Dan.

D

Wow, so this may be something to discuss more at another time, but I feel like we're, leaning too much on the idea of optional features like in in Gateway. They absolutely need that because you have people building, Gateway implementations on top of you know, gcp's load balancers or on top of nginx, or you know, on top of whatever, and they don't have the ability to add features like regex. You know HTTP path, matching to the underlying software that they're using.

D

So some of these Gateway features have to be optional, but that's not really the case for us, like people are writing. People are going to be writing their admin, Network policy implementations from scratch, and, yes, some people might have existing Network.

D

Networking policy features that don't do everything that we do, but they could still write the features that we have. We can say you need to implement all of these features: To Be, A, conformant and P implementation um and again maybe this is a conversation for for later, but I I think saying that that wild cards are are an optional feature is a bad idea like either. It should be a feature that all users can rely on in all clusters, or it should not be part of the feature at all.

A

Okay, I agree: the the the the design constraints are different um I think us, as a community will kind of develop like what we think around that right. I I tend to agree with you right like if we're as an option, Community saying amp should work with fqdn names like it shouldn't. Definitely shouldn't be implementation specific like if you are implementing ANP the API for you to be fully conformant, you better support, fqdn names but I think we're kind of in the weeds of like do you support FKA fqdn names, and does it pick up?

A

You know uh Records with really low TTL timelines like like we're kind of in the weeds and I. Don't know if we need to be right. Now is basically what I'm saying.

B

Yeah, if we're okay with it I'll, maybe make a note of um what we've discussed so far and like leave it a little bit open for the time being, um we can revisit it once we write some more tests and crash out the API if everyone's okay with that.

A

B

Awesome, um that's that's all I had on the fqdn side.

B

A

A

I think that is a good point. Dan I'm gonna think more on that one I don't think we should definitely exactly mimic Gateway API and everything.

A

But there definitely seems to be some relationship between you know: different Cloud load, balancers and different cnis right, it's just about where we we as a community, want to draw the line and that's going to be done via performance testing really for the most part and performance profiles.

A

Okay, cool uh next one up uh Nadia: do you want to bring up your end pep, or do you want me to bring it up.

E

Yeah, it would be nice if you could do that, but uh I've actually replied to most of the comments. There are just some recent ones that they're not done yet, so one of them is the discussion around um a use case. I wrote uh to create the so-called non-tenant, which difference which differs from the existing API that we have so now.

E

It is not possible to have this kind of thing and um I actually wrote down that user story, because it is kind of an existing user stories and a user story, and they figured it's not just covered by default right now.

E

So that's why I actually wrote it down, because for now, if you want like, if tenancy rules are strict enough- and let's say they are supposed to deal with security concerns, then currently it relies on on admin for whoever owns namespace labels to label every namespace, probably before any pods are created in there, because otherwise it will not be a part of tenancy and if I want this tendency, let's say like if I want to deny any connections between different tenants, I want every single pod in the cluster.

E

To be a part of some tenant, so that actually brings it down to me as cluster admin, making sure I absolutely always have the right labels and all namespaces, and that's just not really secure so I mean I could create a tenant with by default. If the tenant doesn't have any, even a namespace doesn't have tenancy labels. I may want to put it in a separate non-tenant that I, just let's say, deny to talk to anyone before I actually Define to which tenant it should belong with the specific label.

E

So that's actually kind of about security and I yeah I see we have some comments about that so uh yeah and they say who wants to say something. So let's do that.

B

uh No I didn't mean to cut you off, though, whenever you're done.

E

Oh okay, yeah, then well uh comments or questions about that. Or do you think we should like give that user story makes sense or if we should change it somehow.

B

My my only thought was that at that point it feels like what we really want to write is a default deny right like it seems like what you're describing isn't too unique to tenancy right like I, could do this by saying default deny and then, if you belong to a tenant, you're allowed to talk uh within the tenant and I. Think that would capture is that correct.

E

Well, maybe some use cases can actually be solved like that, but that literally says that if you have this use case, you cannot do that with just the tenancy and that doesn't really implement that fact that I want tenants to not be able to talk to each other, because there are namespaces that are not a part of any tenants of tenancy doesn't apply to them at all. So basically, what actually protects me is the default deny rule which I may or may not want to have.

E

Actually so it's I I mean I for the simple use case. I guess there are ways yeah to implement that with like extra objects. That makes sense, but that's not exactly.

B

E

I mean I just I.

B

Don't want an implicit intent, I'm I, guess I'm, okay, with someone saying if a if the namespace has no such label, then put it all in a bucket and deny traffic there um like as long as they explicitly have to write a rule to that effect. If I don't mind so much I'm, just saying that at that point, that's not specific to the intended the API itself right, like that's just saying. If this label does not exist on this namespace, the pods just cannot talk to anyone.

B

It's kind of like a default denim.

A

Yeah I think I well, plus one or a rubble on this one like when I think of a tenant right like you're, either creating hard or soft tenant boundaries which you covered in earlier user stories, but um kind of a generic general rule is right. If you're a tenant, um you want to be able to set up walls between everything else like like a tenant, knows of itself that you can either allow it to talk to other tenants or you can allow it to talk to maybe even other resources in the cluster.

A

But at the end of the day you should be able to easily express with a default deny like I am a tenant. Don't let me talk to anything else right, so in saying that I think a default deny would make sense in this case, rather than having like thumb, explicit. No none, tenant.

C

Right, I think the pro my problem with that is um think of a really uh uh obvious example. Right Coop system does that Brit belong to your tenant, I, don't I, don't think so right, obviously not now. What's sort of like the four group.

C

Do you want to put it in like if we don't label a coupe system with you know some tenant label, because that gets sort of like isolated by default with the others, and that's really a bad idea right, because that's why you know all the that's how all the tenants would not maybe talk to DNS, for example right. So in my opinion, what the tenant label or the tenancy API should do, is that it defines a sets of label, and you know for the namespaces, which has this label.

C

We consider them as attendance and then you know all the requirements we have on tenants which is written in the rules applies to them, but you know for the namespaces that doesn't have these labels. In most cases, they don't really actually belong to a talent, and you know this those words be sort of like the traffic that isn't sort of. Like captured by this policy- and maybe you know, other policies in the cluster uh at an admin level, policies based on I mean kubernetes.

C

Sl policies will touch base on those kind of like traffic and say if those are allowed or denied. But it's not you know the tenancy policy concern per se right. That's my my take on this and it's sort of like always my take on this stuff, but.

E

Yeah, well, that makes sense, I, I guess. The only difference is that for like this kind, this user story or use case is that if you can implement this with just the tendency or if you need to Define a tendency in a different way in a different way and create some extra ANP or some other rule, that will actually do the security that you want here. That is explained in this story.

B

E

B

B

um Maybe this comes back to another discussion we were having, which you know in my mind, tenancy makes sense as part of A and P, not a several parallel universe in, like you know, I was thinking of tenancy being a type of selector or you know, maybe a more sophisticated one, but it's still part of the same priority rule, in which case I think it's more natural for users to expect that behavior, but they're sort of like Yang said they pick a label that defines tenancy and things can go into tenants and then there's other pods that don't have that label and they I think naturally would fall into writing.

B

A and B roles for them.

G

A

I'm plus I'm, plus an arrow like I, think a lot of people here are thinking about tenancy at this point as an extent as a feature of amp, rather than its own whole separate brand new, sparkly, API um and I know we were trying to push kick that can down the road. um So it's a little bit tricky, but.

C

A

C

Is tricky it is tricky because I I totally see Nadia's one here, because in amps, uh when you do a subject, you either do a namespace connector or a pause selector, and you have a really defined set of parts that you apply in this policy to before a tendency.

C

If we don't Design This correctly, it can potentially apply the policy to the entire cluster and but by the end of the day you know the the the pods that are the namespace that are affected are actually those who has those labels, and this is a fact, that's kind of like implicit in the current API. We we sort of we have it sort of like expanded, pretty clearly so that what that's?

C

Why why it confuses people, but so um I, I I, do I do believe that it's a valid point um I just I, just don't think that uh it makes sense to me to uh to to do something like because we have this problem now. We wanted to extend the tendency to the namespace that I actually don't have this label uh I I, don't sort of I want that we wanted to fix the original problem, but I I don't want it to sort of like extend this to to do something else to.

G

E

C

That's what I'm saying.

E

Yeah that didn't really mean uh separate apis, necessarily when I said that I just mean like you, either need to only talk about tenants, you're, like whatever rule you define, even if it's a part of A and P right. You think what are my talents and then you either have the whole cluster covered, or you need to remember that you literally need to label something to become a part of that and to be selected for tendency and then I guess.

E

The main concern here is that I probably do want to label every namespace that I intend to be a part of a tenant at some point, but to ensure, like the proper security level, I need to make sure I need to label that before I create any pods, as I said like to just make sure they don't have this period of time before I actually decided which tenant I want this namespace to belong to. So that's, basically the main concern.

A

Well, a and I maybe I'm wrong, but in most like real world production deployments right, you have a cluster admin, that's spawning new namespaces or projects for you right. Aren't they in full control of you know: I've created a namespace with labels x, y and z, and the fact that I'm creating a new tenant or I'm, adding this namespace to an existing tenant. I guess it's kind of a use. Question.

F

G

F

Engineering platform, we Grant teams the ability to create their own teams, which creates namespaces, but we enforce certain labels and we do that in in.

E

G

F

E

I mean: is it just an extra level of what you need to make sure you can see? It's like the you need to be sure to label all the tenants just to enforce the correct tenancy right.

G

And I want to do that without.

C

C

Yes, yes, that's true Nadia, but but you know to be honest with you: I don't think this is problem with 10 and sell right. Think of the nail polish we have right now like like. If I wanted to do some, we don't do explicit tendency definitions. We just do a specific tendency scenario where we say engineering cannot talk to accounting, for example right, so the amp will look like. Let's select all the engineering name, spaces which is label equals app because engineering and then we deny traffic to label app equals marketing. Something like that right.

C

So if the admin do you spawn up a network policy that doesn't have label app because engineering that new namespace fell because it doesn't belong to the subject of the admin now policy, so the so the problem described does exist and I I agree. It's a valid problem, but as long as admin now a policy is a label selector based approach for influencing security. We can't really get around that right because it exists now, it's not specific to tenancy, so so so to ensure that the the policy behaves at as what it states.

C

um I do think, there's some sort of like responsibilities on the users to ensure that you know the name spaces that created are correctly labeled so that they belong to um a policy where they wish the namespace to be subject of that's I, I, don't know if.

D

C

That'll make sense yeah.

D

E

Yeah well, and actually I mean there are, as we've discussed, there are basically many ways to kind of implement the same use case with just different extra actions right so I guess the only Improvement that that would give us is just to have this one less conditioned one less action to do. On top of these to just ensure that and well I mean that is not absolutely necessary, just like if we want to allow these or not actually so.

E

Yeah so uh I don't know if I like want to uh what is the correct, like outcome to write down? Maybe we can continue that discussion into these Common Thread and then I can either leave it or remove this user story completely.

C

Maybe yeah uh yeah, let's do it, uh but but I do have one quick question for you um is that um in the default tenant uh story that you're creating right, so what you envisioned is um Let me. Let me know if that makes sense right if you have an amp which applies to all all the all the the entire cluster and you say: I Define the tenant, as you have a team label right, it's team, accounting, team marketing or whatever it's out on the team that you label.

C

So what you're saying that is for the namespaces that doesn't have a team label they're sort of like all the capital Verizon to one. You know, D4, tenant, sort of thing right and let's say the amp has a rule that you know I wanted to just deny Ingress from any other tenant.

C

What that means is that if this policy applies to the entire cluster, then no tenants will talk to DNS by default. Right is that is that a fair assessment office, because because Coop system will belong to the default tenant, which is not part of any other tenants. So if any of the tenants want to initiate a connection to to cube system, uh let's say we can add all egress to to other tenants right. So that will fail right because they are on different boundaries.

E

And how you may want to jerseys right so, first of all like how you may want to only Define the tendency, for, let's say user created tenants, which you can identify in any way or you can just list the names, the system namespaces like DNS or something else like Cube API. If you want just to not be a part of this tendency at all right, so basically everything else.

B

E

Be a part of the town and then another thing is you: can your tenancy May probably be something like trusted and untrusted tenant and then the trusted one will be actually the one that may talk to all of the cube resources right. So.

G

E

Everything that you want to talk to them and then you create a new one, and you want to make sure that only the allowed one The Trusted namespaces can talk to DNS and qbp and I. Don't know what right then, before you labeled it, it actually has access to everything, so you need to go and make sure you apply the right tenant label to that. In that case, if that makes sense, so there are multiple intentions that you actually may have I guess as a cluster admin. For that example,.

C

Okay, yeah I need to I need to think a little bit more on that, but that's definitely a continued discussion on the on the land pad.

A

Okay, I left a note that just said, like lazy consensus is, is creating I can implicit non-tenancy is not the best, so I feel like the room's kind of leaning towards getting rid of that user story, but just know that continue that and if it needs to change change it. um But let's try to like have it nailed down by next meeting, meaning like Let's, either move.

G

A

It with this user story, specifically because it seems like the main discussion, Point um or not sound good.

B

A

You cool great work with us, not yet. This has been a really complex, n-pep and I appreciate you sticking with it because and for everyone for taking a look. It's it's been a good good group effort here. Super awesome.

E

Yeah thanks everyone for reviewing that, and no that's, not the simplest and Pap to talk about, and also if you want to discuss that in a more interactive way. We can try to do that in slack or something like that if You Don't See me or anyone else to be responsive enough, even though.

C

I'm trying yeah yeah the GitHub carbon system is a mess.

A

It's not built for these long-winded review cycles that we end up with in enhanced proposals like it just falls apart completely so.

E

A

E

Make some kind of a rule if sorry, if the discussion is longer than 10 comments in the same thread, we just started a new one in slack. Maybe maybe we can try that.

A

Yeah I, like that idea, like whatever you think, works um as long as we kind of stick to it, I I think if there is room for like quick, iteration and slack it's way better and then in the conversation in GitHub, you can literally copy and paste the slack discussion, link and stay like go.

A

Look here, I mean it might time out in, like a couple years from now, but yeah cool sweet sounds like both these and peps were moving along, um as I said, before kind of in line with npep work, uh the egress support and pet merge last week, so huge kudos to Surya she's on the call for pushing that through and she's opened up some drafts for actually looking at support. So we have some next steps coming down the pipe which is really awesome, um looking forward to seeing those.

A

So, let's keep on pushing um I, think tenancy is kind of the next in line to get merged. So let's try to keep focusing on that. um I'd love, like I, said to have these stories kind of agreed on by next session and then move forward from there fqdn, hopefully shortly thereafter, because otherwise I think. If we don't set like lines on ourselves, we can go around in circles very easily for a long time. I'm not saying we're doing that with tenancy yet, but we could end up there.

A

It seems like we've narrowed down the discussion to one user story. So that's awesome.

A

Cool um and yeah so for folks, not who haven't been here the past couple weeks, Hunter has actually been working on some cyclonus stuff, which is awesome, he's been deep, diving into it, and I saw he left a note that he's not here this morning for doctor stuff, but um he's been working on cyclonus looking at what we need to do to add and P support and seeing how we can kind of break it up into usable bits for us, um so some exciting stuff coming from there, uh if you're interested in kind of the tooling and observability and conformance testing side of things um stay tuned to any issues or PRC opens that'll, be really relevant cool.

A

Okay, that's all I got is anything else. We got seven minutes left the folks can take up.

E

Quickly, reply to the question Brian actually had in the chat, but I didn't have sure it's time to.

A

Replace yeah, please do yeah.

E

A nice example of different teams, let's say like Finance team and that uh check out team services and if they want to actually Grant access or like allow connections between each other I guess that would be a case for admin Maybe if they actually own different things, but like from the. If there is some tenancy defined to just set different teams up and like allow or deny communication between them. That definitely would need to happen on an admin level.

E

But then, if just let's say, one namespace denies everything by default, but then it decides that it wants to allow access from some other namespace. That probably will just be a network policy. If there is nothing else in the cluster that actually restricts that right, because if there is something defined by admin, then probably admin had their reasons to do that. So then you need to go and talk to them. And then, if it's like just the namespace policy, then probably it will stay deep space.

F

Right, I guess more on learning is like it's. The tenancy API itself meant to be used by teams operating within the platform or by the Admin.

E

D

By the Admin yeah, it's part of admin, Network policy, it's for the admin.

B

um There there.

D

Are there are two different user stories? One for I want to create tenants where the tenants can't ever talk to each other, and one for I want to create tenants where the tenants don't talk by default, but they can choose to, and that might be part of what you're you're asking as well yeah.

F

A

There's some really good analogies in the user stories there that help me kind of understand it. That's that's what I was thinking so um should be covered, but admins for the win here. Maybe in developer, Network policy will come up with some crazy developer, oriented tenancy there. You go.

A

Cool anything else.

A

Okay, on that note we'll end a productive meeting, thanks for coming by everyone. I really appreciate it and uh let's keep pushing forward we'll see you next time.