Kubernetes SIG Network, 29 Mar 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Multi-Network Community Sync Meeting for 20230329

Description

Multi-Network Community Sync Meeting for 20230329

A

A

All right that started I think it's dead. Welcome everyone, uh the uh March 29th um community meeting for multi-networking project um today agenda.

A

We want to talk about some stuff related to what we did yesterday, but the first thing is: we are increasing the frequency of the meeting, so it's going to be moved to a weekly meeting from bi-weekly, so be mindful of that, and we try to kind of speed up our work around this project.

A

If there are any other agenda items, you wish to add just feel free to add them. We have some topics to discuss around what we already did um so, let's go over there and then, at the end, we're gonna move back to our regular discussion about the API. So the first thing is something that kind of came up through slack channels: discussions where how the whole kind of object can be used, and what does it mean and I want to kind of maybe give more uh description of that?

A

Let me move back to to.

A

The cap that we are working on to the dock, we are working on what I did I slightly enhanced it with some with additional examples. What I want to say is the Pod Networks parameters. Field is a optional field, and you can imagine about this as a enhancements to someone's implementation, that's kind of and and correct me if someone thinks differently. I just want to kind of uh provide an example of how this can be used with or without the parameters reference.

A

Okay, so uh most of us are used to something like maltus, where you have a plethora of options: combinations of what you can configure with multiple cnis configuration or even different ways of changing things, so you can go. There is a lot of flexibility configuration of this, and this is where the parameters ref is for.

A

This is for how you would reference this, this network attachment definition with whatever combination you wish to have- and this is how you can create your various versions of a pod network, but that is not the only way to go about this. The other way to go about this is just go very uh simple and I'm gonna use here an example as a something like a V switch. Imagine you have a and I'm borrowing here, I think Nathan, something similar to uh to to what you're thinking of doing where uh I haven't.

A

Imagine I have a implementation where I do not expose any configuration. I know how everything is connected. I do everything in the same way, my network is to identify separate networks for isolation. That's the only reason. I have my pod networks and to have a means to properly to just pipe connection to a specific pods and to have a means to identify which parts which want which a network, and for that reason, since I, don't have any extra configuration.

A

The only thing I really need is how and who's going to handle uh ipam uh I see. There is a mistake here, so let me just quickly fix that.

A

It's not internal uh in that case, I just need the means to, as I mentioned reference who's connecting what, because the underlying infrastructure implementation is done in such a it's done in generic way that it's always the same, where I'm just piping a connection from my V switch to my pod and imagine now in my implementation, I make it so that each pod network has its own ID within my implementation in such a way that all the networks are isolated and then they are uniquely identifiable.

A

Is that clear or or did I? um Is that somehow clear for folks.

B

It makes sense the one question I've got is so you've got your provider there. That indicates that this is just a v-switch or whatever it might be. No.

A

B

A

Provider will mean that my implementation and I'm, like company X, that's my implementation to indicate that that's how I'm that I'm handling this network and I'm going and I know what to do the rest right and so that that's only indicates who's doing it handling it not so.

B

But does the kiblet understand that provider field and therefore know who to call out to in order to actually implement the network? No.

A

Cubelet has nothing to do here right, so um it should be depending on how you, okay, maybe step back depending on how you are um handling the, how you piping and handling the whole administration of your uh data plane right in in your node, uh and that's something that we need to get to to drill down to of how far we want to engage cubelet to this whole thing right or the cni right right now we have the very straightforward approach um and the way we have like most some of the implementation done.

A

It is in a ways where it's even multi's, moving towards the ways where we have a demon set and the uh that the cni is only a um just a small client grpc client that just calls to that demon and the demon does the whole thing and it has the full knowledge.

A

So basically, my cni is just I just just to send signal with which namespace I am to operate on and that's it and the rest is handled by the demon set, which is a fully fledged agent which actors to kubernetes API, and it can do everything- and this is a common pattern across implementations uh which uh can be done that way and of course we have Michael Hill, which may be in a future. Cni can change and it can adapt to the spot Network.

A

Where, instead of doing this agent thing, maybe it can be more elaborate and don't Direct in the cni, and this is up to again app implementation, because this is just an API and then we need to Define an implementation defines how it's going to be later on done. All right is that did I answer your question. Yeah.

B

That makes sense so in my mind, the provider is just a tag and from the point of view of the kiblet well, it's got a tag and it's just going to pass that tag over an API and it doesn't understand it. So the whatever is so it is up to the implementer to put something behind that API on the kublet.

A

Would even pass that around anywhere, because you know you have to be Backward Compatible, so the cni config doesn't go away uh with this at least initially I. Don't think this will go away unless uh yeah, that's my thing, because it's it's integrated with CRI as well, so it's not just so that we can all get rid of of cni or just straight away change. It so now pass this instead of something something else so be mindful of that that this is just an API thing.

A

That's in pi and I would not think um cubelet will pass it unless we, we will have a discussion around this, and this can be changed and okay, it should maybe, but for now I don't see this being passed around by cubelet uh Tony.

C

um So so they are looking there above the yamu file. The this provider field seems to be the optional. So today, what does it mean? The empty is this for the Casual World code or some I, don't I, don't know so.

A

I I gave you this description, I'm, not sure whether I did it right or not. uh But yes, it's a it can be a wild car. And basically it's like today's imagine uh today we have implementers for Ingress same story right. You can have an Ingress implementation where it will recognize its own object and that it uh and that it's going to only handle those ingresses to which it's uh it's marked as or it doesn't.

A

It never bothers to look at the provider field and it handles all of them, but that means that the implementation is um not compatible with other implementers. So basically in one cluster, that would mean my implementation cannot coexist with other ones inside that cluster because, let's say I, don't respect and I don't care about the provider field.

A

I I I will Implement all of them, and that means that I am not cooperating with other implementers, because, whatever field you set there I will I will I will try to use all of the Pod networks right, that's what it boils down to and that's what I'm I'm mentioning here right. So it's up to implementation implementers if it's up to the implementation to respect this field, that's what I want to say and if that's something, that's uh that's not the case and I'm I hope I spelled this out here uh it uh correctly.

A

If not, then please refactor. This, give me suggestions on how better write it down, but that's what I would like it to be is that.

C

Kind of a little concerns about the the future case, so so the at the initial deployment of the this mouse, Network or I mean the airport Network. At that time they they. We could expect that the one provider is running in their one. Cluster I mean that, for example, the uh one kubernetes running the one Motors but I'm a little bit afraid that in the future, the more than one I mean that the two or more provider is running in the same cluster.

C

At that time, if the provider a and the provided B have the uh have a different meaning of the World Cup I mean that the photo air cares about the provider empty and about the beakers about the provider empty. At that time, one photo Network May consist with two or more provider, Network I'm a little bit afraid that this stuff, so so.

A

That will be tomorrow that will be on a bit pressure on the implementers. So let's say uh in in let's: let's I will I will pick on multus apologize, but I will pick on multis and what can it be? And you will see someone saying: oh maltus doesn't cooperate with others, so that will be a a con against maltus right. So basically, you would like to remedy that so that you are not you can that you can collaborate with other providers in the Clusters in.

C

That way, like a you.

A

C

But API side the API provide a way to in the interworking with the multiple provider. I think so I mean that if the provider is the kinds of the non-empt the optional, this is the required yeah.

A

C

The each provide the how do I say: So.

C

Currently we have the empty, and then this uh can interpret us the multiple meanings, and then you are thinking that the provider should care about this meaning, but so that if the provider is non-empty at that time, the uh one provider value is for one body, I mean that so the the amount of, if you want, if user, wants to create the amount of network for uh if you create the both Network for models, other time provider, filling the maltes and it's okay and then the other is ignored, so so I'm a little bit afraid that if we accepting the empty I'm not optional at that time, the this empty, having the multiple meaning and- and also the you you ask uh you you mentioned the empty- is depend on the implementations provider.

C

So at that or more provider is uh having a different semantics for a provider empty. At that time, this may be getting the conflict so.

A

It's I see your case and I will tell you. I will counter that with, on the other hand, with the user, uh with the ux of this all right and imagine, uh 90 of use cases for this is going to be used only one provider.

A

You see that right that the 90 of use cases out on the world, it will be I'm, gonna use just one provider, and in that case, why would even I bother to to provide this and complicate my life? So that's that's the trade-off right.

A

We either force it and let's, let's choose one uh I'm, not saying, but my in my opinion having the optional serves the probably 80 90, which will just use only one provider, and then they will never specify a name right um and then it will be out to implementation and how they handle that.

A

So, let's basically pick your poison right and this at this point: either we enforce it and then we we put a constraint on the user to always provide and all the implementers always have to provide this value or we make it optional, and only when you really need to have the coexistence within one cluster. You have to specify that, because that's what it boils down to right, it's it's that's. Why I think it's a it's a matter of ease of use! If and in most cases that will be.

A

That will be that case, where I don't care about the providers, because I'm gonna have just one in the cluster and don't worry about it. An implementation then can handle that same way right uh where uh you can have a flag in your configuration stating okay in this cluster, you have to be uh compatible with other things, so uh just ensures you uh that you only look at the provider specified uh networks or you don't right so, basically up to your implementation that you can support both cases.

D

But shouldn't be API really describe how the implementation should behave and not the other way around.

A

D

E

A

But, but how would you enforce that differently on the uh on the apis here and yes- and this is where Tomo is saying right like make this: let's make this enforce and then every provider should have its own unique value.

A

We can do that. This is but again this boils down to the user user experience, because that's what it will boil damage on, because this is very um if we go to what you're saying Olaf, that that it has to uh I think that was the goal of amateur yeah.

A

If we are saying uh that um we should enforce it, let's do that. Then. Let's change the optionality and go to towards that.

D

I'm not I'm, not even sure, if it's really about an enforcement, it's uh I thought that the comment actually is about that. It just needs to be properly described. What it means different field is optionally, if it's empty and so on. So what is expected from the implementation on the behavior that you could actually kind of according to the API implement this?

D

It shouldn't be so that you have um kind of field here in this API and then um it's up to the implementation, how to handle this.

A

It will be, it will be uh all the fields are always like that right, I can our iPhone 4. It will be up to implementation, whether they respect it or not same story. So on that one, whether we enforce it or not- that's that's can be said with every field. Even if it's mandatory, we can say that weather implementation supports it or not.

A

um This is a recommendation for that field. So right, you do see that right if we say whether implementation respects it or we enforce it or not,.

C

I I answered to your point, so so the initially the uh maybe the user may provide the one provider for in the cluster and then let's, let's see so there, maybe the if user is trying to provide the deploy, the uh two or more provider in the one cluster. At that time we should care about that as the next version. That's.

A

What I'm getting at right, so I, don't want to enforce now every implementation, like, let's say, multis, whenever this comes out now onto the first thing they should have to do, is Implement provider and how to support it right. They don't right that can be something down the line when they will feel that when you will feel that okay, now we want to be there there are. There are issues in GitHub stating oh you're not compatible with implementation blah or for whatever right.

A

So then you go and okay, let's and you can prioritize this accordingly right. That gives you the leeway on how to do that.

F

Does that sound yeah, so is the parameter ref um any structure, or must it be key value.

A

What do you mean this has to be an explicit pointer to an object, key value, but.

F

As you have specified it in the example, you have only like key value, key value, key value, key value four times. Is it possible to have an array or a sub-object or any object, and if it was Json, would it be a an empty object where you can fill in any structure in it? So.

A

Whatever this object is is up to you and then that object can have additional references. I, don't think we should have a list here, because that means that why would you want to have a list here? Your reference to so so a pod Network should be described by one object, a custom object that you wish and then, if you want to have more references, more configuration, do it in your in your uh in your in that object right.

F

A

F

The answer is, uh what is in the provider? Ref is key value, nothing else. No.

A

List so this is nothing yes, that's true, so, basically, this is a specified. This is a specified uh structure right. So this is how you would describe uniquely an object in kubernetes. So this is a group kind, a name and namespace if it's namespaced right, so those are like generic ways. This is a common pattern in kubernetes on how you would point to a specific object.

F

Okay, good thanks.

A

All right, I think uh the next pointer was uh anything else on these params and what I said before we move on and I want to kind of describe this kind of head explanation on this, so that we everyone will are is on the same page on what we are thinking here, all right. Let's so the next topic was I. Think Pete was you about the use cases right user stories.

B

Yeah, that's right so I've been asking around here about user stories and this is kind of following up on our thing in it's like I. Don't think anything we've said so far really impinges upon the user stories. I've got but I've got some user stories from people here that I, don't think are covered with a list of user stories. We have now and I'd like to kind of, submit them to the group and see if we could get them added to the user stories in the cab. I.

C

Don't know if we.

B

Can do that and if there's a process, so what's the processary I mean I, don't have them properly written up yet I have to say and I do want to find out a little bit more about the details of them. But my plan is to what I would propose is I'll write them all up, I'll put them in a Google doc or something and circulate them to this group and then we'll get them into the cap. Is that the right or right way to tackle it? I.

A

Would say so yeah that would be the best, because uh yeah the cap is out there, the current user stories that we so tell you this, as I mentioned, probably in in the chat. uh The user stories were a description of what sort of use cases we want to tackle, but then the requirements were the rift out of them in the level that we want to support them right.

A

So if we said that we want to do uh support a use case where or one of the use cases were to use siobvfs and how we would use the hardware-based stuff input, try to pass that into the Pod, it doesn't mean we wanted to support srlv here in this effort.

A

It just meant that this is one of the use cases and the uh the requirements out of this is that the ipam has to be optional, so that because I can pass a hardware device into the pod which will never have an IP and that's the requirement out of this use case. The main requirement, I, would say out of this use case where I want to have an ability to pass an interface that will never have an IP okay.

A

So that's how the use cases Drive the requirements so make sure that whatever use case, you are proposing what sort of requirements you want to add from it. That should be your ear, because it's just the use case is just a driver to what the requirement has to be okay and we are driving here, the the how this whole work is based on those requirements and what which one we are tackling so right now you can see on the bottom of the cap.

A

There is a phasing that I did and that's what I'm proposing and that's. We are currently focusing on the phase. One.

B

Okay, yeah and I I think I think there are a few. There are a few things that might lead to tweaking of those requirements or potentially tweaking of those requirements.

E

The user stories yeah yeah.

A

B

Get them written up and then we'll discuss them, and then we can. We can take everything. Okay, that sounds good thanks.

A

A

Hey um any other topics before we move on to the API discussion.

A

A

All right, I'm, moving on to the next one uh and my sharing stuff I, am sharing the the doc right. I am sharing the cap file right.

A

Could someone I think I.

A

Which talk am I? Okay? No, not this one share this stuff. Okay, now I see so last week we um uh end our discussion on the conditions and okay I added a section to describe this, and basically when I went through the use cases the and if you look at the object itself right the object here in terms of validation itself, um it can be all done statically through just the inside the um through, even if you think about, if you, if this was a crd, it can be done statically.

A

What I mean you don't need a validation web hook or anything it can be done statically in terms of what I permit is. So this is just a string and a two and two values allowed and then provider has to be just a string.

A

The other thing is the parameters ref, which is just as well as strings, so this will be Auto validated because you have to just provide that and um I was thinking and correct me if I'm wrong I would not want API, because one thing that can be done here is validate if this object exists right.

A

But my concern here would be uh the controller, the core controller trying to to import a custom crd I would not want to do that. I, don't think we don't have Antonio here. He will probably tell us best whether that's a pattern or not, but I would don't want to kind of validate if this object exists in the uh in the kubernetes API uh by by the controller. So, basically, even if you specify this, this is just information for your implementation and this object Handler to know how to kind of match one to the other.

A

uh Any opinions on that are we in agreement on that one I, don't see any voices against so I assume it's okay, so what it boils down to then this ready condition would be only one possible uh failed reason, and that would be if you specify this field- and you did not add the parameters ready only then it will be a reason for the for the ready being false will be params, not ready.

A

So basically, the parameters condition is not true so and it's either you don't have it or or it's set to false for some reason, and this check will be only done when the parameters ref is set to uh to a non-nil value. So there is something defined here. uh Does that sound uh does? Does that sound, reasonable.

A

Tomorrow you had a you had a question about whether this was what this exactly mean. Does that kind of clear this out.

C

Yep, um the yeah today I just started to read this stuff so but roughly I'm, I'm, okay. So today and then the most important things is that we need to have the description about the which status means what and then especially there. This is slightly complicated. I mean that yeah.

A

And, as you can see here and I even add an example here, so imagine if you don't have.

C

That part yeah so yeah, you have it right, make making via career uh make that career and then also they are yeah I, see yeah.

A

So, basically, only if you have parameters rev, this object will be there and then for the params ready. uh It means that the uh yeah parameters ref, is ready to use right. So basically, whatever object, you referenced. It's ready for use. uh Why am I doing this? My reason for this is to give ability to gate Readiness of network by your implementation.

A

That's the kind of main, as you can see, that that's the main reasoning behind this whole kind of complication, because, let's assume I have my my my implementation is going to do something based on this additional parameter and then I want to get the the the the main Network by that right, because I have to do some operations or what's nuts so I have a once. You have a means to gate. Readiness of this object, based on whatever I need to do with my with my custom stuff.

A

So that's the main reason, and basically this is a fully optional, but I want to have this capability to I want to gate it right so before you're gonna become ready before anything. Let me check my stuff first and then I have a generic way, because this is very generic right. Every every implementer just has to implement the same thing which they just have to handle the params ready, type of condition, and they just indicate true or false, and that will switch as well the Readiness flag.

A

So this way we have a kind of a generic interface on how I can gate the Pod networks from my implementation, if I need to right.

A

So that's the main reason on on behind that and that's why that's? Why as well, I want to use ready instead of accepted, because accepted is mostly just uh it's if I'm not mistaken, accept that is used to fully validate, and only when you do that right.

A

So basically I applied an object and then I do additional validation checks on it uh and then yes, we could use just accept it, but we don't really do much of that for this ready, maybe in the future, when we gonna have maybe I'm not I, hope not, but if, if we're gonna expand this spec with more fields which would have to be validated and probably we can introduce the accepted field if there needs to be some additional validations done.

A

But since this uh there is not much to be done around this, that I I think the accepted is not the right thing and especially since the ready is to kind of gauge this and and be dependent on this pattern, ready.

G

Just from a like philosophical standpoint, I um I definitely would uh uh lean towards reducing the number of um fields and parameters here.

H

G

One thing I can see happening. Is the users pick this up? They look at a reference with all the fields um and they go bananas with them and if it was more simplified, then because one thing I see being a like possible reality is that you know we come up with as many ideas as we can for what needs to be in this object. And then it's just a grim reality that you have to Loft more of the instructions to the implementation, so I think yeah.

G

If we simplify it um and we let lofting to the implementation happen where it needs to be, then we do users a favor by not over promising from a huge number of API objects, but yeah.

A

But I see where.

H

G

A

Yeah, okay and but basically that's that's. This is where we all don't have to be on guard in terms of what you're saying, because I agree with you 100, any additional parameters should be pushed to your implementation. uh That's why we, you see our our.

G

A

With the with the initial proposal, which I had like crowds uh Gateway here, all those.

F

We cut them out and I agree on that, because.

A

Because that's a clean and and mean and clean uh that that's really good, so um I think we are agreeing on this one uh right, uh yeah I, agree 400 in terms of uh because this should be uh kind of representing what can be done. What sort of configuration can be done today for uh the default Network and the default Network? Can you can just do this? That's it we. We are expanding the whole thing with this, which kind of gateway to.

A

uh Hopefully it's not Pandora box, but uh but we open a gateway to you. Further configure your stuff, but today default configuration of the default network is kind of video. Just done that can I do I, want to have a ipam done by uh by the KCM kubernetes control manager or not, and then whether I'm gonna do dual stack or not, and that's the only thing you can do today for for the I uh for the network configuration today.

A

um um Okay, moving on uh the examples, conditions, uh the provider I I'm, starting to think, should we just uh since this our discussion uh previously?

A

Should we cut it out and and simplify even this I'm, not sure something to maybe chew on, or should we just keep it in case for future reference, even if it's currently just optional uh from what I'm seeing in terms of implementation on the core side, it's irrelevant because it Implement so on the core side in in case kubernetes, um this field will be just completely ignored, at least I I think, because it's it's for the implementation, so in terms of implementation and amount of work added, there is nothing in this one.

A

It's just a a a helper for all the other implementers to kind of use that so, okay, if no voices something to think about, maybe.

H

For the provider it, it just seems like redundant with the parameters ref and then like. What's the kind of differences between like a provider and like a map of labels,.

A

How how would I do this if I don't have any parameters ref? How do I indicate my my implementation? I, don't I, don't, as I mentioned my previous example. I, don't need any parameters right, I! Don't want to complicate my my implementation mind, but it's very simple. How do I indicate that this object is mine, then my my mind in terms of my implementation handling, it.

H

Yeah and I think.

A

H

Saying is like it feels like redundant or like a duplicate. That's just my like quick thoughts on the matter.

A

You could yeah I would say that. Okay, you identify your Provider by the the value here right. It's Network attachment definition. You definitely know it's multis right, but what exactly? This won't cover the case the the upper here case this generic example where okay I don't have any providers. How do I indicate I, don't have any parameters in my implementation. How do I indicate that this object belongs to me to my implementation? So that's the reason to have it if we were to kind of have the whole provider thinking.

A

So it's not fully redundant uh Michael, because this is optional right.

C

Yeah but I I think the in the future. If someone so two or more provider is utilizing the one shared crd other time, the provider is pretty useful to identify identify which provider is.

A

Fairpoint a good point: yeah there is that case as well yeah you can have two guys using the same crd as well. Yeah.

C

Thank you, so maybe the Palms left is kind of the additional optional information supplying the additional information to the airport network from the provider point of view, yeah.

A

I think that's it! That's exactly what you just said. That's exactly that's hit on the nail all right all right. Moving on then so.

E

Yeah, so just on understanding the significance.

C

E

Here right, so this external anyway, the parameter reference will be always present. So I guess this parameters. Reference is something like um right.

E

So I mean without this parameters reference uh only the kubernetes I mean if only the iPhone time, iPhone type is kubernetes only then we don't have this parameters, reference object, in which cases I mean if it is, if it is going to be external, definitely there would be some Network attachment definition right, otherwise, otherwise, uh so.

A

It will be whatever you're implementing, so this is if I know- and this is these are examples- and maybe I will do this, because I see this might be missed so make sure that you are aware iPhone is completely independent out of I. Will do that, let's say maltus is, can just leverage the kubernetes ipam story and and leverage that I'm not sure, that's possible. That will have to be some liquidation, but iPhone doesn't indicate who is doing it so so I'm I'm just doing something like that.

A

So ipam is completely independent, but now to what you're saying uh this will have to be right now: I'm providing multiple Network attachment definitions, because this is an existing Upstream example that most of us are aware of, but you can come up with your full object. That's supported by your implementation, which doesn't it's not multis, it's something that you grew up created in-house and then you can. You can specify that.

E

Yeah I can specify that in parameters reference so I I'm trying to get like what is the? What is the significance of Provider here like if I go to parameter reference and I will check the kind object. Then I know like what is the implementation there.

A

Right I mean right, but but you keep in mind, we have to support all the use cases and- and this this use case, how would I know that this object belongs to me if my implementation doesn't provide any parameters, all right and what's Tomo said this is an extension to what your implementation needs. It's not the indicator of your of your implementation, okay, uh because if I, let's use in a case where my implementation doesn't need additional parameters, how would I then indicate that this object belongs to my provider right?

A

That's the reason to have it right, so you have to support both cases.

A

Yeah thanks. Okay,.

A

All right so contentious thing, uh moving on characteristic of this whole object. The two main things one I want to ensure this object is immutable, so once created, it cannot be changed. So basically my reference, uh my reference, my ipam story. It's once created it's, it cannot be changed. You have to delete the whole object to change anything, that's one part, but then there is another bullet to this whole thing. This object cannot be removed if at least one pod is referencing it so I.

A

We need to do something and handle a case where I have a pod, referencing a network and then I cut the network out out of that pod. Is that possible? So in, however, your implementation does it? We need to ensure that we are preventing users from you know, cutting out an object, that's still being used by by the Pod, so.

C

Who, who, who is the responsible to uh enforce studies rule I mean that their core kubernetes and.

A

That will be. This will be core object, so it should be easy to just if you try to I I will have a flag on an object. Maybe this is something that we need to spell out here. How I will indicate that this object is used by a pod? So that's something that let me let me uh talk with. uh Maybe someone has, unless someone has a idea over here, how we can indicate that the network is used by a pod, but then uh inside.

B

C

A

Have to deal with Fi I would think finalizers if this is a crd. But since this is not a crd, it's a core component I can tweak the API to say. Oh, this object is used by a pod. You I will throw you an error. You cannot delete that it's used by a pot.

C

Yeah, so from the uh from the functionality of point of view, I make sense, but I'm I'm really wondering if who cares about that? I mean that the quack one it is I mean that the KCM will care about that or provider needs to be careful about that by having the uh the animation controller or something exactly.

A

So so so so uh who would care? I would say just the, and this is something that maybe someone will push back from the um API side, saying that oh, this is a very custom handling. We should not do this. Maybe that will be the case and then okay, maybe not, but who would care I, would say the sanity of like user experience, all right, so I have a pod and I assume with with today with maltus network attachment definition can be deleted right, yeah.

C

A

So imagine I created my network I created my pod, that references, a network at nod and then the Pod is up and running and then I deleted that nut right and now I'm, looking at the Pod who what you are connected to and now, okay, where is that not and it doesn't exist anymore, yeah.

C

A

I, don't have the state of that guy right or if my you see, that's that's the problem and then it's up to your implementation to handle it right. So what I mean by that is? Okay, if you delete that object, I will do something about this right now you decided to do nothing, but imagine if I want to be like a properly sanitized thing to have some properly sanitization. What I will do if you try to delete this object, let's say I will just delete all the thoughts with that Network, because.

E

A

Can be one of the decisions for the implement station as well yeah.

C

So before that that will be divided, okay, so so for each provider that they may cares about some of that stuff, but to the in the generic case, I mean that, as you mentioned in the uh this cap, the all providers care the there is no difference among the providers at.

A

This point it doesn't matter yes, yeah.

C

And then the code network is uh cannot be removed when the uh one one more portal is used. So this should be I mean that this is a pretty common stuff, so they are. Maybe these stuff should be care about the core kubernetes, as the functionality I think otherwise, yeah yeah, maybe they have each provider May, skipping or something or stuff, so I'm, pretty afraid that.

A

Yeah, so that's that's for the reason as well right eventually, maybe some providers will have the capability of what I want to say of mutation of the object in in any.

C

Way, yeah something something special: maybe the provider at the several provider could add the additional functionality but to the both these function. This this tool is the pretty minimum, and then this this is the required yeah.

A

Exactly so so what I'm trying to say is if someone eventually gets there that oh I need this, they will have to just create a cap and say: okay, let's, let's loosen up this requirements because I we have a means to now be more Dynamic, but I think that should be like uh in in three years. If someone comes up with something and uh and will try to kind of create that uh Nathan.

I

um Maybe a question somewhere related to that. um Do we want so we we have status for all networks coming up. Do we also want to expose statuses for networks going down so being removed, so maybe basically provide a way to for the implementation to tell uh that network is being deleted and that network is fully deleted now uh the thing the reason I'm thinking about that is: if objects are immutable, people wanting to update the object will probably delete and recreate um probably race condition can happen in those areas.

A

H

A

Sort of what do you have in mind? What's so?

A

What race condition you have in mind, because if we those two conditions together right, if I, if I, don't have any pod reference in this network, um what sort of race condition you have in mind.

A

Referencing it or is it about between Electronics yeah.

I

The network itself being created, I, don't know so, maybe when say it's not a problem and uh and say it's it's subject to the implementation to handle, handle this internally.

I

uh I was thinking about to programming physical device, or something like that that you you need to to wait before it has happened before being able to do it again.

A

So this is depending on how far you want to go with that right. That's why? uh If you have the parameters reference, you could always do this and get it through the params ready right. That's what that's! What you're saying that I need to program some data plane right, so I have a centralized, and this keep in mind. This has to be sent in centralized means right. It cannot be uh per node, it's only in centralized ways.

A

uh We will get in Phase two a case where we want to do status per node where okay, this node.

B

A

Still programming and then only then I will report something to the node stating this network is ready on the Node. I want to get there. So maybe that will help you with what you're saying, because then we can indicate Readiness of a network per node and.

J

It's something that.

A

We want to- and this is one of the requirements- this is phase two, where we have a selective uh availability of the network on the in the cluster right, so maybe that should that should probably uh support and help better your case keep in mind. This is all centralized, so be mindful of that that that's the case.

I

B

I was going to extend what you were saying is there? There is one use case that maybe we might want to consider, which is what happens when we want to get rid of a network. So I've got a network out there. There's 100 pods on it and I'm now going to shut down and make that Network inaccessible. Now I can't just delete it because pods are relying on it, but I do want to maybe stop what's being scheduled on it is there? Is there a way of doing that with.

A

This that's a fair point, so uh no like right like now, we wouldn't have that capability right. How to what you want to do is block creation of Newports on it right.

B

Yeah exactly so that, ultimately, I can delete it when I have manually gone through and found all those pods and deleted them or whatever.

A

Should there be a uh so sorry Doug, but should we have another flag inside the stack indicate like up down and that field can be monumentable?

A

um Should there be that that, like a administrative, app down of a and then basically that will indicate that flag would indicate Readiness right with control readiness uh stating uh that if the flag is is down, then I switch my Readiness flag down because- and there will be another reason administratively down.

A

Should that be the case because I see Pete, that's that's a fair use case and we don't have a means to prevent. Okay, don't don't create anymore. It's like um it's like with the today draining capabilities right inside pods and deployments. uh We do have some sort of ability to State. Okay, don't no more new connections to that to that workflow, because I need the.

B

Other area where it's something like that might be interesting is I'm, just think of error conditions if I want to delete my network, so I want to delete my network, but I can't delete my network because there's some pods on it, but I can't delete those pods because the node's in a bad State, those kind of conditions- I, don't know if there are error cases there that are going to be problematic, being able to at least say, stop stop putting pods on this network, so I can clean them up and delete.

B

It might be very useful, so.

A

B

This stage, but it's certainly something that feels like it might be useful, so.

A

Now the other thing when I think about it metadata has deletes timestamp right. If I were to delete the object, we we will block the deletion, but we will set. We can set the deletion timestamp and that can be the indicator that I am deleting this object and and then, where the basically I would set my ready uh to uh in a deletion in progress or something like that right, because I should we do that. Let.

B

Me write it down I like that. Actually I think that that would cover the use case it covers both. It means that if I want to delete it on the reports, I can do the delete and say right, nothing else exactly.

C

B

Then out of band I can get rid of the pods, and once the pods are going it'll, clear.

A

Up yeah and then the reason will indicate that uh deletion in progress and then ready will be false, and then it will be up to the implementation on how they handle it right, whether they do whatever they do right, but but basically that will mean that it's up to your implementation and you'll have that signal. Okay, let me try to add that.

B

A

Any other comments on this on that immutability and and adapt removal.

A

Silence as an acceptance as usual, okay, let's, let's move on good I, wrote this whole app because we are, we are going crazy today, going fast, uh we.

J

Have four more minutes so.

J

If you think of Unix file system and so on, you can remove the file, but that doesn't remove the reference to the file object. That disappears uh when the last reference to this, but to say that you cannot do a management operation until the specific running condition. I've been met this a little bit weird.

J

A

The comparison.

J

Of the file, how do I know if I ever can remove a network? Because how do you stop someone from attaching to it?.

A

So this is what we were saying right: via deletion timestamp yeah. That was uh what Pete was bringing up as well. How you would indicate that, but I would you want to when you delete the object, we will set the deletion timestamp right and.

J

A

And then deletions timestamps with with, if you have that field set, we would change the Readiness to addition in progress. Yeah.

J

That's fine I can't really see today's I'm just listening. Oh.

A

Oh okay, yeah I will update I I left myself a comment to to add a new reason for Readiness uh to say that okay, we are trying to delete this object and the the the just the fact keep in mind that just that will mean that just the facts of me deleting the object, makes the object, not ready. That's how it will behave. That will be a a very non-standard like if you think about pods.

A

That is not the case for pods right pods will say that pause is ready because it's controlled by its Readiness flag, and this will be completely differently of how that will behave.

A

J

This is probably a reference, a network so yeah. Well, that's fine! It's nothing! It's nothing! That's gonna!

J

Stop anyone from removing the availability of a network on a node right, but that's outside the what we can specify here in a way yeah. That's fine! I'm! Sorry for.

A

No, that's fine, all right. Any other comments on that deletion.

A

Approaches: okay, so moving on.

A

So we have last minute: I just want to kind of note a default Network here and I did added copied from our uh the cap here at the top terminology, something that keep in mind that we are introducing to two new terms here. So we have a cluster default pod Network, and this is the network that Today class. Today's clusters are treated with, so there is a cluster default Network, so this is and that network will be. This is what I'm describing below this is the default Network and then the notion of a primary network.

A

uh This is the network inside the pod, which has the default gateway set. So basically, today, default by today, default network is the primary Network all the time. That's because we have only one network. It's always.

J

A

Default gateway, yes, I.

J

Think you need to clarify a little more because there's nothing that you can have many default routes if you use the RFS right, so we are just don't do it now, but I think it needs to be more precise.

A

On on a default, okay, vrf.

J

Yes, you see what I mean right if I start using vrfs I will have one before graph per vrf.

A

uh Let me just say that interface is used to the default gateway on the uh is that table one or does just if I say default, vrf that will.

J

Be sufficient, I think that's sort of that that should be enough. For now we can figure. I will think of something better, but at least yeah.

A

I will give you a comments.

J

They don't call the default thing vrf, they call it L3 context or.

B

J

Like that, but uh who care.

A

Left you a comment to look at it.

C

uh Sorry, the yeah time seems to be up, but also yeah, maybe they're from at the next meeting. Let's talk about the the default Network I'm a little bit wondering that the some customer won't some user wants to create the uh the network without the default. Network I mean that, like the across the network,.

C

A

That's yes, let's, uh let's get there, but I think uh we are not currently when we're gonna go to pod attachments, uh we I think we are. We are not um doing that and by the way I did yeah. Let's, let's end at this, let's I will add a pointer that we're gonna continue from the default network, uh but uh anyone please I did added.

A

You know discussion uh some text on the how I'm thinking of to attach to a pod our pod Network to report so how to change the Pod spec, how that would look like and then some status reporting about this. So uh please read through that and see leave comments. So that's something that we can discuss next week. I will update this more with the because we are going uh quite fast, so I will try to add even more on on um the next thing.

A

I think uh what I need to add on is um attach and the r pack how we will handle that. So that's the next thing and of course, if something's missing anywhere, please let me know or or try to add stuff on by yourself, um because that's kind of as we go I'm thinking about some of those, but maybe I'm missing something so uh I'm not trying to just dictate what has to be handled. Maybe there are some other use cases that we need to look at. So please help me with this all right.

A

Let's call it um thanks. Everyone.

C

D

B