Kubernetes SIG Cluster Lifecycle, 2 Feb 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes [SIG Cluster Lifecycle] Cluster API Provider Nested Office Hours 20210202

Description

#sig-cluster-lifecycle
#capn
#capi

A

All right, good morning, everybody, uh this is the february 2nd uh wow we're in the second month of of 2021. Already. uh This is the february 2nd recording of the cluster api provider nested office hours.

A

This meeting is being recorded and we post to youtube. So please make sure you don't say anything: you don't want posted for the whole world to see and we can get started there. Isn't a ton on the agenda so make sure that you go add yourself to the attendees list. If you're here, I dropped a link into there. I'll drop another just make sure that it's the top for everybody in the chat and zoom as well. If you have items that you'd like to add, please make sure you add those to the agenda.

A

Sweet and I think uh for right now, the first one I don't know ciao. Would you like to talk about the nested lcd controller that you created so far or do you wanna yeah.

B

Yeah um I can I can. I can start talking about it so so. Last week I implemented the nested etcd controller. There are still some minor issues left, but I think I generally finish the nasda adcd controller and I have already tested locally. It works as expect, but when I explain but when I'm implementing the controller I run into some problems or issues or maybe some features, I require from other controllers, like nested control, plane controllers. So I think today we can discuss all these features.

B

I think need to be complete by the nasty, the control plan controller and the crease can give me some suggestions if that sounds reasonable, so the the first one when I when I when I try to implement the nested etcd controller, um the problem is uh we? If, if you guys remember we, maybe I can share the screen that can be. Do you think I can share the screen.

A

You can, let me let me make.

B

Sure, and give you access to do.

A

That real, quick.

B

Your screen item is green, so.

A

B

Think maybe I can do it.

A

All right, you want to give it a shot.

B

Okay, I'm trying to share the screen.

C

A

Try it again, it probably work now.

B

So um could you guys see my terminal yep, okay, so.

B

B

So we, um if you guys remember when we define all this nested nested crd, we put the nested commons back components back inside each nested components back so inside this one. We have borrowed some fields, some some destruct from the um kobe builder declarative patent repository the first one is the common spec.

B

So we used to think we can use this one to um the version and there are two fields inside this state struct the version and then the channel, and I think we are going to use these two fields to specify which templates we are going to use to create this component right chris. So um so I'm not sure how to do that. Actually, I cannot find any like example, working example online. So so currently I just not implement this part.

B

So if you guys can see on a.

B

On the right hand, side I just don't I just don't do anything to.

A

Gotcha so you're just loading up the actual resources in uh as yaml and then parsing them and then applying them to the cluster. That way.

B

Right, correct, correct cool, so, okay, so if so right now, I just do nothing. So I assume that uh if the version and the channel does not set by the user, we are going to use the default template and we are going to host these default templates in under the captain repository. That's what I think so. Currently I only achieved this part uh using the default template to generate the nasir edcd egcd component.

B

um So I wanted to see. Do you think we should support the version and the channel feature in our first release or first version.

A

uh We can we can probably hold on this if you feel like it's, uh I mean so. I added to the to the uh office hours agenda doc. I added a couple notes as you're going through this so for for usage patterns and for well actually for usage of the declarative pattern. Stuff, there's a whole repo of examples that I dropped in there. That should specify how to how to structure these and how to use the declarative patterns like reconcilers that are pre-built.

A

um That might be something to check out. uh I'm definitely open to the idea of moving forward uh with what you have if it's, if that does not fit those, if that does not fit yeah.

B

Okay, okay, sure, okay, so.

A

Because yeah that'll that'll give us the ability to you know, have a single have a single controller that can suffice for 118, 119, 120, 120, so on and so forth. All at the same time, um I guess we technically don't have to vince. You have your hand up.

B

C

Yeah, so um I didn't see like a cluster atom for fcd, except I mean probably like at the best of control plane, but um from experiencing the cluster api with the tube again control plane like xcd, is really picky on how to do your operations, so I wouldn't I'm not sure, if, like it would work out like in the end like to have a generic controller.

C

That does those things as an example like in 118, like one of like we had to like do a special case like for fcd to like only up like upgrade in a certain order, so we might wanna. I mean I do like the suggestions, like maybe just start simple right now, like learn from that and then like. Maybe we could move to a better pattern later, but um also hi. Everyone.

B

Yeah uh thanks for the suggestion. Yes, we do think a general controller cannot cannot fulfill other the final goal, because we know that there are a lot of like, etc, controller out there like different adc controller. They are using different kind of crd. They are using different logic to generate htc components, so this one is like a very basic, very native one, so we assume, when user does not specify any, uh uh does not specify any specific, etc controller.

B

They are going to use this very native one and it's going to create a etcd uh container cluster cluster. Can cluster own native owners super cluster as basic in a container and in the future would think each vendor or each users if they wanted to uh if they wanted to use some more powerful, etc controller, they should stop this native small controller first and then use their own atcd controller to do their jobs. We just wanted to have some like workable working example for the first first version, yeah.

A

Make that first run experience at least useful and then allow you to use uh more more advanced controls later, as you start to build out your solution using this. Yes, correct, yeah,.

D

I think our default result is definitely you need to have a default anyway, so but for the performance back version and the channel, my original thought is, the version is just taken in the formally released epcd version, and the assumption is, if you only satisfy that few numbers.

D

And the assumption is, we already have rows, templated ready in our captain repo, so that version you just used to define. If, if you don't have that particular version on our repo, we should just give an error. So it doesn't stick there and I an assumption- and I assume that the channel is even flexible. People can use that channel put into a customized uh yammer for the edcd. You can give it hands. That's my original thought.

A

Yeah, that was what I was going for with that um yeah, because then you could mount your own. Your own slice of of files into this controller and technically apply whatever you wanted. If you wanted to, if you wanted to swap things like crazy, you could yeah.

A

So the main, the main thing that breaks down there is the rbac rules uh and and what you actually grant that controller access to, which is something that that the add-ons group has already uh had some run-ins with in terms of like, if you make it so that it's so flexible, that you can apply any any type of file to it, ingress service, secret, config map, so on and so forth.

A

That controller needs to have permissions to do all of those which can be kind of a little bit scary, but that ends up being on the provider or on the implementer anyways. So.

D

Carol, you know you just put a yamaha file, you can put everything in the you might yeah and I think the controller just blindly apply, including our back. Whatever services you have so that's totally doable and then coding-wise there's no much complexity compared to just yellow. So um all right, yeah. I don't worry that a part.

D

So as long as we have a default that we can easily can do some at least inversion, the channel part is maybe on a version part. uh You can have a few building templates. So far we only have one. You can add a few others yeah.

A

Yeah, the other thing about channels is it's more just a um it's a string representation of just like of whatever the latest version is or something like that you can think of this as stable is the channel, and so that would be whatever the whatever that most stable version is uh deployed into that operator. Think latest, as as you would in docker images or or runtimes, probably don't want to use it that much, but it's at least something to get. You started, okay, so.

D

Yeah, I think, as long as we agree that so because yes, they ask how so whether we want to have a every every we have a separate ecg controller or just use one but implement the interface to allow the integration. So um I think I think I I kind of remember we have a conclusion it has. Even people may just follow the same api but implement their own controller right completely, swap so I think that's also. Okay, I'm making the native one pretty.

D

You know simple and neat, and it just do the minimum job making blindness. That's all the.

A

I was muted, luckily we're at a place where we have some flexibility, that we can make mistakes and and roll back on those too around this. Since this isn't, this is completely new, um so yeah as we move forward, I think choosing one to go with finding making sure that that value validate that that works as we go forward and uh don't don't go through any one-way doors that we can't go back on so cool. This is awesome. I'm I'm stoked that that to see this, I gave it a quick, a very, very short review.

A

I only had one single knit in there and then I wanted to just open the the communication here. uh You did have one thing that I added as an agenda item, um which vince had some feedback on too um related to what you were talking about about certificate management and how we need to how we probably need to actually do that at the ncp controller.

A

um Right now before you go into that or actually give the background on that, I think would be would be useful.

B

Oh okay, so um we used to think each component should generate each component to generate their own pki resource by themselves and the cluster api do provide several secrets: general secrets for store all the ca root uh certificate.

B

But when I implement the nasd controller, I run into some problem and I think there should be a central controller. Specific controller or nasty controller plan should generate all this pki resource in advance, because when we start the nested atc components or star and atcd, we have to load in the uh api. We need to load in the ca of the api server, because when running when etc start running, it may receive the request from the api server and it need to use the ca. The api server ca to authenticate uh authenticate.

B

This also uh authenticate uh authorize authorize this request. So so I think there should be a a specific controller or an sda control. Plane controller should generate this pki in advance and then we can start creating each nested component. What do you think of greece.

A

I think that makes sense and and vince I popped this question over since vince right before this uh to ask how how the rest of cappy providers do this. Can you give that background for everybody, vince.

C

Yeah, so I'm probably going to talk about like um the entry, bootstrap and control plane providers. So, like you, you might have heard like kcp, which is the cubed m control plane and then there's cappy k, which is the blue, strap for cube. Edm um cappy k actually generates the certificates today um and we're trying to move that around, like I don't remember but like if, like we actually moved it or not, um but as you mentioned like, we do generate this.

C

The ca so like the mean ca, which is has like a 10 year, expiration date, hopefully like you're, not going to have that cluster running for that long um and then jupiter em actually generates like the certificates uh for when you run kubernetes, for example, or join for xcd control, plane, etc. So you have to replicate some of that logic which is already available in q by dm for that, so it uses the ca um to generate those other servers.

C

um The lcd certs are um passed in uh for cuba dm to actually use those as a base. um So close api already has like most of these available and then this will get stored in secrets. The nice thing about is that, like, if they're already in the secret, we just use them instead of like creating them, so that the user can supply their own um today. That is not there. It's its own um controller.

C

If you, if you want to vote like that path, I would just use circ manager at that point and just create that cr to say, like hey, I need a certificate and get us the certificate back from certain manager. um Now the copy in alpha 4 is going to move to cert manager. 1.0, that's actually a great way for us to segue into this, because uh that will give us like a long-term stability that for those apis that the v1 apis for surf manager.

C

um So there's like couple ways to tackle this problem, um I would suggest to like try to replicate as much as possible what kappy's doing up upstream today, um mostly to kind of get in line, because you know if at some point like we merge and we meet um yeah. That would be like a great way to integrate.

C

Doesn't make sense.

A

That makes sense to me. Were you saying just to clarify one thing, though you were saying that in uh v1 alpha 4, uh it's moving to cert manager 1.0. Does that mean that the long term plans for like the kcp implementation would be using cert manager too?

A

C

Is it just an option like it? You can create the cert either like I don't think we'll will change the certificate process as it is today like in at least for this foreseeable future, um especially because this can be just an internal change like users won't have to see uh how we get to a certificate. It's it's more like now, we'll do it in a different way than before, um but the end result is like we're. Gonna have the secret somewhere in a secret like and with the prefix name.

C

C

A

B

Else so um I'm not quite familiar with the certificate manager. So if we are going to use the certificate manager, does that mean we still need to generate the ca secret by ourselves and then store it in the secret into the central secret, the global secret, and then a certificate manager will read this ca and generate a corresponding certificate for different components.

A

If you vince, you want to take that so so I I believe and correct me where I'm wrong the idea there would be, you would use certificate manager to create an issuer and you'd have that you'd have. uh This could be outside of the actual implementation in the nested or in cap n, because it.

C

A

You can go and deploy certificates using cert manager, which.

C

You just deploy.

A

An issuer cr and then you deploy certificates that reference the issuer and that generates it can be cas or keys inserts and that certificate resource would go and have a secret name in the actual spec that defines what secret it's gonna. It's gonna get created as, and so um I dropped two links from chatting with vince before this to the two cert implementations in the kcp. So it sounds like the idea would be continue to go forward with potentially reusing what we have in kcp, but always make sure that it's always supporting this.

A

The proper secret names that cappy expects, and if a provider wanted to and already has cert manager deployed, they could deploy those. It creates the secrets for them, and it's not. It's kind of outside of the cap ends scope, but it at least would would work, um and then the internal could use the kcp setup as long as that fits what we need and it doesn't and we don't have to hack too much of it since we're not using cube, am bits. Okay,.

D

So um yeah, I'm I'm okay with it, but uh I think uh a few consonants is do we. Does that mean that if, if people want to build a campaign- and you know you see from scratch- uh he has to install the manager.

A

No, the idea would be it's just an option it just it's just that we support it. You'd still have the built-in entry like what what chaos already doing, but instead uh reusing some of the cappy functions for it.

D

Yeah, but for the entry uh now who is going to do that so talk? Yes,.

A

D

A

We probably should file some radars and or sorry, I'm speaking, different languages right now. uh We should probably file some some issues on github uh for for those oops.

D

Yeah exactly so, so I think that uh so the media is pretty straightforward. In my opinion, it's just uh you. You just have a contract about the secret name. That's all about it! You just assume this doesn't exist using the big building one, but for everyone. I think for charles you know, question is who's going to create a beauty. One right. I think this program is kind of tied to the, um I would say, the ncp control plan workflow.

D

So somehow we need to define that workflow first and just where is the best place to inject the ca creation uh intuitively? I thought it should be happening in the sap control plan. If it is the centralizing arches.

A

I think you're, I think, you're right there. I agree at least um so I put a note on um because I have pull request. Number 30 is open right now, which has like the first draft that I wrote, which was pretty lightweight of the ncp nc docs. um That still is posted as a draft in github, but I'm open for reviews. I added a comment after reviewing chaos, pr for the nested etcd implementation that we should probably add uh the certificate management stuff into that.

A

So I'll spend some time and write up what that looks like, so we can get that in place. For the time being, it seems like it sounds like and uh well. It sounds like we could potentially go down the path of within the nested scd that you already have.

A

If you want to, if it's easier to keep that implementation of of generating the certs in there, we can go, we can move forward with that or, if you want to just basically put in the checks to see if the the secret exists and for testing we just use cert manager for, like the the time being, to just make sure the secrets enter in place, and then we build out the proper implementation that puts that lays the secrets down. Whichever way you want to go, I'm I'm open to um both. We would.

A

We would probably just pull out the entry implementation and replace it with with and uh with kcp in the long run,.

D

Okay, yeah, I would prefer the second one, so don't do if we agree that you know: net net controller is the right place to generate road search, so we should remove those codes, but even we can make an assumption that whatever the secret exists, that you can just mark them or we created them during testing which is inconvenient in our opinion, so which also means that if we want to the next immediate controller we need. Even when is this, it's a dki related controller.

D

We need to do it in the first before we implement the rest of the thing. Otherwise we have a long dependency change on that for.

A

Sure that makes a lot of.

D

Sense, yeah, I think I I would suggest getting rid of the pca holding your vr.

B

Okay, yeah, that makes sense yeah. I will remove the pki implementation uh in this pr and assume that there is a secret has already been generated and it will check the secret if the secret exist. Okay, chris, do you have anything you wanted to add.

A

No, I just wanted to say that that that sounds good to me. um I just added those notes to the agenda as well um that I'll go back to the ncp dock and make sure to fill out what that's going to look like for generating certs vince I'll, probably poke you a couple more times to get more information.

C

So does just does that mean, like the user has to create the certificates on their own.

A

For right now, until we get the ncp controller in place, but we don't have any of the other control, the other component controller is built. Yet, okay, so for testing the for testing, the scd implementation, we would you'd have to generate the certs and then call the nested scd, uh see or apply a nested at cdcr.

A

uh And then once we build the ncp controller. That'll lay down this the secrets for you and you don't have to think about it in the nested sd implementation.

C

Okay, yeah that make that makes sense. um The I was looking at kcp actually kcp, also regenerates the cubed m uh cubeconfig, um if it's like six months like before expiration, so you might want to pull that in too. There is like a cube config package. I'm gonna link to it that you can find um these utils in.

C

Here and I'm gonna press it in chat like if you're gonna take a look but okay. That makes sense like I was just like confused like if we were gonna require like uh folks to just bring in their own search all the time, because that can be like a very high bar.

A

So yeah, no not long term, not long term agreed okay. So so the link that you just sent so that I can make sure that I get this documented so ncp. Also sorry, the kcp uh implementation will generate the cube config for you, similar to what we were doing in the vc manager or what trout did in the vc manager. So we'll basically do the exact same implementation, and so I'll add that to the ncp dock as well, that we need to generate the cube configs for components.

C

Okay, cool, that's good, to know the cube conveyors it like you can get it back with cluster cuddle with classic config, but this generates and regenerates because that expires after one year, if everyone correctly, which is probably good, perfect.

B

Okay, so so I have one more feature: features I think in the nasty the control plane, controller should do for other nasty controller. The thing is the honor reference.

B

If you remember in the first division, first version we designed to have the user to apply all this crd separately and how to link all this cr together, we are using uh on a reference so inside each nested controller controller, crd uh nested, nested component crd. We have on a reference uh point to the nested control plan crd.

B

uh So this thing has to be set by the nasd controller uh control plan, controller, yeah.

A

Makes perfect sense, I will definitely add that to the dock as well.

B

Cool yeah. It took me a lot of time to debug these things, because if you do not set it on a reference and you apply a crd, it will successfully apply the kuvi. Ctr will not return any arrow, but the crt will not appear on the cluster anywhere.

A

Yeah: okay, interesting cool, that's a good callout.

B

Yeah, so I think that's that's all for my pr for now and yeah.

A

All right uh anybody else on the call, uh if you have a chance, make sure you go and go and review chaos pr about. uh They had nested etsy implementation, uh and then it sounds like so you'll you're gonna remove the pki implementation from in there and just expect that cert secrets exist at a specific location.

A

Okay, cool um and that'll be the from your implementation or you- and I can talk about this after. um I just want to make sure that I have. I create all the proper secrets for you within our within the ncp doc.

A

Perfect cool: does anybody have anything else, we're at the bottom of the agenda.

D

um Yeah, just just refresh your memory, so I kind of, let me think about it. So the idea is the uh the usually supposed to create all crs, including cpcr, and the component crs and controller a bunch of kind of coordination. It says that the sap controller need to first set on a reference based on my code reading about the child transmutation. If the auto reference is not there, the controller is not going to come proceed right so uh so so then uh so have we discussed what is the right way to.

D

Link all the cr together, so what I'm saying so how the ncp?

D

How do you make sure the sap controller find that this particular component cr belongs to this? It's the pcr, so that should be kind of the id kind of thing. So are we using labels vector or just use name to do the matching.

A

So right now in the ncp dock, what I wrote, uh which is similar, which is uh a lot of it, was pulled from what chao was writing at the beginning um in the nested component doc, but basically just referencing using object references. So we basically have in the ncp controller spec. We have a namespace prefix. We have a control, plane, endpoint uh for the capi implementation, and then we have the scdraft, the api server ref and the controller manager ref.

A

And so, when you have all three of those it's going to go, find the resource and then set up the owner. References from that.

D

Oh okay, so so so so are we assuming that the user needs to provide the roads rep, at least the names and.

A

At least the names yeah.

C

At least, is that enough.

D

It would be, there would be.

C

Api group kind and the name usually.

A

Good call, okay, so then it would be all three of all of those.

C

Yeah so like we, we usually have templates so that then you can and var substitute like these things, so that, like users, don't have to like manually put things in um so like this could be like a part of like a release like you have like a template like that's predefined, with all these objects, so that it you just have to substitute the name of like resources, and then it would auto create all these things for you. But we do have examples like when you get there like. I can point you to the right direction.

D

Yeah, that's also my thoughts about you know is this uh leave a burden to let people feel rose field? um If, because I'm afraid people- because I remember the ref- has at least a kind of uid kind of thing. So even now you people think I need to provide everything. You need to feel a lot of things, but I don't think it's necessary to feel those.

C

Many things um uiv should never be failed by a user, because that changes from cluster to cluster so like it wouldn't work in a backup, restore scenario, for example.

C

um So it would, it would probably just be api group, um the kind and the name and the reason I'm saying api group mankind is that like. If someone wants to do a different implementation, you could default to the ones you have in trade for xcd control plane. But if someone wants to bring in a different implementation of the suv nested controller, they could do so. That's why there is that pluggable interface.

C

If you don't want to support that, just go by name. That's that's what we did upstream for some for some things that you don't.

A

But then yeah, so that's probably something we need to talk a little bit about if we're. If I think this is this is where we get into the interface versus how we're going to actually support those other implementations for for fcd.

A

um I think a big piece of that is what actual what data actually needs to get shared back, um because if we, if we do so correct me, if I'm wrong, there's there's a bunch of field selecting within the capi implementation, where we're like that's how we're getting the control plane uh endpoint uh from from any dock correct. So if we wanted to go that route of being able to select a specific field out of there say, for instance, being a little grab, the address is out of the nested cd uh status field.

A

We could do selecting of those using the actual field selector in object graph two, but I don't know if that will support that that wouldn't really work. If we have multiple pieces to select.

A

I don't know what you mean about selectors, uh so being able to get like specific attributes from from a net from. I don't want to call it a nested but a a an object, referenced uh cr.

A

So if there's, if there's attributes that we need to share between ncp ncp, nsc cd and k, api server for a nested api server, for example, because we're going to have to get the the lcd addresses out of the ncd implementation to provide those to the cube api server crs as well, and so we need to have we that's the big, the big hop that we need to be able to make sure you support it. One question.

C

That I have here is like do you? Are you planning to support um like a different implementation of a tv.

A

That's the goal to be able to still support it, but not have it built in.

C

So that somebody could replace it so yeah in that case, like you, probably definitely need api group in kind um and yeah. The name like that's uh that's obvious so for um I've been calling this data interfaces, so you would use some some sort of like unstructured client to like get the object, that's referenced somewhere and then you're saying like this object is conformant.

C

If it has these um like, if it has these fields at this path and they're, like the one marshall in the json object that you expose yeah, that's that's like what we do in copy as well like you could look at the machine controller or like the cluster controller.

C

uh There is utilities there as well, which are not exported today. That say, give go. Get me this external reference, which is a reference that I don't know anything about, but I expect it to behave in a certain way.

C

And that's why you, I think you're.

A

Yeah, that's exactly what I was thinking. What I was thinking there. I think we just need to make sure that we know what exactly what components we need to be able to get and what those need. I don't have that off the top of my head for, what's going to need to be shared in between the two, but we'll want to define what that's going to look like for sure.

A

I just I know at least it's addresses and we obviously have pki, but we're going to be sharing that down from the ncp implementation through secrets, so that one's a little bit different, um but ciao fey. If you have, if you have thoughts there on other components that were other attributes that we're going to need to share between those two.

D

No, it's pretty much yeah just look at what is it dependency here? I think you just service service class, ip, an ip right and uh yeah just service class ip, because the api server has a services post right.

A

The other way around, though the fcd implementation we need to know about the uh you're right. We do need to know about the we need to know about how to address it.

A

That's all from from that side of things um to to be able to give to the cube api server, I'm trying to think if there's other, if there's other fields, that we need to be sharing between, at least at cd and the cube api server, similar we're going to have to the same conversation between what nc, what the cube api server is going to need to share to the controller manager, for example like syncing these service cluster ips um or the siders, because those are the same.

D

Yeah, I don't see others, but maybe we can expose them.

C

One suggestion that I have like around sharing data is like um one potential solution.

C

Like we found pretty useful is it sounds awful, but copy data from like where, if you have an ncd reference and you want to expose it to others, um the set endpoint um or whatever else you want exposed from ncd, I would make a field inside the nested control plane status, so as a status field known as a spec that copies that data from there and exposes to others. So, like your interface between all the components is always the the struct.

C

That's shared that, like you, can actually on martial ads type safe, um so that you you have. You have to only unmarshal this unstructured data in one place and then expose it to others. So, as more data like comes up like you may probably want to add more fields, but you avoid, like other implementation, to also do the same on marshalling, because then you have these massive strings that, like you know like, are everywhere, and you like kind of have to change it like in all places.

D

That's also sounds reasonable to me, so it's if the shared information is limited and- uh and I think hopefully now it's- they are limited, so we can put in the status at least.

A

Yeah, I agree, I think that makes sense.

D

Yeah the only doing trouble, but I think we don't have a conclusion right now is: um how did you upgrade it to upgrade? But I believe, if you upgrade is either the component to do their own upgrade or if we really want to have a centralized update base, which is sap controller. uh It is the ncb controller need to do the armature for the customized cr, but uh at least the components in your composition doesn't have to do so.

A

Yeah, I'm hoping that we can leave those to the individual controllers, doing the updates and and those be somewhat prescribed uh in terms of okay, I'm gonna go update, ncd and I'm gonna deploy a new cr or I'm gonna, update the cr and have that do whatever it needs to roll out. Similarly, allow allow that to happen with the cube api server and then the controller manager as individual components and then still still do the sharing back through.

A

So we can have it so that if ncp gets changes, we're also catching that in the net the next the next layer. um So if the, if the certificates need to be rolled, we we update it at the ncp and that downstream affects everything or if the scd implementation just needs to upgrade to the next minor version it doesn't. It potentially doesn't need to change anything unless it's changing the addresses um on the ncp and if the addresses happen, then we it kind of yeah.

A

I think that could be a pretty easy yeah yeah to work with.

D

Yeah, I think I think in the beginning, we can realize one component do that on upgrade, but in the end, if you really want to do something, you know uh real in the production anyway, you think about it.

D

You, you probably will do you know the whole clusterwise, the upgrade, which means, uh if you, if your controller, doesn't do this, which means you have to implement another raw controller, to use all components so upgrade it's okay, if you think this problem is too you know, but user-specific or whatever we can leave it to the user to implement that or otherwise we can think about. uh Have a simpler.

D

You know centralized place to do the upgrade to do the exact thing you know you find a kind of a cluster version, which means a version of all components.

D

Somehow, somehow I still feel that it would be a useful feature because for the majority of the user they don't have that they don't need that in a sophisticated upgrade for each components, um they probably will just do the major upgrade like you know. Kubernetes goes to 119 120, just to update at least aps server and controller manager. At the same time, right by one click would be nice, but lcd may be slightly different, so, but on our other hand, we only have two uh video component api server and the controller manager.

D

If you use separate, upgrade it's still. Okay, it's not that troublesome, um but if we, if we uh extend it or with more, for example, with a zone, if we have more a lot of ads on so um I don't know if individual, which is okay but maybe a kind of uh in reality, people want me to implement their own upgrade controller.

A

Yeah, I I mean, I definitely still agree agree. The template is going to be super useful. um I think I still, I still think, having the the bass bass components in place to allow us to individually do everything and then adding that as like a as like a composite cr on top is going to be really simple um and it reduces it reduces the amount that that that template cr has to do because now it can just downstream to a bunch of other controllers that can parallelize everything. So.

D

Yeah but uh yeah the ordering behavior that I can think of is if something is broken, so maybe a little bit difficult to debug or find the root cause.

C

Yeah, true, you could probably use condition to percolate information back up so that when you describe, for example, the and we have utilities to do so- to aggregate conditions from other objects to two and different objects. For example, the cluster does that to control plane or the machine deployment. Does that to machines um so that we aggregate that information up and the user has to only go look into one place um to get that information.

C

So if you want to debug something- and you have a failure like you- could percolate that up and also block the process for other upgrades, because I'm I'm sure that at some point you might want some some operations to be sequential, uh especially when it comes down to xcd and control plane. You probably don't want to do different. uh You know exact same yeah.

D

That's exactly what I was talking about, so you you, when you do the actual roll out of the entire cluster. You have to have orchestrator. Somehow you you, there are some other other other implications. There are some, you know, comparability implications, because people, if they do that individual component upgrade they may not do sufficiently. You know compatible tests against these other components, because if you really leave all the you know, powers to individual component owner there's no centralized guys to do the whole cluster-wise, very validation, about all the upgrades.

D

So that that's my my worry, but I know at the beginning: it's not a problem because I believe in the beginning, every upgrade is done by the experts, so they they shouldn't know exactly what is consequence of doing that, um but from the complete device, for you know from the projector-wise we can make such a strong assumption.

D

I'd like to have some loose assumption, like making the controller more strict on this upgrade process. But but but I agree, this is a bigger, tougher problem. Maybe we can address later, but but I think, we'd better. Keep it in mind when we do kind of design, so I see, I still think we better in the end, we'd better, provide a wage building easy upgrade, follow the controller control user controller as orchestrator, and I don't want people to build another controller to do this.

C

Yeah we have the exact same we're still having the exact same problem in cluster api, like you have to change a bunch of versions like in other places. If you want to upgrade a whole cluster, we haven't pushed for the easy button.

C

Yet because, honestly, like we take a lot of um we assume user intent at that point and that's hard to like grasp it with crds, um but I do agree like that might be like kind of like uh maybe one plus year goal like that, like you want to get there, but getting the foundation in place today is probably the most important thing so like getting these controllers, like do the right thing first, and then you build on top of it, because you can trust them. That's yeah! That's great!.

C

Cool, so I have a question about namespace prefix, though.

A

C

What is it, uh what.

A

Yeah, so this is a little bit related to how the syncing works, um and I don't know if it sits in this controller or if it's somewhere else, uh because we really haven't talked about how how the sinking objects works, um at least not from from the cap and side of things, there's a lot of a lot of information on the vc side, but the idea there is in in vc what we do is it generates a a prefix based on the the name space that the actual virtual cluster cr is, and then it hashes that, and it basically creates a a unique name- space prefix that gets added to every namespace that gets created from a virtual cluster uh to stop from collisions happening when say, for instance, the default namespace is created in every single cluster.

A

So when that gets shared down to the super cluster, those prefix that prefix gets added to the to it um yeah, and so that's the the idea there is to be able to specify those uh or have them auto generated um similar to the way that vc works. So you can have a little bit more predictive, yeah, predictable namespaces.

A

If you want them versus being able to have auto generated.

D

But I'm guessing this prefix may maybe.

A

It's not directly related to cap n, that's the one thing, and- and I and I want to figure out how how we intermingle those um I'm open to not putting it into these crs. I tend to.

D

Think we anyway, so I think when we really put vc on it, is kind of. We still need a kind of vccr anyway, because we can describe all the you know: vc related field in kpm, which doesn't make sense, so I would suggest to move that prefix to the vccr and the make appearance. You know save campaign I like that works for me.

D

So it's just just like you know. We need to refer a little bit later, so the vccr need to refer to a cluster cr, but that's fine, so the vc controller needs to do the. You know: coordination with vc objective class cr, which is we sound like one more layer compared with. Currently we don't have this layer, but which is completely fine to me.

A

Yeah that makes sense, which will actually be good, because that will make it so supported across all the components that vcrd has and we don't have to do much work.

A

All right is there anything else that anybody wants to bring up. We have roughly nine minutes left.

A

Cool well thanks everybody. This has been fantastic, uh a good, a lot of good conversation and uh we we've been up and down on the amount of content we have. So I'm super stoked to to have all of this uh thanks, everybody and I'll. Let you all, uh I think, they're free for an extra 10 minutes.

D

Go ahead, yeah just just give a wrap-up, so next I think uh the priority-wise is. We need to have a ncp design. uh I think it's kind of okay right now so because, because we all agree, we start with the simple stuff with the simple version, so uh I think we can start once we make sure the workflow wise. What is exactly workflow, for example, if the people fresh people come to use this campaign, what's this a handbook, they should what's this instruction, they should follow.

D

It's like a tutorial kind of thing, but we'd better put somewhere formula in the design, doc, saying: okay, you need to create a row cr first and the water would be expected, like you know, pk is going to be created and the sap controller will add our own reference to everything and the rest component controller will generate exact component part for you and just get it ready.

D

I think we can move on to at least start a ncp controller, because I think we need to get this pki stuff down uh to unplug the rest of thing. um If, if the pk is kind of ready, the rest of the thing seems straightforward to me, like the aps: server controller and controller manager controller.

D

I just want to give a kind of a quick wrap up about. What's what what should be the next steps.

A

Yeah agreed, I added that as an action item at the bottom, uh so there's a a bunch of a bunch of notes that need to get added into that uh that that um ncp dock as well cool uh yeah, and if you want to just if you, if you can, if there's anything you want to call out in that doc as well. Adam, has comments and I'll make sure that all those get addressed.

A

Cool thanks. Everybody all right have a great rest of the day.