Kubernetes SIG Cluster Lifecycle, 12 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes [SIG Cluster Lifecycle] Cluster API Provider Nested Office Hours 20210112

Description

#sig-cluster-lifecycle
#capn
#capi

A

All right, good morning, everybody, uh this is january, 12th 2021. uh This is the cap and uh office hours call. um As always. This is a recorded meeting. So don't say anything you wouldn't want to be put up on youtube later and if you can, I've dropped a link and I'll drop another one in case you folks that have just joined, don't have it yet, but in the chat for zoom there is a link to the actual attendee or the uh the agenda. Doc.

A

If you can add yourself to the attendees, and if you have anything you want to talk about, we do have a short agenda. um Yeah. If you have anything, you want to bring up, please add those to the agenda and we can get.

A

Started cool and the first thing is: there's a couple: new pho or there's one new folk right now, um scott, if you'd like to you, can give yourself an introduction.

B

uh Yeah hi everyone, uh scott rosenberg, uh from terra sky, a systems, integration company, um leading the kubernetes efforts uh for implementation by our customers, uh dealing a lot with cluster api and excited to uh get to learn. uh Cap n as well and start implementing it.

A

Cool thanks for that uh and I don't have any psas faye. I don't know if you have anything you want to bring up to the group before we go into the two agenda items.

C

um Oh no also yeah, I I was just looking at a while uh in the slack channel you discuss with, uh I think uh bright right, yep yeah yeah. I probably will follow up on that. If I didn't misunderstood, what it said is that the campaign points to cappy.

A

Yeah, so it sounded like um the I had that as the first agenda item, because I wanted to to start us off here with that, at least um so it sounded like breit was trying to figure out exactly if we need to move all of vcs all of the functions around vc that are related to syncing actually into this project. um They were saying that, let me see yeah, so if you click on the link in the in the agenda here, I can actually pull it up too. Let me share my screen.

A

So I think, there's a good. Oh! I actually can't, because it's slacking, I'm not gonna open up slack. uh So if you click on that link, you can actually go see the conversation, but breit was bringing up that uh it. It doesn't technically have to be homed within uh cap n for all of the syncing logic and all the all the custom stuff, because realistically cap, the way that we're the way that vc is built and the way that you all have implemented already doesn't actually use the same provider.

A

Work that uh we're doing from the upstream to provision clusters, for example, and so bright, was kind of saying what? If what if there was a different split in the project where it was, you still have a you still have a new way to provision nested control planes, but then also still have a separate project that was doing the sinking logic so that it didn't really matter where the control plane provider was coming from.

A

So the kappa provider, for example, could work or you could use eks or you could use, align or anything like that any of those providers, but make it so that it's independent of the project um I see and.

C

A

C

Yeah that that's what uh yeah I didn't realize uh he did I I was thinking he just want to say that kind of extending the current canon operator, because colonel turner operator has two mode right is: why is the audi chronometer otherwise in a local mode, so we were trying to replacing the local mode with captain and- and maybe he suggested that maybe we can. We should continue the cloud mode, current cloud mode thing to different cloud providers.

C

That's my understanding.

C

Yeah now I see what you're saying so saying that if we do that direction, we want to support more cloud vendors, that piece of logic maybe make sure to come along with uh vc instead of kpn yeah.

A

Yeah, it's an it's a it's a different approach, um I'm not against it.

A

I think my current thought processes is we're probably going to have a a couple iterations on this whole project as we go um and giving it a place like at least giving us a place to solve some of the problems we had early on with vc, where we couldn't actually instill uh proper, um like build pipelines and like the problems of just basic uh development within the the multi-tenancy repo and and being able to get to an actual virtual cluster sub-project, that's separate.

A

um I think this is at least gives us a progression where, once we have a cap, end provider built in place- and we have some of the functions around syncing and all that re-implemented and it works for both uh the older functions and the new stuff that we could kind of potentially propose another project. It's the way that I was approaching, or what the way that I was expecting that we would approach it.

C

Yeah yeah yeah, I agree so yeah. I think we need to have a place single place to combine our efforts. So I'm thinking because maori tennessee is not really a good place to holding everything to share the ripple with other projects anyway. So um yeah, I agree. So I think we should first approach is moving everything spread out.

C

Yeah, um I I think that's so maybe we should have a another thing if it is pretty much just movie moving vc stuff to another repo, if that is that is possible or that's worth doing so because I was, I see these two things are tied together. um I don't I I don't. I don't have a strong uh feeling that the synchro, because synchro alone doesn't work- I mean if you only have a sinker, we don't have all you don't have all the roundup uh captain stuff.

C

You know control plan, it won't work yeah, um but I do see. Let me let us think about it. So should we, you know, keep that interface.

C

You know that interface in the sense that has a cloud mode provider kind of thing uh is existing, but.

A

I wonder yeah, I wonder like, as we go down this path, uh if we, if we do end up with something where it's you could deploy a nested cluster and a nested cluster, is just a reference to a control, plane, endpoint type, then it doesn't really. Then it doesn't really matter what the provisioning the provisioning tool is there? It's just. We need that one cr that represents where you're going to get your cube config file and where you're going to get the the endpoint that you're going to connect to for that nested cluster to work.

C

Yeah- and I think that is actually the point to the way that which how we should define the ncp crd right, it's like an ncbcrd probably needs to provide some flexibility. Just point your an existing control plane.

A

Yeah yeah because we yeah, if they, if you don't, need to actually go and do that uh and you want to use a managed control plane. So it should be totally fine to do that um and uh yeah. So as long as we build that way, it doesn't seem like we're we're stopping that, but that from working.

C

Yes, okay, I agree. So it's just interface thing. If we it's just a controller kind of thing, so maybe.

A

Yeah cool, okay- um I don't know if you want to if you want to respond.

C

Okay he's the last group.

A

Cool as well, if anybody else is not that is interested in adding anything to this, um and and has seen that conversation that's going on uh you're more than welcome to chime in.

C

Yeah yeah, maybe just give some background so currently. The way that I implement the this the tenant operator is, I have two mode, the local mode and the cloud mode. So it's just a crd is who provides the exactly what chris said. Who gives you the credential to access to access the con, the the the um control plan? That's all about it.

C

uh We are capping trying to replacing the entire local mode kind of thing, because local mode, there is no way to do the upgrade and we use the you know: in-memory udcb, there's no precision, storage at all. It's just for you know protecting purpose, um but we do have some usage internally usage. We because the the intel vc has been in some of the cloud product already in that. In that thing we use the cloudable provider. So do exactly um that! You know you know we don't have the problem of. You know persistent storage.

C

How do you manage it? It's all separated! It's like it's like you know. We see product with the eks, you can think about that or um yeah, but but we don't. Currently we don't have a supporter for uks. Yet uh because you need to write a bunch of new controllers to call the eks api, um which we haven't done uh by the way. If anyone wants to try that write, write a small controller to codecast api, that's more than welcome.

A

I think that's one of the nice things about about kind of shifting us towards towards cap towards cappy's apis as well, because if we use the standard, the standard, location and path for cubeconfig files from a management cluster, it makes it pretty easy, because every every provider should be using that same secret, name uh and location associated with their their cluster, which I think uh helps because then we don't even need to have anybody, implement a special provider for any of the managed, managed solutions or even like cops or anything like that.

A

We could just use off-the-shelf tools and just say: here's, the new endpoint and you already, because it's cappy deployed you already know what the secret name is going to be for the cubeconfig file um for the admin config at least, which I think is beneficial.

D

Hey chris, what do you mean? uh What are you referred to by cappy.

A

uh I more so just mean the actual the generic interfaces so being able to to use any of the other providers is what I mean by that, so being able to use kappa and and cap v and so on and so forth.

D

So what sorry, what are the acronyms referring to.

A

uh Specifically, like the the cluster api provider for aws or for vsphere or any other provider that would provide a control plane.

D

Okay, gotcha yeah, I'm I'm just thinking about I'm familiar familiar with the the auto scaler and how the different cloud providers are. You know they each provide their own implementation. According to the interface, I thought that was a nice solution. I don't know seems like you're describing something where you wouldn't even have to do something like that. But that's also, I thought a nice option.

A

Let's see yeah, so this is kind of what I'm, what I'm, what I'm getting at, and I still sharing my screen, I believe so or at least chrome. um So the things that I think are interesting here is because we'll have something like when you go and generate your cube. Config all of the providers should have some sort of path like this to have generated a cube, config file, um there's locations for them.

A

Developer guide, so all of the actual cube config files should end up as a is this, the right one.

B

Yeah it's at the cluster name, minuscube config in the namespace in which the cluster was created. Thank you, scott. Yes,.

A

Yep, which makes it easy because then, because then our implementation doesn't have to think about anything special. It just knows that it's the cluster name yeah, exactly as scott said.

A

Cool all right, uh I feel like that, makes sense. uh I also wanted to bring up another thing. uh Our our agendas are super lightweight right now. I think it's beginning of the year and all that uh as well, but I'm I'm contemplating whether we should move these to bi-weekly, um because we don't have a ton that we're bringing up and there's only a handful of us working on the project right. This very second um interested everybody else's thoughts on that.

C

C

Every two weeks bi-weekly- I mean I understand, but I I would still prefer five weeks for just a couple more weeks, because uh I think outside will have more things to to talk uh after child come back from his location. um I think we do need to push on cp design. uh I think we still have kind of four things to discuss um people I would suggest once we do the real development mode we probably can can yeah. If we can predict in the next two weeks we are just going to coding. So that's okay,.

A

All right that sounds good yeah. I just wanted to throw that out there as a as a suggestion, because we kind of go back and forth on whether on what our agendas really look like um all right- and I don't have anything to share on the ncp side of things- I'm still dragging my feet on that. So I need to get. I need to need to get on to writing that doc, um a bit more.

C

Yeah yeah, since there are some new guys here, I probably give a heads up, so I was trying to move uh working on the you know. If somebody watches for the multi-tenancy report, you you guys can notice that I'm going, I'm I'm currently working on a self-explanatory experimental project in terms of vcs by supporting multiple super clusters instead of one um so so yeah. I I think I can expose more.

C

Maybe next meeting I can show more details about what I, what I'm trying to do, um yeah just give you guys a heads up, so um it's kind of you can think about it. It's kind of another way to. I will define it as another way to do. The multi cluster management, uh which is different from uh could be fed, could be federation, um but uh under the condition that is, is fitting more like in the vc scenario. It's like a serverless.

C

You don't need to you, don't need to understand the underlying cluster details from the tenant perspective. It's more multi-tenant solution, um so so for federation, it's kind of slightly different is like you, you know everything. You already know you how many classes you have so, but you just need to have a easy convenient. You know place to manage those clusters, like you have to specify how many replicas in class a however and class b, so in vc with modis cluster.

C

I was trying to hide this kind of complexity in the sense that you only work on your virtual cluster. You just see one cluster but underline for the part provider perspective. You have have multiple clusters, uh yeah, that's my my goal and there are indeed some use cases for that.

C

Just give you guys a heads up for the new guys.

B

Cool yeah and uh chris just one note, also on the back to the cap end and using like kappa or cap v, or something like that is the control plane. The one issue that uh I can already think of with that just in terms of a complexity with the design will be that uh cappy. Currently, you can only have one cappy provider uh watching a single namespace um so because it uses also the capi uh base crds for machine control, plane things like that.

B

um The secret for the cube config is going to be in a different name space, um because if you have cap v, for example, on a specific name space, that's where the cube config is going to be in not in the cap end name space, so you're going to have to copy over the secrets or whatever it is uh in order to use them.

C

I see interesting.

A

That's a really.

C

A

Yeah um and I think so uh fei, I think that also aligns with what we're doing currently with at least with vc, because all of the secrets for that end up being within the.

C

A

As the control that even in space, you see, there's the.

C

A

Control plane, but not as the not as the crs yeah. Why do.

C

You miss it. Okay, you know control, plane, cr.

B

Okay, so yeah that should work. That's just something to note that has come up in cappy multiple times.

A

Okay, yeah that'll be interesting.

A

I wonder if that's going to cause us any and I don't think that'll cause us any issues as of yet but that's good to know. Okay,.

C

uh Yeah, I don't see copy has at least I don't see obvious problem is copy, but do the same because so if there is no naming conflict, so that probably should be okay, yeah.

A

Cool all right, I'm gonna open the floor. Is there any other any other things that people want to bring up talk about. Ask questions because we're at the end of our agenda already.

D

Nothing really from me. I joined 15 minutes late anyway, so I think I've relinquished my right.

A

All right, I think, action items this week. I'm gonna finally get myself around to doing the ncp dock I'll share the the hack md with you face so that you can check out what I've written so far yeah. We can move forward on that and then.

A

After that, uh if we, if there's still some time, I'm gonna, I'm gonna get back to starting on that um the nested api server implementation, at least um because I can build that without the the scd implementation in place, since I can just pass in those uh values and we'll sub them out later. So as long as I can get the ncp dock in place, I'll get working on that.

C

That would be most important thing, because I would expect that it needs some more debates or discussions.

A

A

All righty we can give everybody back some time. Then yeah, okay, see you all right, see. Y'all later have a great yesterday, bye.

C