Kubernetes SIG Cluster Lifecycle, 8 Feb 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Cluster Lifecycle - Cluster API Provider AWS Office Hours - 20210208

Description

SIG Cluster Lifecycle Cluster API Provider AWS Office Hours 20210208

A

Hi this is the cluster api provider aws meeting of february 8 2021 as part of the kubernetes cluster life cycle, and please be aware: we're fighting by the cncf code of conduct uh be excellent to each other, um very sparse attendance. um So, should we kick this off uh psas uh yeah the release notes plug in.

B

Yeah, it's been enabled in impro so and the template updated in our repo api template that is so um yeah. It does require you to include that block now for any aprs, a little block uh emerging until you explicitly state. No, I don't have uh release notes or yes, I have already said this is what it is and there is actually required as well so and actually relate to that.

B

Today, I've been looking at the automation that will sit behind that um to rip all those out as part of the release, and we can just basically reuse the uh kubernetes release. uh They've got release, notes, uh commands uh cli util that give them two to um then two tags, essentially it'll get all the pr's with the release, notes that will create a markdown document or json file that we can use. So it's.

A

All good- that's super cool, I think, might even be worth putting on the agenda for the cappy upstream meeting. um One of us can take that and see we want to do that on the main cluster api repo. I never make sick release happy if you're all using the same tooling so.

B

Yeah yeah cool yeah I'll, get it working first, um so I had got it working locally. Just on my machine, I guess my replays um but yeah I'll get one hour one. uh Hopefully, tomorrow.

A

That's super cool, it's really cool um all right. um I've changed the agenda, but just put the review action items um so um we had created a new doc outside corporate google workspace update community. That is done uh thanks to uh library and tim saw sorting it out. For me, I did not create an issue for the mic id so well over onto the action items and you, where are we with the e2e test failures.

B

uh I've started to look into it, but I haven't made much progress. I've probably raised more additional issues and not actually you know fix that one. But this thing I'm looking at currently.

C

Yeah, fine um cool.

A

Oh uh me and steph put the same items.

B

A

Or do you like what or the same so the first of all is um made a change to the proposal with the multi-tenancy. Do you wanna talk us.

D

Yeah so um previously we were not limiting all the users to access controller credentials. We had no crds blocking that, so we added aws cluster controller principle, so that we can only allow a set of namespaces to access controller principles similar to static principles and role principles.

D

So by this way we have a namespace level, uh at least security.

A

Do you want to um the bits that you want to talk about that sort of singleton thing and like how we do the deployment uh and how we handle the case of upgrades just briefly,.

D

Yeah, so as another suggested, uh we are creating that new crd as a single, because we only need to have uh one one of that object, because there's only one controller credentials since we do not know, since we are currently using a default um provider chain, we actually do not. uh I mean we do not know which credentials are being used uh by the controller.

D

So it is, it will be only containing.

D

A lot of namespaces and a name, we also made it again cluster scope so that only users that has access to create objects in the cluster scope will be able to create it.

D

Also for the upgrade not to break existing clusters. For example, uh the existing aws clusters will not have a principal draft so with the defaulting web groups, we are gonna default, their principal ref to this new singleton controller principle, so that existing clusters will continue to operate using controllers principle.

D

And users during upgrade has to create this new object and make sure that they allow namespaces that they wanna allow the usage of um controller principle from.

A

Yeah and finally, we uh we we're going for we are now for free. We will include the blank uh credit blank singleton so which will default to allowing all namespaces right.

D

uh Which will default.

A

That will we will include an infrastructure components by definition, that includes all name spaces so that on upgrade it doesn't break and it's immutable as well so won't get over.

A

D

So for uh yeah, so for being it immutable, uh what what if uh I mean users want to change the namespace a lot of namespaces? Later I mean I wish, I'm sure we wanna make it immutable.

A

They would have to delete and recreate it. Isn't it so. That's not that's not ideal, and also does it not cause an error for cue card or reply.

D

So if we yeah, if we do not make it immutable, um what are the thing? Why are we trying to like make it immutable again.

A

I think you had it on open api spec right, so you marked it as immutable. There.

D

Yeah but yeah I'm just uh discussing if we can, if there's a like, if it is not needed, we can make it mutable so that uh users do not need to like recreate it.

A

Yeah, I think what we need to consider here is so I think this quite ties back to our discussion earlier today that label selector versus array of namespaces as per the cap c design. So the original kappa, multi-color multi density design was around there using uh namespace, selectors, so label selectors and.

A

If, if we only support listed namespaces and the default is nothing then basically the cluster will break on upgrade. Well, I mean it's not not that it will break on upgrade, but we'll stop. Capital will basically refuse to do anything on upgrade, which is interesting.

D

uh Yeah but uh since in the infrastructure yaml we will add an empty alloy namespaces, which will allow all the name spaces. Then nothing will break so.

A

But then every time they're doing say the user then restricts the namespaces, and then they apply new do cluster. They upgrade the infrastructure, components and infrastructure components defaults back to all namespaces that, basically we break the security every upgrade, which is not ideal.

D

Yeah, how about a custom block there if yeah, if there are, if the scope of the alarm name spaces, is less than all namespaces do not allow it to switch to all all name spaces like increasing this. A lot of names, basically, maybe.

A

It puts a lot of again we run into the issue of inferring customer intent in the controller when it comes to security, which is less than ideal any. We should never.

A

It's never been magical when it comes to security.

A

I think we need some further four on this. Don't we.

D

Okay, so yeah: let's, let's uh do it like it for now and see if there are any any other comments from maybe andrew who started the pr.

D

He may get more comments.

A

A

D

uh Yeah also, I will not discuss one one thing regarding uh caching, the providers so uh currently in multi-tenancy proposal.

D

We want to catch the providers so that we do not make the calls every time we we wanna create a session, but so the way we are allowing uh chaining uh in the new multi-tenancy, for example, we may use a role that is assumed by another role, um that is assumed by say a synthetic principle. So this is a chain right, so I mean it is not very easy to uh make the retrieve operation to know what is the next previous step in the chain so.

D

That's why caching automatically triggering that process is, and it looks a little challenging. As far as I see I'll, I will uh look into it a bit more.

D

A

Yeah, you might need to make each principal know its position in the cache yeah.

B

Itself, update.

A

Its own cache position.

D

Or or we can just do it ourselves, we can take that responsibility in the controller so that if, if a provider is expired, we just recreate those providers because we know the chain already.

A

uh Yeah, I'm not at that yeah.

D

I'll I'll try yeah I'll, try that, but I mean there's yeah there's a tricky piece there about um using uh aws is automatic uh like retrieve and is expired. Logic is not very suitable for order. Chaining.

C

A

A

um Maybe also you might want to look at like k-I-a-m or I don't think they do the full chaining, but they do some elements of this they might have. There might be some prior art in some of the other projects that do similar things to this.

D

Okay, yeah I'll check out.

B

Stupid question: there's anything changed in the the v2 sdk around the credentials and the session.

E

In the aws sdk, I hope not too much it should. It should be.

A

Mostly the same because the proposal I wrote actually based on the ruby version which I've done before so they tend to have a like common approach across all of the sdks, um so it shouldn't be a massive change.

A

um Actually, you raised the v2 issue. Do you want to what do you want to just talk to that? Slightly more? What we get from that.

B

I guess just to say it's amazing thing: well, the main thing is, you can just uh obviously import the service that you want. Only the services you want and then you can get those included in the binary.

B

The big downside is, they haven't included any interfaces or you know each service has an I face in the v1 sdk, um so they have published those um in v2. So if you want to use interface based programming like we do and inject, you know for testing mocks and all those type of things you have to create those interfaces yourself at the moment.

B

There's an open issue.

E

B

An open issue and they're like oh: no, we don't want to do that. If you want to use interfaces, then you just create an interface with only the methods. You want, let's cycle along those lines, so.

A

That's really funny, because that's exactly what we did and like if you look at the first few hundred commits of camper, that's exactly what we did is.

B

A

A

And then we ripped it out when we went out for two, we became lazy.

B

uh I guess there's some other stuff in there as well, as I can't remember what I put in the issue to our list.

B

C

B

Just unusual, let's having to use the aws dot string, type thing for the pointers and all that which helps.

A

Yeah I agree yeah um we should get around to it eventually. Yeah. Definitely.

B

um Yeah there are, there are clies that you can. We could point it, but you know these the services and it will generate the interfaces for us and then we can, just you know, call it a bit if we, if we went down that route, so it wouldn't be too onerous.

A

Yeah, okay, all right, let's explore that, I don't maybe we went out- is that that's a really run out for four things: yeah, I guess the v1 sdk is not going anywhere anytime soon anyway,.

B

I can't yeah, I guess, there's got to be some sort of support life cycle on there hasn't there.

A

All right cool come today and the virgin is there. I said I'm only action item I mean you're gonna, which is good, ongoing, diagnosing etv test failures. I will join in uh once so. Let's freed up someone splits pieces, I still need to create open issue, for that am I crd, um I'm actually starting to have some conversations at and we're about what like use cases and stuff which will help inform that um yeah.

B

I said the only thing I had to catch up with mike earlier in the week, um so I know he's going to be working on adding support for fargate, so there'll be a like a aws, fargate pool or something um so you see you can then spin up vks clusters with you know the managed machine pools and the gate as well so sort of like burst. I don't know compute or something which would be quite interesting um yeah, but it could be quite cool.

A

Yeah, um I've not no idea how kappa gets involved in any of that, but um that'd be intriguing to see. So it's going to be a sort of a variation of machine pool, yeah, somehow yeah, basically,.

E

A

How does that work in terms of like, because I thought you'd spin up a bar gate for every pods, rather than like? Who is that.

B

Yeah you basically dude yeah, it's exposed as a is it a virtual cubelet.

A

B

But yeah, but underneath it yeah, you get a fire in this fargate instance per per pod. Don't you the equivalent.

A

So it's like a far gate pool to represent like the fargate provisioning thing yeah, but we don't care about the individual fargate pod.

B

uh Now your correct profile will say: yeah we're gonna. This is you know, yeah and you attach it. So these these are the uh like the specs and then it will just run whatever is required. Yeah.

A

Oh, I see yes, I think we have a similar concept in vsphere, so in research, seven we have the sphere and pod vms that do the same thing, absolutely yeah. I think it's similar yeah, okay cool, make.

C

Sense all right, uh that's all call it a day, cool thanks, yeah, that's all soon later, bye.