Kubernetes Kubernetes AWS Provider Subproject, 21 Jan 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes - AWS Provider - Meeting 20220121

Description

Recording of the AWS Provider subproject meeting held on 20220121

Subproject updates:
CCM - 1.23 release.
kOps - IPv6 work
AWS Load Balancer Controller - 2.4.0 release
Karpenter - Intro, RBN support

A

All right, I believe we are recording. um It is january. 21St, 2022, uh welcome to the provider aws meeting um so to kick off 2022 we're going to change the format. A little bit do a little bit more project updates at the beginning of the meeting and we'll try to do a better job of recording and publishing this year. um What do you think about that? Justin.

B

Good goal: good goal: I, like new year's resolutions,.

A

Yeah, exactly starting off strong, um so I'll go ahead and go first. um So for the the cloud provider aws, we had a 123 alpha release and uh I think all the prs for that are merged, but I will double check after I finish the update, um so yeah, that's, I think, that's the really! The only thing I have other than that, let's see justin, do you want to say anything for cops.

B

um I I don't know I mean I think maybe john might want to do things. I think we've done a ton recently around ipv6, uh particularly on aws. I think john carey tell us exactly where we are and I think that's been driving a lot of the things we've been looking at. Like I see, john has two items there uh around lbc, which I assume are might have a back 56 john. Do you want to jump in.

C

uh Okay, uh so in ipv6 we just recently merged support for uh private topology, with v6 only worker nodes, um the I've been working on trying to get lbc working with that uh been hitting some roadblocks but and haven't had time to look into them.

C

uh Let's see, we also have a problem with external dns, um which there's an open pr out, which uh needs to be looked at by those folks, uh and I've also been looking into uh using lbc as default service controller in ipv6, because this dcm service controller is completely useless for ipv6. Only.

D

There's one limitation, though, for ipv6 there's no support, for instance target yet from the nlp side for ipv6, uh so we'll have to use ip targets for ipv6.

C

Yes, and so that means that I can't use ccm, I would have to use lpc correct.

E

C

Correct correct.

D

C

F

Way, thank you so much.

D

For uh contributing the v1 changes to lbc, it was in our roadmap. Definitely it helps appreciate greatly and uh getting back to the load balancer control release, uh so we are trying to get the 2.4.0 release out with v1 support. uh There's one change that I'm currently working on, uh which is uh adding support for the load, balancer class and uh also going through abstract security.

G

D

uh So that's what's uh causing a slight delay in the release, but this 2.4.0 release would have these two major features and we are trying to cut down all of the other planned work. uh So we support uh kubernetes 1.22 as soon as possible and we're working on it. Definitely.

E

D

C

So chaos is desperately you're meeting that.

D

Sure sure yeah, I will try to do whatever possible, from my end, uh to meet the dates and uh we can definitely chat further on kubernetes slack. If there's anything.

D

For making the load balancer control as default service controller, that will take some time.

D

We because it's an optional component, uh so we'd have to like look through like what all work is involved there, uh unfortunately like for other ipv4 customers uh like there is no way that we can make it default right, because customers have to take some action after the clusters. uh Cluster comes up. uh What I'm hoping is like with the support for load balancer class. We can at least uh say like without annotations, like for certain load balancer class.

D

It will be reconciled exclusively by the load, balancer controller, that's where I want to begin and uh for the long term uh we still need to work with the cloud, controller manager and load balancer controller story. uh I think that's where we have some gap right now. We need to focus on that story.

C

So talking about the lbc and ipv6 see that uh we're gonna need two features. One we're gonna need it to be able to uh just take on the being the default uh service controller, which is my open pr, and then uh we are also going to need a feature so that it defaults. The load. Balancer is the dual stack.

E

C

uh And I think that's a separate vr.

E

C

But I, but you know chaops, would uh just uh in an ipv6 cluster. It would just we would have it automatically shut off the service controller in ccm and automatically uh configure lbc to be the default service controller.

C

D

uh Are there any flags that you use to shut off like the service controller uh side or uh like? How do you do that uh currently or are there any feature gates? uh That's what I was wondering as well.

C

Yeah, I I put that in one of the tickets, uh there are two flags you send this tcm uh to tell it to uh first, uh you have to explicitly tell it to run the default controllers and you have to give it a flag to tell it to stop running the service controller. Okay,.

C

But yeah any pretty much any uh ipv86. Only uh our cluster of ip66 only uh pod.

E

C

Is going to want to turn off the cdcm service controller because it just doesn't do anything yeah.

D

Correct correct I mean customers have to take some action to make it work right, so which is not something we want. We want something to work out of the box, of course, so it's going to be a nice usability improvement. If we do this.

C

F

But that, on the other hand, doesn't.

C

Mean that you should prevent people who uh want to take action to change the defaults from being able to do that.

A

Yeah long term, I'd like to move all the load, balancer uh code to load, monster controller and just deprecate it what's in the ccm. But.

C

In that case yeah, we you'd probably want to add some code for uh classic load balancers into lbc so that it can take over that function.

D

uh As far as the clv is concerned, like uh our uh stance is like, since clb itself will be deprecated by aws eventually, so uh will not even move uh the clv or will not have code for the clv in the new controller, uh we'll specifically focus on nlp. uh That's uh that's what our long-term plan is.

C

uh Yeah, unfortunately, there's still some stuff that it'll be generally still have some shortcomings compared to.

D

Security groups and like the hairpin issue, I mean we. We are working with the nlp folks as well, see like how they can provide us a solution, uh and we are definitely working on those as well.

B

Yeah we I assume on that. We also have to figure out what we do about clusters that have clbs right. If we're going to just stop it working, I don't know what we would do there so.

D

Ccm, so that's the reason why we can't completely move to lbc we'd still have some role from the ccm side and like the way that I have thought about. It is uh for existing service. uh Ccm will continue to provision and update the clvs, but for new services we can provision nlb and then what role ccm is in that or do we want to entirely offload to lbc? Still we we haven't, have clarity on that.

D

Yet we need to have that story out and yeah.

C

The other question is: what happens if you don't use any annotations, because uh what's the default behavior, because currently that's uh you get a clb so.

D

Once we decide to support nlp like nlp will be the default when we switch nlp to default right, so without any annotation it will provision nlp. uh So for that specific reason like we still want some limited nlp support from the cloud provider, uh the ccm and for advanced features like target group binding and other improvements. uh Customers can use aws, load, balancer controller.

C

Because I'm thinking the long term you want to go to is that ccm doesn't have any load balancer, it doesn't have a service controller and that it.

D

Would work very limited like a very limited one like uh for customers who don't need all the features they just need? One load balancer out. We still plan to have something in the ccm.

A

That still sounds like we have duplicate like nlp code, at least um so, if we could get rid of that, and just.

D

E

Like use the library.

D

Like some load balancer controller as a module in ccm, some some of the modules we can import and do it at least.

E

Like that will minimize the.

D

Amount of code that we have to write for ccm.

C

The uh yeah, the other possible wrinkle there is, uh or with the idea of getting all the service control out of ccm, would be if support for load balancer services is required for kubernetes compartments, which would mean that you would have to have lbc installed in order to pass the performance test.

F

D

That's that's also true. Yes,.

A

Yeah I mean if, if, if we do, this lbc is going to need to be more um like always installed component, uh so we might wanna. I mean it's easy for us for eks, um but we might wanna think about other uh provisioners like cops.

C

I think caps wouldn't be too uh disconcerted with having to always install ldc.

C

A

Okay, anything else on.

D

So after the 2.4.0 lvc release, I will put some time I will try to see like if we can have a refined story around this area and we can discuss further after the 2.4.0 release.

D

And if you have any suggestions, feedback always welcome. If it works out.

A

All right, um so else, do you guys want to give an update on carpenter or anything else that you happen to be working on.

H

Hey, um I can say, hi, I'm ellis. I work at aws on scalability in the carpenter project. I've been bugging nick over the last year for any clout, ito's cloud provider, questions or feature requests. So I I guess I have a little bit of tangential experience with you guys, but it's kind of fun to sit in on a on another seg and see what it's like.

H

um I don't think I have too much to to announce about carpenter. I expect you guys, probably don't even know what the project is. It's, okay, you know: okay, cool um yeah, it's an open source vendor neutral, kubernetes native, auto scaler, creates the nodes directly as opposed to using auto scaling. Groups has its own cloud provider api, which we've tried really hard to refine, are about to go through the whole entry out of tree problem for various cloud providers. Luckily, it's not that big of a problem yet um but yeah.

H

I think the the most interesting thing about me or about the adidas cloud provider for me is um just being able to refine things around the node startup process to make things more smooth for auto scaling. So, for example, right now, nodes are named using the public pns or the private dns name, which requires an extra api call to ec2, given an instance id to ask what that is and there's some race conditions where sometimes ec2 takes. You know 15 seconds to even populate that value. So we can.

H

We have to handle all the timeouts and back offs and make sure that we can get that public for that private dns name the node name, but if it was just the instance id which I think is the path that you guys have been going down, it just works immediately with no problems.

H

um So there's very there's various things like that that I think, are potentially opportunities for our projects to align, to improve the customer experience, but other than that, I don't. I don't have any major ass, I'm just kind of here to see. What's going on.

C

uh Yeah, I would say uh chaops uh if you're running, uh dcm123 or newer and that may go down to 122., uh we'll use rbn for all working or for all nodes. So.

H

Yeah marcus from the cops team implemented support for rbn and in carpenter. um I'm sure you know who he is as well. Yeah.

C

Yeah yeah, our main concern was perfect. Right now is uh there's noise about removing support for custom, launched templates and we absolutely rely on them.

H

Yeah, that's a big long conversation. Our two projects need to have together, I think, perhaps with a wider audience than just myself and marcus too.

A

Cool thanks for the the intro alice.

A

I think that basically wraps up our uh agenda unless there was something I missed.

D

We also have like two new members joining the meeting, uh howe and olivia, who work with me in the networking team and they help with the aws load balancer controller. So how and olivia if you can introduce yourselves as well. Thank you.

F

Hey good morning, everyone, this is how I'm from the uk's networking game uh just to join this project, to work with fisher on the balance a little bit nice to meet. You guys.

G

Hi everyone- this is olivia, I'm from eks networking team too, and thanks uh keisha for inviting me to this meeting uh and I've been working on aws load, balancer controller with kisher. Since I joined the team. Thank you.

A

Thanks for joining nice to meet you guys, yeah.

F

Welcome welcome.

E

Yeah, I don't know some of the people here, so I'm just going to say hi. My name is pratik, I'm also in the aws scalability team same as ls, so just wanted to come and say hi.

F

A

All right: well, thanks everybody for joining. um I think that wraps up our agenda so we'll see you in two weeks.

A

B