Kubernetes SIG Scalability, 5 Aug 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2021-08-05 Kubernetes SIG Scalability Meeting

Description

Agenda and meeting notes - https://docs.google.com/document/d/1hEpf25qifVWztaeZPFmjNiJvPo-5JX1z0LSvvVY5G2g/edit?ts=5d1e2a5b

A

Okay, hello, everyone: this is six scalability regular meeting august. uh What is it today august? 5Th um yeah. I don't think we have anything in the agenda today, but I guess abu you. You mentioned that you have some question. So yeah go ahead. Yeah.

B

So um the the default values for the max in-flight settings uh in kubernetes is, I think, around like 200 and 100, but I was just wondering like for gke or other big clusters like how do you tune those values? uh Do you do it based on like cpu, core or.

A

um Yes, I think magic will know it better, I'm not sure how much we uh we we we can actually tell, but yes, it's more or less. We are tuning in with the size of the master.

C

Yes, so basically, this is a function. I think this is a function of the number of cores. I don't really remember specifics and I also like not sure what part of that is actually public or not. So I see okay, if.

B

C

B

C

Offline and maybe we can go back to you like, like maybe on either on slack or like next week or like yeah,.

A

Yeah, but it's basically some function of like size of the machine, yeah.

B

I see okay yeah, I don't know how much of it is public. uh In the case, I could take a look at it. The problem we have in open shift is um the default. Values are very high. It's like three thousand and 1000, and even though we have pf enabled uh like pnf enabled it doesn't help because the cluster gets killed anyway, because yeah the limit is so so high.

B

So I was, we are thinking about. You know tuning these values based on the number of cores you have, and I was looking for any reference implementation.

A

Yeah, I don't think there is any open source reference implementation here. But but yes, but if but.

C

A

Had the same problem, I mean yes like when we initially switched on like um pnf. It wasn't doing anything. It wasn't really doing anything because, like every request was like starting immediately anyway or almost right.

B

Yeah yeah, the other problem is: if we tune it too low, then um they might be starving. Right. Request may be starving, because you know the concurrence limit is so low and we don't have borrowing in pnf. Yet so that's the other concern, but I think right now more so in openshift we need. We need to lower this value, because most of the clusters are very small and, like I don't think, there's any cluster on earth that can like have 4000 requests like in flight. Just like huge.

B

Yeah cool, that's good to know, so I think we'll implement something similar on our side. Do you know, roughly, like uh our cpu code, how many requests you allocate for the limit.

C

B

I see yeah, I I think at our side we might do some testing with a mix up of different requests and see like what a baseline like you know, four core uh masternode can handle and then maybe like.

A

B

Out a value or heuristic value, based on that, um I was thinking of error 100 like parkour, but we'll.

A

A

Okay, anything else what we can probably get 20 minutes back.

A

Okay, thanks a lot for today and see you in.

C

C