Kubernetes SIG Testing, 5 Sep 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: SIG-Testing Biweekly Meeting for 20230905

Description

SIG-Testing Biweekly Meeting for 20230905

A

A

Okay, um I think we can go ahead and get started. It is recording um hi everyone. Thank you for joining today, September 5th, for the Sig testing bi-weekly meeting. If you have anything we want to discuss, please go ahead and add it to the agenda and if there's anybody new that wants to introduce themselves, please go ahead. It'd be great.

B

Hi, my name is Ben and I'm. A very novice contributor I am a master student at the uwaterloo in software engineering and then nice to meet you all.

A

Welcome it's nice to have you thanks.

A

I'll give you a second.

B

Antonio and.

A

We could go ahead and get started on your point.

C

Yeah so I didn't prepare during this week and things and secret providers started the migration of the all the CI to use this thermal crowd providers, but basically, right now you know that most of the CI is is GCE means that all the jobs that were using this Cube app script that you can find in the repo that is in clusters, class, Cube, app or something like that is going to use by default, the cloud provider gcpe um the cloud provided DCP controller instead of the the loops that were living in the Q controller manager and the cube I suspected a lot of things broke, I will I, don't know, I didn't have all the details.

C

There are like four or five issues related there was CI signal were detecting these failures. I think that that's today everything is is fine, running and smooth and Angling, but well, just as you know is the goal is, is to move together in in 199, so we can completely remove the internal Cloud providers.

B

C

Entry crowd providers and from kubernetes.

A

A

Anybody have any thoughts on that; they don't have.

D

Yeah I have a question um so right now, um when we run Ed tests against Google Cloud, um we use the scripts that live in the cluster folder to create the Clusters, with any plans on replacing that.

C

That's what I love too, and there was an issue I and several discussions. This is a neighboring discussion right, so there was one point in the time that cluster API wanted to do that. But then the cluster API jobs were very unstable, cops the project offered to do that. I, don't know, I'll talk with Justin Santa Barbara about in in the coupon in in Amsterdam and and there there was a sun discussion, but I don't know at what point is that and I know? Cuba I mean wanted to do that.

C

The problem is, is this in all when the tester becomes the testing part right? The thing is of this scripts is that they are very stable. As I mean there has no moving parts right.

C

The problem of this scripts is that is brush and it doesn't grow and I, don't know things and I had to spend hours and just to debugging. How do I look this configuration? Then you may be doing things that are wrong and you don't even know so. I, don't know how this chicken and egg problem is going to be solved, but we should for sure have something better than these Cuba script.

D

Yeah, it's not a hypothetical question, because I actually have an open PR in KK right now about running arm, 64 edv tests um against the scripts and it's a big fat mess. It's gonna work out when it's going on.

C

Can you link that yeah yeah I will see a second yeah? That's the problem and you only.

D

And the reason I'm asking is like, as you know, teams put together a keep test, two ec2 launcher for cluster E3 tests and it's and there's arm stuff very nicely. um And it's not leaner, because it's a bunch of girl code and it's very easy to reason about what nothing's going on.

C

D

C

The problem is, is not the tool in The, Proven is maintaining the tooling right, and it's like you test. We have q test. Michelle is here and has more contests tonight we have cutest. Now we have cutest two. We have the cluster apps script. We have kind, we have cops, is running out of things. Cube admin guys have kind of that is a version of kind, so we have cluster API. We have Cube spray.

C

The only fact is that the only thing is stable for the CI, that is guaranteeing is dispatch, script right, yeah.

D

I think we are in the chat um and yeah. It looks a bit scary, like some of the diffs that I did to it. um It's.

C

Like yeah clusters to support or whatever that's that's the thing, it's bad right. You cannot touch unless you guys think Hawkins and gives you all these weird commands that nobody knows that is using Bash.

D

Right, so if you look at the diffs I'm making to like manifest files right, um why do we put the arch like right at the end of the image, um instead of just referencing as like a multi-arch image?

D

B

D

One of the things I've found. So if you look at some of the files I'm changing like the dot manifest files um like I'm, just removing the architecture, I'm just referencing it directly I'm like you, would um in an image, that's obviously multiple architecture right.

C

D

C

The SE are not the dimension of this problem. That is the the architectural thing right. The problem with the architect is that nobody step up to to so, for example, with arm. We have this issue with kind two right and like the arm was everything like two three years ago and the young people came and we went to RCI and blah blah blah and they are the CI and after one month it started to fail, then I fixed. It then I fix it with Ben like three times and I told them. Look.

C

If you have CI you want signal, you need to support this thing right and sign it. So the problem of the multi-architectural thing is: who is going to maintain it.

C

Okay, so and- and then there is an statement in in somewhere of of the project, saying that I think that officially I don't know how many so, for example, we drop it all PPC or Atomic photic combination that they started to fail because one gold and back and nobody step up and we will block it, who can spend hours on debugging the scenes. So I don't know right now where we are in terms of architectures that we guarantee.

D

C

D

With you, this is not even much about architecture support right. It's like I need to get the shell scripts to work with both architectures yeah, but.

C

But that's the thing is that that's that's really pretty slow right.

C

So I think that Dave Dave is the one that does the CI forearm recently.

C

But I mean what is your their problem, that this this change is not working.

D

um It's not working there's some initial findings that are commented on there and it's yeah. It kind of makes sense of what is going on.

A

A

C

Know Cube test changes, but is: is there more people involved that is committing to support arm or why? Why do you.

D

Want it, there are people supporting, um so we've got to answer properly. Node um uh we have almond nodes in the noity test is working fine and there are some images of healthy okay.

D

If there's another person as well for each mentioned as.

C

D

The problem is, isn't that I think the E to e tests are are running successfully when I? Do this? It's fine, it's just the headache is our cluster scripts need to create all classes and because of some Google Cloud Shenanigans you gotta speak for example. The way you create the VMS.

C

But the the question is.

D

C

This is like the one-time.

D

Setup right, uh you need to set this up one time and then it's an ongoing thing is like the business. There is.

B

D

It gets to application right um your app which already compiles to arm, and we know the tests are running. Fine, uh green, um like we already testing this on today and it's all green.

C

This, if, if I got paid for every time, that this is one time thing, I heard that and.

B

C

I mean it seems very. What I see is that you are changing time out, don't change timeouts.

D

So that was the temporary fix of something else um like.

B

D

Want to wait 30 minutes to do something.

C

Anyway, so but then I can take a look later.

C

You can see if I spot something, but my suggestion is don't spend on the cube up scripts. We need to eventually remove that.

D

Yeah I mean like personally I prefer cops is great. um It's very stable, especially if there's a policy you're going to use the last major last minor versions, test head and newer versions.

C

But then, if, if they just kind of a CI with cops, they just use cops I mean is.

D

Pretty good for Upstream, though I didn't see. Why not.

C

I think that this is an issue to add cops to lose. You guys I need to chat with.

B

D

That's all we need from. There is the business logic that creates virtual infrastructure in Google Cloud. We don't really need anything else from it. Everything else is developing in head, so she should have.

C

What I said document.

C

I I I cannot find this I try to find this related and I'll. Send it back to you about the decisions. Let me ask, can somebody add me an art uniting on the Gen I can't do it myself? Okay, sure find this around using.

A

C

Cops on this here, I will share it like that.

C

I don't know, I mean the less that we need to touch on.

C

It disappeared.

A

Sorry I couldn't approve or I thought I did.

C

I think that they had the writing for me and.

A

Sorry about that Antonio.

C

Yeah no I was having to remember.

C

Myself, too, to find this issue to share with.

C

Muhammad, you are drawing whatever in there.

D

C

Because I already know people by the GitHub countries and then when you have different name, I I'm confused.

C

Sorry about that.

D

Sounds fun um because my name is on the handle, like you could type my name because I'm the old memory I'm an old member, you can use that and pop these.

C

D

Instead of moving these legs well not legacy, but the cluster EDU jobs to keep test. Two, which is just a wrapper for the shell Scripts and jumping straight to Cops, is, is a much better use of our Molly, but he's at the time that we are.

C

The the way it's the same I mean if we are going to move things like that. The start from the scratch like arm are the perfect candidates. Well, we should not do is is to uh the rapid changes on the system, because that causes a lot of pain on development right because at the end is two or three people that we know are. We are changing the scene in CI, but the rest of the people is sending piers and they send them failing, and then we try and then this is flight.

C

This is not flake and the CI sinner comes back and say: oh, we have I, don't know how many blacks, so the best thing is always this way, the same that we did with the AWS I. Think that's perfect right. You start moving things that are the less usage or canaries or whatever, and once you have them stable, you just have to migrate the other. So if we are going to move to another things like cops Cuban, whatever you name it, we kind of start with these things like art that they already don't have anything.

C

So it's okay! For that to to be experimental and be unstable, once we have seen that oh, this is working perfectly, so we're going to start migrating the rest of the things.

C

So, if you have because I told you it's it's the same thing that with this you know renaming the master to Main, Branch I mean I, don't think I. Don't think that it is the only problem there is that.

D

For the most part,.

C

Yeah, but the problem is not the chromatic chain, it's I mean nobody disagrees and, and I mean it was easy that should be done like three or four releases ago. The problem is, you know how many scripts depend on that I mean I, don't think that they listen anybody that is able to to know the brush radius of that we're. Quite we have like thousands of jobs. I mean, depending on on where things.

B

C

So that that's an example of why that's not done I mean who, who is going to be able to to narrow down all the problems? Yeah.

D

I'm more than happy to, instead of trying to get the old stuff playing nice with arm s run it with cops, cops very stable. We use that to deploy the arm tests and once those are very green and stable, then we revisit this conversation.

C

I'm going to find the link uh today's issue that was discussing the new and I'm going to mention you and and this uh approach and if there is no pushback or anything just go ahead.

C

We we can discuss that right, I mean at the end is I I. Think Benjamin, Justin and teams should be interested in this topic.

C

Good I'm monopolizing the conversation anything else.

C

Sorry, of course,.

A

Thank you guys does.

B

Anybody have anything.

A

A

Okay, I guess uh we can go ahead and wrap it up. um Thank you. Everyone for attending and I will see you all in two weeks. Bye.

C

A