Kubernetes SIG Testing, 14 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG-Testing Bi-Weekly Meeting for 20220614

Description

SIG-Testing Bi-Weekly Meeting for 20220614

A

A

B

This is dave.

A

I don't hear you, it probably is me, though,.

B

Hello, can you hear me.

C

A

Hello is that working ready? Yes,.

D

A

My gosh okay, thank you, pipewire or whatever linux just switched to.

A

So I'm not sure if ben is joining.

D

Do one repeat: do you want me to ping him.

A

Yeah, I'm just going to send him a text, real, quick, all right.

A

We do have an agenda.

A

Ben said, he'll be late on slack, okay cool. Well, we can get started with the usual and introductions. uh I'm gonna click record nope. I don't have host privilege. Does anyone have the host key ciao? Anyone it it's already recording? Oh, is it I don't see that but I'll believe you.

A

Cool well just a quick reminder that this call is recorded. So don't say anything you don't want on the public internet um and just a reminder that this meeting all kubernetes meetings, abide by the cncf code of conduct. So please be excellent to each other.

A

uh With that we have a handful of folks on. The call seems to be the first call like two or three weeks, so I guess we can start off with some introductions. While we wait for ben to show up is anyone here for the first time that would like to introduce themselves.

E

Hi, I'm roger, I am a junior student b-tech student from india and I am currently learning go and go testing, so I'm also.

F

Learning kubernetes.

E

As a part of my gsoc project that is in jenkins x, so I thought it would be great to just come here and know a little bit more about cuban race and how testing works here. Yeah, that's from my site,.

A

Awesome welcome.

C

To my first video.

A

Well, welcome. Thank you for joining us.

B

uh uh This is dave chang, uh I work for I'm limited and I'm an open source engineer, and currently um most of my my time is all- is focused on the kubernetes and the other project around the code. That is I'm a docker, so this is. I think this is the first time I joined the single testing regular meeting, uh it's very uh happy to meet you here.

B

This is that's all from us. Thank you. Thank you for joining.

A

A

Cool uh dave, I see that you had a item on the agenda. Do you want to start with what you have there.

B

uh Yeah, yes, yes, uh what I'm talking is about the gingko migration.

B

Currently, the pi is ready and it has been revealed by several maintainers and it has pending for more than three three months. I think so. uh uh Basically I I think that the pr is in good shape, and I just need some some some ois on cpr and the background for the pi is that the ginkgo v1 has been uh formally deprecated. The bin code is the framework of the e2e.

B

uh We can benefit from the new features provided by the ginkgo a2 and some walk-around, I think, there's some more corn uh in kubernetes, which which, which is uh uh exists for exists to to address some some issues for the ginkgo in gekko.

B

So I I think it's better better for us to migrate to kinkovito, but I understand that gingko ginkgo is the framework of the e2 testing world test. The framework the impact will be a little bigger, so uh we we need. uh I think we need. We need more, more review comments on the pr.

B

uh The the current study is that we have a pr for the communities and another pr for the tax infra, and also we have one pr to address.

B

Ginkgo v1 dependencies in uh open api repository overall, I think we have 3p already to migrate the ginkgo from the v12.

B

So this is a the overall status of this feature, so.

B

Is there any comments on that pr or.

B

Or if you are interested in the change, you might provide some uh leave some comments on that piazz directory. So.

A

I see that you did a mailing list, uh email back in april, about telling people about this yeah yeah. It might be a good time to to bump that to the top of everyone's inbox and let them know that we need some more eyes on it.

B

Yeah, yeah uh sure sure, so uh I hope uh ben is here so I I could ask his opinion on that, so I I can move forward the the move forward, this feature, so I'm not sure I use anything concerns from others. So I just want to bring this to this meeting and I want to get some feedback um on this.

A

Gotcha yeah ben just texted me back. He said he's on his way to his desk right now. So hopefully he joined shortly. Okay,.

B

Okay, okay, no problem, so I I I can. I can pick him his neck.

B

uh Oh, I can pin him on the pr yeah, so I I also thought uh I always encourage other other guys who is interested in the bingo migration could take a look at the pr and just leave your comments here. I think I will. I will address these comments as soon as possible in my site,.

A

Is there a this needs top level approval right.

B

Yeah yeah jordan has has a.

B

Has done some reveals in the past several months and dropped some good suggestions, and I think most of the comments has been addressed, so uh we we. We need some guys from the from the testing to uh leave some uh leave a lg to me labels or approve labels. So I can pin jordan for the final review.

B

A

At chow or linus, do you have anyone who could take a look at this on your side too?.

D

D

Okay, um I don't I'm not sure so uh we we are the uh proud team. To be more specific. I just took a look at the testing for uh pr. It's not controversial, it's pretty minimal, but I don't really have much or our team doesn't have much contacts on ginkgo. So I'm afraid we probably won't be that helpful in this space.

B

Okay, no problem, so the pr unit test, infrared uh just the one approach we talked to migration. So if we.

B

Take a look at the pr in the kubernetes. We have finally added another commit in the kubernetes to add ns for the build. So with that change. There's no need to uh uh no need to add another change in the case, in fact, so we can just abandon the change intensifier. Finally, if we got the pr in the code that is merged. Finally, so uh the big problem from my side is that we should first find someone who could uh uh approve the pr for the uh gingko migration in the commentaries yeah.

B

That is the first priority boom for my side. Okay, no problem- I I will I will. I will try to ping ping, some stakeholders, uh offline, yeah,.

A

I think ben just joined us actually.

G

Hi, it's me um like cindy joined from the office. Sorry y'all um yeah, we're talking about the ginkgo v2 pr, yes, yeah! That's very much on my radar, um I'm trying to get to some cabs first and I've got a couple of things going on um personally, but I'm hoping to get back to that. um If not, we actually have a lot of approvers in the test package. Last I checked. um We can probably uh poke someone else if I'm not getting too quick enough.

B

H

Okay, thanks mute. Do we also need these changes to to it to a framework project.

A

Then you're, muted.

G

That's a good question. We probably do there, um though we're not kubernetes as a project isn't currently using that sort of more like we're, exploring what would make sense for other people to use.

G

um So uh I don't think that I think um it's been a little while, since I've put up with that project, but last I saw it wasn't using these things, um but we should uh we should double check with them and make sure that we're staying in sync on like supported versions of things, I'm pretty sure it's actually more of a like just standard go test framework, um but it's been a while. I I stepped out of that project.

G

We should check with them.

A

Yeah we use the end-to-end test framework for all the six cli cube control tests.

G

uh Which framework, though, because a lot of times when people talk about the ud framework, they're, referring to the thing that's in the kubernetes repo um there's another project testing has to try to write one. That's actually meant to be used outside of the main repo, the one on the main repo we've voted against like staging or something.

A

Yeah we use the one that's built in.

A

Cool, so the action there is, we need some more eyes on it. um uh Dave. You said that you will ping some people offline as well, and I would also bump your mailing list thread telling folks. We need some more reviews and eyes on it too.

G

Yeah, it's a big enough change that I feel like. I would kind of prefer that we get like multiple people to to check everything um as opposed to the usual just like okay, you know we met approval bar um yeah. It would be good to get a couple of folks. Looking at that who've been, uh you know, writing evs recently as well. I think we've got a few on there, but I think it would be a good call to action to the community because also, I think, there's a few changes that will impact writing tests.

G

If I recall correctly,.

A

Cool any other thoughts on the v2 ginkgo topic.

H

Just a question: um are there any significant changes between v1 and v2 uh when it comes to just uh test writing things that we should be concerned about or something like that.

B

Are you able to answer that dave uh which kind of concerns you mention.

H

Yeah, for example, um if we are interested in writing tests right, um we are pretty much used to the v1 way to write them. Are there any any significant changes or things that we'd have to look for, or look after when switching to v2.

B

As I think for most of functionality, there's no change at all, just for the it it brings some new features uh to uh improve the test framework. So if you are just a uh ordinary user, there's nothing change at all, but uh if you are looking for some brand new features, I I think the v2 is better.

B

C

There's not long.

B

Long list of the features provided by the gingko v2 you can, you can re, you can go to the uh ginkgo official website to check out the vision list. So.

B

Yeah, so there's no much changes for if you just want to write test case and check the uh and verify some functional functionality for the e2e yeah there's a little virus at all. I think so so most of the uh uh change we need for the community is for the customer reporter migration and we have have uh uh rewrite. Actually we write the the reporter with v2, so it will be.

B

It will be okay for me if we, if we, if we start to use gingko b2 in kubernetes,.

A

Cool uh claudia, do you want to move on to your topics on here.

H

Yeah, um so we're currently planning to have unitas running for at least communist kubernetes, and I have sent a couple of full requests which address that there's a couple of issues which causes unit tests to fail on windows, the first triple requests are fairly straightforward.

H

uh Most of them are basically related to the fact that, um for some reason, file path that join was not used when constructing paths or stuff, like that pretty much trivial stuff uh things can get a bit more complicated in pull request form which might actually contain some debatable stuff. It might also contain a few bug fixes for various things because of how uh they've been written.

H

So basically, I can clearly need some help with the review for for those requests and then afterwards, we can potentially think about uh maybe add a piece of meat unit test for witness as well, because I have noticed quite a few interesting bugs as well for windows, which we might want to avoid in the future.

H

For example um this this something like a pattern that I saw um quite a few times. um There's path and there's also path, slash file path, as modules there case, which path is used, functions from path are being used and they do not function properly. On windows, for example, path.

H

The join just puts a normal slash between the paths that it joins instead of the voice, specific path separator, which should be a backslash which causes quite a few issues, for example other example, but that deer simply returns point instead of the the parent, folder and so on and so forth.

H

uh Stuff like this could be avoided in the future. If we actually enable so many tests, I have sent five requests in total.

H

um I still have some like five units that are failing uh which I'm currently investigating, but it seems to be um some sort of press conditions or flakiness there, which I cannot yet pinpoint, but I will at one point and send the 64 requests afterwards.

H

Then we could um add the windows unitas jobs afterwards, but having those frequents merged until then would uh greatly help.

A

And what type of approvals do you need right now? uh These are sig approvals or top level.

H

um They basically span the entire kubernetes project, so mostly top level, because having uh per module itself uh getting approvals that might be complicated, be a bit more complicated for some of the first requests, mostly deals with uh cubelet uh unit tests, the second one second one for volumes and the rest of them are various uh modules for pretty much and anyone and everything again. First, three uh requests are mostly unitas and trivial changes and the fourth one onwards, a bit more complicated and including actual code changes to how things work.

A

Do are we able to turn on the do? We have a windows job running. Can we can we turn a windows, job, that's informing and not blocking for prs.

H

um There's um an e3 windows job that is currently running. There should be already one called something like cab, z, windows, something like that.

H

uh We can have that one running.

G

For example, yeah we have some e2e coverage, but we've got wait. We've got unit test, breakage.

A

Yeah, I'm just I'm trying to think for for new code that gets put in. I don't want this to be a a rat race, for you know what I mean claudio tracking them down as new tests get added.

G

Yeah, um I think that we should probably be looking to actually so so. The problem is.

G

We don't want an explosion of uh precedent, we're trying to avoid everything being impersonated unless we see it's breaking really frequently, so we should so. First of all, we should start in like periodic or post, submit and see like how like how was the signal, um uh if anything, we're trying to look at. Can we remove some of the jobs because getting appear emerged is a nightmare.

G

um I think this could be a good candidate for pre smith. We should start in impossibly periodic that will also possibly make it a little bit easier from an infrastructure perspective.

G

Unless we're going to ask more folks since prow doesn't have windows, we either have to have prow spin up a windows machine and then like remotely clone kubernetes and run the tests and basically re-implement all of the useful ci things by like ssh or whatever equivalent.

H

For when it was too late, you can you you can ssh into windows, notes um pretty much. All windows should already have open ssh installed on them as well right.

G

We either we have to set up some kind of windows infrastructure like this, either on a per job, temporary basis, or someone has to like set up a windows cluster in the infrastructure.

G

I think that, depending on how much work we want to ask of folks that might be reasonable to suggest um actions uh just to get some like periodic signal.

G

Yeah, uh but at this point I don't think there. I think that if you wanted to run windows directly in prow, um I don't think anybody would post that. But I think it's asking a fair bit of whoever wants to set up you're gonna have to like you're, also.

F

Going to have to get test infra.

G

Everything functioning there you're going to have to get windows, container images built for the all the pro components and get ci running for that. We currently do most of the testing that project with like kind ed's and unit tests that are all linux as well um or instead of like fully getting pro running on it like have a pro linux job that remotes into some windows infrastructure.

G

I think I don't think we'd be opposed any of those things. I think it might be a lot of work, um so my first suggestion would be it's probably reasonable to start getting some signal by just using github actions as a uh like, a periodic or a post, submit.

G

The only real downside there is that you can't get test grid integration, so if you want release blocking, then it's probably that I think probably getting it onto prow is the is the best route, one of the one of the options for getting it.

D

On-Prem, the other small downside is that uh tide can't re-test those github actions as well.

G

But if we're starting not in pre-submit, that's fine, because that's null true but yeah. If you want, if you want, if you're planning, to get it in pre-submit or you're planning to get into release blocking having the tight integra having the merge, robot integration and having the test screen integration are both, um I think, pretty important for either of those things.

G

um In which case, I think we're either talking about a pro linux job that remotes to some probably ephemeral windows infrastructure, uh which is gonna, be a bit of a pain, slow, probably or um figuring out, getting proud of itself to have windows no pools, um which is also probably a pretty big diff, because things like we'll need to make sure that all the existing job pods scheduled to the correct os, and I think, there's a pretty large long tail there to actually spin that up. I imagine it would uh whoever works on this.

G

That's gonna take a while.

A

Are we able to get uh linting rules for those types of things not like not using file path, join and stuff.

G

um That's pretty hard to to lint correctly, because there's cases where like if we were to, for example, if you're talking about like cube, cuddle, exec or cp, or something and you're doing things with container paths, then it it should be path or if it's like urls, there's lots of valid uses of either.

G

We could maybe come up with some way to say, like this package should be using it, but I think I don't think you're going to have a very accurate heuristic for this.

G

Also, I can say from past experience of like just our linux unit tests not passing reliably if we clone the repo and run them. That's a pretty frustrating. First interaction to hacking on the project. um It would be great to actually run the windows unit tests and know that they work.

G

I I think that is a very reasonable thing for us to be doing it just depending on how far we want to take it. That might be pretty involved, given that we don't have any windows infrastructure today,.

H

Okay, um I think we might have some plans for that as well. um I will have to check up with my colleagues regarding that, but.

H

Yeah, those are some good ideas to be to be had yeah.

A

Any other thoughts on windows testing here.

A

Cool that was what was on the agenda any other last minute items people want to discuss.

A

F

uh I do want to just briefly mention: um if you type docs.proud.case.io, there is a documentation site for prow. It's been up for a while, but we are in the middle of migrating, a number of readme files from the test, infra repo to this new docs repo, that that is basically the host for all of these. What you're? Seeing on that site?

F

There is a legacy snapshot like top level folder that has like a snapshot of various readmes from pro from the pro repo. The idea is to kind of move those out from that directory and make it more accessible from a different location on that site.

F

uh As far as I know, I'm the only one who was working on this, but it would be nice to have some help here. um Yeah.

F

A

Any other thoughts questions concerns.

A

All right, not ben, unless you say otherwise, I guess we can give everyone 30 minutes back.

G

I've got nothing, I'm uh I'm happy to see that we've had the topics we have had and so much attendance today. Thank you all for coming and uh sorry's late thanks eddie. So much.

A

Well, thanks: everyone cheers have a great one.