Kubernetes SIG Testing, 3 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG-Testing Weekly Meeting for 20220503

Description

SIG-Testing Weekly Meeting for 20220503

A

A

Hey uh are we waiting for people to join.

B

C

D

B

Let me give it one more minute.

B

Okay, well hi. Everyone welcome to today's kubernetes sig testing. Meeting, uh I'm benjamin elder be hosting. um This meeting is recorded, will be uploaded to youtube, and this meeting is under the cncf code of conduct essentially be excellent to each other.

B

So we have a couple items today. First up nebrun rajas, do you want to talk about test grids.

A

Hey yeah, so uh so I basically, I had two questions. uh One on like uh open sourcing, the front end of test grid like uh are there any timelines around it? uh What is the community thinking about open sourcing front end of test grid? I know the front end of test grid can be used as a services of now, but that requires giving explicit read access to one of the.

A

I think the kate's descript service account to the to the publicly available pocket, which has the config of test grid um and also like it'd, be cool to have like a standalone test. Kit of I mean that's standalone uh instance of disk grid up right, not using disk dot case dot, io, so primary questions around like. Are there any uh timelines or discussions on open sourcing front end of the screen.

B

uh So my understanding is, they currently know the components that you see in the repo are the components that uh will be open source this time, um I'm not directly involved in that decision making, but I asked one of the developers about this earlier.

B

I think it's just a matter of approval.

E

B

A

E

Community or cmcf can do to get that pushed up because test grid is such a vital piece to our entire release and project that yeah.

B

Yeah, so you can you, can you can file uh issues voicing your support for this? I think, but um I'm not sure if there's like a whole lot more, we can do um it's been discussed a few times internally.

B

uh There is also I'm gonna. Add some seduct. We do have test credit, has some stuff about like sort of test grid as a service and how to set it up for additional dashboards. um I think that's. The closest reminders we have at the moment is that, like you can add additional usage.

A

Yep yep, that's that's true, like you can use the front end as a service, but again that's like uh having uh tesca dot gates, dot ion a reference to that right right, so yeah also, if there's any help required um yeah after that as well right.

B

I think, as far um let's see, I think as far as like getting open source about the main thing you can do today is like file an issue outlining like why you think the front end should be open sourced, um so they have something to reference, but um yeah. I don't think it's a. I don't think it's like a resource issue. I think it's just approvals.

A

Got it got it cool, so I'll, probably uh file an issue and also start a thread on the mailing list, so that and and maybe link things so that we can have something running.

B

Yeah, I think, uh having discussion on it is that you just written down somewhere, is helpful for being able to point to in terms of like arguing in favor of this, but um I think that's probably is about. As far as we could go.

A

God got it cool all right, uh uh so so the second question I had was uh mainly around like adding support for like project logs for s3, so so the ongoing issue right, which you have linked in in the dock right now uh it it also mentions that s3 data is not something that the web app can render.

A

So I think this again points to uh like uh the front end of test grid. So uh if there's any help required in this aspect as well right, um I'm up for that.

B

Yeah, that's to know, um I would like mention that somewhere but yeah. I appreciate the choir.

B

I I don't know why um I've never understood this, but, uh like I can't officially speak to that. I just tell you what I know currently, which is that it's as far as I know, it's all the existing components are in source and if we're, if testgrid is not running on those, it is moving towards running those, um but that I don't believe, there's any plans to move any further components, and I think that just means the front end, um but I'm not certain.

A

B

All right and the reference thing for anyone just watching the video we linked in the doc there's a there's, a thread about there's an issue in the repo about um s3 support and uh it ties back to the to the front end of the discussion. There.

A

So sorry is it. Was there a question for me because I think I lost you for a bit.

B

No, I just uh some of what we were discussing is like only in the doc, so I was just briefly mentioning what what we were discussing.

A

All right all right, so uh I I think I missed what you said around like uh s3 support for the project logs in tesco. So is there any help required in any specific area around that.

B

The so that I was mentioning that we have there's an issue there's an existing issue in the test grid repo discussing that and that, um as you mentioned, the current discussion cycles back to the front end.

A

All right, so I think I think that the best way to go around this would be like filing an issue for the front end and then following it on the emailing list or something.

B

Yeah, um I think I think filing another issue and the test grid repo and uh and maybe bringing it up in the sig testing email list.

B

I yeah, let me see so michelle suggested filing an issue in the repo, um but I don't think filing and mailing this thread will hurt. um I think we're kind of gonna see the same thing, but anywhere we can collect. Support is probably helpful for making the case.

A

B

Cole, do you have anything to add.

C

um I guess what I can add is that uh I can't speak for the test grid folks, but I suspect that that will not be prioritized um in the near future, um but yeah, um given that it's the front end is not open source and uh only googlers can develop on that. I'm not sure um if that'll be prioritized.

B

Yeah, I I'm not part of the uh team. That was it so I can just I've just spoken to them about it, but uh yeah it's been a long-standing issue. I I honestly think it should be open source. I think that makes sense. I don't see any reason not to, but clearly others have so uh we can raise it again. um To be honest, I wouldn't really bet on traction there.

B

Like you know, it's not a new topic, um unfortunately, but it's certainly worth trying.

C

And I do think that uh it is an extra data point and argument for open sourcing, so the test grid folks would like to have that because they they are trying to make that argument.

B

Yeah, as far as I know, everyone working on it would love to have it all open source, but they don't get to decide.

B

Okay, well, let's take this to the mailing list and the issues uh collect everyone's comments there um arno, do you want to talk about greenhouse.

F

Yes hi, so the issue was open a while- and I did a I opened a pier about this- about greenhouse so most of the jobs using that now builds images without binaries and images without using great greenhouse as a basal cache. So I'm curious about what's a timeline to move forward about this.

F

How do we need an announcement from the sig chair and tl.

B

I think we can move forward with this a book we've uh that we found that cops would still appear to be the only user and we've spoken to them. um The other user was kubernetes itself, which is the original reason uh that we like funded this um and uh we've since stopped using it there completely. So.

F

uh No one's is yelling at the moment, so I think we.

B

Are right since then, I think since a soft disable went smoothly, and we only expect it to be one project. We've already spoken to the owners of. I think we are plenty good to just quietly move forward.

B

Okay, if anyone has if anyone comes back later and has any concerns, we can roll back out an instance and discuss that with them. We probably roll out a much smaller one and um you know we're not depleting code or anything, and the cache itself is already like a best effort component of the build. So uh there should there should be absolutely no one actually broken by this. There might be some job somewhere that, like lose a little bit of performance.

A

B

But I would also say the community has moved away from using bazel nearly everywhere. um So probably anyone that's still interested in the cache should be considering that as well.

F

B

For uniformity and project development.

B

Make things a little easier for contributors.

F

Okay, uh I think that's all for me.

B

uh Okay, great well, let's move forward with that. um I see antonio, has just dropped in something about going silent and get up integration.

D

Yeah I'm trying to find the pr so paulie. I don't think that he's in the meeting he's trying to to implement a more street so to run the golan linter with an strict mode and he wants to do an integration with you have to represent something, and I see that he is trying to reach out to different people, and I was wondering if we can provide some guidance I'm.

D

I really think that we should be more restricted on the lintel and I I I'm fine. With looking onto into the rocks.

B

Yeah, I I think I commented on the thread um what what he's talking about with having a second pass of the lint- that's more restrictive um but isn't blocking uh initially potentially makes sense to me. um I think that I think the remaining thing that patrick was looking for is how can we enable the commenting, online's functionality from growing sea island um yeah? It has support for doing this and github actions. I think it, I think. Probably it looks something like we find out.

B

How does what is the format in which github actions supplies the token like what environment variable or whatever does it use? um I know we could set that on a pro job. We need to pick an appropriate token. um I don't think we have one at the moment um like for the triaging issues. We have an account that we pass those that's just an unprivileged account that has nothing more than the ability to comment um has no permissions in the project uh we could we could.

B

I think we want another one of those and that's more of a question for the to like a re-race to the github management team, um how they want to handle that.

D

So so there is no technically, there is no technical problem. It's just a problem of of getting this token.

B

I can't see for certain but minus my rough understanding of github actions is that they inject a token that can be used during the action, and you can configure what permissions it has um to potentially go interact with the repo there's. Nothing that prevents us from there shouldn't be anything that prevents us from injecting a similar token. I think the question is more like what token, probably we need to make a new account and get a token from it, um because.

F

B

They all have because they'll have rate limiting and they have names and things I don't think like kate's triage robot is appropriate to be going commenting, lit issues, and we can't give it something like the main ci robot, because that one is privileged.

D

Okay, now I was wondering if how how can he render the data my.

B

My my biggest problem here remains that, when we're running things in pre-when we're running things in untrusted ci that comment, um we always run the risk of someone just like hijacking the account and doing things like as us right like someone could just. I could send a pr to kubernetes that replaces going sea island with comment, a bunch of ridiculous nonsense and.

F

Use the same account.

B

Yeah, so that that's I I agree that can be easier for contributors to understand it um if it comments in line, but uh even if we use like github actions or something we run the same, we work. We fundamentally run the risk that, like we are running code from the repo on a pull request and then letting it comment. um So it's worth considering, but it like.

B

We have other places where it's feasible, that you could probably steal like the triage bot account or something it would be a little bit harder because it doesn't run on pull requests, but not a lot harder, because it's in the same cluster.

D

No, it's fine. I mean okay.

B

So I think the so I think that's the main thing here is that we're moving it to specifically being a pull request. Running thing I don't know like have actions aspect of this. I I feel like I wouldn't be thrilled to be doing it there, either um and like in independent of any other reason to choose between them.

B

I think just the idea of of pull request code commenting sounds kind of bad, maybe github action scopes it better or something we should check into that before moving forward, um uh I mean they would have to be doing something special, but I imagine that might might be the case.

D

um But then, then, is it better to postpone that and let's split that part of the yeah okay yeah.

B

I think we should.

D

B

I think we I think patrick's concern is that if we, if it doesn't comment and it's not blocking, then people aren't going to pay attention to it, but it wants to turn on lints that maybe are not ready to be blocking.

D

This this is going to take time, but we definitely have to do something because sooner or later we you need to fix all this amount of tech that we have.

B

Yeah, my thinking is that we should. We should say that these are good lints and that we should just make them incrementally blocking by using the run on new code feature, and we should just make them pull request blocking and sure you have to go to the logs to see what's broken. But that's true of all of our links today and it would be cool to make that better. But we should probably make that like a follow-up instead of the instead of blocking this effort,.

D

The last time that when, when I enabled the the lint of the ci linter, it was about 50 comments, copies fixing things. So I think that this is going to turn out.

B

Great yeah, I think the golem clint, having the ability to tell it this is the this is the commit that we're starting from and run only on new code with a different config is definitely something we should leverage um to enable more links, but.

D

I think we should just do it.

B

Blocking and make sorting out commenting it follow up.

D

Okay, so everybody agrees on on on making the linters modesty.

B

uh Well, I haven't heard anyone else's opinion, yet uh I don't know if anyone else here has one.

C

I I agree that that's a that's a good best practice.

B

I I'll have a little color commentary that in the past, we've had to do this by just saying we know all these things are failing and adding um and configuring that and saying, okay, we'll just ignore these packages or these files that we know are failing the lin when we add the lint or we try to do like a mass fix.

B

I think it's much better to start by saying, let's start by just applying it to new code, I would still like to see people go back and fix it for old code, though um the one thing about doing the approach where we just say these are the known failures is that we know where all the failures are and we can track them burning down on the uh and have like a like a ratchet on the flip side.

B

I think this gives us a ratchet for new code and that file was always a bit of a merge conflict.

D

B

This way we can, we can still kind of ratchet new changes, not violating the the new linters and not have the merge conflict on the known failures file.

B

So it seems like a straightforward improvement. I think we should probably split up the commenting part but very curious if anyone else on the call has thoughts.

B

Okay, do we have any lingering topics that weren't on the agenda generally, I would say, let's put them on ahead of time, but we have a lot of time left in our slot. So.

D

Yeah we have the bingo b2 thing, that's going to be a and I.

B

Yeah, I think we don't have all the people here for this, but that's quite a uh quite a change coming.

D

Yeah, that's more heads up to the people, I'm.

B

I'm not thrilled that the level of breaking change. This is going to be for us that it's kind of understandable, but I think we've left ourselves in a place where this is going to be a little painful with things like there's a different binary. You need to build now and um the the size of the pr that makes the changes quite large.

D

But we we, we are publishing the to it. Binary.

B

uh No, you have to build the ginkgo binary, the the ginkgo runner binary. um So right now, if you want to run e2e tests, you can you, like people, have scripted invocations that, like we did in the kind repo where you you need to build eating.test, you need to build cube pedal and you need to build the uh ginkgo binary, but the import path to the ginkgo binary changed and the way that you tell the kubernetes build system. What you want to build is generally the import path with uwe.test being special case.

D

Yeah, but the thing is: that's that's it. You want to to use the gingko binary as runner, but you can run the e2e test, binary direct.

B

It's true: I haven't really seen people doing that, though. So that's like.

F

Another aspect of it.

B

Being a breaking change um that.

F

Is going to be a little bit.

B

Of a long tail to track down in retrospect, it might have been good for us to for things that are worth building from vendor, to alias them like to say like make what equal ginkgo, instead of make, what equal and then like the full vendor path. But um I think it's too late for that now. So.

D

Okay, the best people build the gingko binary from the ripple, so.

A

D

Not going to work right, they can go big. One is not going to work against the right.

B

So you have to I'll. Let me just point to the diff in the company, but I think that will make more sense.

B

uh So this change.

B

So that sort of thing is going to have to happen in other places, uh because the way that we tell the build system to build it includes the import path.

D

But it's because of the scripts I mean people can download the gingko binary by itself.

B

uh But we also, but they wouldn't be doing that currently and they shouldn't, because we want.

D

To make sure they get the right.

B

D

Yeah well, no I I was. I was wondering it was something more at the level of the code of the test.

B

I think this is the only place that we're exposing like that people are doing that uh outside of just using one of our development scripts, so it might be adding a make target for this, so that you so if there's like a v3 in the future, you don't have to do this again.

D

B

Sorry puppy he wants to play.

B

um Yeah, well, I guess this is more of a call. Actually everyone, then the the uh if you're interested, there's a rather large change coming in kubernetes to migrate us from ginkgo v1 to ginkgo v2.

B

Currently 21 commits and there's a lot to look through. We could use eyes on it.

B

Okay, well thanks: everyone for coming um seems, like we've run through our agenda items, we'll let everybody have about half an hour back, enjoy the rest of your day.