Kubernetes SIG Node, 6 Sep 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Node CI 20230906

Description

SIG Node CI weekly meeting. Agenda and notes: https://docs.google.com/document/d/1fb-ugvgdSVIkkuJ388_nhp2pBTy_4HEVg5848Xy7n5U/edit#heading=h.2v8vzknys4nk

GMT20230906-170716_Recording_1572x1018.mp4

A

B

C

D

Being recorded.

A

Hello uh today is September 6 2023. uh This is signaled CI meeting. Welcome everybody.

A

Don't have anybody signing up for uh hosting test or bug part of this meeting, so I will be watching that uh and I looked briefly on test grid. It seems the same so all the same errors there. So I don't want to spend time there, um especially when we have like 24 items that they are here. I really appreciate people start working on different aspects of tests, so let's jump right into it. um I pre-opened a few bugs.

A

um So let's take a look okay, so this is modifying some. This definition to me, I think it's fair, um okay, I will take a look later.

A

So we're trying to test sidecars- and uh these are all sorts of definitions: okay, fix, multi-normal test for topology management memory, manager, okay,.

D

This is interesting.

D

Sorry, can you hear.

A

D

Okay, yeah so This Is Us, trying to fix the topology manager and memory manager, jobs that are fulfilling um there was a contribute tutor who tried to fix the CPU manager jobs and he essentially identified the issue that we had previously so I've and because his change was focused specifically on CPU manager, jobs I tried to have a follow-up PR for technology manager and memory manager jobs.

D

So we need to merge this one and I'm going to rebase my PR on that, and you know we should be able to see if this has fixed the issue.

A

Okay, yeah I approved this one. Let me move that. Okay, it's already here and for this one, so you just I, think you have a hold on that right.

D

Yeah because I would like to see how the change that is made for CPU manager job reflects in test grid. If we see green I think then we know for sure that the rest of the test would be fixed as well.

A

Yeah, let me approve it ahead of time, because it feels I mean it's definitely making things better. It doesn't make anything worse, and since there is a hold, uh you can unport it any moment. Yeah.

D

All right, thank you.

A

Another I think from gunjo, and it's also the restaurant start.

A

E

A

Flake and sidecar tests I think.

A

Those uh life cycle tests that we added last release uh when we've been working on sidecars facing General life cycle of uh how they, when post start starts and uh was it, runs through the uh everything um foreign.

E

Yes and I put on all the just, because the the contributor asked for specific reviewers, so just let them chime in any time. Other than that, it looks good to me.

A

Okay, so just increase.

E

A

A

Was it me a reviewer or somebody else? Oh somebody else. Okay um I actually find to keep approval here.

A

Assignment then, just in case I would I need to look a little bit deeper.

A

Whinge again, uh sidecars I think this is work in progress, and this is um yeah. This Dash is a like a code of um product. Did you have a chance to take a look or yes.

E

I I, actually, uh don't you find uh found a bug in Project's API, which is relevant in this context, because he was trying to use the producers API tool for the end-to-end test, which is a good idea by, in my opinion, so I suggested him. Okay feel free to rewrite the test. It's a way. You don't you're, not blocked on that and I think that we should fix the API. But it's not too urgent.

E

It's not something it's needed now, which could be, for example, a better item, better blocker or whatever other than that for the product code itself. Yes, I will be reviewing most likely tomorrow, but yeah I'm on it, and even if I'm, not this I know, I am assigned so okay I was about to say, feel free to send me thanks.

A

So if I would promise the eye board and I think that's wrong.

A

B

Those edit levels, those tests.

A

Were added to check the reason? Because the jobs really needs us.

B

Yeah so I'm not so this one's just uh skips the reason check only on container D, with a link to the containerdy issue that hopefully, when it's fixed, we can sort of undo this whole change. Basically,.

B

D

I didn't see a good weight yeah and the.

B

Other one was just the linters wanted me to change the bill. Mega expect um I, didn't I, couldn't find a good way to detect containerdy other than seeing the container deep binary binary is running.

B

But um get a validated against one of my nodes for uncontinently.

A

Would be the same for Docker, because it will still be running.

B

Because, if you're yeah, if you're in the continuity binary, um this will feel like ice technically, we can't tell that Cube. Let's talk in the continuity binary, I didn't know. If there's any method to detect that well, I didn't I started to do this earlier and then uh Paco said hey. This is really flicking a lot. So I submitted this just to sort of get some discussion on. Is there a better way to just detect container-d, because this is causing a bunch of failures at the moment.

A

Do you know how Kyle Works around it or is it was still busy because.

A

Peter, do you know about this bug.

C

uh uh Oh, is this the one where whom we sometimes like Miss um kills yeah.

A

C

Is actually I don't know? Do you either of you know, so we hit this when we set the system d c group driver and we set inactive or failed as the collect mode or scope units. I, don't know if you all do this as well. We did this to prevent leaks a while ago, but this actually makes a race happen.

C

Where system d is too quick to delete the C Group after the unit fails from a noon kill, and so our the equivalent of our Shem kanman doesn't catch it early enough and fails to Market as it goes.

C

um So we do hit this, but we do hit um failures in the um pass and there is a fix that we are tracking in run C. That will allow us to fix this. That um I, don't actually remember where it's at now, but.

D

C

Don't know it's totally possible because you kind of you.

E

Know I don't know if.

C

You're setting a nectar failed on your unit, so I'd say that that would be something to check.

B

Yes, I guess: I'm I noticed it a lot more on the containerd, but it might just be that there are more containerity tests running I'm. Looking now and it looks like uh there's a cryo test failing as well the.

C

Same reason, yeah we have our own uh in the cryo CI. We have a couple of varied like you pass.

B

B

Let's see uh any, is there an issue against Ron seeks that I think continuity was still trying to investigate this thinking. It was a problem off their end, but if it's a race with system, D I'm trying to look at the all the.

C

Yeah, so if you uh this is uh here's uh I'll post it in chat the 1.1 cherry pick to the pr that fixes this. So if you vendor one dot or if you have the I think it would be, it would have to have the binary on the Node be run C 1.1.8. Then you can remove the inactive or failed and then.

D

Like her has a.

C

Whole bunch of information about like what went on here, okay,.

B

Not for the goggle I'll go read that after a meeting, so let's uh CR may not be necessary unless we want to skip the reason check entirely for now, I'm not sure just to deflect things or try to fix it quickly.

A

Yeah I'll just add a link here, so we won't forget, uh for this part, I think we need to just keep it all together without check for continuity, yeah like quite a lot and who just explicitly add a skip there. It will be better yeah I'll do that yeah maybe even have a check performed and if check fails, don't skip so it will run at least sometimes most of the time. Yeah.

B

A

We'll just lose this job yeah.

B

Yeah that works.

A

A

If you have the same time like bug tracking it for cryov uh I would appreciate, if you share it, uh we can just type it together.

C

Yeah I think I mean the language should be quite different, because we were just tracking it for, like the Upstream.

E

C

Failures but I, you should have an issue it'd be better if we did uh yeah yeah um and you tagged me on that bus, so I can find that.

A

A

A

Okay, so for test it just must be reloading and some resource connection.

A

A

And this has also worked in progress.

A

Duplicated great Airway, timeout.

A

Yeah for no teams there are a couple of files.

A

A

E

Anybody interested.

B

Yeah I can look at this one.

A

Yeah I, just don't know like I, just need like some time to read about this.

A

Similar cleanup yeah um this oops okay.

A

One minute.

B

Yeah as well um yeah I guess you might as well, because if you happen to touch a test that modifies one of those lines, the lint fails so yeah.

A

Foreign yeah I hate this one. um We get PR like that, every now and then I'll just archive it out. It never succeeded. So people don't really like comparisons like wooden competitions with like equal, equal, true and send this PRS and it never been merged. Like I haven't seen once it will be. What was successful.

B

Especially when it crosses a whole bunch of owners like that other VR, that was fixing the test like it was just in test CDE. So it's kind of much.

E

B

More likely to go in when you only need one person to prove it.

A

Yeah yeah: this is not.

A

I think I double up on this one. So.

A

As I said, we have a lot of things to do yesterday, so keep going.

A

Okay, 95 and static Point detectors are missing during Google startup. Why do you need to get notified?

A

It's a bomb. Okay. We can do part of back trash right now. um The final reporting.

A

Trying to get there, but no login without errors.

A

So we started couplet.

A

Okay, so person saying that uh some warnings needs to be updated.

A

It's not a test. Okay,.

A

Okay, I am confused. uh Let me archive it out from here. If anybody's interested, to take a look, please do um it seems to be a just a debugability Improvement.

A

This one I will remove kind back.

A

A

Almost got my bugs.

A

Okay, uh next one is, uh we can go into, wants to switch uh test or something else.

A

But you already reviewed it right.

B

Yeah so we were using the termination log, but he has some uh like longer running tests and wanted to just pull it from the container log itself. So he didn't have to wait for the container to actually exit coach seats gone.

A

How do we pull it from Continental work.

B

um Actually uses that uh I actually use the logs API, you can get it from a container log. So.

A

B

Does this yeah still does the same thing right into a file and recetting it when the thing restarts, but.

A

Is it available when it's terminated.

B

Yes should be yeah analogs.

A

A

Yeah so Fusion duplicate, okay, another one of those.

A

And as well only for nodes so yeah, they probably won't approver uh origin in here. So.

A

A

A

Another one of those cleanups foreign.

A

Yeah, quite a few.

A

Very interested.

A

um Project actually want some feedback on what is better.

A

I'll take a look.

A

D

A

X parameter, it could last longer looks.

D

Like a product.

A

Change I wonder if they need LG TMC. Sorry.

A

Yeah I don't think it's related just a few unit tests that attached in most of the projects like that.

A

Another issue Canada outside come on second kind of we hope both start of the first container completed.

A

I think this is what gunju is working on yeah we have somebody assigned to it.

A

Teaching log monitor should generate no other conditions and events for corresponding errors.

B

So I think I fix this. One um I didn't notice issues there. The I had a change merged in yesterday, I think where the someone updated a node problem. Detector image in the arm. 64 version was uh not an arm64 version.

B

It actually had a x64 binary in it.

A

Okay, I sent it to if you can call it yeah.

A

A

That is fine, um okay, uh we done reviewing like crash on this board and we already half an hour past uh the hour so yeah at some point. We need to really use this in progress and issues to do um to make sure we not forget anything here, um yeah for now, let's switch the box today.

A

A

A resonance between scheduler and device plug industry- oh.

C

I think we reviewed it last time.

A

A

It's Kind feature, so let me just remove come here.

A

Well to mount the horse was coconut, it's from June, so yeah something going on. Let me just okay: it used to be six storage, but now we just added six node.

A

I wanted to play a port in my cluster to monitor with information nodes net.

A

Okay, that may be Qatar. This thing.

A

A

So, let's try to know which and then maybe.

A

Okay, oh this is the box that we discussed yesterday on Signal meeting I believe.

A

A

A

E

To delete Port.

A

Doesn't queue ports process immediately when specifying grace period equals zero aversion version? We need to know what version 27 right so I remember. There was some fixes in this space.

A

B

A

A

I want I mean, maybe we just need to try to reproduce, but also may want to ask how Google Talks.

A

I remember: we moved the grace period override like few lines up in the past release in 128.

A

And I don't remember the.

A

Me try to find a bug how to use this file.

A

Let's do everything in browser.

C

A

Yeah this no! What is the difference.

A

I feel like I'm wasting time off everybody on this call, so.

A

When Max Sports count is set a note how to get the list of ports of websites that are considered in that limit, okay pick, all the ports are considered in this limit.

A

A

Yeah the person was confused was a bot accounted in Max pod count so yeah. There is a reply already now job files that time is 19 service, okay, Linux, Time, Zero,.

A

A

A

We might have fix something big started at I. Don't remember if we take it from um continue. Runtime.

A

D

A

A

A

In the point of some PR.

A

Well, I have a comment here.

A

So I think um it may be related to container runtime.

D

A

If it's important, due to MTG Wireless returns as completed.

B

C

Think I looked.

B

At this, one um I think it's as expected um yeah, so he was in his test he's using nginx which handles the Sig term, gracefully and exits with a status of zero, which then costs the job to be completed successfully. If you swap it out for something like bash sleep, that's exactly like anything. It'll reliably have a failure status. Basically,.

A

Interesting, so, even if you have a job, let me eviction, it still can be successful.

B

Yeah, it seems.

A

Weird there's a discussion.

B

On signaled a few weeks ago about this, but basically yeah doing a big, it's a sick term, and it if your pod gracefully handles the Sig term and exits with a zero status. It's successful.

B

Which I think that is a change, since it was only through seven I forget.

A

I'm, sorry, is it the.

B

Regression you're saying uh I think it was intentional um trying to find the the notes from the sigma code reading.

A

A

And uh another box.

A

A

A

There's only four of us, so if you have anything, please pick up. Otherwise, let's get 10 minutes back.

A

Okay, thank you have a good day.