Kubernetes SIG Node, 14 Mar 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Node Sidecar WG 2023-03-14

Description

Meeting notes and agenda: https://docs.google.com/document/d/1E1guvFJ5KBQIGcjCrQqFywU9_cBQHRtHvjuqcVbCXvU/edit#heading=h.m8xoiv5t6qma

GMT20230314-160421_Recording_1656x1120.mp4

A

Hello, hello, it's uh Tuesday, March, 14 2023, it's signaled, sidecar working group I, welcome everybody.

A

Let's get going so I think we are at Finish Line and um we need to decide I think at this meeting, whether we trying to shoot for this really so, uh but we will be able to do that. uh Who wants to uh ganja? Do you want to say what's going on with this PR.

B

uh Yes, yes, uh I opened this PR, this big PR to kubernetes last week with Saturday, Matthews and told, and some people reviewed this and I think they are all addressed so and now the action items in the list are or addressed, I think the topology and CPU manager thing I'm, not sure.

B

A

And so for topology manager, I think um we handled first part uh told change uh like changes, how they calculate total size. uh I, don't think we handled second part I, think uh topology and CPU manager has special logic. You need containers that uh we may need to modify.

B

A

Am I making sense like? Is it uh you updated the topology manager total size right calculation.

C

I believe so obviously uh check um yeah I'm not familiar with their their other logic, but yeah well. I can verify that.

A

Okay, and uh did you try this um idea with chickens and tacos for the next container to understand where the sidecar was already started before.

B

Yes, I I already merged it to this branch, so I changed the condition to start the next any container. From the previous side, query started now to the previous study. The car has ever been started before so I think it works. Well.

D

B

D

It this this morning it looks okay, yes,.

A

I'm trying to find it anyway, um okay, so you already looked at it and Test passing I assume right.

B

A

B

Is already a test case and also I I complete the test cases you made Sergey so that PR is in your Repository.

B

Yes, I I think most of them pass, but some some are not like bad image. The sidecar with bad image will never start, so it should be removed. I think, and there is one test case that is not that is failed, so the button up this this page.

B

So there is only one job.

B

This job is, um if, if you use a part with restart policy on failure and the sidecar container is continuously failing, then, when the main container completed with Mexico general, we should Mark this part to succeed, but the the sidecar container is in Waiting state. So in my logic, we can I I can't Mark that part to succeed, so I think I think I can change. I can fix it easily.

A

Yeah I think we catching a lot of good issues with uh with the test uh have I think I mean it would make sense to succeed the port right when a sidecar is failing.

B

Do you think that is it okay to Mark part to succeed when sidecar is still learning.

A

I think it would make sense. Imagine you have a login container.

A

Do you want to know that uh login container failed or you want to know that, like main logic, succeeded, so I think main logic succeeded is more important right, I.

B

A

With making sense.

B

The current logic is waiting for main container to complete and also waiting for all sidecar container or terminated.

B

Being terminated so.

A

Yeah I guess it's. uh It was interesting challenge because when main container stops and you started uh terminating sidecar, if you were lucky and the side car was running, then you will succeed. The port, if you wasn't like in sidecar, was just crashed like uh then you will fail the port right and I think it. This inconsistency is not ideal.

B

Oh actually, the the execodile cycle does not affect the parts succeeded or failed state it. The I think main container succeeds, then part should be succeed, regardless of the state of Sidecar.

B

But but I think that part should be marked as succeed.

B

When all containers are terminated.

B

C

So this is the case where the sidecar comes up and like repeatedly crashes or something, but then the main container succeeds.

B

Yes, yes, but but it stops restarting, but in Waiting state not restart again, but the part phrase is still in running now, so I can just I can just fix this easily.

A

Okay, so we have some life cycle issues that needs to be addressed, um thought you just posted something in the chat. I will post it here.

A

You're saying that technology manager uses new compilation, but there is changes required.

A

A

All right um for this logic right for this part, yeah.

C

Yeah, that's there might be more it it's a little hard to try to find every place throughout the code base where it tries to add up resources, but.

A

A

Yeah I think I mean theoretically we can just say that this is a limitation of current implementation. The topology management, CPU manager wouldn't work as expected. Recite course, foreign.

A

What about uh limiter name, you.

C

The limit Ranger um yeah that should be done um so I ended up just copying the code and changing the type from the core type or API tap to the core type. Basically, I couldn't find any other elegant way to actually make it work.

A

C

Yeah it's already it's already.

A

In here, in this big car or separately uh yeah, it.

C

A

Yeah I'm just thinking, maybe we can merge it separately. uh It shouldn't be a problem.

A

B

And yes, and also there is another clear surrogate you you made, you merge the E3 node thing, the third Pier it breaks. No, that is not not that one third one, okay,.

A

B

After your European merged the the job that.

B

That enables all Alpha features break so.

D

B

Think we have to separate them.

A

D

A

B

And also, we had to enable only sidecar container feature to API service in testing fraud.

A

So you're saying there should be PR for sidecar. Specifically this one right.

B

A

Okay, is it for pull requests or CIA pool? Okay,.

A

Okay, great uh I will review that this, so we have end to end so Almighty is what's the status of props and pre-stop hooks. Is it uh something you worked on? You said it's a very small change in logic, right.

D

Yes, so it's it's merged uh in your branch. The unit test, but I haven't done an end-to-end test, but I I think it was covered by uh some of the work from gunjuno.

A

B

uh Ice I, don't think so, maybe find next any container.

B

I I don't have any any tests for or Provo for the test.

D

Okay, I I need to look uh further yeah, otherwise uh I will check to edit tomorrow.

B

A

Yeah there is a still to do. uh I I put this to do uh for the case of uh when we will introduce uh site, uh oh by the way um I'm going to. What did you say, is not working with bad image. uh Is it uh not stopping, or is it.

B

It is never starting, never start yeah.

A

I should not start the spacebar. You should not start regular container.

B

uh Okay, you said they should not restart or sidecar, but sidebar.

A

Never starts yes, okay, it should not start. It should be shouldn't start right. Yes, yes, okay, yeah, if it shouldn't start, it will work right, so it will keep trying to instantiate this uh port and uh container and I will get into image. Pull back off.

B

Okay, I can change it.

C

So I didn't see uh your PR and then I also did that test I just pasted the commit. um So it's uh yes, I've done the same thing. So if the sidecar has a bad image, basically attack or it goes into image, pulled back off and then uh side chord never starts and then the regular container never starts it's sort of the two tests I had. So if you want to look at mine, see.

B

The search inside okay.

A

I was thinking, oh wait, nice, so new weight method, um I think it would be wise to make it here, so I can merge it faster. Otherwise, we need sick testing to be involved.

A

That's why I put one with the restart count here. I, don't remember if you saw it yeah just I put it here, because it was hard to find somebody from sick testing to review fast enough.

A

A

All right, so, if we check for image pullback off specifically here make sense.

A

Okay, um so what else we have to do? It feels that we covered plenty.

A

So topology manager- maybe we can uh work around it by saying we're not supporting this limit. Ranger is done, okay, um so the only thing left is uh to make sure that success status quote will be success when a sidecar failed, even when sidecar failed, and we need to have end-to-end tests for uh probes and uh hooks.

D

Would it make it easier to have the the tests today because yeah then there is like one more day for the uh for the approver.

A

Yeah I'm gonna bring it up on the next music agenda, asking the opinion. The problem right now is um I got heavily involved in this in place vertical pot skating, because people want to um and reported so like uh removed from the T and like it's a big change. So that's why and most what it broke it broke. The Standalone mode, Standalone mode is apparently heavily used, but we never tested.

A

So it was Zero tests in open source repository, but we um uh people heavily use it and it's considered GA feature so I was involved into fixing that and making sure we are testing it. So yeah I think this will be done by this music and then my question to everybody will be how comfortable they have is um engaging this uh targeted, but uh significant change. So I think what the reviewers will look for is how well we protect every single chord change like. Is it unguarded by feature gate?

A

Is it like clearly guarded by Future gate and what will happen if feature gate is said and whether we will break any uh scenarios that we don't test I? Think, with the amount of testing we have, we should be covered.

A

um Let's see how it goes. um People really want sidecar, it's it's not like uh um nobody wants it. It just uh made it harder that uh Bruno wasn't around to help us run faster.

D

And and also we have like two more weeks for test trees now.

A

That's true, but nobody wants to take anything at quote: freeze time, but yeah you're right. uh That is theoretically.

D

A success with.

A

Different controls.

A

Okay, so I think this uh two: what we need.

D

So so those are covered in the in the to do comments in the container lifecycle test that you showed before, or there are more.

A

I didn't at least any uh pass for pre-stop I, don't know how important it is. I, don't think it's critical important, but uh you may be interested to add those. uh What I really want to test is uh startup logic like uh when startup, uh if startup probes never succeed like how we behave or.

D

Yeah, so those are the to do like the 20 or the 30 to do in the in the file. Yeah.

A

Yeah just logic um massage: it could be useful to have okay.

A

um Yeah I think this.

B

A

Left to do right so um and ganji, you said you will fix this one and uh my TSU plan to write the test like tomorrow. Okay,.

D

I think I would even try tonight to start uh yeah. Thank.

C

A

C

um I can try to update the CPU manager um and then really look at the topology manner manager if it will help with the merge. I, don't know if it's more, if it helps make it more or less likely to go in if it works. But it's it's also more code to review if it goes in so.

A

Yeah I think we may get more success if we will um just polish logic we have rather than trying to expand into this area. um I would expect a lot of uh I mean we'll need people will be hesitant to reuse as part. Of course um it's not that, like we have like two or three uh reviewers and one approver who are dedicated to the space, so I, don't I, don't want to complicate this uh PR emerge.

A

um But if you let's wait another hour, if it will be they'll say that we have to do that, we will need to okay understanding sounds.

C

A

To me, um okay and yeah, if you, if you have some PRS that can be moved into main tree and we can merge them faster. Let's do that, um because uh smaller PR will be easier. It will be to review and emergence.

A

Okay, um I think we've done for now. Let's get another half an hour and then we'll have signal meeting. um Thank you, everybody. Thank you. Thank.

B