Kubernetes SIG Node, 12 Sep 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Node Sidecar WG 2023-09-12

Description

Meeting notes and agenda: https://docs.google.com/document/d/1E1guvFJ5KBQIGcjCrQqFywU9_cBQHRtHvjuqcVbCXvU/edit#heading=h.m8xoiv5t6qma

GMT20230905-160425_Recording_1920x1018.mp4

A

This meeting is being recorded.

B

Oh, it's um Tuesday, September, 12 2023. It's a signaled, sidecar working group meeting. Welcome everybody um I think the first matter of uh conversations can uh country determination. So we have a slide deck from Gonzo. Do.

C

B

I can make you a presenter.

C

C

C

You see my screen. Yes,.

C

Yes, I just made this slide to make sure that we all are in the same page, so each slide represents the container termination scenario, a specific container termination scenario. So first one is graceful termination.

C

uh If one, if a container terminate, then that application will see the system signal and if the application can handle that gracefully and terminate in that line, which is the termination grace period seconds in pass pack.

C

So then we can just terminate gracefully. That is the happy case, and the second case is if the application cannot handle that victim in within the deadline or it doesn't handle it at all, then it will receive the CQ signal, so it ensure that the container termination can finish within the termination grade through period seconds, and if the container has Crystal Brook, then the container termination starts with executing the Chris topic Handler and then the app receive the system after it finish the prista proc.

C

So I just want to point out that there's the cap, like part life cycle, sleep action to delay the app termination to handle the request, gracefully, and it also is based on the assumption that container is terminated after the Crystal Brook so and the next case.

C

If, if you have, if the container has the free stuff, then the application will get the remaining time so it it may not handle the signal gracefully and get killed by CQ, because Christopher will consume the termination grasswork period and if Christopher takes longer than termination graceful period, then it will. The cubelet will just ignore that pristaprook and and terminate the container with minimum graceful period, which is 2 seconds.

C

So this is the exceptional case.

C

So so we can just say that those scenario as a container termination, then in terms of per termination, we can say that there is a moment at the pot. Is termination requested and cubelet sync start, but each container is guaranteed to have its own termination duration for current implementation.

C

So when you say uh so, the current behavior recycle containers feature gate is enabled. Then it will just terminate all container in parallel and each container can has its own termination period.

C

So if we say that if someone say that terminate, we will terminate sidecars after all, regular containers and in reverse order, then to me the most predictable way is just moving. The container termination start time to backwards, not changing the termination duration there to me, there's no reason to change each containers, termination, duration.

C

So in worst case we will get very longer termination grace period for in terms of pot, but that is the current behavior. So if we don't want to break the current implementation, I think we need other ways to handle it like introducing new fields for this.

C

So my proposals are, we can just make each content current part. Dot termination gracefully appear seconds applies to each container. So if you want a specific container to have different termination graphs period Then, we can introduce new field like termination grace period for a container or other way is.

C

We can just introduce the part level termination period because we don't have it now. So that is, there is the second solution and third one is I, think each one is independent to each other. So third, one is: we can introduce the termination policy like if you want to terminate all containers in parallel or reverse ordered.

C

Those three are my my proposals for now.

B

um Great presentation, um I think my immediate thought on that is. uh We really need to talk about scenarios rather than what is logical, um but otherwise you practically you have to use cases first use cases you want container to germinate, gracefully and you're ready to give as much time for this graceful determination as possible.

B

um So you basically given like five minutes and then like or during this five minutes.

B

You explain that everything you complete and you hope it will complete in like one minute, but do you still give in five just in case, um and the second scenario is that you either want port to come to finish in certain time so you're given like 30 seconds like in 30 seconds, you have to finish like whatever happens, it's done uh or there's a graceful determination can even be um enforced by you uh to you by, uh let's say note will be gone like you have a spot instance in Amazon or like a g key, and this audiences will be gone in 30 seconds.

B

So you have 30 seconds to terminate uh do whatever you want, but, like you, don't get any more. um So first scenario with uh please terminate, terminate, gracefully and I will give you as much time as possible. uh This picture feels perfect, so you can terminate main containers and like inside the first side, car second like sort of like last side, car previous side, car and, like so forth. This is uh certifies.

B

This scenario very well, however, scenario when you have to terminate in certain time is not required at all, like I mean either satisfied to the extent like I mean on not anyone satisfied, because you you're saying that you cannot control how fast Port will terminate so now you're saying I'm, giving you graceful termination time, but uh Port can take as much time as once as long as it has enough sidecar containers.

B

A

B

Is um I, don't think it's as far as it's not at all and I think, like whatever solution we will propose, we need to think about this scenario like how does swipe Boston that is first and then second, how well this study will be working, so I think proposals that we had on last meeting with Sikh term uh interruption. Pre-Stop satisfy bostonitos but like it satisfies first scenario in not ideal way.

B

So, like um you, don't it's a recorder and you um you need to think about implementation very carefully, but the second scenario satisfied very well because for second scenario, you give each container enough time, including sidecars, to terminate even though like termination period is very short.

B

What do you think.

C

Actually, I'm not sure about the second scenario, uh if you, if we use the pre-stop group for signaling the sidecars to terminate gracefully then there's there is, is an ambiguity about.

C

When is the real termination start moment for sidecar container like if it starts from the cubeless things? Third time or the moment that the parties termination requested or the first container starts termination, I, I, I'm, not sure about the base? Third time for the termination.

B

I think all three that you said is pretty much the same time. So when deletion timestamp was set then like in the closest thing, I will send like we'll either start with stops or six terms for all the containers, including sidecars, and then, when all the main candidates are gone, we will send a sixth term to a rest of sidecar containers.

C

So what I am saying is that the current behavior doesn't start the container termination from the time it says the deletion timestamp. So if we move that way, then we had to change the current behaviors, so it will be the breaking change.

B

B

So when our deletion timestamp is set, we will start determination of all the containers it's just like it may be, like not immediate, but like it's very close to that deletion. Timestamp equal now right.

C

uh But but the duration timestamp is set by Q, API server, I think. Actually it doesn't matter. But if we want to do that, then we have to change the code, because the code doesn't care about the deletion timestamp. It only cares about its own termination process, so we had to.

C

We have to pass the termination duration by calculating the deletion, timestamp and parts termination previous seconds and pass that termination duration to contain a long time, which is not the current implementation. So that is the reason. I say that this is the breaking change.

B

I'm not addressing that I'm just suggesting keep the current logic of deleting everything at the same time as we delete now, but the only difference is sidecars.

B

We will receive a sick term when all the main characterists completed, even if they're still running, please stop hook.

C

But then how can we, how can you calculate the deadline for Sidecar when, when does it start.

A

I think it's the same time as the initial like there's one pod level termination grace period seconds and if your main pods use it all up and sorry, your sidecars get the minimum of what two seconds so.

C

It means that we have to make one Baseline, which is not currently implementation.

A

Yeah, are you going to the kernel protection again, async calls that like a kill pot or whatever, and then basically, they all the run and go routines, the weight group and they all run the pre-stop book and then end up calling the stock container or whatever but yeah like in there. It would have to wait basically just realize the termination order and.

D

A

Things you know, basically, if you, uh if your maintainers, take too long, you may not get any time which it's not ideal, but it's sort of something you could fix on the user side as well, like I, think like one problem, what this one look at on the screen is, if you have like this cover or something enforcing like a termination, grace period of 30 seconds on pods and I'm, a malicious user like now I can extend the lifetime mobile, pods and start stacking inside car containers.

A

And now you can't you know: I put 100 of them in there and now I have like a 3 000 seconds per month for my uh workload to run and I'll just run it as a side caller and I can run for much longer. Basically, so it's some side effects of multiplying that plot termination grace period I, don't think you can change the semantics of the poly termination base period really.

C

Yes, but for me the path termination grace period is actually not the population, a patternation duration, but it just ensures that each container can has that amount of termination, duration periods.

C

We can interpret like that way. I think then, there's no problem.

A

So look at the dogs and someone describes it value. It's the optional duration in seconds for the Paw to terminate gracefully and.

C

So for me it's one.

A

Level construct, okay,.

C

A

Entire pod has to be gone in that second sergey's, um like spot Interruption I. Think it's a really good use case of you get 30 seconds and the machine's going down like we're, taking it away from you so finishing that time or not your processes or get killed or shutting the BM off and that sort of works and then, like the other case of uh if you could just multiply that out, like I, don't know if they're like multi-tenant kubernetes users, that sort of enforce like all right you're out of time.

A

Let me kill your pot like, but if you consult all of a sudden just add side colors to multiply that odd termination out like you can start to abuse that I guess they could do a kill it with a zero grace period. Maybe but I just walk around.

C

Yes, but technically, that is the current behavior, because we cannot ensure that the party is terminated within that termination grace period because cubelet main may not start is termination immediately after it requests the part deletion.

C

A

Yeah I guess but I mean hopefully between your two left arrows there. It's a really small sort of small amount.

C

A

C

So if there is logically inconsistency, then it is hard to implement that so.

C

Yes, that is what I'm saying when we, when we decide to implement that way, then we had to make some breaking change, which is very little, but yes, we need to do that. I think.

B

What kind of breaking change I.

C

Didn't get like like current behavior is if the cube plus sync starts uh after, like cupid is too late to terminate container after five seconds, then it will get more five seconds termination duration period. But if we decide to to terminate container from deletion timestamp, then it will be the breaking change for the users like before we. We will get 35 seconds of termination grace period if the cubelet is delayed.

C

For some reason, but after that change we will get 30 seconds of termination duration seconds and the container will get 25 seconds of termination, duration time.

B

I'm not suggesting to change that what like. Ideally it should be same behavior, but we need to distribute it across sidecars and Main containers somehow.

B

So the only question is how we distributed and I think the I will always like in distributed systems, you always have some delays and like inconsistencies and timestamps, but um I think what is implemented right now, working for most scenarios. So even if there is a delay, then I haven't heard many complaints about this delay. Yet I don't know if you have any Radio.

A

C

B

Delay was important.

C

Yes, yes, the current implementation is okay for many users. So, but if we change.

C

It depends on how to change the behavior, but.

B

I, don't think we want to change it like we just want like so initial deletion will be the same if sidecar has a pre-stop cook, we'll start executing this free stop hook, um and then we will send a sixth term at the moment when all the main containers are gone and key like uh Port will be killed, no matter what, uh when Grace determination is completed. So there is like no inconsistency here.

C

But Christopher doesn't if the container termination doesn't start just after the first approve, then it will be the problem for this cat. Like this cat hot life cycle, sleep action is based on the assumption that termination starts immediately after the pre-stop group.

A

uh So I read that one I think that one was really about just an easier way of adding a sleep as a freestyle book, and so it was the primary jobber. Is that so that when you delete the pod, the endpoints sort of get dropped? Basically, so you quit routing new traffic to it, and it's just so. You can use a like distortless container and don't even have to put a sleep in there or something.

A

Basically it's to stop routing traffic immediately upon deletion, which already occurs but to still give the container and pod a few more seconds to continue to try to handle existing connections. Basically.

C

Yes, that is the main motivation, but in terms of is semantics, it just delays the app termination. It is just for delaying the app termination. But if we separate the Crystal from container termination, then it will not work.

B

So I mean if somebody will decide to use sleep a pre-stop hook for sidecar, then it will behave differently. That's what you're.

C

Saying yes, yes,.

B

And I think it's fine I mean yes, we change into symmetric and I think this is the main question for API approvers. If we change this semantic for all, pre-stop hooks for sidecars isn't acceptable.

B

Alternative is to implement some other means like maybe, instead of pre-stop or some something else like have a new life cycle hook for sidecar. So some introduce some new signal for sidecars or.

A

B

Two sick terms.

A

I guess if you just change the like and just say like well, Sig term for sad cars comes after the pre-store after the pre-stop hook finishes or after all, the main containers complete. Whichever occurs last. Is that solve it? Because then, if your sleep extends further in the pre-stop book, you'll still get it when it's finished, basically, sidecars can always assume if they got a Sig term.

A

The uh main containers are dead, which is all they really want to know, which is sort of the useful piece of information that you want to know if you're a sidecar and that would work with sleeps or whatever.

B

Yeah it may work, but it may delays. um There's many like if you implement some generic sidecar like shutter smash. You want uh to know when termination started, but then you also want to know all the maker has a gun because you are not useful. After all, containers are gone, so, whatever your pre-stop duration, you want to interrupt it and get out of it.

A

I mean I, guess most guys just say: well, don't use a freestyle book or.

D

A

Use a sleep freestyle book for those cases right if you're like we're using pre-stop, to tell your sidecar only.

B

A

Sleep will be very special.

B

A

Yeah, because you're using sleep I mean the reason is to give your thing to give your container more time to process existing connections and parasite car like use it. If you want to, it, probably doesn't make a whole lot of sense, because by the time we by the time we send you like we're, already delaying your Sig term anyway, like we've, we've got more information than you do. We know all the containers were dead. Basically,.

B

Yeah, that's maybe debatable like what the semantic of sleep will be um for this ignore six terminal doesn't Exchange.

A

B

D

A

um We're sort of constrained by the fact that you have like the Pod level termination grace period. You don't want to extend that for reasons and then, but you do want to try to give a stock market standards as much notice as possible, but they also need to know when all the maintainers are dead, so they can actually quit early.

A

B

Yeah I remember it is in a bug, but I, don't remember the exact Behavior. If you trying to delete port with the graceful determination of like five minutes, and then you said it's too long too long, it's taking too long and you start another deletion with like one. Second, then I think what you expect to have is Port being deleted in one second, so I think you would expect that sick term from second deletion will interrupt sleep, foreign.

B

But um there's also scenarios that you need to think about.

A

C

And I think that I I said it before in the comment we we had to care about the restart scenarios of sidecar containers.

C

So if the prista proof starts before the actual container termination, then we have, we may have to restart the sidecar container after the Press taboo.

C

So that would be one problem. I think.

B

Are you convinced that this scenario needs to be satisfied that, uh like short termination period with a given limited time,.

C

uh I think that if someone wants to terminate sidecar containers after all regular containers, it means that it needs more time it. It essentially needs more time so.

C

There's no way to just terminate in the same termination duration seconds with using the cycle container I. Think users should know that they will get longer termination duration. Second, if it if we want their sidecars terminate after all, regular containers, so I just want to I just want to make a way for a user to just change. Each containers termination duration seconds.

C

I think this is the clear way.

B

Yeah I think we can make it in future. We can make it work and.

C

B

Mind we need to implement it right now in this uh you know we already have one termination period that we write in a live on this probe. You can specify grace period every right, so if your liveness profile, then we can apply different termination period for the entire port. So this is implemented. So you can say that if loudness probe failed, it's a catastrophic failure and we don't know how to gracefully terminate you need to just kill the port completely.

B

A

B

The same vein, we may be introducing some Fair container, otherwise, in future.

C

B

But I I, don't I think for sidecar, specifically Proposal with pre-stop being interrupted with sick term may work for all the scenarios.

C

But what I'm worrying is that if we change the semantic, then.

C

C

um I'm just worrying about the inconsistency with the regular containers, but let's see how reviews or approvers saying okay, they will. They will give give us the review.

B

And I, are you convinced that this may work for all the cases like? Are you fine with that or you have a very strong feeling about it because, like if we are not together on that, like it's, uh it's hard to argue with uh foreign.

C

Actually, I'm not convinced about about changing these I'm, not sure how to implement these container determination. If we don't assigned termination grace period for each container.

C

That is my my main concern.

B

Do you have specific sidecar container in mind that you have problems to implement because I talked to Easter people and they said like even today, behavior is fine with them. uh If we'll make it better by separating two signals, then it will be ideal for them and they they done like they. They happy um for logging.

B

um I haven't talked to anybody yet I've been at Rotten to login a lot so I I think I can represent it uh to some extent, but maybe I need to speak with current maintainers, but I think I can convince them that this Behavior will be working for them. You have any do.

A

B

Any sidecars in mind that you want to implement, and you don't know how, with this Behavior.

C

I, just I just don't know how to implement these container termination ordering without making.

C

The the base third time for that, like you, think that we have to make the base start time for part termination.

B

I mean we don't want, do you mean I, don't think we want to implement ordering right now? Do you think you need ordering for some scenarios that you have in mind.

C

I made I may don't understand what you're saying.

C

So we we are planning to implement the termination ordering for sidecars.

C

And I'm worrying about that implementation.

C

That is my my concern.

B

Wait are we planning to do termination ordering that's where I think we are not aligned Matthias? What is in your cap? What is your in your PR? Is there a termination ordering.

D

uh Yes, we yeah yeah, but I mean it. Can change. No, no worries.

D

I I think we said we need it for beta the termination ordering now.

B

Like how do we Implement ordering I thought we will just start terminating at one time and then, whenever all main containers are done, we will start start you'll, send sick charm to all the sidecars, so I thought that uh we agreed on that I guess: I need to read it carefully.

D

Yeah but I I think that that was okay for Alpha.

D

Let's, let's reopen the kit.

C

To me, the hardest point is that when we decide to terminate sidecars after all, regular containers, then how much time we had to assign this sidecars to terminate how to calculate the time which is start from the first regular container, terminate search or the part termination starts or cubelet.

C

Sync start.

B

I think a robot termination shouldn't change like whatever it was. What was the behavior before it should be the same for overall Port termination period, thought I. Remember you drawing this picture. You drew this picture last meeting. What is your recollection of uh how we want to implement it.

A

So we I thought we were doing the pre-stop for everything and then a sick term for six arm wants the main containers, but I also thought we were doing the ordering as well and I was going to read the cap because I thought I read language in there. That's.

D

So so, in like multiple place in the cape, we we say that we we need ordering for beta yeah, even in the graduation criteria. First point Implement proper termination ordering.

B

Ordering may be different like ordering, maybe between sidecars or ordering, between mine containers and sidecars.

D

B

D

In details uh in a different place,.

B

My water here is the same as gunjo like I, don't know how to properly Implement ordering and how to like how long to give to each side car. Once we send the signature.

A

Yeah I think like, if you had to do it today, like I'm looking at this kill container method. Like you probably end up, um you probably end up with some sort of uh state, something that maintains State object that you pass through the kill container method, so they can each update and then everything sort of Waits on it.

A

So we can kind of get the correct ordering and correct timing, but it's solely to sort of handle the case of I'm trying to shut down the entire pod and I need to just realize the termination of these things. The only issue I've seen is like kill, Tanner's called in a few other places and in those cases you just wouldn't pass that in. In which case the containers gets killed like in GC and containers. You don't care about ordering um somewhere else.

D

But but I I think if we say that the ordering is not guaranteed. If you take too long to terminate like.

D

You you could get like a sick term uh before get a secure before getting the signal right. If you, if one of the sidecast takes too long to to finish or even the main container, if it takes too long, then the sidecars will get a C kill at the end, but but I I, don't think I have a problem with this.

A

So just implementing pod termination ordering between the main containers and the sad card container. So it was two separate sets I'm wondering if just that complexity is enough, that it's it's not much more, to implement sidecar container ordering I, don't know no.

D

No, no, no, there should be the ordering between sidecars as well. Yes, but but I I I. Think the the the main issue for gun show is that if we take too long, then we might not get the sick turn before the C kill.

D

But that's that's. Okay, I think no I mean at the end. We have one hard limit, which is the grace period so.

B

So we generally to have a signal when a port termination started. So we need to tell sidecar that you may start cleaning up now because you may be killed so.

B

But then sick charm, ordering I, don't know how long between six terms can give each Sidecar, even if you do it in reverse order like how long do we would give a couple like percent of what's left.

D

No, no, no, no I would only send the the sick term when the previous one has has finished.

C

D

Let's, let's imagine we have like a an airplane which will hit the floor in 30 seconds and we have like five passengers to to to jump uh with parachute.

D

Well, if, if, if one takes too long to jump, the others won't have a chance to jump and it will crash with the airplane, so I mean that's for me. That's determination.

B

So in this analogy, when airplane starts failing, we tell everybody start putting on parachutes and they already did so now. Instead,.

D

Of yes, they kind of jump. If you see the the the jump and before they hit the floor is like the graceful termination of a pod. Okay, the only difference is that uh the the the next one can only jump when the the previous one has touched the floor with the parachute okay, but.

B

The problem with with the airplane has one door or multiple doors like. Why would we wait.

B

So if one side car is implemented poorly then it will yeah. Maybe it wanted to delay everything.

D

I mean if, if it's implemented poorly too bad for him,.

D

D

And I think it's easier to to explain and easier to to document than saying that ah yeah. But in this case we will add, like this amount of seconds, because if someone does not Implement correctly the the shutdown and it takes too long, then it will delay the rest and yeah.

B

I can see that in this case, if you have login container, then it start like it try to send as much information clear all the buffers as soon as possible, but then, when all the maintenance is done, it knows that no more containers. So it will take another like couple seconds to send all the rest of the data that it accumulated during shutdown and then istio will be the next like first container and it will terminate immediately so yeah I can see it helping yeah and then.

A

You don't need to.

B

I mean yeah: you don't need to give any special handling for that. Okay, yeah.

D

B

See it working it's great, but we still need to tell them when when Port termination started, you need to tell everybody like everybody needs to know about this happening, and then when maintenance is completed, then we started sending six terms, even if they still running a free stuff.

D

A

No yeah I agree, yes and I wonder if users can use the freestyle book to like work around these badly behaving sidecars sleep for 10 seconds on. You know whatever kill all that's not on whatever process inside there as as their pre-stop book. Basically, and you can sort of implement um whatever you want. I think that way like if you wanted to give all right container a gets 10 like sidebar a gets or like in this case Type R A gets 20 seconds.

A

It's like our two gets 10 seconds or whatever you can change your values and basically use your pre-stop book. Send your own Sig term, basically and set your own Sig kill and your freestyle book it's kind of ugly, but, like you could you could work around some stuff for these badly behave inside cars until you get around to fixing the code.

B

Yeah, so within the signal is not necessarily needed. You can just Implement whatever logic you want and your freestyle yeah, it's, the only complication is it's a scenario of login container and H2O like and serious mesh. So Logan wants couple more seconds and it wants Network to be up. How does they tell istio that it needs to be out for two seconds and so on? The complication and I wanted to ignore that I wanted to say that, like it's, not our problem like we're, not solving this problem but I think my tears have a point.

B

uh There is only one door.

D

I mean it's true right: yeah yeah.

A

I guess Master is in conjunction with this: it's just. If you have a badly behaving sidecar like you, have a an escape patch of use. Your freestyle book to manually kill it.

A

You know we're still serious uh serialized termination ordering, and that way you can sort of time bound how long it takes for it that sidecar to end.

B

A

In future, we can have per container determination.

B

Period right and I've write it whatever we want.

B

Yeah, okay: yeah: do you have a problem with this approach like I? Think this will be easy to complement. You don't need to wait for specific time. You just wait for completion. Artistic term may just say like it's not been restarted any longer. It's done and you go to the next side. Car.

C

I'm not sure I, understand what you are saying so you're saying that approached.

C

Should be applied like Christopher should be executed, after all, regular containers are after The, Path termination requested.

C

Yes, yes, but for what about the restart behavior of the container? We can just ignore the crystal.

C

If the, if, if the sidecar container crash during the the part termination, then how can you handle that the prestock food is already executed, but containers should be restarted then? What should we do.

B

I didn't get a.

B

D

Container is not never restarted right today,.

C

Please stop means that container determination starts so so.

D

There is no no more restart yes.

A

Okay, yeah, like it's, don't crash, basically started.

A

Seems reasonable.

D

I will and actually we need. We need this one if, if we want uh to to use studs uh determination using the pre-stop hook, because imagine if we kill a sidecar before it gets thermal C kill using your sleep in the crystal book and then sending your own. But then we don't want the cubelete to restart the sidecar because we wanted to kill it.

D

C

If the container has pre-stopper Handler, then we don't restart it in termination period right.

D

Yes, I would say yes.

A

I think it's consistent with like everything else about sidecars it's best effort. We try to keep it running and like if your sidecar starts failing. We don't stop everything else anyway, right if your sidecar's just repeatedly crashing, we don't stop them in containers. Sort of the same thing for me.

A

But there are situations where, if your side or behaves badly, it's not going to be running and if it shuts down during termination or crashes during termination, it's not gonna We're, Not, Gonna, restart it because we're in the process of shutting down anyway, I.

D

Think it's I think it makes.

A

C

But what about if the termination growth period is too long like like 30 minutes than one of the sidecar May crash.

C

During that amount of time, 30 minutes.

A

Still happen, nothing even with a short time period like if you've.

A

So your suggestion is, don't send the pre-stop. The downside of that is that your sidecars get sort of no notification of the pot is shutting down and when they get the pre-stop, basically they can assume that maintainers are all shut down already.

A

Not only that main containers are not previously started.

C

I think this is very complicated problem.

D

A

D

If you, if you want like a graceful shutdown of 30 minutes,.

D

C

Mean right once.

D

Once we start the budget shutdown, we stop sending traffic to it and everything right so.

B

Yeah example uh that Ronald gave last week when we discussing graceful termination, uh they're running some drivers, installation or uninstallation, so it may take a while. So this sport May gracefully terminate for such a minutes uh legitimately the uninstalling some drivers in the machine.

D

Yeah, but do you need a sidecar for that.

B

And if you have a login side, cars and it's supposed to be working all this time, so the way you implement the login containers this way in case you on pre-stop, you say like: oh, maybe I will be terminated soon. So let me send off all the buffers and then it went into this mode of sending all the logs immediately without like buffering its uh without buffering too much so I think it's fine.

B

And then, when driver installation completed, then you be in terminated immediately because you receive a sixth term.

B

The problem with that is typically, you give this condition a very long time to terminate like, let's say, 30 minutes, but then, if driver install, an installation took only 15 minutes. You wanted to complete, like you, don't want to wait for a Whole 30 minutes, um so you want the sidecars to extend it to insignificant amount of time instead of for the whole duration, foreign.

A

You have a long grace period and istio crashes right after that grace period starts basically and since it's the side part we're not going to restart it and there's something else after it, a logging container that needs it to be able to actually communicate. It's logged off in a case logged here, logging pair is just going to fail.

B

Yeah I think we need to document this case saying like this is what the side effects of this will be.

B

D

Yeah but today without the sidecars, is it even guaranteed to restart.

B

No, nothing will restart after grade school termination.

D

Okay, anyways, nothing will restart anyways yeah. It's.

A

No worse than right now.

D

B

Okay, so you have a couple more uh questions. I think uh taught is from you: okay, I I posted some answers, I think uh for first one we discussed a bug uh from like two meetings back um and we decided we need to have a test written for that. So.

A

I, looked at that note and I thought it was. um It was only talking about around. Oh, maybe it was a bug. Sorry, maybe I didn't read the book. um I looked at the psychometer notes and I thought it was only talking about like if the startup, okay, never was successful or something.

B

A

B

Wanted to have this problem with sidecars implemented differently, so yeah, it's a good idea to Define the behavior and test it yeah.

B

um So I think it should be one of the issues uh in the track and then for second question yeah. Definitely, yes, and we have a test for this.

A

B

Like yesterday, I.

A

Did not know the answer.

A

D

A

uh Let's see I can write a test for that first case. It's not done already.

B

No he's done. Thank you, yeah. We only have three minutes till the next meeting, so I think we can finish now. Thank you. So much as you will clean up the uh yes.

D

Yes, yes, yes, yes, yes, yes, now now, I know what to write and uh yeah I will I will do it like quickly and just send you so you can uh review. Thank you. Bye. Thank.

A