Kubernetes SIG Node, 15 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Node Sidecar WG 2022-11-15

Description

Meeting notes and agenda: https://docs.google.com/document/d/1E1guvFJ5KBQIGcjCrQqFywU9_cBQHRtHvjuqcVbCXvU/edit#heading=h.m8xoiv5t6qma

GMT20221115 170457 Recording 1738x1286

A

Hello, hello, it's uh signal. Sidecar working group today is November 15 2022.

A

welcome everybody uh yeah this meeting has been recorded. um We have a as I said before I started. Recording I didn't expect that much interest for this topic, I'm, really glad that we have so many people joining um yeah today will be. um It is kickoff meeting I hope that we can get some progress I. You may not take whole hour uh and we'll see uh how it goes and what frequency you need for those meetings and I want it to be open Forum.

A

So if you want to speak up just like um do it, we need to I mean it's a working group. We want to have open uh communication, so I started. This document of a working group and um I may need some help with filling up Charter of a charter. We want to I mean why we all uh here today. We want to understand what would be our goals and what would be the exit criteria?

A

I think uh when we just been discussing it um at signaled meeting, we said that the main goal for this working group is to come up with the design that you will be approving for 127. Ideally, that's why um we need to start early because practice shows that if you start when enhancement starts, uh then we don't have enough time to discuss all the aspects of it and get all the necessary approvals. So yeah um I think the goal will be 127 um design, uh I, don't know whether we will um get beyond that.

A

um Maybe we'll have so many open questions and so many desired discuss that you keep going or maybe we will just dissolve this group, so I think executed will be at least um have cap approved and the work started on a sidecar in 127..

A

Any comments on each other.

A

Okay, um I posted a few links and, if you're aware of any other documents that uh may be interesting to share in this context, please add more links. I know so much was produced during this. uh How many years um I just failed to put all the links here um yeah. There are also public talks on like water sidecar. So if you have those, uh please also uh maybe put it in this list.

A

So people who are not familiar with the concept or problem space um you'll be able to catch up quickly, yeah um yeah, magic links, okay,.

A

A

um Let's go to agenda I wanted to recap some history of how we get here and why we decided work group is a good time for our group, um foreign.

A

Over time we had some conversations on what sidecars may be, and everybody wanted to solve problems beside car. So for some people it was a problem with jobs not being terminated and for some people it was um uh that sidecar needs to start early for some sad car needs to collect some logs from any containers, so many problems were raised and people were trying to address those problems um and it was uh hanging for many years. At some point uh we almost started the um cap going.

A

uh We almost approved it for one of the releases I think it was from 119, but at that point um we said that uh there are two cancers first concern. We didn't understand what to do. Next after we will implement the very first step of sidecars. We didn't uh see the whole picture and second, we still had a lot of doubts at that time. uh What the final picture will be, and second, uh our test infrastructure was so um fragile and many tests were so broken.

B

A

We wasn't sure that uh if you add anything uh that big uh will um will be reliable enough and will break anything else beyond the uh sidecars I think at that point, uh we'll we confirmed both. Our worries, like first word, is that we don't know what it will be. uh We still don't know and more discussions. We have more ideas, people bring and these ideas uh needs to be satisfied and uh some designs were better for those ideas and some worse like one of the biggest ideas that I heard uh recently is security isolation.

A

We need to isolate uh continuous, different security models.

A

um So yeah uh this uh was, uh this fear was uh almost realized and second here was materialized about the reliability of tests. uh I think uh everybody uh heard some uh at least some uh talks about regressions that we had after we changed something in the Pod life cycle.

A

We've been uh fixing bugs and finding so many regressions, um it was uh I mean minor refactoring caused all these problems um that there's also materialized, fear and I think we are at a better stage right now um we have more people knowing what what they're doing and we uh went through many regression investigations. So we we know where to look so it will be yeah okay, so it will be. It is a good thing to do so.

B

A

um Two fears they both materialize I, think we are the better station right now um and um in 125 we've been trying to have a sidecar containers kept back um we now at that time we attended uh to minimize the scope and make Scopes so smooth that uh it would be uncontroversial, but it didn't pan out as well. So even that uh didn't work, we truly believe that sidecars are needed um and having sidecars implemented as a as a hub or as a like surgery.

A

uh Basically initially it was an idea to even do it with annotations to don't change any apis at all.

A

But all those ideas are across of reality like with one side, cars, and we want to invest into that, but we don't want to do to put any code that will be uh partially baked and not clear how to like uh scale it into so yeah uh and during the discussion 125, we realized that uh we already ruled out so many uh implementation possibilities that uh we at the stage that we pretty clear what we want to do uh so, maybe for me the sidecar will help us.

A

Approve some design that is uh future proof, so yeah I can tell what the rejected designs were. I have some uh notes here. So uh some of the rejected designs was uh uh we don't want to do systems. Do you like before and after? Like depends on kind of uh annotation I mean there are different names. Binds too depends on, um so this was rejected as something that uh may be needed for advanced scenarios, but clearly uh not working for sidecar pattern, uh because inside car pattern we want to have.

A

uh We want to have clear way to inject container, sidecars and clear way to recognize that this container sidecar and this country is not sidecar. So this was rejected as an idea for implementing sidecars. It still may be something that can come up in future as not as a sidecar as some Advanced feature, but uh for sidecars we don't want. uh We don't want to implement uh sidecars as a binds to uh semantic or before after semantic.

A

Second idea was rejected that we don't want to implement sidecars through uh different variants like termination of specific container. One like restart policy of their container, so idea was to here idea, was to minimize the scope uh to mark that certain container will terminate the entire port or that every container will have its own restart policy and it will be controlled per container.

A

So you can Mark some containers uh be in a sidecar via this characteristics of container like restart, always uh even if it's a job and then I terminate if it's uh not the job, so that was rejected because uh it's not solving um I mean uh yeah. You can read more on this link, but basically good design of this uh container Flags uh implies that uh there are many limitations with typical sidecar cases, so to only solves very specific uh cases for jobs or very Advanced scenarios for advanced uh container interoperability and design.

A

So we don't want that. So we want something that categorize containers and um we also discussed that uh we don't want to implement sidecars as a regular containers that will die faster. So you don't want. We really need to cover. You need container scenario. So if we will Implement sidecars, um the only possibility we implemented is uh it will have. You need containers implemented as well, so sidecars needs to be able to run while you need containers uh initializing uh main port, so that was another rejected idea.

C

Sorry cycle you need to run before, after or at the same time than any containers. What is the requirement there.

A

So requirement is to allow containers to run while you need containers are running, so you should be able to configure. You need uh sidecars to cover unique initialization stage, and um there isn't like, like to to.

C

Be ready before you meet containers to start.

A

Yeah, you should be able to express ideas that uh some sidecar needs to be fully functioning when certain Indian containers are running, so you need to be able to invest it.

C

A

Thanks yeah and the main idea is, uh if you want to enable Envoy as a sidecars, and we need to be able to start and work before any container starting.

A

Okay, um yeah, um so that two of the rejected design said ah I think uh having those designs. Are it getting pretty clear that we want some category of containers, so we uh can easily inject them? There should be category and it should be uh categories that, apart from regular containers and uh something maybe it containers, maybe a part of your containers, um uh so I mean I, think it gets in pretty close to shaping its form uh by just looking at the rejected designs.

A

I also was trying to put put up some requirements that uh uh we may have for this um outside cars. So if we yeah first and foremost, requirement is uh sidecar should not block determination of a port, um even if it's still running uh second is uh it can start before any containers uh key here is, can I I, think um I mean if design will allow it to start only allowed to start before all in it, containers I think it's fine, uh but uh there is no restriction here.

A

We can, as a goal is to make sure that you can express that they start before summing in container so like specify, which needs to run well uh restart behavior.

A

um So the result Behavior can be different for sidecars, so this idea is if Port is marked as never restart, then you should sidecars still should be able to restart, uh because you still need login car, even if it was unkilled or like crashed. You still want to restart those sidecars and then, uh when we've been talking about uh sidecars like once, we introduce this category of sidecars.

A

uh There may be other requirements that are coming into play and as I mentioned, security resolution is a big idea here, because if you inject something in customer Port, maybe you want to isolate it, um then the resource installation. uh There are many examples when sidecars are starving the main container by like taking over all CPU uh Padres Instagram. So if sidecar is not tradies and maybe Port should not be ready, uh that is also what's coming up as a requirement and may clip out into sidecar semantic uh awareness and resonance cross-dependencies.

A

uh It's another variant of that like if sidecar is not running. Maybe we should say that uh other continents are not running that kind of idea and uh one Behavior should be a little bit different.

A

uh Also like we don't want to kill sidecars how to terminate sidecars uh through um kill uh when main container is not being terminated by umkyo, so maybe it's better to kill all uh um kill all the containers and sidecars of a whole Port, rather than kill all the side, cars and not touch main containers, so that also make rip out- and you know design.

A

So once we have a concept of sidecars, this all will uh get into uh into play and uh also the idea was to establish the pattern like we may not address all those scenarios but- uh and we might not address all the scenario sidecars, but uh there was really big concern of against blockchain output spec.

A

um We don't want to have Port spec updated for every use case. So the idea here it was, if you have side cars that running after you need containers sidecar running before any containers. You don't want two categories of the sidecars because, like sorry, category will be uh sidecars running during any container stage, but uh like termination differently or like not having some restarted policy. So we don't want that kind of uh bloat.

A

We really want to make sure that we have a clear design that covers most of scenarios, and this design is uh I mean it's expressive now, but it's not uh bloating into too many moving parts and too many Concepts. That needs to be explained so.

D

A

Think those are requirements I heard. uh If um anybody wants to add something. uh Please do.

A

I think yeah I'm mute it yourself. So.

C

No, no, no uh I just wanted to like my my thoughts when, when you write these requirements, is that if we don't want to have a flag, we don't want to have a system deal like dependencies and we don't want to only have determination or restore policy signal when it's a cycle. I think the only alternative is have some kind of run levels. It can be profits.

C

Prefix names like uh in team, with teams proposal or in an INT, but I think we don't have many more options to fulfill these requirements. We need the one level root and we need to follow that route. I think there's no other way that I can see. Okay,.

A

If you disagree with the rejected designs, please speak up, because uh this is my understanding of what we uh came up with. uh After all, the discussions um and I wanted to make sure that we agree that this is also designs are ejected. Let's work uh in this assumption, but if I misrepresent something- uh and you refuse it into reopen this some discussions- please pick up.

C

Yeah, so I didn't follow the reasoning for rejecting the system being like up before and after uh designs, because that is so. That also feels very well with the government number five that basically, you can express whatever use case. You want without bloating the spots and the parts back, because you can basically Express whatever so now that I see that requirement and that design being rejected and curious. What was the reasoning, foreign.

A

Yeah, so this uh a slide deck was created to express to explain why uh dependencies and binds to semantic was rejected, um and here I can walk through that really quickly. um um So we today we have init containers and app containers and uh one category of containers running before other category of containers.

A

So you can express the same concept like this, but it it is a mess of like you need to have every container explaining what it needs to do uh and if you try to create new init container you you need to make sure that this unit container is defining all this binds to and dependencies which is not trivial like it's. uh It's making uh injection container into ports back really painful.

A

You can also express it like that. So it's a little bit better uh design, because in this design you kind of uh line up all you need containers. uh So it's easier to inject one new, unique container, but it's still like injection up containers to hard to to do like you need to make sure that it's defining all those dependencies. Otherwise it may start in incorrect fashion.

A

So um yeah uh injector concept was explained here and then uh it's much easier to express just uh two category of containers, uh because you have category or candidates in it and then, once all these categories uh completed it's easier to just start all the app containers.

A

A

Yeah there's more about new containers, so it's not my side deck I um yeah. uh It was explained how to inject some app container here and uh yeah. It's not very straightforward.

A

A

uh Yeah and removing containers also I was being brought up here. It's it's hard to remove content. Once you remove them, you need to revisit all the dependencies again because one container may be blocking other container to start before any containers and then, like you or like before other containers, and you remove one and you you get a mass of re-evaluating all the links uh so yeah.

E

There was too much complexity seen for the current odds, developments, but.

A

Yes and then, uh when we introduce a notion of helper containers like uh in this sidecars or helpers, then we get into a similar situation when we need to for every helper container, you need to Define that it starts before all app containers, and then you may forget someone and uh it's much easier to just say that helper containers kind of go before app containers.

A

It's a little bit easier, like I mean it's much easier to inject this way um uh because, like this is typical design like you typically want all sidecars before all up containers, so you typically will introduce all those dependencies you will never. You will rarely do sequential uh definition of containers uh and.

D

A

Much easier designed uh to uh express the same concept, uh at least for most of the scenarios and then a slide that goes into more details and like run levels uh um yeah, so we may have more run levels to solve heat problems and stuff like that, so um yeah I think. Basically, the main idea here is that all scenarios that you want for sidecars looks like that, and this is not ideal. uh Yeah uh is uh somebody.

B

Hi yeah um I'm, Claudia hi, everyone.

D

B

So I was looking at the slide. I think it's 24. uh You showed that really quickly and there is like a cleanup container yeah. So um the question mark is there because it doesn't exist or or for what? Because that's something that actually uh I wanted to ask to the group if the something like the cleanup containers do exist because I wasn't able to find any of that, like the counterpart of the init containers, for something that you want to do once the app containers are completed.

A

Yeah, it was one of the ideas that it doesn't exist. First of all, it doesn't exist.

B

A

It's present, um I mean there is workarounds, but like it generally doesn't exist. um The idea here is that once you introduce this different uh run levels or different uh stages of initialization, then we will. We may want to solve this problem as well uh clean up containers and uh um when I was saying that we don't want to bloat of ports back. uh That was one of the concepts that maybe we want to invest, cleanup containers later and uh Define them, but we don't want to like.

A

We want to establish some patterns that wouldn't require to have cleaner containers in different uh I mean it should be easy to follow the same pattern for cleanup containers. Okay,.

A

F

Hi um so along the uh so we have startup, we have sidecar containers and we have a patch that does ordering during startup as well as ordering during shutdown. So, ideally, we want the app container to First terminate and then the helper containers to terminate. So we have ordering dependency on the shutdown phase as well. So I was wondering if that will be covered in this um in this uh windows. Work group.

A

Yeah requirement here um yeah. It definitely came up many times.

F

And for us, the init container, the site, init containers, help initialize the sidecar containers. So when the init container comes off, uh then only the sidecar container comes up. So the initially we're talking about how the sidecar containers would come up first and then the init containers would come, but actually it would mess up our startup order, because the init container initializes all the certificate, things and whatnot and then istio and everything else starts up.

F

So it would be nice if both are separated like first cycle inner container and then sidecar container, and then, if someone else wants the side container first and then in its container I wonder if you want to support both cases.

A

Yeah, that's why I put this uh like this I can start before any containers uh if I would put the differences and we generally lose in the scenario- and this is quite common like if you have some sidecars, that updates uh you know credentials, uh you wanted some storage to be insized first and yeah. You definitely want that um yeah.

A

um But uh when we just we're discussing it, we said that we cannot propose a design and start implementing designs that doesn't have ability to start before any containers. So we need to have this ability and uh opposite design wouldn't be accepted. So that's uh kind of a message right now.

F

C

Thank you. If we are thinking in that many run, levels will be, might be neither in the future should we right. The only downside that I see with run levels or the potential downside is that uh the body startup latency might might grow too much when you start injecting, like seven, eight sidecars to a pod, because basically you're forcing like each run level.

C

The next one level starts when all in the previous one level so you're, basically yeah making it as the slower side carry in a running level.

C

uh If we're thinking about the future, this might be something to take into account, but but yeah I I, don't think, there's any way around it. uh If we don't want to have more granular dependencies, so it might be just okay to just acknowledge that the process art of latency in my growth, a lot um that is out of scope for now, or something like that.

A

Yeah um I mean alternative, maybe also like. If you want to start some, you need containers before side cars, maybe some side cars can be uh used as init containers. So if they start and like will never shut down, but then, like uh you'll, do only initialization on containerization, it may be a good enough design. So if you satisfies it,.

C

But you need containers run until completion. You need to maybe change that too.

A

um You can like today um the pattern that people follow to start campaigns sequentially in a single category like main containers uh is um they Implement, restart hook and restart hook, will block all other containers from running until they complete it right.

C

C

A

So um yeah we may like.

A

Let's say we only have round levels that is pre-unit containers and we don't have. uh Is that one of the big containers we may design in such a way that pretty new containers can be uh authored in such a way that it will become init container?

A

Almost it will not uh complete and Like Me Maybe for a long time, but at least it will run before, like initial Edge before any other sidecar containers, so that may be another alternative design, but yeah uh I'm not sure, what's better uh how you need containers that never uh and or have pretty neat containers all I have for two stages: pretty neat and helpers containers, so that this is one of the open questions, but uh yeah I think uh main point of this um slide deck.

A

Was that uh all the helper, like all the sidecar scenarios typically go, looks like that and uh the much better way to express this is uh to do that and I would agree with Jordan uh saying that uh sidecar can be injected by Five. Hooks I think it's uh it's coming up all the time. So let me add it as a requirement.

A

I think keyword here is reliably.

A

Okay, um yeah I hope it answers the question why this was rejected. um Is there any more questions on easy requirements or what was rejected already I.

E

Just for arguments sake, one could argue that having n levels right run levels is still a graph there's still a dependency graph. It's just more linear.

A

Yeah, it's uh kind of uh opinionated Advanced graph. You can only.

B

E

So if you needed more, you'd have to do something else, like plugins in the container runtimes.

A

uh What do you mean something like I was something else like.

E

If I've got two in the containers or two two init tasks right into cleanup tasks that have to go in a particular sequence right, one before the other and then on destruction, you know the opposite direction.

E

A

So if do you mean if we have cleanup containers, so you mean, uh if you have pretty containers that need to.

E

So you've got free in it and you've got a knit and then you've got the cleanup. One could argue if I'm gonna have pre and internet, then I need to clean up and post cleanup.

E

Okay, that's fair! You see, you see what you and then, if I've got two sets of you know one. If I got a dependency between two of the containers and the unit and the pre in it, then I would need another level right which basically you're just walking your graph out into a long line, a long dependency line as we find out. We need more run levels for argument's sake right.

E

It may seem that we're making it simpler by not having a graph right with two dependencies on one, but really we're just making one long linear graph.

E

Or argument's sake, I've been I've, been here before Sergey right, 40 years, it's nice to have it simple in the graph to be linear, but there's somebody's going to say what? If and there will be what ifs we're going to have plugins in the container runtime. So it will be ways to do what you know. Other ways to to you know Resource Management, uh for example.

E

um You know in in a dependent way, yeah or.

A

E

Of these containers and yeah right.

A

And so goes back to a question what we as a kubernetes want to orchestrate and what customers will need to implement themselves. So, um even today, people get uh uh very um uh Implement, very interesting Solutions, with uh even like two stages of initial containers and Implement all sorts of cleanup tasks uh with uh hooks. So we it's just uh how much of it we want to take uh and express as a kubernetes and how much of it uh we will keep offer people to implement as uh as a code.

A

So that's a big question and I understand that it may blow up if you will decide that we want to support all the scenarios. But uh if you only support very generic pattern and then let people use this pattern to implement more elaborate uh scenarios and I think it may be fine.

A

Okay, um I think we um uh we are, we have 20 minutes left I, think we need understand how to press it. On that uh we uh we need to come up with a write-up. What exactly it will look like um so I would suggest I'm. Sorry, I am not uh catching up on chat. If there is something that needs to be discussed Let's. uh uh What do you do? You want to discuss hooks uh sorry.

C

No, no, no I think we can continue, but I was just. We can have that question in the background answer by chat. Fine thanks.

A

Okay, so I think people will mean please stop hooks that will be used as a clean up uh for Canadian, but it's not reliably working.

A

um Okay, um so yeah I suggest. uh Maybe we can uh brainstorm on uh some ideas right now or maybe we can start a brainstorming and understand what we can prepare for the next uh meeting um to have a better understanding like uh first of all, um is there anybody on this chat, who wants to write up like who understands all these requirements and see a clear picture, how it should be implemented?

A

I I'm, not raising my hand, but so I still don't have some uh reservations.

A

Okay, um maybe you can try this mental exercise. We will take uh this picture and like uh have one stage and uh try to Advocate whether we'll have the stage how it may look like in pots pack and uh um what will be um um basically how we, uh whether it will satisfy all the requirements.

A

Let's work, this way.

A

Okay, so let's say we have only this.

A

Okay, keep it deletion almost there.

A

Okay, let's say we only introduce the three containers in 127. Does it satisfy all the requirements.

G

F

They terminate before the internet containers or do they have a termination like in it containers.

A

um Let me answer that after I will figure out how to do a screenshot. Okay,.

A

A

uh Sequentially.

A

um Yeah I don't know um so, if you have like, let's say we have category of containers called pre init. That um I think uh idea would be that as they start before, all they need containers and they run through uh completion uh of like entire port, and uh they will need to probably they will need to terminate even like uh they wouldn't block Port termination I, think that would be a requirement for them.

A

So if you have that, um do we need to uh to ordering so I think biggest uh promise kubernetes can make is uh to terminate them after all, up containers terminated, so that will be clearly promised.

A

uh Ordering of termination is rarely guaranteed in kubernetes I, don't know whether we want to do it in this uh um uh cap. Like let's say we have a termination order like, even if we will try to do that, we still yeah I I, don't see how we'll do that, like? Typically, we don't do it today.

C

Okay, so I go so if we don't guarantee that bring it containers will be terminated after app containers. Then sorry, information use cases, for example, will not work because the service expansion might be terminated before your app so and so the last request won't work.

C

So I think we we don't necessarily need sequentially, but we need uh termination dependency between pretty containers and the rest and app containers.

H

Also, if we only have three Net before a net, um it won't satisfy number two where it's cancer for in a Canadians, because in this case they only start reporting in the Canadians. We don't have the possibility of something after in the containers.

H

I

X container does it mean that we signal the C term after we verify that the process has exited, because there's a very big difference between guaranteeing that we signal them in different orders and guaranteeing that we send the signal only after the app has exited right. But.

F

Right now, the way the lift patch works is we signal the main container wait for the main container to terminate and once after the main terminal main container is terminated. uh We started sick killing all the style and Envoy side sidecar container, so we may actually wait for the main container to shut down.

I

Yeah, so that's what we have right now too, but then we have a work around this behavior that we don't really want.

I

That basically allows us to send the signal to the cycle, because we want the sidecar to know that the part is shutting down so that so that it can stop draining the Ingress listeners, because some grpc clients, for instance, will try to reconnect otherwise.

I

So what I'm saying is it's not always obvious? The exact ordering you want, or at least I I want the sidechart to know that the part is stopping because it might be responsible for draining the Ingress traffic and.

D

I

Aggressive work, but the Ingress needs to start draining yeah.

C

Exactly you need in the side card to know that the ball is starting, but the termination order of the container is actually uh the one suggested. So there might like what that means is not that the order is not that necessary.

C

What that means is that we may need, for example, another hook, a termination, a termination cook, or something like that so and sidecars that need to drain can Define the television cook and said: okay, call this script as we have restart and press stop hooks, but uh we don't need to change the determination order because in fact you want termination in order to be the same. In that case too.

C

B

D

You kill the network before.

C

The other containers, then they won't be able to respond to requests, for example, right.

I

Right but again, I'm not sure kubernetes makes this kind of um oh. What? What the depending on the like you're suggesting we add an extra way to signal to the sidecar that it is stopping instead on top of singing sending it sick time, but if you just send sick term to the sidecar right now, in our context at least, we would have zero problem. The the cycle knows how to stay alive for the duration of the termination Grace termination of the pod I, don't understand.

I

What's the actual need for for this ordering to be done by the complete itself.

D

Yeah I think the issue is that you don't want the side card to stick around for the full termination grain period like, for example, what if I set it to an hour, because sometimes my app takes a really long time to shut down, but other times it just takes 10 seconds, and it terminates right. If you just wait for the entire drain period, the sidecar will the whole pod will be stuck around for an hour right.

D

We want to be able to optimistically shut down as soon as possible, but be able to live up to as long as the grace period not always live. That long.

I

A

So John, do you mean terminate like exit side cars when up Cutters are still in Grace with termination or.

D

Yeah I mean once it once the app container is done, at least for, like the service mesh use case. There's absolutely no reason to run the sidecar right, it's useless without the app container, so there's no points to keeping it around.

D

um So we want to exit exactly once the app container is done, but no no earlier or no later.

A

But um uh going back to the signal about determination: um do you want side cars to be notified when uh view like, after their termination, so like we sent a signal to up continents to start terminating um and they enter this? Like please stop uh hook. Do you want to start sidecar uh termination at that point or you want to wait till exit and then start determination.

D

um That's a good question: it might be kind of nice to have two levels of signals because, like what we would probably do in that case is once the app container um started shutting down, we could start sending like goaways um to responses to the proxy. So we start kind of draining connections.

D

um Then later once the app is drained either send another signal and we'll shut down. Maybe it's sick term first and then sick kill later. Maybe it's something else. I don't know, um but it would be kind of nice to have I suppose I mean we could still handle it. It would just be a bit less graceful if we didn't know when the app was starting to shut down.

C

But do you see uh like it should be a signal that is sent or maybe a hook like a tradition cook and.

E

C

B

The thing with signal is that.

C

Is no standard signal right now to use for that and if we go into seek term and then secure, as you suggested, then you're killing the the sidecaring kind of a violent way when yeah.

D

No I mean you're absolutely right. um I I, don't have a good answer for that. um Okay,.

H

Also, if we did, if we did a signal, would it be on the developers to essentially capture that signal and and do the the graceful shutdown, as opposed to like um having a hook.

A

Okay, so student was better.

H

Well, I was saying the hook might be a little better, just because uh a signal would would put it on the developers to capture that signal and and uh know what signals can be happening like make. It I mean. Essentially you have to make it companies aware which I mean you kind of already are doing that, but it's more tied to uh kubernetes implementations, foreign.

A

I think we I I want to discuss this like initialization pattern um and whether we can uh initially like use some pretty neat containers that swipe through the duration as a actual init containers. If you need to do them earlier.

A

um So if you will have some startup sequence, that, uh like we guarantee that some pretty neat containers will not like one prediction, can block a start of, another cleaning container will be enough for this number too, or we clearly need this like specific unit containers to run to completion, and then everything else runs.

A

Is there any uh tell us about it?.

C

What I don't feel about that is that requirement.

A

C

Two said it says: can't start before you need containers, and here the conservator like, if they start like underground in the chart in the drawing is starting before so I. Don't see why it wouldn't stop Define number two, because it's talking before what am I missing so.

F

uh We wanted in it containers to we wanted the pre-init containers. Pre-Init containers might have dependencies on init containers in terms of certificate issue and all that, but we can just move all that initialization into the pre-init Container itself and then start the main process in the three init container.

F

Actually, the difference between init container and unpre unit container is kind of uh like converging in the sense that the only difference between inert and pre in it is pre in it is lives for the entire life cycle of the Pod and not terminate so what I'm trying to get us like if you have, if you depend on initialization scripts on init container, you can just move that into the pre in it container. So Point number two may not be, um may not be a blocker, but.

C

Can you always move it like if you have some certificates like SSL certificates that are different, you need container is doing like.

C

uh Can you move that last week inside some a completely different container that needs them, I, don't say, but maybe I'm missing something.

F

ah I think it can be moved, but I'm not thinking fully like I'm, not having the full picture. So.

C

So let's say you have some certificates in some Vault, you have some Vault, you need containers and some sidecar pre-unit container that needs those certificates to run. How would you move it? You need to create a combined container image that has.

F

So the unit container is just a script for us, so I would execute those scripts and events init, containers, typically and so I would wait for those to end and then start the sidecar container process in the same process. Namespace.

C

G

C

Need all the dependencies from the any.

G

C

Like you need to create a is not clear for me at all or.

G

C

G

Think actually actually matches up with what you're saying Rodrigo we have two primary sidecars. We use a lot. One is providing identity Services. um You know spiffy stuff, emulated, imds, just things of that nature, like you're talking about um and mostly serves out over a local housebound server to other containers in the Pod. Then we also have a sort of service, meshy, side car, but it's getting certificates from the sort of identity, side car so like ordering. We need the identity one to be there.

G

Then the service mesh, one than anything else so sort of you know a proposal that just has a single set of like a single phase for enveloping containers, whether it's called printed or whatnot uh would still sort of not necessarily work super well for us in that use case, because we do have that. You know two different enveloping sidecars around our app cart containers, sorry um that have dependencies on each other, so we need at least like two two phases there for how we use them.

H

Also, the reason why I said it wouldn't test by two is because the wording of two is can start before in it, but that also suggests is stress can start after net, but in the case of like in the graph, specifically, we wouldn't be able to start putting it after knit. But if that's not, if, if people are saying that we can kind of move things around and move logic around, it wouldn't be a blocker, then my point isn't really valid.

A

I extended this requirement just allow in it like behavior before side cars, so that maybe um clarifies things but yeah radical to your point. It will not be three like it's not a trivial real name of.

I

Init container.

A

To pre-init, because behavior is different.

A

You need to move code around. You do actually hold um so looking at comments in the chat. Okay, so um yeah, we have two minutes left I. Think uh it's a good mental exercise to go through this uh picture. um I I really surprised that most discussions were around uh determination of sidecars rather than uh on initialization.

A

um I think are people find me this time for next week to continue discussion and like uh get to the um uh specifics and maybe write something down. uh Is this time? Okay,.

A

Okay, I see one's about two thumbs up. um Yeah next week is uh shorter week because of Thanksgiving in the U.S for U.S people, uh um but um yeah. If you cannot join, we will record everything and write down all the notes.

A

Yeah Mike kind of no change to Siri noted so far.

A

um Yes, there is no changes to see right, uh I, think one uh difference with CRI and uh for uh that maybe interesting that I'm thinking about is, um if we have this uh playing it, he needs and up containers uh CPU manager will be very tricky to implement, so CPU manager will need to think about playing it and app containers how they will allocate CPU with pneuma uh kind of uh call to closer to each other and uh pre like know about those I know about like think about how it will run in future, but then run on some other CPUs for any initialization stage.

A

E

It will be, do we have an annotation, so we can identify the classification of the container I can't remember, we don't have yeah yeah, we might that we might just need an annotation, and then we can pass that to the to the managers.

A

No I, what is I will try to copy chat, so let yeah, let me send an invite for next week and uh we will continue this discussion. um Thank you. Everybody who who came here today.

E

H

F