Kubernetes SIG Node, 22 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SIG Node Sidecar WG 2022-11-22

Description

Meeting notes and agenda: https://docs.google.com/document/d/1E1guvFJ5KBQIGcjCrQqFywU9_cBQHRtHvjuqcVbCXvU/edit#heading=h.m8xoiv5t6qma

GMT20221122 170443 Recording 1920x1008

A

Thank you, hello. It's uh uh short week in the U.S on November 22nd 2022.. uh It's a signal. Sidecar work group um welcome everybody.

A

So last time we discussed. um Let me share my screen first, um so it will be on the recorded as well.

A

Okay, dude see my screen now yeah, so last time, perfect. Last time we discussed uh What uh design will not be. uh We decided to discussed that this is not as a graph. This is not a per container uh single property uh and it's not a regular container beside car flag uh because it needs to cover in it containers as well. So we want to cover in its stage, show inside cars.

A

Somehow, and we came up with this set of requirements, uh we discussed that uh it needs to have some property to block uh both termination uh do not block termination, so it will be uh deleted when app content is deleted, uh it should be able to start before you need containers you've been discussing that uh there is a requirement for some uh to initialize something before uh sidecars, but this installation, theoretically, can be moved into uh into a container itself, so container itself will do initialization, uh which more burden on developer but uh easier to implement for us, um then restart Behavior needs to be taken into consideration.

A

uh Probably it is very started, even though whole pause is not really started and then semantically different Scopes. In future, we consider more features or more properties to be added to this kind of containers, uh including security reservation. Some resource resolution um uh put the Readiness at the right when ports affects whether, uh when sidecar affects differently, whether Port is already or not ready and then Wireless can be also affected in a different way as well as Behavior, and then we need.

A

We need to establish patterns that we can reliably inject uh this sidecar into port and then determination was a Hot Topic. uh Last time we've been discussing germination order and it was important that uh sidecars will somehow be notified when main containers uh start being deleted. It's not like hard requirement, but it's very nice to have to implement many scenarios.

A

uh Yeah I think uh seven is repetition of five. Oh yeah.

A

Okay, so this was the requirement and we uh started this mental exercise. We said that uh out of the uh slide Deck with all the stages, let's discuss single stage like pretty neat containers like let's say we have pretty neat containers and then we've been discussing. What does it mean and whether it covers all the requirements?

A

So we went uh through some requirements around termination and we went through some requirements around uh um initialization uh and finally, we are stumbled on CPU managers that may be tricky to implement because CPU manager needs to account for like today it uh allocates some CPUs for any containers and then any stage finishes and allocates proper CPUs for up stages like for main stage now it will need to pre-allocate some init containers and then they need to play very nicely with app containers.

A

One day, you'll come up and that may be uh not trivial task to accomplish, but other than that we didn't find any problems. So this time we wanted to continue the discussion. uh We want to take this picture and continue discussion.

A

What requirements will not be satisfied and then Matthias and Ramya worked offline on trying to come up with like different options, how it can be represented in Podium, and if you get to that, you can discuss it. uh If we're not ready, we can do it next time, because not that many people today on the call, um let me copy requirements again.

A

There are any questions before we go into recasting the requirements.

A

A

So yeah um again, let's reiterate requirements, I think what we can say is it's uh will not block termination yeah. This can be accomplished like, as we discussed three containers. We have this characteristics that they start uh in order before any containers, and then they uh will be terminated when app containers are completed. So this will be completed. It's easy to accomplish foreign.

B

I have a question: uh sorry I wasn't here last time. Last week um you mentioned real briefly in the in the intro um pre init pre-pre in it I as an idea like we added in it containers to make a strong guarantee that this always happens before that right.

B

um Do we feel, like that's going to happen here? Will we need pre-pre in it.

A

Yeah, this is uh second requirement uh cancer. Before any containers, we um we recognize that there is a requirement that they should be able to start before a new containers. There are scenarios when we want to reliably uh in inject containers such a way that they will cover in each stage as well um and the question was uh people want to initialize some certificates for service mesh, or something like that. So um today it's accomplished through init containers and in the future. What we've been discussing is.

A

uh We can either have two stages: try to come up with two stages like one three, three like one three unit containers, another post, init containers or we can make developers code more, so developers will need to take the logic of init containers and move them into pretty mid stage, so they will either move it into existing container that will initialize itself or they will move it to separate containers that will consume resources through throughout the whole um Port execution, but the same, but it may be very little resources so does.

B

That solve the requirement, though I mean the the requirement that I have in my head is around networking um where we've often said the. We need a guarantee that very, very, very early before or anything quote real runs. We install traffic capture right now.

B

If we, if we're saying that, there's a pre init phase that things can inject themselves into like what are the guarantees we're making there are there any or can somebody come along and and basically will, will people end up arm wrestling between two different pre-init containers as to which one goes? First,.

B

A

Very few questions.

B

A

Yeah we didn't get uh into that discussion. uh We just um the only thing we discussed is that this logic can be moved into pre-made stage, and uh it should be uh it can be accomplished. uh Second, question is uh whether we will have different side cars so fighting for the for the first place like to be first, uh we didn't discuss that far and we didn't discuss it as a scenario that have to be accomplished.

B

The reason I ask is I think if we figure that out it will inform the rest of the design a little bit as to whether we need a single stage here or whether we actually need like a series of stages with different guarantees or whether we simply say we're just not going to handle that, and you can arm wrestle in whoever wins. Good luck to them.

A

Yeah um last time uh we also discussed this uh concept of uh like Mike Brown, came and said like if you start expressing different concepts in podiamo uh like we want to be this used to be first, it needs to be a little bit faster than this one. Then.

C

A

End up this expression, uh almost a graph inside the podium so like, even though it's both phases, we still ending up with some sort of a graph, because we have so many Port phases to accomplish every single scenario, and the discussion was how much you want to accomplish. Declaratively saying this is scenario support uh as opposed to how much we want to accomplish through um some prescriptions like this is how it's supposed to be working.

A

Please be nice or uh record a little bit and then it will be easier for you, like you, need to code more, but you still can accomplish this scenario. So uh yeah I agree that we need to understand. Is it hard requirement or if it's not, then we can, uh it can be easier for us to implement. If it is, then it will informs the design for sure.

A

Is there anybody on the call who has an opinion and who has this requirement in mind? I I think John John you're from istio right, so you may.

B

You know, John, are you here.

D

Sorry I just got back and heard John are you here so I.

B

We were talking about the arm wrestling. The jockeying for First Position uh requirement. The idea that, uh like traffic capture installation has to has to has to happen first.

D

Yeah I mean It's tricky, because even if you say that, like maybe you have a I, don't know a vault in it container that sets up this certificates for the surface mesh. So maybe we don't even always want to be the first verse. We want to be the slightly less first um I I feel like you've run into this. With anything that's like in CSS, you have like the weight or priority or whatever right, and then you have people putting 999 and then 9999 and it's kind of a mess.

D

I don't know, I, don't know that. There's a there's, a clear solution. There.

E

Well, one best practice that I've observed in that kind of space is almost subnetting in a way of kind of breaking it up, so that, like indexes 0 through 100 or 0 through 10, are dedicated for one purpose or one layer and then 11 through 20 or whatever, so that, then you kind of have a a range and generally in practice.

E

You don't like know like eight or three or some specific thing, but it's kind of you either pick in the middle or on the the boundary point of each one and then, if you're having to fight you're, only you know adjusting one or one or uh two above other things similar in that layer.

E

So if there's like some alerting thing that always has to be on top you're not going to even risk fighting that unless you, you know, you start bumping and going up to 999 or something because generally that's where that kind of stuff would be dedicated.

E

um So perhaps perhaps it's a kind of a free-for-all, but if we give them some kind of best practice, then at least for for most purposes because I mean, if you have. If you at that point, need that many different orderings, then it's kind of you're kind of a unique situation, almost Beyond like three or four orderings I, would I would think yeah.

A

uh In kubernetes is typically accomplished through ordering of containers and yaml file, so we respect the order and uh they will be started in this order. So it's kind of indexes uh without declaring the index.

B

John D: do you have? um Is the the case that she's decided the Vault needed before mesh setup? Is that a real thing that you've encountered, or did you just make that up.

D

um I think some users use kind of sidecars to the sidecar to handle certificates, but it's pretty rare yeah.

F

Which case well something like this? Where we have a, we have a sidecar that like serve as a Cloud, API, metadata, stuff and certificates, and the proxy for the service mesh depends on it.

B

But like does that itself use Network.

F

Wow uh yeah because it basically it captures traffic on the like uh local uh IP, that the cloud providers expose their metadata servers on and there's Magic on this.

B

So you need to use the network before you get the certificate before you can configure the network. Yes,.

D

Cool I feel like the ideal flow would be like in at least in Easter. Today, like we would block Readiness until we have certificates. I feel like in this specific case. The ideal thing would be the sidecar. The network proxy actually runs first, but it doesn't block until certificates, um but it I I. It's a bit. Fuzzy I feel.

E

Like that's, why the.

D

Way, it really works, because otherwise you have the circle or dependency right.

B

Right and like the requirement that I've heard is I really want nothing to egress from my container or for my pod until the proxy is installed and so install the traffic capture, and if something else starts just block it right or fail it or do something.

B

So I don't know how to crack that way. It sounds like they're in conflict.

A

Yeah I think this kind of requirement can be accomplished through custom policies on specific customer saying like if you are that strict about networks- and you need to be to have a web hook and move this container into first position.

D

Yeah I mean as long as we can order them. Then those two cases like there's ways of either the Vault container or the service mesh container or whatever they're using they could somehow be modified. It make it work.

D

I, don't think kubernetes can solve the fact that we have two codependent applications right. Some of the applications I.

B

Don't think they are codependent I. Think we're just not expressing it clearly like it sounds, like uh sorry was that Felix speaking before, yes, okay, I, think what you're expressing was you get? The the first part needs to go to the local metadata server right. So it's like not real Network traffic. It's like link local traffic, correct! Yes, so maybe that's the carve out or something like that.

F

Yeah yeah I mean to be clear: I we don't do traffic interception for the mesh itself, so we have a weird setup compared to like uh Easter or Lincoln, but um I'm just saying that um there are other cases of I need this container to stop before this other container will start before X, then just setting up their table rules to do traffic interception.

F

In our context, we don't have a loop of the balances we have like a depends on. B depends on C, the everyone depends on the sidecar to do volt, magic and Cloud apis magic, and then the sidecar for the proxy depends on that and everyone depends on both.

F

A

Containers and this first A and B, like volt installation, will probably consume resources forever uh for entire duration of port. um That is it fine. Is it uh What uh will break anything or it's fine.

F

Well, in that context, it's fine, because this does not just do any stuff it weren't far. It runs forever. It's a sidecar.

A

Okay, so it's like generally, you can combine it into one container.

D

In theory, if the pre-knit container did just exit afterwards would would that be fine for people that want knit like semantics and bring it.

B

Okay, so that's an interesting question um and I don't know if this was discussed last week, but we have this property today, where the Pod has a restart policy and all of the app containers for lack of a better word uh follow that restart policy and all of the init containers are specifically defined to have a restart on failure policy right and you can't configure that you can't change.

B

It um is the thinking here that pre in it would have a configurable per container policy, or does it always do all pre and it containers follow the uh restart on failure. Run Forever policy.

A

Team, your master of transitions to the next topic like to start a policy. It's the next on the screen, yeah yeah.

B

I think this is really one of the most interesting discussions, because it it has. The at least seems to me from who's not super familiar with the cubelets Pod lifecycle code anymore. um It seems to me like this, could have the biggest impact on the the qual, the the ability to do this reliably.

A

Yep um last time we discussed it, uh all three containers should be uh running uh like it started always uh so they never terminated and we didn't come up with a scenario when we don't need this. uh Behavior Uh we've been discussing like I mean we didn't discuss explicitly. Was it initialization that you do in pre-made containers and you don't need this any longer? You want to shut it down. We didn't discuss that in details, but uh nobody seems to be very worried about uh one.

A

uh Those kind of fish cases.

A

I wonder if anybody on this call today can comment about it.

B

I, don't have a concrete use case for it, um but I do worry that um we're defining it sort of recursively like if we do this, then I suspect we'll end up with pre-pre in it which follows the Run once semantic.

B

Just you know extrapolating from now two data points. It indicates a direction at least.

G

I'm I'm wondering if we should not allow the network provider to to provide us a set of init containers himself, and then we incorporate it into the Pod spec, like you know, delegating to someone who knows more than just us like in the case of istio, if you say yeah, we we need this certificate. We need this key. We need this information before and and and if you just provides all those pre-pre-init containers for us, I, don't know.

G

A

Sorry I didn't get your comment like are you.

E

G

So so I I I, think I, don't know, maybe an idea if, if we would allow Easter to to provide a list of containers to run before their own container, like like a better mechanism than just doing the the Pod mutation at admission, that adds one unit container something more powerful than this I. Don't know how it would look like, but.

A

Basically, you're saying some candidates may just do nothing. Unless some you need container run to completion.

G

Maybe, or or yeah or giving too noisy here sorry.

B

We're not hearing any noise from you, so it's okay.

G

Okay cool so um like saying so, I I need to insert this container before the main application. So that's istio, but in order to run I need a container that handles the certificates or I need a container that handles this. And then this can be gathered in the Pod specs. Somehow, like a composition, I I don't exactly see how it would work, but it's interesting maybe to to think about inversing. The the configuration saying like I need the component, and this component knows what it needs and an upfront.

G

The guy using the pod doesn't know it. So uh like a dedication of dependencies.

A

uh Do you think container will know in its code that uh something is completed so, if I think in implementing that I will just start playing it container and watch for some things to appear and then, when it appears, I can start running or we need to have explicit delegation on like uh our orchestration of the things and have a notification to bring containers.

G

Yeah, maybe it's getting too complicated, I think for the the one percent. uh We should not bloat the implementation.

B

Yeah I mean that's a good point right at the end of this there's going to be Corner cases, no matter what we do where we're going to have to provide. You know the escape hatch. That is you've done. Something really weird or your environment is really strange, and you have to configure things yourself that way.

B

um So it's good to keep in mind what that escape hatch is.

B

And then solve maybe for the you know, 80 case.

A

Because we've been discussing the three unit, containers want to have notification when uh app content is completed, um so they will like received a termination uh uh callback. So putting contents will also receive the same callback or like similar callback. So maybe we need to have notification when you need stages stage has completed so maybe another option.

A

No no takers, I.

C

A

Specific scenario in mind, so I don't want to put this requirement if somebody has, uh if it resonates with anybody. Please pick up.

B

I've left the thinking about what all the life cycle stages means until I have a better handle on what I think the general model should be. But I do think that what we currently do with like pre-stop and pre-start will have to get more sophisticated in order to do any sort of intrapod messaging.

F

C

Yeah I think I think I'd, like what Tim was saying he. If we're not going to do a dependency graph, then we at least need a dependency to you know line so that we can. We can have some way to define you know and sets of dependencies right and as soon as you have one you're going to have an hour, it just happens that way and then we need on the destruction to be able to walk it back. The other direction.

C

Use case example, you know we went from containers to pods and that introduced the dependency on pod initialization volume management, other group managers inside of the pods, um and that that's all being held in Violet today, but it would be nice if you could extract that into more uh of a common container model. If we could so these you know, pods could become more interesting.

A

Okay, I think I'm going to put X here and I think we can accomplish this. We we still have open question. Is uh anybody like really need this uh uh network setup stage to be uh before everything else, but I I heard that this on this call? Nobody have strong uh requirement like that like uh if some, if customer has a requirement to uh customer and then I suggest, we can discuss this uh kind of uh enhancement like featuring hazards when we have a Podium example.

A

So when we have a Podium, then we can discuss whether this kind of properties will just fit into expanding this uh definition of sidecar or like green it or like ambient whatever container or we will need to have, or it will just go to a boat and you'll need to declare them right now will be to discuss whether this kind of requirements will be applied specifically to print it or it will be applied to any container. Because if it's applied to any containers, then we don't need to take care of it right now.

A

We can discuss it later. If it's only specific for pre-knit uh uniquely, then that may be a new requirement.

A

um Security solution, any opinion like uh like let's say we want uh premium containers to run in different runtime Handler.

A

My opinion that this requirement is not super unique to pre-init Containers, uh it's more like everything that is injected should be able to run in security, isolation.

B

Can you describe, then what you mean by security, isolation.

A

um Run some containers in different runtime calendar.

B

In a different container, runtime yeah.

A

Like gy differences.

H

A

B

Yeah is that really the same pod like that feels like a different um proposal here to have Potter pod assistance, but they're not really in the same pot like running something in a g. Visor puts you in a different network space right where you can't really coordinate ports, and things like that um or or you know, if you run it in Kata, you've got more of a vmish thing in between you right um is. Is that the same POD at that point or is it a different concept.

A

Yeah, that's uh that became complicated once you have this uh I heard it even uh people dreaming about side cars running on different machine, but somehow have still has access to Port. So it's like.

B

um Yeah, that's a that's a different thing: I'm I'm, not even against the thing that that is, but I. Don't think that that's a sidecar.

A

Okay, yeah um and I- don't think it's specific to sidecar, so I think that maybe applicable to any container.

A

One question, though, is uh whether the security, whether it is any different, like I, think resources evaluation, maybe uh something that we may want to apply specifically for sidecars.

A

uh One of the complaints was uh I mean we have this issue for a while that uh CPU on throttle think is defined uh for entire portals and for individual uh containers, and uh there wasn't even a talk at some point on coupon about uh Envoy eating up all the CPU quarters.

A

um So that may be in specific for Primitives.

A

Do you want to speak up about security context.

H

Yeah, my my question is more related to what we do actually like. If we have like a container acting as a sidecar, then we apply the same pattern.

B

Well, she can't explain what you mean a little bit more or no.

E

Like which pattern.

B

And what does acting like a sidecar mean.

B

H

You still there yeah sorry I have to open the door for someone- oh no Yeah. So basically, if in some basically Cloud providers, say okay, you should run a agent as a sidecar, which means you define a new controller in a pod with a different security context.

H

So I'm trying to see if, if we have like a sidecar per se, can we Define different things in a new security context applied to the sidecar like? Is the sidecar can run as a different user or something like that.

D

Isn't that something we can already do with containers today and we have different resource requests and limits and different security context. Yeah.

H

But can we because right now, I, don't in the specification, if we Define sideka in the bot specification, how this is look like from a security perspective.

D

I see yeah I mean I would think that we would need all the fields that are in container um today, or at least the vast majority. Maybe if you don't make sense, uh which would include you know, resource and security context.

A

Yeah and if you need enforcement, we can probably a customer, can do it through web hook and try to enforce something.

A

C

I think it's I think it's fair to say that the runtime handlers and the things that can be configured with respect to resources need to be expanded. um Today, there's a little bit of limitation around the image resource. You know: access polls, storage, that sort of thing that needs to be murderous as well as snapshotters needs to be merged into the Run handle or context, and that needs to be expanded and expanded and accuplet, but I'm not sure if it's in it containers.

C

You know sidecar container specific, it's more of a the concept that you know these runtime handlers need to be expanded across the entire node, so so that we can have confidential pods so right and in the kublet. You know the node manager needs to be able to handle that as well as the container runtime.

B

I want to be careful that we don't bite off too too much here, like the idea of a confidential pod is a very interesting idea again, I'm not sure the same thing as a sidecar right exactly.

C

A

Caesar that are very specific to uh pretty neat containers. What about resource resolution? Is there any um I know we have an issue suggestions that we want to do. Cpu throws container, does it make?

A

Will we have pretty neat because there's a special case for this kind of isolation, so all will be.

B

Two: do we not set up uh CPU limits per container.

A

D

Don't think so: oh they.

A

D

A

Yes, so what's the issue then, um but I think the topic about CPU quota.

D

um I don't know, I haven't I, haven't seen issues, since every container can Define their own limits and requests, and if you don't Define like well, an issue could be that users, don't typically Define guaranteed um whatever quality of service I. Think for sidecars, so like they can't dig into the quota of the other container I believe that's how it works, but that's not something. That's that that's a user config that they've chosen to do that because they want you know bursting or whatever I. Don't know that we need to solve that.

D

For for this problem work we do it's. At least it seems orthogonal right if there's maybe a better way to have multiple container resource allocation. That's fine! That's kind of unrelated to the ordering problem. I! Think.

F

F

G

A

Remember will sorry.

B

I was just say: I agree that we don't do a great job overall, of defining the exact uh semantics of resources per container per pod and how the Overflow of those things Works in kubernetes and in fact it may not even be kubernetes job to Define, that it may actually be the container run times job to Define. What those mean.

A

Yeah I posted the link to this issue here. So it's about CPU, throttling uh uh and uh yeah um suggestion was to do it uh to improve it. Somehow.

F

I think the idea here was to do the opposite, which is like to maybe allow a way to define CPU limits and or requests, but rather it's really about limits here um at the Pod level in some way,.

B

Yes, exactly there's a cap for that. Somebody.

G

Wants to bite off.

E

B

Wants to bite off another chunk of meat uh that kept was was fairly well thought out, um and then the contributor disappeared um and okay, it's just waiting for somebody to take it over. If somebody wants to yeah and I thought, I thought it was a good cap. I was hopeful that somebody would pick it up.

A

Yeah um started for misrepresentings that uh my question was: uh do you want to have special throating uh special treatment for premium containers with regards to CPU floating? Let's say we always have CPU more CPU for main contents, app containers and for pre-unit containers, because uh here the issue is described that and why is still accepting traffic when app containers are just do nothing, because it is important.

B

But I mean isn't that the correct Behavior like if some Upstream balancer is sending traffic to your pod, but your pod is not able to handle it like. Don't you want the load balancer to at least Knack all those incoming requests?

B

There's a semantic question here, I guess is, is what I'm getting to um with respect to the per pod resources here, I'll, throw out some crazy idea that I don't think we should do but I feel like I should say it out loud now, um we've gotta we've already got an actual hierarchy.

B

When we look at something like c groups right, we've got a pod, C group and then per container c groups like perhaps it's worthwhile to actually interject another level in between which is like phased uh group like these are all the pre in it things, and these are all the app things and um that allows us to with the semantics that c groups offers allows us to sort of slush resources around between containers, but only within a set right. So all the pre and containers could have.

B

This could share some amount of resources, but they can't starve the app containers. I, don't know I I, it's it's fairly complicated, even just to think about.

B

um So almost certainly nobody will get it right um and the semantics will change across kernel versions and across container run times, and so it's probably a terrible idea, but I felt, like I, had to say it out loud.

C

So I think it was a great idea um and we've got this class resources cup. You know work going on I, think that would be related to this possibly provide a solution.

B

I think maybe the better answer is. We should just Define what we do with c groups better and actually use them better. Like there's a lot of capabilities that c groups offer that we don't take advantage of and there's a lot of things that we would love to do in terms of isolation, that c groups doesn't offer and- and maybe we should go back and just like focus on that and then and then worry about this later. The the the subgrouping of resources later, okay, because.

A

B

This like how many people are confused, how many blog posts a week. Do you see that argues about whether or not to set CPU limits right, um as the answer is larger than one um and uh people don't understand? The semantics of the c groups that we already have so maybe making it more complicated is a bad idea.

A

Okay, uh I just needed to talk about it, just to offer complete mistake um and I think this is one of the requirements that can be applied for only containers and maybe a requirement in the future, um but uh yeah we can discuss it when we have Podium like how the main Charities can be implemented later um rights um I think this uh is applicable for any containers like we don't not necessarily will single out specific uh stage like not all pretty neat containers.

A

It will have the same weight of uh like attracts in Port Readiness, so it's likely it will be per container in future.

A

um That's why I think this may be a good requirements, but it's unlikely will be sometimes we put as a like force like it's, not an opinionated layout layers that all pre-init containers Express this kind of behavior. All printing containers like loudness is uh affection. All other containers, violence and stuff, like that, okay,.

B

So is the proposal that we changed. What pod Readiness means again, no.

A

Proposal is to not do that. um I'm I heard this requirement that uh and boy Readiness will affect Potter and his differences and any other container, but I think if you start playing with that, we will need to do it per container not per stage. So it's not like all three unit containers who have same effect on portraisiness as uh it's likely to be single, like specific container, will have different uh effects on powderedness I, see.

B

Yeah I could get behind the idea, maybe that some containers don't affect pod. Readiness like you could opt in to say this container is not really that important, but uh that's certainly not a default.

D

Yeah, okay, it's! Why do they need a Readiness probe at all yeah? Is it to say that I've started up, but not that I'm ready well.

B

There's more than just Readiness probes right like you could be restarting or something which could affect your your Pod radio, okay, fair enough yeah right like if you're, a very crashy, little helper, app or or you're, particularly subject to I, don't know out of memory or disc full or something um it just. You know it's software, it's written by people that we're all stupid, so uh yeah.

D

Okay, if I say.

B

This thing's, not this thing's, not important. It doesn't matter if it's Crash looping keep my pod running.

A

Okay, so those are not uh pretty new stage specific, it will be there and uh own behavior. That is different for preheat I. Think it's the same category we probably is: maybe is it in category of resources, relations that uh we will apply a little bit different um behavior for all contents, but likely to be per container rather than their stage.

B

This is one of those places where I think we don't do a very good job of using what c groups offers us yet yeah.

A

Especially secret video yeah okay, so we found one that maybe we'll need to discuss when we'll uh get into podiamo. um Everything else is see what it sounds to be covered and can be accomplished as pretty unique. Containers.

A

Savage is a pattern um okay, so uh this one is talking about. uh If you will Implement just that, if you implement just bringing containers um when new requirements will come, do you know any requirements that will come that will not fit into this model uh and that we will need to introduce a new stage of containers and if you will need to implement new stage of containers and whether the pattern that we will establish will be uh repeatable. So we can do similar things.

A

Prediction strategy.

B

Yeah I think one of the big questions I'm trying to avoid like mentally designing what an API would look like, but I think one of the big questions here is whether we think we're doing this one time for pre init or whatever we're going to call it God help us.

B

We need to find a better name than pre in it, um but uh whether we're doing this one time or whether we actually believe that there'll be another one and maybe another one over time like if there's going to be pre in it and then pre-pre in it and then Network pre in it like that, will change how we think about the API.

A

Yeah I think so far we've been discussing that uh everything Beyond standard requirements can be accomplished by ordering containers and quoting something inside this containers, so I think we should be covered.

A

Maybe we can say that we didn't come up with any requirement in the mid future that will help the required this thing.

B

When I wrote this doc a couple years ago about phases, I wrote it the way. I wrote it because I was trying to think through the future. Like the directionality right, two points indicate a line and uh if, if the direction is that we keep getting more reasons for different different uh stages that um we should I was trying to Pivot the conversation from what to why, instead of saying well, we have pre and it containers. That's the what it's more like. Why?

B

What are people doing with these things and I tried to define the stages in a way that said, okay, look, here's the assumptions you can make during this stage and here's the assumptions you can make during the next stage. Like the first stage assumes you can't do Network traffic. So if you have to do something and it's okay not to use Network traffic, then you can go there. But if you need Network, you have to go in the next stage and that level of thinking broke down into you know four or five potential stages.

B

I didn't have concrete use cases for all of them, though,.

A

Yeah and we can come up with use cases in different during those meetings- wider email asking for those- and uh if there is none, then you can just call this conversation for a few years, at least.

A

um Okay, I reliably injected I think this is covered with uh most of uh thinking like most of most of proposals. How to implement this premium stage, uh Works nicely with injecting uh containers and then.

G

I just have a question regarding the the ordering uh the does yaml guarantee the ordering of in lists. Yes, yes and.

B

Like Jason right, yes within a list, but not within a map yeah, if you define.

A

It as a list- and it's.

B

Fine I, still I find lists. I I think in the API are good. If you have a single author and they're cumbersome, when you have multiple authors, um anything that is simpler than either prepend or append becomes complicated. If you want to try to insert something into the middle- um and this goes back to the arm wrestling for like who gets to be first right.

A

And probably from Jasmine about indexing I, don't think we I would be against in the intelligent indexing uh just for premium containers. It seems to be a big change that uh may not be uh Justified with scenarios yet.

E

Yeah, it feels like Overkill.

B

I didn't see the proposal. Sorry. What can you give me like a 10 word summary.

E

Basically, like um it was really just like weighted orderings uh and then the problem was proposed of people just do 99999 to get priority.

A

E

Least, priority um and I just had mentioned that there are ways it's kind of like a song, because this is already kind of you're you're, going out of the way to to do something funky with with many many pre-ordered containers, but um to just kind of educate them in the documentation of like an example of you could bucket it, um and if you, if you kind of put put out in maybe brackets, is a better word for your indexes um and you dedicate those it's basically just recreating your stages like stage one would be with, like you know, zero through ten stage, two would be whatever, and the idea is just.

E

um You typically would pick like in the middle and if you do need to budge, because somebody else is at that stage, but you need to be just before them. You just tick up by one or take down by one, and then you just don't risk entering into the other phase. But you still have a little bit of jiggle room um and that's how you pick the size of the bracket is how big you want your your room um but yeah to his point, just just especially just for in it containers.

E

If it was a pattern that we deployed everywhere in the API, it would probably make more sense, but.

C

E

It's also very much that's kind of a a convention driven thing right and and Convention isn't a great way to yeah to do an API.

A

Okay, thanks and lastly, uh termination ordering I think we discussed it, uh it should be uh sidecars last and uh we will. We can Implement containers this way. So that should be fine. So.

B

I think sorry, question on termination. um It's I think it's a requirement that sidecars get some reasonable termination grace period um above and beyond the applications termination grace period. So here's here's the thing I'm concerned about. If I have a logging, uh squirter sidecar, that's going to send logs somewhere else and my application says: I need 60 seconds of termination grace period.

B

It could consume that full 60 seconds the application could and still be generating logs and then I need some time after it finishes to finish sending off the logs and I. Don't know where the right expression of that is, but I do think. We need to capture it somewhere as a requirement. Okay,.

B

Wait and then, of course, the question comes in in chaining these things right like if my log sender captures the logs from some other container and that other container runs for its the app runs for its 60 seconds, and this other thing runs for its 60 seconds. Does the logger need 60 more seconds like I? Don't know how to sequence that yet I haven't thought about it.

F

E

That does make sense, that's kind of like how the container resource stuff does for the pot. It just is the sum of all the containers in it. So then it would be like your pods graceful termination is really the sum of all of its containers and pre in it. Containers, graceful termination periods.

A

We have three minutes left and I'm really glad that we went through all the requirements this time, uh so we um I think we covered them all and we pretty confident that pre-made containers concept may work for foreseeable future uh as an implementation. Now I think we have two directions. First is uh all right: um those requirements and discussion a little bit more formal and maybe presented on Signal meeting.

A

uh So if there are any more requirements that people can think of that doesn't fit in this model, then we can discuss those requirements uh and uh if not, then we can close this conversation and like move into discussion of yaml like how yamo may look like, and whether all these requirements is just one property or like one list that we have, or this can be service requirements, can be split into property of any container and apply to any uh container here, um yeah. So much as you I think you started discussing this Ramya right.

A

How podiamo may look like do you want to continue and like present next week.

G

I can try to to work on this, but unfortunately we yeah, we didn't advance so much on it.

G

uh Rather, we discussed the the use case for the the lift patch uh and and made sure that uh the brain it uh kept would would tick all the boxes that they need as their use case and then I wanted to check with the other patch the other Forks of communities, maybe the caverno one and I don't know if there are other ones to to see if our proposal would would fit the needs as well before going to the yaml, because I know this one will be super controversial.

A

um Okay, um what how about that like? We will start with next time with Podium that will simply have a different set of containers, and then maybe you can go to requirements and understand if any of these requirements should be built-in feature of the containers, and this is how we describe what these containers are. All these features and properties can be described as a separate yaml field that can be turned on and off for the premium containers and that can be applied potentially for other containers.

A

How does that sound.

G

Yeah and we need the naming as well.

B

Let's leave late naming for the last discussion, that's the least important thing here unless we think that the naming influences how we think about the semantics, which you know is a maybe a fair argument. Well.

F

It it sort of makes sense to stop talking about it at some point, because I think one of the point of trying to this approach is like. We have a good abstract concept to represent um this stage, and if we can't name it then.

E

We at least know there's some dissatisfaction with pray in it as a maybe.

A

We can uh at least all the options.

B

Next time next week, correct pre in it is one of those names that you use as an obviously bad placeholder that tends to stick yeah.

A

Okay, um thank you. Thank you. Everybody uh talk to you next week and maybe earlier sorry yeah and if you're in the U.S Happy Thanksgiving thanks. Sorry.

B

The same this same group, not the regular signal group right. Yes, this is uh specific size. Cars, perfect.

A

B

It if anybody's playing with API over the next week and wants to bounce some ideas around slack me.

A

Yeah and um I put all the participants from last week from Zoom just uh grab the zoom uh airport. So if you want to put your name and Company, uh please put it here. Otherwise, I'll just put the other participant just Consumer Reports.

A

Thank you. Thank.

F

A

F