VMware Carvel Community Meetings, 15 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Carvel Community Meeting - March 15, 2021

Description

Carvel Community Meeting - March 15, 2021

Discussions around recursive bundles for imgpkg, kapp enhancements, and new proposals process. Details here: https://hackmd.io/F7g3RT2hR3OcIh-Iznk2hw

A

Just a reminder, this is being recorded and it's going to be added to our youtube playlist afterwards and to adhere to our code of conduct, which is listed in the agenda here today is march, 15th 2021 and I will um kick off this to aaron- to um go over the rest of the agenda items and have him start sharing his screen to drive.

B

Cool, can you see my screen all right um so happy monday, we'll start off with a few announcements. uh We released cap v036 last week um a couple of interesting things in here. I think this. This first feature filter out. Mislabeled resources returned from ktpi. um This is, I think, a pretty nice win for for various.

B

I guess you know bits of software that were not returning filtered, not returning resources filtered by labels, so, depending on how you implement your if, depending if how you implement the aggregated api, um this could have been an issue that that folks are in we're running into.

B

um We have some other things to just mention, but uh yeah and that's that's kind of the main just here.

B

um We renamed our carnival repo to carvel community uh trying to make these things with these repos a little more clear, and in addition to that, we also published a proposals process uh and have a few active proposals uh already in flight. So please feel free to check those out.

B

Another note is we're hiring a staff engineer. So if you're here, it seems like you have some interest in karvel. uh If you would like to work with this lovely team and on these exciting tools, then please click that link and learn more. We don't.

C

B

And then last announcement is uh just asking for for any feedback, if you have it regarding this community meeting, um you know in the interest of iterating will likely introduce some some changes in coming weeks, but we'd like to hear from y'all what y'all like what you would like to see anything like that.

B

Cool then we'll zoom through the status updates, so we we did finish up development on the image package, performance improvements track of work. uh It's still on the develop branch right now, but expect it to be in the next release.

B

The image package recursive bundles track is very much in flight same with ytt schemas, we'll talk about both of those a bit more in detail. Today we have some cap enhancements that are also in progress just due to team members being out this week. We may make limited progress here, so just wanted to call that one out and then cap controller.

B

We are continue to work on the package manager, api um and as well as some more stability focused enhancements, so feel free to click any of those links to dive into more um before we get started with this week's plans. If, if you have anything that you want to bring up today, we'll try and make sure that we leave some time at the end to go through both discussion topics and we'll try and get to at least one of these triage helps.

B

So with that said, let's get started, I guess. Is there anything else from anyone before we start.

B

A

Sorry, I'm actually curious, so um 6.0 do you have any like expected time frame for when that might be released.

B

um We, I guess I haven't really communicated one yet at this point. It's likely to to be released when the recursive bundles track is done. So it's more feature scoped within a few weeks.

A

B

All right so jump into the backlog, how we start with recursive bundles today, um so we can kind of see we have a three-pointer, that's in review. We've got a one-pointer progress. We already pointed this one, so maybe we'll try to get through a couple of these today.

B

So starting with this first one pull update all image, lock files uh dennis. Would you like to walk through this.

D

I could, but I'd have to read it in as well. um I can try, though, unless you've already read it aaron, um okay, so the use case. So this is about pulling updates. So when you do an image package pool with a recursive bundles, it's going to update, I guess all the image log files recursively in all the bundles that it's bringing in so we're going to create a bundle to distribute to multiple applications on a part of a deployment.

D

So prerequisites looks very similar to the other stories. This, basically just outlining a bundle that references, another bundle and there's a couple of images, essentially as leaf nodes. As part of these two bundled references of bundle going through the acceptance criteria, scenario number one.

D

We want to make sure that updates all the image log files for all bundles after copy. So we push a bundle that references, another bundle and then we run copy from one bundle to another repo and we have the experimental flag turned on and so that when we run image package pool with a dash r recursively referencing the bundle we copied uh and to a directory, then we should see then that this output has already been done for us, but at the very bottom of the output. One difference is changing.

D

You know, updating all images in the image log file, so changing all images registry repository references in this to other registry. So we already have a logic that updates image lock references in the images yaml file, but we want to do that for all the recursive bundles as well. It looks like so like the bottom.

D

The last three four lines is essentially a delta between what we have today and what we want tomorrow and then when we cat and then when I execute so when we look into the contents of an images yaml file inside the recursive bundles image package directory, and then we see, I guess the uh the contents being rewritten.

D

I think, and then, when we cat the images yaml that got brought in by the nested bundle, then we also see the images yaml also being uh updated as well to the other registry.io bundle and then, when we cat the another bundle images yaml. So all the images yellow that we've got brought in need to be updated, essentially and then scenario, number two uh does not update image log files. If no relocation has occurred so in this scenario we're going to copy it, but it's not going to.

D

Essentially, I guess, uh update the image demo file because I guess we we only push it, but we don't do a copy. Essentially, so I'm guessing so like. Let's go let's we look at the images yaml file and then it's everything's in my registry and then we push up an r bundle in my registry, but we haven't done a copy, but now we now do a pool dash r- and here it's just it's not mentioning anything about updating the images yaml file, it's just pulling all the bundles recursively.

D

So now, when we cap the images yaml file, we it shouldn't say other registry. It stays with myregistry.io and then we're going to cat the nested bundle, images, yaml and that still says myregistry.io, and then we cat the other nested bundle images.yaml file and that still says myregistry.io.

D

It hasn't changed to either registry because we didn't copy it. That seems to be the.

E

Story, so in that second scenario, if one of your images in your images lock file does not exist at my registry.io, you would expect that not to change as well.

D

um If one of the images does not exist in my registry dot o, I o like it- should just.

B

Delete the image.

D

um I think it's whatever the current logic is, which is it's just stays the same. It only checks to see whether it's being relocated and if it has been relocated to the destination repo, then it rewrites the updates yaml file. I think that's what it does. I think we want to preserve that for this recursive logic,.

C

Yeah, I think the answer is yes. Then right like it sounds like scenario, two is sort of a pedantic way of saying. If you didn't copy it, nothing new occurred, nothing, no behavior changes just being really precisely stated: okay,.

B

Any other questions.

B

All right: do you want a point.

B

Cool three two one shoot.

B

You got twos and a one. Either side want to speak to their pointing.

D

I put it to I mean I think, the simple, the simple implementation, the the simplest thing about this implementation is like updating the images log file recursively. The thing that made me go to a 2 is just the way we want to structure the output. It's just all grouped together. I mean I just I've, just yeah, it's just the um just I'm at another point just for the logging. I've had problems with the logging in the past.

D

That's all, but it seems like a lot of the implementation has already been done in a previous story, which is we're already recursively going through all the bundles. It's now just also making sure to call this other thing, which is update the images yaml file and that code has already been written.

C

I see so. Are you saying that there's like some essential complexity in doing the bookkeeping of like what got relocated? Where that, because we're reporting that, at the very end, that that means there's work, we have to change in the way. We're collecting that information in yeah.

D

C

So I have a feeling.

D

Just intuition here, like just the fact that we're grouping all the output at the bottom and not interleaving it like we, you know you see in this bundle. We extract the layer that that's maybe where we would have written the images log file, but we want to consolidate it at the end. I feel like that. Might just drive up the complexity a little bit, I'm not sure.

C

Yeah- and it seems like that's- that is nearly essential, because if we um did that interstitially, then this really important piece of information about the fact that we that the uh lock files are getting rewritten gets buried in all the minutia of the individual moves. So yeah seems essential.

C

Cool yeah, I totally pop up to it too. I didn't think about that book. Keeping peace.

C

B

Sounds like we're landing at a two then, assuming I guess, does anyone want to change their vote.

B

I certainly led the group in a direction that way, but we'll keep going.

B

All right dennis, would you like to speak this one as well.

D

I can I that will get to it this week, though I don't know if you want to spend more time on things we're going to get to, or I can definitely just read it as well up to you.

B

uh It might be useful just just to point it out.

D

B

D

I haven't read about document: recursive bundle feature so yeah. We want to document this really cool feature. We've been working on when we access image package and the cover dev website, there's a documentation page. So we can see documentation about the new feature, some dev notes. We can piggyback on the proposal that dwell wrote and we should explain the pull command to inform the reader where the bundles will be placed on the disk.

D

So when you do a pull with a r and like just explained that just a convention about where the nested bundles are going to be pulled into a.

C

C

Any questions on this story any sense of whether or not this should be something that could sort of easily be placed in existing structure. This might involve calling out or if it's distributed throughout documentation, or that we want to talk about this as like, just a completely separate topic.

C

Any thoughts on that.

D

Or whether there should be a workflow like is this, this is like a workflow we're trying to solve similar to an aircraft environment or is yeah. Oh, like you said, it's like a nested point in the pool, documentation.

D

Could be both like you know, in the pool, doc, there's a sub bullet that explains the dash r flag and and then what it does and just some pointing out things that might not be completely self evident that you know where the bundles, the nested bundles, are going to be placed. The new dot bundle directory structure. I'm talking about here and if we are solving a workflow or use case, which we are maybe a workflow, a scenarios.md.

D

That outlines why you would want to use recursive bundles. That might be useful as well.

C

C

B

Anything else from folks.

B

We are ready to point.

B

Cool pointing three two one shoot got ones and a two opposite from from last time.

C

I think getting words of heart.

C

I I think that there's probably enough in uh in there to like try and convey this succinctly and that, yes, I think the proposal will definitely like help not having a blank canvas problem which, maybe maybe that makes it like a large one instead of a small two, but that's right on the edge for me um and we see how how much it matters being able when we provide clear documentation, it's precise and it's wording and is written thinking about from the end user's perspective and not just describing the feature that can take some work.

C

So that's those are the things I think about when it's not just putting words on paper. It doesn't have to get compiled, but in some respect that sort of makes it even harder. Sometimes.

C

That's my thought.

B

Anyone from the one side.

F

um The reason I pointed as a one was because we haven't added any new. Well, I don't know about this wording, but new functionality like pull copy and push all already existed.

F

So there's a lot of work there prior that we can use. When writing this documentation like we won't be going through a whole new feature or something that intense. I think it's just going to be sort of modifying what we have already with this new workflow.

B

How about we throw points once more? Three two one shoot.

B

All right, cool, good.

B

Conversation all right, so, let's move away from recursive bundles, take a peek at schemas.

B

um So we have a two. That's currently under review, a three pointer, that's in progress and the two already pointed do we think we need to point another story.

C

I don't all right, I think we think we've got enough horizon here.

B

Cool, um so then, I think there is, let's see.

B

B

I was wondering if there's a cap story, that we should point.

B

See so we do have a cap story in flight um how's this going garrett. Do you want to talk about this? I don't know if we we have discussed this as a group.

E

um Yeah we could give it a talk as a group just saying, what's kind of missing in our documentation at the moment and how we could be changing it. um I picked this up about an hour ago, so um I haven't made any progress towards changing the documentation yet, but um essentially, there's also a flat conversation in our carnival uh channel, saying explaining the same kind of issues of not being able to understand uh these fields in these rebase rules for caps.

E

So um for the type field, it's in the examples, but it's not explained whatsoever and um that could be fixed as well as it seems like there's, not really a clear distinction between what new and existing means under the sources field. So some investigation into what really those things mean as well as changing the documentation, I think, is the steps.

B

B

Cool and it looks like there is a little bit of conversation going on in here.

B

Does anyone else want to add anything.

E

E

I myself am still trying to understand it, so if anyone does have an understanding that they can clarify here, that would be pretty nice.

C

I think that the tl, dr, is that um from cold, it's hard to know what new and existing means and like what does changing the ordering of those do, uh if that's not detailed in the docs and then um and then the the the type, uh I think just doesn't have any documentation at all around it so like even just having something would be an improvement just literally like what are the values um so and then a little bit of description about what the expected behavior of that type is.

C

um I think bronson did a pretty good job of like kind of pulling out like contextualizing those bits, but that's the high level.

C

B

Do folks want to point the story.

B

Cool three two one shoot ones ones will take. It.

B

um Let's see so we do have another cap story, that's coming up. That would be after this, which is number 11.

B

Which we maybe could get to this week.

B

Does anyone have significant context that they want to present the story.

C

How does this compare priority wise to the explorer around, um or is this the the initial story for uh externally applied resources like like cluster fulfilled resources? It's the same thing.

B

I think 194.

C

Yeah that's custom logic.

C

Oh, so 11 is above that, okay.

G

Oh um so I know 11 came from dustin as well and for him um one, the 194 was a higher priority.

B

G

So I think 194 explore story kind of spun off. As from the there's another cap story, camp number 52, I believe so before working on weight on wait for crd's existence at one time. We wanted to play this 194 explorer story, so I think that's higher priority yeah.

B

Cool well, thank you um great. I think we have sufficient runway for the week, see any discussion topics added.

B

Doesn't look like it, um but I think we could switch over to the triage help topics that sounds good to people.

B

uh Maybe starting with this first one.

F

um I did provide an update for that one. We mentioned it at the community forum last week. um It was something we talked about in a prior community meeting, so I added some notes. There um feel free to check it out.

F

The tl, dr, is basically adding this feature would change the ux of ytt in terms of how files get output and then also there may be a security complexity with it, because um it's a common use case for people to template um like, for example, kubernetes resources, secrets, um and so the output of ytt could contain sensitive information. So writing that to disk is um something that I think we don't want to support for now, um and that was that is sort of a consensus um from my own opinion and also from the community meeting.

F

um If others have thoughts, feel free to reply in this thread or check it.

F

B

That sounds good. Anyone have any thoughts on that one right now or should we move to the next.

B

Cool so move to cap number.

B

B

Which is actually.

B

B

See looks like there might be a bit of context here.

B

Okay, there's another person running into the same issue.

B

So, at a high level, it's when using null um well, I guess, uh because null is not what in containers is expecting. Is that correct.

C

Yeah this has to do with, like some templating, that cap does, when you have version resources, it's trying to update the references to those version, resources that, when you go to do another deployment, gets another revision um and uh yeah.

C

I I'm not I'm not clear on how to connect that with the init containers, but it's effectively that where cap currently is is, is not relaxing the it doesn't know what type that thing would need to be if it was updating, um ending containers um or any anything that that expects to have a particular like collection.

C

And so I think this is about either just not tripping over the fact that it's null um or, uh I think that's the heart of.

C

It it can be helpful to note that there's like a work around that, like me, of just basically people removing that field when it is null, so uh in some sense the fix would be. I don't know that would be removing that field, but it would be like making sure that we just don't trip over it.

E

I found a thread in our private slack where dimitri comments on us talking about this issue, saying that both of the issue before in this issue 89 relates to cap's use of template rules that, when cap tries to update a reference, it's going to walk through some expected yaml and those template rules are what specify what cap expects, and so we would have to relax some of those template rules in order to allow values like null to be in this field, and I think that's kind of the comment at the end of the previous issue is that we needed to discuss like what are the risks of relaxing these template rules.

B

So I think I guess just more of a meta note, but the first time we were kind of doing this triage help section in this meeting um and we have time. Do you want to.

B

Just kind of brainstorm and and see you know what risks surface to people or do we want to tackle this in a different way.

F

I think that's a great idea and a good use of our time here. um I don't know about others on the team, but I might need some more information about how template rules work currently in order to come up with those risks, um because the the problem with the null types is something that I'm not really sure what the root causes.

C

Interesting so yeah, it would be like um I clicked on that that reference, um let's see, can we point to that so from that from the private slack, here's the line number that can be traded pointed to um in the code base.

C

Basically what he's showing is like. Well, we have all these rules that are defined and notice that there's init containers in some of those rules, so the rules are going to attempt to traverse down this path, and the question is: if we effectively.

C

If we say well, it's not an error. If you can't get any further, because we're going to treat null as equivalent of empty list.

C

Is that a problem like does that accident? For example, does it accidentally cover for what would otherwise have been a defect? It would be really helpful to know that it was null. For example, is there a problem with taking the case of null an empty list and treating them as the same, or is that okay.

G

That's helpful thanks.

C

Yeah, that's not all of what you were asking carrie, but that that's like one step closer of like okay. Well, this, if we were to like dig in through the code, we could see that that's a a big piece of what's happening here is you're not going to be finding it. A line in the code says if it's in it containers like, but in general um yeah. Maybe that's that's the more broad thing is, while we're traversing a path.

C

If we expect it to be an array, can we? How would we know them? Would it when a particular path is an array.

C

Like there's one clue here where the subsequent path element talks about how to traverse an array, almost all of these, I think, are all indexes which means do this once for each uh search through each element.

D

In in that list, this for me also spawns off like a another question, which is so we have these optional fields that might, as a value, have an array, and one of these optional fields is in it containers another one is online 269 volumes, and so, if we decide that null for input containers means an empty array, should we make it consistent for the other optional fields to have arrays as values like volumes.

C

Yeah, I guess I guess when I'm I'm noting is exactly that that actually the way that we would get that done for init containers, there isn't specialized logic for containers. It's.

D

C

That, whatever is doing that walk today, thinks that it needs to traverse over a an array. It was expecting an array and it's not present. Okay,.

D

I thought it was maybe a change in in this file, but it's it's a code change that interprets this file differently.

C

Well, that's another angle: right is basically having like an elvis operator. uh So, like a thing that says: well, it's okay! If this is null yeah or not okay, if it's null.

D

Yeah, I feel like an all index, is true or like something that says: it's: okay, not to treat knowledge, an empty array or something.

C

Like I know they do that in kotlin, so you'd like throw a question mark at the end of the identifier, and it says this may or may not be null and it's called optional operator or colloquially elvis operator may or may not be there.

D

I mean with containers: that's a required field for a we're looking at a pod, but uh you know you need to give it a non empty array for containers. I think- and so, if you gave it null today, I think it's going to break at template rendering time, whereas if we just skip over the null containers array, I think we get an error at deploy time. Kubernetes will say.

D

Actually you got to give me a non-empty array so like, in that case, we're getting feedback and error feedback later, rather than sooner, if we just apply it broadly to every you know, array, treating knowledge and empty as an empty array.

D

That's a great.

D

F

I'm trying to imagine what it means to be an empty array in this context, so like, for example, 274 if we're trying to match on uh this containers with config map key ref. If containers is an empty array, then we'll never match any further down than that an environment or config map key ref, and so is our options like not to error out. If we don't find that item or would we error when we hit the next key that doesn't exist like, for example, m.

C

Oh, I see yeah, I think yeah, so these paths are meant to describe places to go, look to see if there's a reference, and so therefore he has a really good question. So, therefore, um if there weren't any containers, then well, then there won't be any config map key references, so you're you're good, because, like the parents way up, the tree is not there.

C

um So yeah the like restating like what I. What I think is sort of the core question here is: can we and like contain containers like for the reason that you said then, is actually like a really like both motivating and and um as a uh as an example, but also like one that you can't imagine, because you can't have uh um to play without or a pod with pod spec without containers, but init containers you could so the question is: if, if a nit connectors is null, um would that be considered like it's?

C

Currently, an error like cap won't handle that that you have to go in yourself and either remove that field or you have to put in an empty.

D

List and kubernetes is totally fine with in a container's colon null in a pod spec, so we could also just have a way for it when we define this rule saying you know it's okay, for this value to be a normal array, because kubernetes is okay with it being a null array, but we, you wouldn't want that logic to apply to containers, because you want to know that something's wrong earlier, like during template, rendering time, if you give containers column null kubernetes is going to fail.

D

But cap can also tell you that before we even talk to kubernetes as well.

C

Yeah, I wonder, is, is the it's truly a genuine question. Like is this the mechanism that should be detecting that like, if, if, if we're passing in a manifest that has an either null or empty list, containers like that's going to be pardon regardless.

D

Right now, it's blowing up doing I'm guessing I'm just I'm assuming here is during the template, rendering time it blew up before it even talked to kubernetes. Is that true, or where did it fail? Well, it's not it's not blowing up on containers. It's blowing up! Oh wait!

D

Even in your example, if you give containers a null, wouldn't it blow up for the same reason? Yes right, so where does it blow up into like? Is it before it talks to kubernetes, or is it when it's trying to talk to kubernetes.

C

Well, well, we've got: we've got the resources in hand.

C

C

uh We're trying to think about like the order of operations here, whether or not we're in the midst of has to be happening before we apply right.

D

That's the assumption I was going with.

C

Calculating a difference.

C

D

So like with this triage section, maybe it's just as a risk is we might like making decision, might impact how early we you know we can get an error feedback back to the operator, but it still requires digging into.

D

There might be implications as to how early we can give feedback to the user before talking to kubernetes or not.

C

Yeah and I'm wondering if it's even a new risk, because what happens if somebody supplies a containers, that's an empty list.

D

Empty array or no empty array yeah, it makes it to kubernetes and kubernetes, then complaints yeah. So this is not new.

C

In that sense, like that cap already isn't trying to protect you from uh violating the api schema per se, got it that's fair.

E

How is this situation different with version resources, because I know that was mentioned in 189? Was it also mentioned in 89.

E

C

Yeah, what it's talking about is these template rules are being triggered because the subject resource is versioned specifically there's a config map like if we go back to that um those rules.

C

Oh, that's like a.

E

Config map matcher right.

C

And the next one.

E

Is like a secret.

C

Master yeah this this thing here exactly so. This is saying when, when you have a new version of a configmap, here's all the possible places where that config map might have been referenced, so go through this entire path and make sure those are updated too, because cap is generating that new name for that config map, and so all the references have to get updated as well.

C

So that's why this whole thing gets triggered with the versioned resource feature getting enabled for something, and it is by default for commit maps.

E

E

C

Yeah, so I think this really is this is uh probably the question on the table is like uh I mean explore, explore what what's going to happen, the change in behavior, if you just treat something, that's null as an empty list in the path walker.

D

It's like an exploratory testing or like do we have like some testing? Can we can we just like rely on our testing infrastructure around cap, to give us even more confidence, or what does that look like? I.

C

Don't know that's a good question yeah, I'm not sure either. I don't know the top of my head, but um I would hope so I would hope so. But yeah, that's actually you know that's a good excuse to get in there and get an inventory on that.

C

Yeah, because uh did you want to do this for anything that could be null or just collections.

C

Specifically uh sequences like this, I think it's that.

F

Yeah, I would base my decision off of testing and see what the implications of allowing that is.

C

Cool okay, so um does anybody feel confident that they can like update this ticket with, like that, as like a next action step that can be taken here doesn't have to be something um more than that at the moment in order in in terms of like trying to close the loop on this question, you could sort of summarize.

D

What we've talked about.

C

Here thanks gary yeah, I can take that action right. Awesome.

B

Cool, thank you garrett. um We do have one more triage help.

B

Let's see see if this rings any bells for folks, we do have less than 10 minutes, though. So, if no one's familiar with this, we might just postpone until next.

B

C

Yeah image reference in the middle of a string, yahoo.

C

So today, kbill gives you the ability to tell it here's the different keys that you should look within a yaml structure to find its value is, is likely an image reference.

C

But what happens if that value is inside a string somewhere like right here where it's like? It's, not the string. It's a thing inside the string.

C

So scan through a.

C

You know you can, like maybe name name, a key like containers or args, something like that and then say: okay well, within that here's, a here's, a regex that like if this regex matches, then um the match group that comes out of that is the image reference.

D

Would that require a change to k, build I'm wondering? Could they leverage environment a container and okay inside a container entry leveraging an environment variable so like the environment? Variable will be defined as part of the deployment spec and then inside the cli. They would just reference the environment variable and then cable could just find the that environment key and just do what cable does that's not like that that wouldn't require any changes to k, build nice.

C

Nice is there more context on this.

E

Yeah, I remember trying to um kind of provide a fix for his problem for their problem. I believe carrie was a. Is that true?

E

I remember, um but I remember that we couldn't replace the we couldn't replace the image argument without getting rid of that dash dash some image arg prefix, like we could successfully identify like hey match on this image. This is what it's going to look like, um so we provided that search rule, but it didn't exactly do what they wanted. So.

C

Oh yeah, there you go projects, roll nice.

E

And I know we've also been talking about um having some like way to provide starlark functions as arguments in some of our tools. So maybe that's a thought here as well. Instead of having like this regex, it's starlock.

E

C

Yeah, that's interesting right, so I like uh we've, talked about. We have sort of the more obvious strategies and then there's a um all-in strategy. All the rope you can handle.

C

Knock yourself out go get your image reference.

C

um I do. I do really like the way you're thinking dennis about um what do. They call that when you there's there's sort of um there's essential complexity, there's optional complexity and then there's accidental complexity around like um and so there's a piece in here about like well. If it's optional functionality, where, like you, could truly like get what you need with existing functionality and the workaround is not onerous, um it's an important caveat, then you actually can help keep a lid on on complexity.

C

On that particular flavor of complexity, which is really important to the community like the simpler, the tools are, the fewer bugs will be just hands down so um where we have opportunities to reasonably provide workarounds, be maybe an interesting thing to suggest here and see if that flies.

D

Yeah, I'm happy to pair on with somebody to you know, reply um to this issue. If someone's interested.

F

I'm I can help you with that cool.

B

B

Well, I think we made it through the triage list. Thank you all.

B

All right: well, we did it any anyone else. Have anything stop in mind for him.

B

All right, nancy take us home.

A

uh Okay, thank you, everyone for joining this week's edition of the carvel community meeting. This will be uploaded to our youtube playlist and we uh hope that you join us for the next community meeting, which is march 22nd. We meet every monday at 11, 30 a.m, pacific time, 2, 30, p.m. Eastern time we also have carmel office hours where we meet every second and fourth thursday of the month. At the same time, 11 30 am pacific time, 2 30 pm eastern time.

A

So we hope that you are able to join us at one of those options in the future. um Otherwise you can find us on slack the kubernetes slack channel, which is the carville slack channel and we're happy to engage with you in any way have a great week.

A