VMware Carvel Community Meetings, 8 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Carvel Office Hours - April 8, 2021

Description

We meet every 2nd and 4th Thursday at 11:30am PT / 2:30pm ET for Carvel Office Hours. We look forward to seeing you at the next one!

This week's office hours centered around "New scenario on Copy with Rename Proposal for imgpkg."
Details here: https://hackmd.io/5Bh2IXwTShSrA0YdBY4AGg

A

So did you out.

B

uh All right welcome to uh carnival office hours, which is a place for you to come and bring any discussion topics you wish to have with the maintainers of karvel or, if there's any questions or help needed to use the cargo tool suite. This is that opportunity to do so. We meet every second and fourth thursday of the month at 11, 30 a.m. Pacific time, 2, 30, p.m. Eastern time these meetings are being recorded um please and read, and abide by our code of conduct.

B

When you attend these meetings uh today, we have a little bit of a light agenda, but you know, as as we go through the meeting, you know different things that might pop up. I encourage you to add them to the discussion topics or, if there's any questions that arise, you can add them down here in the open questions, help needed section.

B

So looks like we do have something that just was added new scenario on copy with rename proposal for image package. The proposal url.

B

Okay, who wants to go through this, and would anyone like for me to stop sharing my screen so that would it be better suited for them to drive.

C

um I I know that john and eli love to comment on this, but I'm gonna take a stab on it.

C

uh So basically, what we're talking here about is the proposal to when you copy images from one place to another, to one register, to another registry you're able to change the the default renaming. That is done so currently by default, all the images when you copy a bundle from registry a to registry b. All the images are added into the the repository of the bundle, so they lose all the information about where they're from and and that that is helpful in some cases. So that's that's the the general um gist of it.

C

It has a lot on on this particular proposal and to be fair, I don't think we should like cover everything I should people should look into it, but uh in talking with dmitry today uh he realized that there was something that was that could happen that we were not thinking about, and that is if you do have a registry that is replicating the images to another registry.

C

The the way we had this proposal would not work because we were storing the urls of the location or or where the images were being placed in like an article in the place and because, when you do replicate the replication, the other registry might have a different name. Might.

A

Have a different.

C

Url, which means that we would be when you'd be doing a pull or we're doing another copy, you'd be getting the images from the original registry instead of the replicated one. Even though you are trying to get from the replicated one.

C

Does does this make sense as a.

C

A

I have one sort of I'm following along the front and the tail end of the problem, there's like one point in the middle, where I'm not quite connecting, and that is what what is the user doing when or not able to do, there's something in happening in there in the middle, where they have replication configured yet they're trying to do an image package.

A

A

So they want to, they want to relocate the the ultimately is that about. I want my the uh the image images lock when I pull down to be pointing back to the replica registry b that that internal repo, I want my when I pull to get my images locked to like point back to that. Is that that's what they're trying to accomplish.

C

It's like the easiest way I think about this is, let's imagine you have all your images in a registry in north america, okay, but you are in asia and you are replicating this registry to asia right.

A

C

When you're pulling an image from aja, you want to download the images from there because it's going to be much faster. For example,.

A

C

So the problem that we currently have is that we are storing the information about the where the images are like the location of the images in another oci image.

D

C

It's pointing to the north america registry, so when you're trying to pull the bundle from a replicated um in this case asia registry, you will get the images from north america and not from the not from the asia.

C

Registry, so that's a problem that we kind of realized today. So maybe maybe it's helpful and I want me to share my screen.

C

A

So the where's where's, the bundle- live that I do a poll against. In this scenario, am I aiming I'm aiming at the the asia registry right.

C

Yeah you're pulling from the asia registry or you're.

A

C

Any operation from there from that.

A

Yeah yeah so, and part of the challenge is well if it gets replicated the the images.yaml. The images lock all that thing that list of images well that got replicated, doesn't get changed, and so all of those references are pointing back to north america. Yeah right.

C

So maybe it's easier if we just look at how we are storing that information?

C

Okay, so as part of of this change, we are, um we are talking about like there's this kind of problem that we don't know where the images live after the moment that you copied them for for the first time, because they could be all in the same repository or they can be in different repositories that might not even be related to the name that they originally had right. So the proposal what was trying to and the way it was trying to fix or to help there was it- was going to create.

C

Here it was going to create an oci image that contained a file that is called. Let me see if I can find it. I forgot the name of the file images location. It will be in the root of the file system. It's going so it's going to be the only thing that the image has and what is going to have is basically a translation from this image.

C

It is now here if we see if we think about like the example that we were talking about just right now, the yet another registry.io it is our north america registry, and we have an asia registry.io that we are trying to pull our images from, and this is a replicated.

C

uh This is a replicator registry, so image package does not do any operation to move the images to this asia registry. So now, when you're pulling the inform, the only information that we have about this image is that this image now lives in this yet another registry that I owe.

C

And we don't want that, we want it to be on the replicated place right. So this is. This was something that uh today, uh dimitri and I were talking about the new. He thought about this scenario. So this is. This is like a a problem.

C

It is a major problem. I don't think it is like a major issue, but it is something that we will have to address, and this came up because we were talking about another another point here that was this destination. So let me see if I can have like a more interesting okay. So let's take a look at the api, so when you're trying to copy from from, like a registry a to a registry b, an image, what happens is that you can create a couple of overrides of images.

C

So basically, what you're saying image package is that this particular image is going to be copied into this particular place and by having the the registry here you're kind of telling people that it would be. Okay for you to copy one image to a different registry, because you have to put it there right and that could be a problem.

C

So one thing that um yeah- and that was one of the open questions that we had.

A

Can I just want to make sure I want to keep being able to stay stay following here, so the the pro the the the meaning of that first override? Is that anytime, you see this repo reference, so any image that is sourced at that repro reference.

A

It will, during a copy, get relocated to a completely different registry.

A

Which may not be the location that I gave on the command line.

C

Yes, that was something that was an open question that we had. Let me see if I still have it here as an open question, or if I just push this to change. That answer the question.

C

No, I just pushed the answer to the question. So should copying an oci image to a different registry be allowed, so that's, basically the crux of it and like my initial inclination, was not to allow it and, in the end, so after some back and forth kind of decided not to allow it, so you will not be able to so it's like I'm just saying this like this is like a change that hap that happened today as well.

C

So you all are aware, when you're reviewing uh this this proposal, but that was what prompted all all this uh discussion about, the the the.

D

Yeah, the replication.

A

C

So, in the end, you, when you see so this api in here, we are going to remove the registry from here, so that you know that you will not be able to change the registry right and but that prompted the question about what happens. If you replicate the registry so.

C

That was a problem that just bubbled up today that eventually will have to address in this proposal. So if.

A

If we can gain this out just a little bit, it'll help me better understand where the sticking point is so, if the image, if, if the image has been replicated from north america to asia, and then I go to do a copy from the north america to cop my bundle from the north america uh registry to the asian registry, what happens.

C

So let me see if I understood what you're saying so you have your bundle in the north american registry and by means of the registry. The images were were copied, so they were replicated into the asia registry.

C

Okay, so no intervention from image package images.

A

C

Up in the asia site right, so what you're asking what happens if you copy from the the america, the north american registry, to the asian one again.

A

Oh wait now I think, okay, but by me trying to propose that I'm now think I'm seeing better the the scenario, the the problematic scenario. The problem is: let's see, if I understand it, is that the person who then so the replication happens and then the person who then is pulling from the asian registry having no idea how that those images got there got it. So I go to do a pull of this of of the the image pointing at the image. That is the bundle that had gotten replicated.

A

I go to do a pool and all those references are now pointing back to north america. I see.

A

Yeah I mean I, I think we sort of have a general, a really interesting general problem, around replication in general, um like being able to bring image package up to date. That is, update images lock, with whatever happened in replication, like in general,.

D

When you guys say replication, do you mean copy or is there a difference between replicating a registry and copying and bundle.

A

Yeah, like like imagine, you have installed jfrog artifactory, so like a commercial oci registry in these two countries, you can configure these. These things have like rich feature sets. You can do all kinds of stuff with them, and one of them is, you could say well if somebody pushes an image to the north america instance of my registry, I'm a multinational company, maybe I'll.

A

uh Oh, in the background, I want the registries to share like with no nobody touches automatically uh in the background copy that to the asia one and maybe even vice versa, who knows like, but all this stuff, like happens without any user intervention. It's configuration of the registries themselves.

D

Okay, so image package will not replicate a registry, but it will copy a bundle. Okay,.

C

Great question so there's like, for example, like azure one, the azure container registry. Does this seemingly, like you, don't have an idea. You can just have like a one url for the registry, and you will pick up the best, depending on your location, to give you the images. So this, for example, it would not apply on on a case like this. They have like geo geo replication but, for example, in arbor. You can have two registries that are connected in this particular way that have two different urls.

C

So in order for you to get the image from a particular arbor instance, for example, you'd have to get it directly from there.

E

Does this use case still.

E

Maintain the constraint that anytime you're pulling a bundle, there's.

E

I can't think of how to word this, but um like right now we rely on the fact that, when you copy a bundle, all the images will be in the same registry and I'm assuming a replication sort of still keeps that guarantee that you're not just going to move the bundle image over. You know all the images will go along with it, so why?

E

Why do we, even necessarily because we're disallowing the case of copying two different registries?

E

Why do we care about the registry?

E

Why can't we just use the repo from the destination or like wherever and then always do that same sort of fallback, behavior of we'll look in where we're pulling from first, if it's there we'll use that otherwise, we'll use like the original location or something like that.

E

D

I haven't read.

E

Through this proposal, so if it's something that's addressed, stop me and I can just read through it on my own.

C

No, no, no, that's exactly the problem right like it's going to be a question of ordering and that's not captured yet, so let me try to go into uh so one thing that I realized that I didn't. I didn't. I don't think I write wrote explicitly like what is the order in which image pack is going to try to find your images at so the the original intent was.

C

There will be a tag like a nausea image in the same repository as your bundle that has the following tag: sha 256, then the show the bundle image package locations that contains a file that contains the basically like the where the images were where the images like what? What is the original place, where the images are. That's because that's the information we have in the images log file and where it is right now, so that would be the first place where image package would try to find this particular image. This one here is.

E

Is that where it is so that where it is currently gets updated on a copy or something like that to replace the registry domain, it's not just like what repo should we look for it in exactly.

C

So this was the original proposal like this. If you change, if you copied again this bundle that now leaves somewhere inside this yet another registry to the I don't know, uh blueregistry.io this location in here. The change would be that this would become blue.registry.io.

C

E

C

So, and so this is the first place where to try to see okay, so I'm going to try to find this image that you care about and I'm going to search on this particular location first, so the second place where you to try to find uh the image would be. I think it is just right there.

C

I think that was what the original proposal was.

C

So there are like two possible locations or eventually three possible locations.

A

C

Two two, I think two is enough because we are always going to create, depending on it, doesn't really matter what the strategy is for you to copy from a to d. We are always going to create this tag. We're always going to create this file, so this file will always be there as long as you copied a bundle from one place to another.

C

So we know for sure that there are only two places where this image can be, so it is in this place or in that one place.

E

So this is sort of removing that implicit location of bundle, repo added digest. Yes,.

C

So in in today's world, because we copy all the images when you copy from registry, a to registry b, you copy all the images to the same repository as the bundle and you can access the image just by using the sha of that particular image that you're looking for right and this changes that the way it works. So it's it's all. It is also possible. That's like already false track.

C

If you don't do anything, if you don't say like, I want to use strategy blah, it will do that, so the the default behavior is going to be the same.

C

But now you can specify uh now that there's another strategy that allow you to keep the repository name of uh of the image by default, and then there are like a list of overrides that you can say. I want this particular image to be placed in that particular place.

C

And so yeah, so the problem here is that now we have like a possible third registry on the mix that we don't have any clue about, because the image package was not the one responsible for moving the image there.

C

So that's that's like that's an issue.

E

And we want to keep this this.

E

Behavior able to handle different registries.

E

Like I guess what I'm thinking is why why do we even need the full location? Why can't we do something similar to how it does like fall back to this fall back to the bundle repo look for the digest? If it's there use that, and instead have the location, just be a repo and fall back to bundle, location registry that repo digest and if it's there use that otherwise use.

E

The original location or a different different variation, but, like you've, mentioned that sort of fails in the case that we allow different registry copying and stuff like that, like once you once you introduce the ability to do that.

C

So that that was more or less my idea, so my idea was to remove this from here and the location is going to be only there. The repository, so the namespace, plus the repository where you put the image at the only caveat here, is that we need one extra information. That is where this, where, where where was this at in the moment of the copy? So what is the registry that you had that you placed the image at when you did the copy to start off?

C

So, let's imagine you are in registry: hey you do an image package copy to registry b. You have a replication from registry b to registry c.

C

So now, when you pull from registry c, you want to copy the images not from here, not from registry b but from registry c.

C

Nevertheless, if not all images got replicated because you can. This is where it becomes very hard because, like you can define whatever you want to replicate and you might not replicate all the images. So we need to have a fallback to registry b to get the images from registry b and if they don't exist there, then it goes to registry.

C

C

Did that make sense.

E

Yeah yeah, so so there are replication techniques where all images don't get brought over.

C

Yeah so like, for example, arbor arbor has like another another interesting issue somewhat. That could be somewhat more complicated for us. Let's see if I can find the link for their application, so you can set replication rules in arbor, for example, you can define which tag you want to copy, and if that's the case, we because we create an image that has its own tag.

C

It might not get copied at all when when it's being replicated so this can cause us like another, completely different range of problems, but I think I think we're going to ignore these for now and we're just going to say: okay, we will do our best effort, then our best effort is where, where the the bundle is, then we are going to try to search and see if it exists in the bundle in the in the registry where it was copied to first and if it's not if it doesn't exist there, it goes to the default.

C

That's like the original.

E

E

Yeah, that makes sense, I feel, like there's always going to be a configuration of replication that can break the tool, so I think, like controlling, for all of those cases, is probably pretty extreme and out of scope. So.

A

Yeah, I think, is worth digging in in so much that if we can provide a good user experience when that happens, that, like we're clear about what we attempted, like oh tried, to find this image here and then looked over here and then looked over here, and it can't find it um it's better than couldn't find the image quitting.

A

It's like there might be things like that.

A

C

Yeah, in the end, this would be like somewhat like transparent for the user, because in one hand it might have like an impact on you on your when you're doing something, because it might take like a lot more time. Let's imagine it to have a 10 gig image that you need to transport all over the world right like it becomes very time consuming. So that's a problem, but in the end it shouldn't fail you in in any substantial way, because we should be able to get your image into place so.

A

So the so the image location is sort of an attempt to try and help support the pull operation. In doing this, yes, look up, and it's just it's just so that it can update the image's lock file right on the poll.

A

That's what this is about. Okay, so so you could there there is, there might be some options to sort of like eventually make this better and better. Like uh you know, a warning or a linting that says hey. I want to make sure that this thing is only pointing to a particular registry or something.

C

Yeah, like the only case that this will fail, is going to be an air gap environment because you might not have access to to the wide internet where we where the image exists right like so. Let's imagine that you replicate from outside your firewall to inside your firewall, and you do not replicate all the images. We will not be able to get you one of the images because you're this outside of the firewall right, but in that case there isn't much.

C

We can do like as a tool will not be able to to to save you in this particular case.

A

Yeah, that probably, I think, that's a feature like exactly if.

C

D

A

You're in that mode yeah, although I guess there's another part about that, like if you're, if you're in an air-gapped environment, I think folks that live in that world are pretty sensitive to any tool. That's trying to reach out to a location that they did yeah that whole thing.

E

This seems like not even a problem.

E

Of the proposal like image package currently would also suffer from the same problem if, like something's, partially relocated.

E

So it seems like something we just like document that like, if you do have replication rules or something like ensure.

E

All the things get moved or something or else image package has like undefined behavior or something like that- maybe not undefined. That's a little strong, but.

A

Yeah, like I, I wonder if it helps simplify the whole thing. If I don't know if this is oversimplifying it, if the essence of the feature that you're uh scoping out joel is purely focused like that, there's no mention whatsoever of registry in any of the configuration that what we're doing is providing there a an alternate structure, repository structure that things could land on a copy where images could land on a copy.

C

So I'm going to repeat that last part.

A

Yeah that all we're doing is we're saying: okay, it used to be that all the images would land in a single repo in the in the target registry.

A

uh What you could do is take the simple case: you're like okay. Well, I want to move just one image into a different repository must be in the same registry, because you're copying to that registry. So I get to say very special image goes into this one other repository and then motivate another one's like well. I got some overrides like multiple overrides and then you get a bunch of overrides and then it's like the ability to just describe the shape of what set of repos.

A

You want images to land in, but the assumption is well, it's always one registry. The target is always one registry.

A

And that's that's just sort of a base assumption.

A

Now there might be something that follows on from this, so this, like helps with the issue of I've, got everything stuck in a single repo and now it's I I can't. As a human being, I can't tell heads or tails about what is what and maybe there's this so that that so that that's this proposal solves that problem and then maybe there's this follow-on.

A

That says, oh yeah, so things happen outside of the tool outside of image package and we like to give it more be able to give it more information, somehow to be able to say, oh well, well, this stuff got replicated. I want to update you on where these things are located when you go to do a poll, so maybe it's something that does uh generate an image destination image.

A

uh Or, or what have you that? Like help that that, like helps support that that rewrite strategy on the poll.

C

Yeah, I think it makes sense uh so in one hand, I think that, having a somewhat some sort of like a two registry or like a registry field in this file that would have like this could help us have like one more place to check, uh and I don't think it's going to be too expensive for us to build.

C

So I think I would like to incorporate that uh and then you'd have like three places where you can find search for it. It is in the current location in this repo current registry, in this repo on the two location on the repository plus this repo to see if the image is there, if they are different, like the the first and the second choice, are different and then the third location, where we try to find the image it's right, there.

A

And where did where did that value come from? Is that the original value.

C

So this value is the value that exists exactly yes, its original value that you put in the images log file. Okay, so that's already.

A

In the the images lock yeah, but it's a way it's the key into the image.

C

It's the key to connect.

B

A

Part got it, that's pretty crucial because you need that digest. Yep yeah.

C

So yeah so like I was planning on adding that and.

A

You need you need more than the digest there.

C

In here yeah in theory no, but it makes it easier just you can just compare two keys. You have a key you can just compare to you. Don't need to do manipulation of the keys you don't need to like, because on the on the on the images log side of the world, you have the full the repository with namespace registry in the sha. It's like the full image location.

C

So you'd have to be trimming the that string, while in here you can just do a simple compare so in theory it it should be able to be to be just a shot. Yes,.

A

Yeah and I'm wondering if that might help make the intent of this mapping a more clear and b, potentially more extensible, that, like um uh that, the that the key would have a list of locations like imagine. If the digest itself was the key and in what it contained was a list and those were all locations.

C

You'd have to add always one location. Nevertheless, right would you.

A

What would that be.

C

The current registry, where the bundle is because it might not match so like this location, is always going to be just one place.

C

Even if we do a list, it is going to be just one place and it is the place where you copy the image to right so and for that you need, to always add the origin that is equal to the. What you have in the images lock and you'll also have to add one extra. That is the current location, where the bundle is the current registry or the bundle is if it is different from this one. Here.

A

But don't I I'm I'm just trying to understand so, but but don't I have that in hand when I do a pull don't I have that.

C

Oh yeah yeah yeah, when you do the pull? Yes, yes, but this is not a. This is something that we're going to store on the registry.

C

When you do the copy.

A

C

When you do the copy, you don't have like all this.

D

C

Because of replication.

D

D

A

I have to I have to chew on this.

C

So one other thing that I'd like kiel's opinion here is so we do have a as the api say. Let me just go up a little bit, so we do have some overwrites. You have a matcher, so we have. They are the same as the cable, the cabled one, so image repo matches on the on the image, repo plus registry and then there's like image that matches exactly the image with the shot and then there's this destination field that we are going to remove the registry from it.

C

So what do y'all think this key should be called.

A

I mean it's a repo right, it's always a repo.

C

A

Yes, so dust repo.

E

Could it be confusing, though, that that first key is called image repo, but also contains the registry.

E

Because, like one of them, they both have the repo in it, but one of them contains the registry, and one of them wouldn't be able to contain a registry.

E

If we change destination to destination, you go or something interesting.

E

I'm curious why that one is imagery, though,.

C

um It is because it's the same in cabled, that's the only reason why it's called like previously. This was called something like this was called exact match, and this was called something like match, registering repo something like that.

C

But then, after talking after talking a little bit, we decided to go and try to have the same matcher names for all the tools and try to so that people that use cable know how to use image package in reverse. So that's more or less. That's why it is called image repo here and image. There.

E

C

Okay, then, so is there like any other topic? I don't want to be using the full time.

C

Oh 10 minutes there's another topic here, so maybe let's just spark this, if you all have any suggestion for this uh for this field, just let me know I'll be working on this today and since john and eli are going to approve this by tomorrow. uh We have like this buffer right cool right. Thank you.

B

uh I don't see any other topics.

A

Yeah, I I don't either.

C

That's fun because I was looking at the previous week so there's this week.

B

Yeah yeah, so we can. We can still chat if you really want to talk about that. um Continuing on with that topic, or do you feel like we're in a comfortable spot.

A

Yeah I mean: are there? Are there any more open questions? Yet you had a list of open questions. There were there any other ones that were worth kind of teeing up.

C

So I wrote a message on that particular um on that, so I wrote a message in slack, but then I didn't send it uh this the screen correct screen. Yes,.

A

Which is weird because I didn't get it.

C

Yeah, it was just poking y'all to look at it, but then I didn't do it because, because this question rose but basically there are three open questions at this point in this in this uh proposal, one it is, uh should we wait or not for artifact oci in order to implement this, because in theory, if you wait for for the artifact oci you, we don't need to create this.

C

This image locations image that contain only that file when they have the translations, because in theory the oci artifact is going to be a it's going to kind of solve that problem. The other question is if it would make sense to have like an intermediaries in intermediate step that would just using a flag instead of having to provide a file to select a strategy and the other one was.

C

Oh, I think yeah, that's that's the second one. So in the end I think these these are together, but in the end I think these are implementation, related questions and also priority related questions. So I don't think we need to answer them before we get the proposal out uh because to be fair, the artifact the ocean artifact.

C

I think it's going to take some time for it to be out so, and I don't think we want to hold for now, but it's good to have this information that maybe eventually, when the artifact of ci comes out, there might be a uh easier way for us to implement this.

A

Yeah, am I right in thinking that the the image location, tagged image is just an implementation detail of how we carry that? How we store that information, the registry and okay? Well, maybe maybe it'll get officially supported in some other location, and we would just migrate over there to be able to support both for a while.

C

Yeah with with the artifact the oci artifact, we can just add it to the to the image, to our bundle as a dependent image there, and then it would be. It would be much easier because now we are relying on a tag in the text.

C

We all know that they they are not immutable, so people could change the tag through to another image, completely break everything, or something like that, and when the oci artifact comes out, we have the ability to keep this as an image and like branded there in the in the place there where it should be. So in the end, I think it would make it better, but it's not something that we need.

A

Yeah I see and then the other piece about the the flags like whether or not to specify that be able to specify this on the command line. Is that my understanding that other question? So what that is essentially, I mean.

C

I created like a a bunch of like steps for us to implement so like I have an implementation breakout of what I think it should be like the the the most important like the mvps for each phase, and when we implement the strategies we are kind of forcing you to provide a file with a strategy that you want to to select, but you will not be able to do overrides on the first mvp, so only on the second mvp will be able to do overwrites.

C

So the question is: would it make sense for us to be able to just toggle this with a flag in a temporary fashion or in like a more long-term fashion? I don't know, but I feel um that it's more like an implementation detail, that the proposal doesn't need to prescribe for for that right. Now, that's fair.

A

It doesn't, it doesn't seem I mean I think, the one thing that's interesting. There would just be exploring the possible some some possibilities, but you're right. That's that's sort of an added value thing, as opposed to you need this to decide whether or not to move forward with the.

C

Idea, cool. This is also a good time. If anyone has like any generic question about this proposal, you can just ask as well.

A

When do I get my starlark expressions.

C

Mvp4 or three: what is that, like it's the final one? There we go.

C

But it will look neat, but it is the last last step.

E

We're gonna have to install ytt.

C

To use that no you will not ytt will magically just run inside your image package. In this case, it's magic fancy.

A

It's gonna be like a wasm module.

C

Okay, okay, so if there is nothing that comes to mind, I think from my side I'm good.

A

This is great work, appreciate you putting all this thought detail in it and turning all these rocks over. Thank you.

B

Cool uh before we end things, we have five minutes left. Is there anything that popped up that anyone wants to bring bring up? Just talk about.

B

Good, okay! Well, thanks again for joining uh this week's edition of the office hours. If you're watching this from home, we do meet for these cargo office hours every second and fourth thursday of the month at the 11 30 a.m. Pacific time, 2 30 p.m. Eastern time and it's just an opportunity for you to join the maintainers, and um you know if there's any help, you need any questions you have regarding the carnival tool suite it's a it's a great opportunity for you to to bring that up here.

B

If you aren't able to make the office hours, we also have our carville community meetings that happen every monday at 11, 30 a.m, pacific time, 2, 30, p.m. Eastern time and that's just another opportunity for you to join, and you can listen in. You can bring topics up for discussion whatever you might need from the maintainers, and with that we do hope to see you at one of those options and have a great week.

D