VMware Carvel Community Meetings, 22 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Carvel Community Meeting - March 22, 2021

Description

Carvel Community Meeting - March 22, 2021

Discussion topics include vendir v0.17.0 release, kapp-controller v0.17.0 release, and other items. Full details here: https://hackmd.io/F7g3RT2hR3OcIh-Iznk2hw

A

All right welcome everyone to uh this week's edition of cargo community meeting today is march 22nd 2021, just a reminder that this meeting is being recorded and adding to our cargill youtube playlist.

A

uh Please read and abide by our code of conduct, which is listed right here in the agenda.

A

um If there's anything that we don't get to in this agenda, uh we do have carnival office hours that happen every second and fourth thursday of the month, starting at 11 30 a.m. Pacific time we do have one this this week, which is march 25th.

A

So uh please join us for that as well for anything that we might um need to move from this meeting to there or, if there's anything that you need help with regarding karmal, if you're interested in becoming a contributor or anything like that, that's just some more in-depth time that the maintainers have available to you to really get into the project tool suite, and with that, I will hit it off to aaron um to go over the announcements and other status updates, as well as uh the black backlog of different things.

A

B

All right, happy monday: let's see I'll change this view.

B

Here we go um so a few announcements. We got a couple releases last week. um First, one we'll mention is vendor, so adding authentication arguments to the home repo ad command, and then we also cut a release for cap controller, consuming that vendor change, as well as a recent cap bump that we talked about last week as well as maybe the most noteworthy item is this issue that was delivered immediately. Reconciling apps when associated secrets or config maps change.

B

I guess I don't know daniel or eli anything else you want to add to either of these.

C

Only other thing is that um there was the addition of adding an image package bundle as a fetch source, and this has actually been around for a bit, but I I think we just had some um time between uh when our last release was cut and um when this was added. So that's the only other thing in there.

D

I've been taking a look at community issues this morning and one came in about vendor version 17, which no longer has the versions uh package which k build now depends on. Is there any uh consideration or plan for that, or is that something that we'll just have to address.

E

um I'm not sure when it was removed, but I can look at that real, quick and see, I think, if it's removed, we should probably update k build to use wherever the new part is or the wherever that code moved to.

B

Cool sounds like you might need to make an issue.

E

Awesome, oh yeah, okay, so this was done because we had to generate some uh like kate's specific code for that using the code generator. So it should just be updated. It's now in like a v1 alpha one package instead, so.

B

Cool thanks for the live troubleshooting, um it's one other thing to mention was we also cut a new alpha of cap controller last week as well, including some of the changes that we just mentioned in zero um and likely some kind of stability experience uh enhancements as well.

B

uh So on to oh, I guess last announcement as we mentioned last week, but we are hiring a staff engineer, so please check it out. If you are interested um status updates, we have a number of things in flight ytt schemas. It's in progress currently folks are working on the type checking private libraries anything else. These people want to add there.

F

Just while we're talking about ytt, there are also a couple of uh nice like ux improvements that we're looking at uh excited about releasing um one was detecting when your yaml is um not a template, just a plain ml file. Now we're automatically recognizing that, so you can just pipe yaml in as part of your set of files that you're sending in and it'll just work. Just for that file, which is great, you can turn off that ignore unknown comments flag uh so that that'll be coming soon.

F

That's exciting and then another improvement that came out of schemas is the ability to overlay. When you have a value, that's currently null to like add in a different value that was like before you would get hey, expecting a map, but was nil, um that'll be uh in in the past soon. So we're excited about a couple improvements that are just being pushed forward by schema stuff. It's great!

F

So look for that in the next release. Awesome thank.

B

You uh we also have recursive bundles track in progress. Any any updates.

G

Doing progress just working on updating the image lock files in this bundles recursively and in parallel with this, is also a ux, a ux improvement, which is ideal like we submitted up here upstream and that got merged we're waiting for that library to get updated so that the ux of image package can change from what we have to date. Where to you can see in real time how many bytes are being uploaded versus the total amount of bites you're waiting for so you can maybe see a progress bar or something to that effect.

G

That's about it.

B

Very cool: uh let's see, we also have some cap enhancements in progress. um I believe. Last week we had a like a ducks improvement story. um I think that was wrapped up yeah anything going to show there yeah.

H

That that pull request is on karl dev and it's open for review, because it did go through an initial round of review and changes were made. So if anyone wants to review that for clarity and correctness, that would be.

B

Great cool um and then cap controller world, uh the package measure, api mvp, is still very much in progress. um I don't know daniel eli anything you want to share at a high level. There.

E

No, I don't think much has really changed in the last week. I think.

E

We just merged a change that was mentioned before I think of now. You can update your values for a package and it'll get pulled in immediately, um but other than that, I don't think there's been any major updates.

B

Cool um and then, similarly with with cap controller, we're also weaving in some some enhancements between all the api mvp work, so those are ongoing as well.

B

If folks have any triage help topics they want to discuss today, please feel free to add a add. A note here same with more general discussion topics we'll try to keep an eye on these lists and if any topics show up we'll uh try to cover some time out at the end of this meeting.

B

But if no one else has any announcements or updates, we can dive into the backlog.

B

Cool, so let's do some filtering.

B

All right, so things are, I guess, from from, like a priorities perspective this week are still fairly stable from what we've been working on recursive bundles, still very much in flight.

B

Let's see, we have a story currently in progress, a documentation story pointed and then we have the, I guess, uh kind of one, one more story which is remove experimental, the the flag and then an additional follow-on story for supporting copying bundles referenced by other bundles and images. Lock.

B

um Let's see, maybe we can, I don't know, discuss one story this week. Does that sound fair.

G

I don't think we'll get to it this week, but we can definitely discuss it and point it, and just that's also. Okay,.

B

Okay, uh I don't know if we don't think we'll get to it. Maybe we can. We can just postpone until next week then.

B

So it sounds like we're good, because bundles front so take a peek at schemas.

B

See so we have this check private library data value story in flight. We have pointed the the next story about supporting floating types is feeling pretty good for the week as well.

F

Yeah, it seems like we got a reasonable amount about, yeah carries nodding as well, so yep.

B

Good stuff, uh this cluster fulfilled resources. That's one that we had already discussed as an explorer story. um Let's take a look at what is it show all issues not in an epic.

B

See so we do have this image package. One recursive pulling should be filterable okay, so this one jumped to the top. So this one might be worth.

B

Discussing see eli since you're here, would you be interested in presenting this one.

E

uh Yeah, so this is an issue that we, from the packaging side opened um essentially adding a feature that allows users to filter which bundles will be recursively pulled from the images lock file.

E

So the specific use case we had in mind is: there's an idea of a package repository in the packaging apis and not every single bundle in your images. Lock file will be a repository, but cap controller only wants to pull repositories.

E

So, instead of spending the work to pull every entry in the images lock file and then go through and manually, determine which ones are repositories or not, this feature will allow it to like be a little more efficient and only pull certain bundles based on the annotations.

E

And then that's the cap controller specific use case, but the generic one is users can add any annotations or other tooling, can add any annotations and you'll be able to pull based on those.

E

I don't know if joao had more details about like implementation or anything, but that's sort of the high level use case that we had in mind when opening this issue.

F

And this is specific to a poll. This is this: wouldn't affect copy or anything.

E

Yeah because you get into someone's pool yeah, if, if it's added to copy, you get into some weird cases, where there's no longer guarantees that all the images are coming along with the bundle and then if people don't know that certain things were filtered beforehand, you'll get like weird failures and stuff. So only for.

E

I

Have we converged on a ux for this yet like how would the user specify a filter.

E

um I don't know the answer to that, but my speculation would be just a flag that like allows specifying an annotation or.

F

Something dennis, do you know whether or not we've already exposed and like a namespace for referencing, a bundle.

H

F

This one is that something that the user has access to.

G

No, not that I'm aware of like when we push a bundle. We label it and we expect a certain project structure, but there's no that I'm aware of yeah and name space for it.

B

Okay, so it sounds like maybe whoever picks up this story might have some discovery work to to do. um I mean there certainly is a proposal as to a way that we could take this. But at this point I don't think we I don't know have converged on a solution yet.

G

Yeah, I think if I was to pick it up, I would definitely work on turning this into a story having acceptance criteria like similar structure to the other stories, we've had get feedback from packaging team, eli, etc, and then, once it's good to go, we could point it.

B

Okay sounds good, um so maybe what we need to do is pull this from our prioritized backlog and uh do some discovery. Work first. Does that make sense.

F

Yeah, I think so because it's adding api, but it's adding adding to the interface so we'd want to be really careful not to commit to a certain approach that later becomes not not ideal. For other cases,.

F

F

F

So we'll we'll look at that story as as a kind of explorer is that is that the way to maybe look at that particular card right now say hey this one is: do the exploration to try and define the story behind this yeah. That's prioritizable.

B

Is there a good way we can like? Should we add a label to this or something.

F

To say it's an explorer yeah like one thing, we've done with other stories is just make that the prefix of the story name, that's a cheap, easy way of doing that, and then we've generally given it like a zero point.

F

Just to like I don't know so, that's one possibility yeah.

F

F

And so then, to I'm look, I'm usually looking for like what are the set of questions that need to be answered.

F

That would like make the story that, like make it possible to write stories, so carrie asked a great one, which is what's the ux, so uh that'd be a good story. That'd be a good question for us to.

F

F

And then I asked a terrific question around: uh is there sort of a general way that we want to refer to bundles.

F

Because this is being recorded, I have to say, like that's, tongue-in-cheek, the greatness of my question anyway, but then that, like the expected outcome, would be either we've come up against some kind of we discovered something that sort of prevents this for some crazy reason. Maybe maybe there is or we've got.

F

B

Cool, does it sound good for folks.

B

All right, I like the idea of just zero and explore and then once it's worked on, then we can revisit it.

B

Okay, um so I think we're we're pretty good in terms of like runway. uh Let's see any topics added.

G

B

Yeah, I don't know, would folks find value in discussing any other stories. Anything that's in flight.

B

Discussing anything at all.

F

Is anything from any of our proposals right now? That would be worthwhile bringing up in this forum.

B

Well, one thing I'll mention, since this is a community meeting and the community will could benefit from this- is.

B

I drafted a pr that describes how our backlog works, um in particular like what the name of the columns are descriptions, any labels we use and how they progress. uh Like I mentioned, this is still under construction, it's in draft state, but but please, please, do give any early feedback. If you have it.

B

That's a plug for my my proposal. Anyone else want to plug theirs.

E

Okay, so where's the um accepting unannotated yaml is data values proposal at.

F

Yeah good good question: eli we we have, uh it looks pretty good. We have an approach, so we believe that at a minimum we'll want to support an additional um yeah thanks, aaron yeah. So this this one. Let me contextualize it real quick. So this this proposal is about there's a there's, a need for many different places, and this is sort of actually a long-standing request. Can I please provide to ytt?

F

Can I provide data values that are in just plain yaml, that is that don't have any annotations on it and that I can somehow tell ytt treat this file as if it were a data values file. um So we have an approach.

F

The sort of fundamental approach is we'll introduce a new file mark type, so file marks are a feature within ytt to be able to say here's a path that describes one or more files, and I want you to change how you process it.

F

Sometimes it's saying well, you'll omit it from the output when by default normally would be included or usually you think this is a template once you treat this as a file that could be loaded by a template and used as data and then this new one we're going to add is usually you might think. This is just plain yaml and you'll process it that way, but actually we want you to treat it as a data values file.

F

So you point to the the file you want to be a data values file and you say, type equal, yaml dash data values. So that's sort of the general approach.

F

There's one thing that this surfaced, which was in the filemark feature itself.

F

The path that you give to identify the file or files is actually a relative path. It's relative to once we load up all of the files from the dash f parameters that you give on the command line, those all get kind of squished into a single tree. You can sort of think of it as like mount points.

F

If you're used to like the unix file system, you can load up and attach different uh drives to make it look like it's one single file system, um the problem with the the challenge that comes up with that is, you could actually have two files that have the exact same path and it's possible that you may want to treat one file one way and another file another way, and today there isn't a way of doing that.

F

So uh so we wanted to solve that before. We went forward with this proposal, because what we don't want to do is accidentally turn multiple files into data values, files and, frankly, it'd be nice to be able to solve that problem. Generically for file marks.

F

So there's an issue uh that uh that we created around that to um sort of like think it was like a mini mini proposal. If you will just saying hey what what, if we accepted uh operating system file paths optionally for that path, value of a file mark.

F

If you did that, then you'd be able to uniquely identify a file because you're back in the operating system, land that place where and you would be able to like point uniquely to a file so that'll be a nice downshift option. If, if you run into a collision, you can name that otherwise it's fully backward compatible.

F

We would continue to support relative paths so that that issue just needs a little bit of review and then turn into sorry to get implement that one block the the proposal uh and and then then we can proceed with that primary implementation.

F

Oh there's another! Half to that! There's that part did. I lay that out well, eli.

E

Yeah yeah that.

F

Was good cool and then there's another half to the proposal or or a second potential approach which would be to introduce yet another flag, but it'd be specialized. It would be dash dash data dash values file and what this flag would do is include a a specific file that you name as as its argument, and it would give it that same treatment as if it would file market as a data values, file and the reason for there's. There's two reasons for that. One is that.

F

First of all, if you cannot think of the of the ux of like the case where somebody's using ytt from the command line, it sure reads nicely if you say: ytt dash, f, config, dash dash data values, file and then here's the data values file and often if I am a configuration consumer, so I'm somebody who's using a ytt library.

F

Then my data values file is probably not inside the library in the first place. So it's it's not unreasonable to think that that file wouldn't have been included um in the config in the first place. So that's kind of nice. It reads nicely the command line, sort of says what it does: it's sort of self-documenting, it's pretty cool and then the the other case is in providing data values um that are pure, that are plain yeml, no annotations that contain sensitive data.

F

Maybe you maybe are including a decryption key or uh secrets bodies, or something like that or something assert or or sorry, private key.

F

For some reason you want to include that, um and so you don't want it written to disk you you'd rather use. What's called um uh process substitution. That's for that! You do that like redirect and then there's a parenthesis and whatever is inside. There actually gets its own file descriptor like it's piped in as if it were like a file, and you can even give a name for that thing, so uh sometimes to be useful, just to be able to not have to give it a name but be able to do that process. Substitution.

F

um So that's that's another! It's sort of a nicety, it's it's! Definitely. The second option is a convenience feature, so I think we're going to split the first part in the second part and make sure that we at least get that file mark approach, completed and then consider the you know the added complexity and maintenance that it might be to include this convenience feature. It would be a little bit more judicious about that.

E

Okay, cool so like the the preferred way, at least initially is going to be file marks and then maybe some sugar comes along later, but cool yup. Thank you for that recap. Yep yeah great.

B

Question cool good stuff, um I think maybe just like one other thing to note with the the image package proposal here. Is that joao, uh I believe, had oh, oh, what did I do.

B

So he he shot this off before taking off for a couple weeks, but is planning to revisit this pr. This discussion next week so still still time to provide any thoughts there, um but yeah anyone else have. Oh, we got some topics now.

B

Okay, we're capturing what we talked about.

B

Cool anything else from folks.

F

All right I'll I'll do one real, quick, uh not to expand the work to fill the time but just uh be on the lookout. There's a another proposal that I'm hoping we'll get resolved pretty quickly going to put in today around how we manage documentation um so I'll make some noise on on the carville channel just be just be on the lookout for that we're really hoping to there's a couple parts that are a little bit bumpy right. Now, it's not a huge deal, but it'd be nice to get that resolved and going.

B

So sounds good cool. I think we can call today's early then.

A

Okay, all right well thanks everyone for joining today, and uh this will be uploaded to our youtube playlist uh later today, so be on the lookout for that. um If you weren't able to attend today's meeting just a reminder, we do have the carnival office hours this week on march. 25Th uh looks like the bulk of that will be available to use for um anything.

A

You might need help with from the maintainers, whether that's learning, how to contribute or if there's something that you're stuck on or are interested in a particular part of the tool suite that's a great opportunity for you to to come and attend those it's going to be at 11, 30 a.m, pacific time, 2, 30 p.m. Eastern time, on march, 25th of this week and again we do meet um every monday for our community meetings. At that same time, 11 30 a.m. Pacific time, 2 30 am 2, 30 p.m.

A

Eastern time, so we hope that we see you at one of those options soon with that have a good week.