VMware Carvel Community Meetings, 2 Dec 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Carvel Community Meeting - December 2, 2021

Description

Carvel Community Meeting - December 2, 2021
We meet every 1st and 3rd Thursday of the month at 10:30am PT. We'd love for you to join us live!

Full agenda on this week's meeting found here: https://hackmd.io/F7g3RT2hR3OcIh-Iznk2hw#December-2-2021-Agenda

A

Hi everyone welcome to this week's edition of the carvel community meeting. If you are watching this from home as a recording, we would love for you to come and join us live as of right. Now we are meeting every first and third thursday of the month uh we meet at 10. 30 am pacific time we're doing this right now, probably until february or kind of tbd, just as things wind down for the year and then start back up for next year.

A

We won't be meeting every week just every first and third thursday, with a bit of a break um starting at the end of december and probably mid-january. So um we encourage you to come and join us live if you're able to for the times that we are meeting it's an opportunity for you to bring up anything. You wish to discuss with the team regarding the carnival tool suite anything you need help with, you can come and meet the maintainers or just listen in you don't have to actively participate.

A

We would just love to have others from outside of the maintainers come and join us and get to know you a little bit.

A

If you aren't able to join any of the meetings in person, you can come reach us on the kubernetes slack workspace in the carville channel or at twitter at carnival underscore dev.

A

For those that are on the call right now, if you have anything to wish, you wish to discuss with the team, just a reminder put that down in the discussion topics section and anytime, you are engaging with us or anybody in the community. We ask that you please read and abide by our code of conduct.

A

For those that are using the cargo tools, we would love to to know more about how you are using karvel.

A

If you go to this issue that we have pinned on the carville repo, you can add a comment in there providing those details regarding the word is any organization you're representing the website to the organization, uh the country where it's based or where you're based any contact information you wish to share and details on how you are using any of the karmal tools.

A

You can also add your logo to that comment, and we will add it to our doctor's file for some promotion of your organization and just to showcase uh to the community what other organizations are using the carvel tools.

A

So for us it helps when we hear about how folks are using the karmal tools how to improve the product. How are they using it? What what could we do better things like that, so we would love to know more about how you're, using those tools for those that are on the call today, please add your name in any organization that you are representing.

A

This will help us keep track of those outside of the maintainers and see who's attending, and we can keep those lines of communication open.

A

Some announcements, uh I'll just as I said at the start of it you know, community meeting is moved to every first and third thursday. We were previously doing every thursday, but, as you know, we're winding down, we are just doing. First and third, thursday of the month, we had a recent release for ytt 0.38.

A

I know carrie is going to do a demo over this in a moment, but carrie did you want to go over anything in particular, regarding this release before we get the demo going.

B

um Well, this is the first release that allows you to generate an open api document from a ytt data value schema. That's, I think, the most important thing to call out here. There were a couple other editions: um we've got a tamil built-in library and um error message, key suggestions for schemas.

B

So if you have a key, that's just maybe slightly off we'll suggest you the actual key name um and then a couple other bug fixes and uh contributions that uh you know really helped out with this release. Yeah. If anybody else wants to call, it me out, feel free to speak up.

C

Let's see the the tunnel stuff is something we've had some requests for, uh for example, container d config is in tamil there's others as well, so it's really sweet that we got a community contribution to just now. You can uh include that uh in your input and output, um nytt, that's super awesome and then some of those uh bug fixes are really relevant like um if you uh um needed to have a percentage sign in your in your formatted yellow.

C

You couldn't do that before, but now you can that's really awesome, um and then um we had some sharp edges uh that we've had some folks on the team really focusing on making the schema stuff the um easier to to work, which was just great we're trying to improve that overall experience so and we're going to keep doing it, there's more things that we want to do to make those improvements very excited.

A

Awesome thanks carrie and john.

A

All right, it looks like we just had image package release, 0.23.1.

D

uh Yeah that was like a quick release that we had to do a couple of days after oh 23 came out because there was a. There was a bug when you were copying um a bundle from a tar to a registry in an air gap environment. It was trying to reach to the origin uh repository due to a change that was done in o230.

D

So if you are using o230, please update to the newer version.

D

A

A

Okay on to project road map updates.

E

um I quickly made some updates to this, so planning is is very actively happening right now. So that's why a lot of these timelines are still tbd, but these are generally some of the things that are going on and we have started at least like some discovery. Some exploration work around the open api document, metadata and ux improvements. This is a follow-up release that will just provide a bit further customization of the exported open api documents from ytt um plan.

E

To start that soon the timeline is still being adjusted, but we we are planning to pick up that work uh as soon as possible. Some of the other work that's going on is the cap controller cli work.

E

I believe sumic and praveen have been running with this, and this is to provide a user interface for interacting with cap controller, feel free to click into that link to see what is all the work that they have scoped out there, but they've been making good progress there and some of the other things that we have coming up schema ytt, schema, validations, carval, asset, signing and verification.

E

I think we can provide probably more more details on that stuff in the future, we're just somewhat actively in the planning stages right now,.

A

um Thanks aaron: uh does anybody have anything else they wanted to share regarding their road map.

A

All right, uh carnival backlog is there anything outside of the roadmap that wasn't touched on that the team is working on this week.

F

You know I can share one thing that we landed in cap controller, which uh you know is going to be of of tremendous interest to everyone at home, who's also developing cap controller- um and uh you know basically, um we have. We have a deploy script for deploying cap controller if you're iterating on your local changes and we now have a dev hyphen deploy script. That runs uh orders of magnitude, uh one order, magnitude faster, so the old deploy script took a few minutes.

F

The new deploy script takes uh about 20 seconds um and so there's a similar one in secret, gen controller. So again you know ever all all the controller developers listening to the recording um you're welcome.

A

Well, good stuff, thanks joe.

A

C

Yeah there's one thing I can add: that's about to land for ytt we've had um a sort of a standing issue around when you have plain yaml that you uses the um anchor and merge operator.

C

So there's a yamo operator where you can say: hey here's a chunk of yaml and I want to reuse that exact chunk later in the document you like, give it an anchor and then you reference that then there's a further feature there, where, like you, can override values for keys that show up in that inserted piece, it's kind of cool, so you can have like a basically like a templated mini little template piece within yaml itself.

C

So it's not supported currently in nytt is uh we're not able to. We haven't been able to support overwriting map keys, but there's a pr. That's in flight, that's just about to land! That's gonna support that and that's a little bit of a mini milestone for us, because at that point most of what you'd expect to be able to do um in yaml. uh It will just pass through uh quite nicely, as as you would hope, uh when running through ytt, uh probably the last piece.

C

There would be like supporting emitting comments um and that's something we're. Definitely. We are thinking about that's down the road uh quite a bit, but um but this is really nice because you might refer to third party yaml in in your setup uh that uses these. These features, these slightly advanced ammo features and you'd have to do some rig a roll beforehand before you can include them in your set of ytt templates and stuff. So soon that won't be so that's kind of exciting.

A

Great thanks john: does anybody have anything else they wish to share.

G

Oh sure, so there was a question from a user uh to set map keys dynamically from data values, and um that can be done using text templating, but a user was not able to find it through faqs or by search. So we added that um in the reference docs and added it in the faq also added uh it added example: reference for such requests- and there is another um part of error message or hint message improvements.

G

We are working on uh to provide special, uh specific hints for uh boolean boolean values that can be replaced by a and d or or symbols. So um these two are uh in progress this week.

H

And one last update that I like to provide is the track of improving authoring, schemas and schema annotations. So in the needs review column, you see 521. That work is basically done of improving these error messages with the actual annotations in the text and at the top of the in progress. There should be a story about uh returning errors when you have a schema, annotation nested under in any type.

H

So there was some confusion and some rough edges as john mentioned earlier around this- and uh this is the last step in this story, so uh we should be looking for some improvements there for authoring schema soon.

A

Wonderful thanks, marcia and garrett.

A

D

uh For image package it landed developed, so it will come out in the next version. The possibility to provide image package with the number of retries that you want image package to do when it fails. It has like a temporary error trying to connect to a registry. That is something that was asked and we do are working, and I hope that we land that before the next session the next week at the next release the ability to always tag um images uh whenever you're you're ever pushing a bundle.

D

So that work is in flight, and I hope that we can land that before we release the next version.

A

Awesome thanks joelle.

A

A

Okay, uh all right carrie. Are you ready to do the demo.

B

Sure yeah I can. I can share my screen.

B

All right cool, so I just wanted to show a little bit about how you can use a ytt data value schema to generate an open api doc that you can use in many different ways. You can use it for documentation for your api. You can use it to deploy a package in your cap, controller cr,.

B

It's a way to just declaratively state what is allowed in your configuration as inputs.

B

So uh let me just start out from the beginning: we need to talk about what a data value schema is before we can get into an open api document created from a data value schema. So on the left. Here we have a data value schema. You can tell, because it's got this annotation.

B

It's got some schema annotations in it.

B

If you're using ytt right now and using data values, you might be more familiar with a data values file which might look like this. So it's got comments in it to describe things. um It's players, values, um inline, so similar differences here are in a data value schema. It provides not only like the values themselves as defaults, but also type information, so, for example, for your load, balancer enabled this value is defaulted to true, but it also provides type information, so this must be a boolean.

B

So if you're a configuration author, you declare a database schema your configuration. Consumers must include that information using the correct types so like, for example,.

B

If we're going to use this, we've got uh this value file, schema file and then just like a simple template here. I'll show you what that looks like it really just outputs the data values.

B

So if we run that it should work- and it's going to be all the defaults in your schema uh overlaid with the values included in your data values, file right so enabled was overwritten to false, because that's allowed it's a boolean um for arrays.

B

You can declare defaults in your schema here and any values included in your data value file will be added right. So it's additive here.

B

One of the benefits here is, if you are totally messing up- and you accidentally put something that is not a blue in here, you try to run the skin you're, going to see an error that actually prevents this from being templated, so it'll catch things before you deploy them.

B

All right so we're going to totally forget about this values file and only going to use the schema, because, really that's all that's needed to create an open api document.

B

So um these fields here um they can create open api keys. So, for example, this can provide us documentation when reading your schema. So you know what this means, but it also will include a description key in your open api schema that has this string.

B

um The schema nullable will allow this value to be either a string or it can be null and you'll see that reflected in your open api schema.

B

So like, let's see what that looks like all right, we're going to take our schema file and we're going to run it through this command in ytt data value, schema, inspect dash, o for output, and it's going to be an open api, v3 format and we're just going to write it to a file.

B

All right, so this was created.

B

So this is what the open api document looks like that ytt just made from your data values schema and you can see everything under component schemas data values. Everything in here is the actual schema for your values.

B

um Everything above that is like header information, that's used by, uh like any other third-party open api tools that you need. This just makes this valid open api. So, if you wanted to, like, I don't know, generate uh html or documentation based on this schema, then you'll know it's in open api, 3.0 format.

B

um And then just yeah to highlight this like, for example, load balance without enabled it's a boolean and it defaults to true. You can see that right here or sorry right here, yep, so it creates all of the type information. That's in your data value schema and any comments as descriptions right.

B

Right so this can be used in a cap controller package. That's one of the use cases that it it supports, and.

B

You might do that by having a package custom resource as a ytt template, for example, and say you want to put your open api schema in this open api v3 field.

B

This acts like documentation for what values are allowed in your package. That, for example, like this package, will get your configuration from an image and then template it using ytt.

B

uh You can add values here, and you can read this open api v3 section to know what values you can put in your templates here.

B

um So what we got here is just this key. Let me show you the command. We're going to run. Make this make a little bit more sense.

B

So we're going to just template this package and we're going to include our open api document as a data values file and we're going to name it open api and include that file as that data value.

B

So that's what this is here. It's going to take that open api key and it's going to get the component schema's data values out of it right. So, if you remember that's everything down here,.

B

um And then this is necessary because uh when reading from the command line, it's going to read it in as a string, so we just need to convert that into yaml so that we can access this field.

B

All right, so, let's template this and see what we get.

B

Yep we get our package out and it's got all of our schema here.

B

So then we can deploy this and use it for uh our configuration.

B

um So that's really like the main use case that I wanted to go over today. The open api schema can be used for other things as well, um but yeah I'm happy to answer any questions about that.

A

Thanks gary does anyone have anything uh any comments, questions for carrie.

C

One thing comes to mind, for me is um give a sense of like what the next steps are there things that um uh that we're gonna do with this that's coming up.

B

Yeah, api yeah, so there's there's more improvements to come. So currently this is the first iteration of this. um You know it supports uh nullable values, um keys that can be any type. um It will generate a description key for you, but there are other things that you want to add like. I think the most immediate thing is adding a field for deprecated and a field for examples, so that you can basically just give an example of what a value might look like and then there's another part which is upcoming for validations.

B

So say you have more specific needs. Your values need to fall within a certain range or need to have certain characteristics that are more complex, that you want to express in, uh like a validation um that will be added in the near future as well.

B

And also like, if there's something that you want, it's an open api that you know is not currently supported by ytt like come, submit an issue or put a thumbs up on the issue so that we know that you want it, and we can prioritize that.

C

Cool the other thing, I noticed, if you didn't wouldn't mind, sharing your screen again, that when you had at one point, you had like both the open api uh description of the schema alongside the ytt schema, um but, like you kind of have to scroll on the left side, to see everything and on the right side, you have all the same information, but it's compressed just thought that was kind of cool like this is a reasonable schema here, um and uh I don't know I'm finding.

C

I don't know if it's the syntax, highlighting or if it's just not as much text to parse but finding the one on the right, pretty pretty approachable.

C

So I thought that's kind of cool like to be able to live in both worlds, um maintain stuff in a concise expression with this people that participate in other tooling in the standard format.

B

Yeah, that's a great call-out.

A

Oh any other questions comments.

A

Nope yeah, that was really great carrie. Thank you for showing us and providing that demo. uh If does anybody have any other anything else, they want to bring up outside of the demo.

A

No okay! uh Well thanks for joining us today and um we'll meet again on the third thursday of december, and then, after that, we'll take a little bit of a break. We'll update the agenda for when to expect us uh to reconvene for these community meetings. So with that, we hope to see you at our last community meeting of the year and um have a good rest of your day. Thank you.